Big data, emerging technologies and the characteristics of ‘good intelligence’

ABSTRACT

What constitutes good intelligence is best understood by practitioners but has not been explored through empirical analysis and in the context of a digital age. This paper presents the first research inside all the agencies that form the Australian National Intelligence Community exploring how they are impacted by big data. Intelligence is often opaque to outsiders, yet understanding the characteristics of good intelligence is important to societies that rely on intelligence agencies for national security. This paper reflects the previously unheard perspectives of members of the agencies that form the Australian National Intelligence Community – where there is a significant empirical gap. Semi-structured interviews with 47 participants explored the impact of big data on intelligence and decision-making in Australia. This paper finds that intelligence must meet the following characteristics, many established in historical literature, in order to be considered good intelligence; (i) timely, (ii) purposeful, (iii) actionable, (iv) accurate, (v) provides value-add for an intended audience, and, (vi) is unbiased. This article explores and unpacks each of these characteristics of good intelligence and finds they remain critical in a big data era.

Acknowledgements

This work was supported by a Data 2 Decisions CRC National Security Big Data PhD Scholarship and a Deakin University Postgraduate Research Scholarship. I am very grateful to those who contributed to – and vastly improved – prior versions of this article. Thank you, Chad, Clinton, Kate, Meredith and Mish. Thank you to the anonymous reviewers for your suggestions. Image design by Susan Beale.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Notes

1. Andrew, The Secret World; Hughes, Jackson and Scott, Exploring Intelligence Archives; Van Puyvelde, ‘Qualitative Research Interviews’, 6–7; and Zegart, Spies, Lies, And Algorithms.

2. Gill and Phythian, ‘Intelligence’.

3. Lundy et al., ‘The Ethics of Applied Intelligence’; DeGennaro, ‘The Gray Zone’.

4. Lundy et al., ‘The Ethics of Applied Intelligence’.

5. Andrew, The Secret World; Hughes, Jackson and Scott, Exploring Intelligence Archives; Van Puyvelde, ‘Qualitative Research Interviews’; and Zegart, Spies, Lies, And Algorithms.

6. Symon, ‘ASIS Director-General Launches New Book’, 4.

7. Walsh, ‘Building Better Intelligence Frameworks’; Walsh, Intelligence and Intelligence Analysis. Five Eyes is a term used to describe the national security intelligence sharing arrangements of Australia, Canada, New Zealand, the United Kingdom, and United States of America.

8. Walsh, Intelligence and Intelligence Analysis.

9. Coyne and Bell, ‘Strategic Intelligence’, 25; Gill, ‘Theories of Intelligence’, 213–4; Kahn, ‘An Historical Theory’; Rolington, Strategic Intelligence, 17–8; Scott and Jackson, ‘The Study of Intelligence’; and Wheaton and Beerbower, ‘Towards a New Definition’.

10. Lowenthal, Intelligence, 1–9.

11. Rolington, Strategic Intelligence, 17.

12. Omand, Securing the state. National Security is defined broadly. Omand sets out three propositions underpinning the modern approach to national security; collective psychological safety, citizen centric view of threats and hazards as well as informed decision-making, adopted in this article.

13. Omand, ‘Reflections on Intelligence Analysts’, 2.

14. Ibid., 9.

15. Ibid.

16. Hayden Center for Intelligence, Policy, and International Security, ‘Conversation with Sue Gordon’, 24.

17. Clapper, “Luncheon Remarks”, 3.

18. Johnson, “National Security Intelligence”, 6.

19. Johnson, “National Security Intelligence”, 6.

20. George, “Fixing the Problem”.

21. Kent, Strategic Intelligence.

22. Ibid., 5.

23. Johnson, “National Security Intelligence”, 5.

24. Ibid.

25. Lowenthal, Intelligence.

26. Wastell, “Cognitive Predispositions”, 451.

27. Haass, Making Intelligence Smarter.

28. Fingar, Reducing Uncertainty, 4.

29. Lowenthal, Intelligence, 158–9.

30. There are an increasing number of articles exploring the impact of technologies, from machine learning to AI on intelligence. These are excellent contributions; however the inclusion of these works would take the discussion away from good intelligence characteristics – the focus of this article.

31. Bennett Moses and Chan, “Using Big Data”; Chan and Bennett Moses, ‘Making Sense’; Chan and Bennett Moses, ‘Is Big Data Challenging’; and Malomo and Sena”,Data Intelligence”.

32. Hammond-Errey, Big Data.

33. Kitchin, The Data Revolution, 68; Laney”,3D Data Management”.

34. Akhgar et al., Application of Big Data; van der Sloot, Broeders & Schrijvers, Exploring the Boundaries.

35. Hammond-Errey, Big Data; Hammond-Errey, Secrecy, Sovereignty and Sharing.

36. Bell, “The Character”, 175.

37. Hammond-Errey, Secrecy, Sovereignty and Sharing.

38. Office of National Intelligence, ‘The National Intelligence Community’. In Australia, there are ten agencies that form the Australian Government’s intelligence enterprise, working to collect, analyse and disseminate intelligence information and advice in accordance with Australia’s interests and national security priorities.

The agencies that form the NIC are: the Office of National Intelligence (ONI), the Australian Signals Directorate (ASD), the Australian Geospatial-Intelligence Organisation (AGO), the Australian Secret Intelligence Service (ASIS), the Australian Security Intelligence Organisation (ASIO), Defence Intelligence Organisation (DIO), the Australian Criminal Intelligence Commission (ACIC) and the intelligence functions of the Australian Federal Police (AFP), Australian Transaction Reports and Analysis Centre (AUSTRAC) and the Department of Home Affairs.

39. Ritchie and Lewis, Qualitative Research Practice, 220.

40. Guest, MacQueen and Namey, Applied Thematic Analysis, 10.

41. Charmaz, Constructing Grounded Theory.

42. Lowenthal, Intelligence; Wheaton and Beerbower, ‘Towards a New Definition’.

43. Omand, ‘Reflections on Intelligence Analysts’, 2.

44. Rolington, Strategic Intelligence, 17.

45. Ibid.

46. Hope, Royal Commission on Intelligence and Security, quoted in Richardson, Comprehensive Review, 155.

47. Kent, Strategic Intelligence, 157–8.

48. Agrell and Treverton, National Intelligence and Science, 35.

49. Vandepeer”,Question-Asking”.

50. Lowenthal, Intelligence, 58–62.

51. Vandepeer, ‘Intelligence and Knowledge Development’, 785.

52. For a full discussion see Hammond-Errey, ‘Big Data, Emerging Technologies and Intelligence. National Security Disrupted’, 86.

53. Johnson, “National Security Intelligence”, 21.

54. Ibid., 22.

55. Gill and Phythian, ‘What Is Intelligence Studies?’, 7.

56. Treverton, ‘Terrorism, Intelligence and Law’.

57. Hammond-Errey and Ray, ‘A New Methodology’.

58. INSA Rebalance Task Force, Expectations of Intelligence, 6.

59. Fingar, Reducing Uncertainty, 4.

60. CBS News, ‘Former top DNI official’. Sue Gordon at the six-minute, thirty-seconds mark.

61. Vandepeer, Applied Thinking, 46.

62. Ibid., 46; Heuer, Psychology of Intelligence Analysis, 62.

63. Agrell and Treverton, National Intelligence and Science, 32–9.

64. Fingar, Reducing Uncertainty, 5.

65. Omand, Bartlett and Miller”,Introducing Social Media Intelligence”, 807.

66. Symon, ‘ASIS Director-General Launches New Book’.

67. Wesley, ‘Intelligence Dissemination’, 111.

68. Johnson, ‘National Security Intelligence’; Walsh”,Building Better Intelligence Frameworks”; and Wesley, ‘Intelligence Dissemination’.

69. Gookins, ‘The Role of Intelligence’, 66.

70. Symon and Tarapore, “Defense Intelligence Analysis”, 8.

71. Coyne, Neal and Bell, ‘Reframing Intelligence’.

72. Wesley, “Intelligence Dissemination”.

73. Ibid.

74. Johnson, “National Security Intelligence”.

75. Agrell and Treverton, National Intelligence and Science, 43–8; Heuer, Psychology of Intelligence Analysis.

76. Herman, “11 September: Legitimizing Intelligence?”, 229.

77. Lefebvre, “A Look at Intelligence”, 243.

78. Omand”,Reflections on Intelligence Analysts”, 6–7.

79. Ibid., 6–7.

80. Hammond-Errey, Secrecy, Sovereignty and Sharing.

81. Hammond-Errey, Secrecy, Sovereignty and Sharing; Sue Gordon in Hammond-Errey, ‘Intelligence, AI and AUKUS’.

82. See Hershkovitz, Future of National Intelligence.

83. Symon, ‘ASIS Director-General’, 4.

84. Richardson, Comprehensive Review, 100.

Additional information

Notes on contributors

Miah Hammond-Errey

Miah Hammond-Errey’s work explores the intersections of emerging technologies and security. She hosts the Technology & Security podcast, where she is joined monthly by global experts to talk technology, security and leadership. Dr Miah Hammond-Errey is the Director of the Emerging Technology Program at the United States Studies Centre at the University of Sydney.