{"id":14092,"date":"2017-12-07T15:43:24","date_gmt":"2017-12-07T20:43:24","guid":{"rendered":"https:\/\/lexingtoninstitute.org\/?p=14092"},"modified":"2017-12-08T10:59:18","modified_gmt":"2017-12-08T15:59:18","slug":"quantitative-analysis-big-data-address-hard-intelligence-problem","status":"publish","type":"post","link":"https:\/\/www.lexingtoninstitute.org\/quantitative-analysis-big-data-address-hard-intelligence-problem\/","title":{"rendered":"How Quantitative Analysis Of Big Data Could Address The Hard Intelligence Problem"},"content":{"rendered":"
Understanding the Democratic People\u2019s Republic of Korea (DPRK), its capabilities, plans and intentions, has always been a particularly difficult challenge for outside observers, including Western intelligence agencies. Access to the country is limited. Information is tightly controlled. The regime has become extremely adept at countering or denying technical means of intelligence collection. According<\/a> to Daniel Coats, the Director of National Intelligence, “It is one of the hardest, if not the hardest, collection nation that we have to collect against.\u201d<\/p>\n This problem is even more acute with respect to the regime\u2019s leader. Recently, The Atlantic <\/em>published<\/a> an article titled \u201cKim Jong Un: The Hardest Intelligence Target.\u201d The author asks several questions that must be at the heart of the Trump Administration\u2019s deliberations over how to respond to North Korea\u2019s nuclear weapons program: \u201cIs Kim Jong Un crazy or hyper-rational? Is he bent on destroying America or deterring America?\u201d The answer is the only one any honest observer of Kim can provide: \u201cNobody knows.\u201d<\/p>\n The DPRK is but one of a series of so-called hard intelligence targets. Russia, China, Iran and most terrorist groups are also considered hard targets. They are hard in the sense that customary collection methods often are not adequate to provide the desired quantity and quality of information. But even if access is available, it is often a challenge to correctly interpret the data collected.<\/p>\n Quantitative modeling of open source \u201cbig data\u201d has the potential to widen the aperture with respect to hard intelligence targets. It could be particularly useful in gaining insights on how decision makers think and their internal operating dynamics. Quantitative models can identify policy trends early, help our understanding of decision making in secretive, closed governments, and support the development of strategies for dealing with target countries or leaders.<\/p>\n The most basic source of quantitative data is open source news stories. In the United States, print outlets such as The New York Times<\/em> and The Washington Post <\/em>produce<\/a> between 250 and 500 news items a day. Television, cable and radio outlets each produce hundreds more. Academics and technical literature provide additional sources of news items that can be exploited, depending on the research question. In countries that impose press controls \u2013 which includes all the hard targets \u2013 fewer news stories are produced but their significance is greater because they are government approved.<\/p>\n Every day, just under 1,000 distinct stories are published on North Korea by a few hundred unique sources both inside that country and internationally. Each story is a source from which information can be extracted. Using models based on elite dynamics, which comb through the dataset to create statistical products based on thousands of observations, it is possible to penetrate the information barriers set up by this hard target.<\/p>\n