Data Mining for National Security: U.S. Government Programs by Michael Erbschloe - HTML preview

All Source Intelligence

All Source Intelligence collection activities are conducted by various agencies in the Intelligence Community (IC). All Source Intelligence professionals analyze threat information from multiple sources, disciplines, and agencies across the Intelligence Community. They synthesize and place intelligence information in context; draws insights about the possible implications. Much of this data collection is performed through data mining open source media including a broad array of information and sources that are generally available, including information obtained from the media (newspapers, radio, television, etc.), professional and academic records (papers, conferences, professional associations, etc.), and public data (government reports, demographics, hearings, speeches, etc.). Collecting open source intelligence is a complex data mining activity that requires a variety of tools and skills.

INTellingence: Open Source Intelligence Program

The president and policymakers rely on insights from the Central Intelligence Agency to inform their foreign policy decisions. CIA officers use a variety of sources in formulating their assessments. The following article is the first in a series that will explore different sources and collection disciplines, which are the building blocks of what we call “finished intelligence.”

Information does not have to be secret to be valuable. Whether in the blogs we browse, the broadcasts we watch, or the specialized journals we read, there is an endless supply of information that contributes to our understanding of the world. The Intelligence Community generally refers to this information as Open Source Intelligence (OSINT). OSINT plays an essential role in giving the national security community as a whole insight and context at a relatively low cost. OSINT is drawn from publicly available material, including:

• The Internet
• Traditional mass media (e.g. television, radio, newspapers, magazines)
• Specialized journals, conference proceedings, and think tank studies
• Photos
• Geospatial information (e.g. maps and commercial imagery products)

The DNI Open Source Center

CIA is responsible for collecting, producing, and promoting open source intelligence through its management of the DNI Open Source Center (OSC). OSC was established on November 1, 2005 in response to recommendations by the Robb-Silberman Commission, and is charged with a unique, Community-wide responsibility.

OSC and its worldwide network of partners have the skills, tools, and access necessary to produce high-quality open source intelligence. These capabilities include translations in over 80 languages; source, trends, and media analyses; specialized video and geospatial services; and rare cultural and subject matter expertise.

OSINT has always been an important part of all-source analysis, but continuing advances in information technology have given a voice to even larger numbers of people and made it possible to address new intelligence questions.

To filter, understand, and analyze the enormous amount of material that comes into OSC 24/7, Open Source Officers (OSO) must be fluent in foreign languages, sensitive to cultural nuances, experts in their field, whether video, geospatial tools, media analysis or library science.

OSC makes most of the information it collects and processes available both to the Intelligence Community and to the entire U.S. Government. Beyond making this “raw” data available to their all-source counterparts, OSC analysts identify and flag for others new insights or trends from open sources.

An experienced OSO is attuned to changes in tone, word choice, and syntax in official messages from foreign governments and organizations. Comparisons with past statements can provide insights into how the foreign actors view an incident or issue. The analysis can also help identify their “hot buttons” or “red lines."

As with all intelligence disciplines, OSINT has its challenges. The sheer volume is daunting, and separating wheat from chaff requires skill, knowledge, and a reliance on sophisticated information technology. It also takes a concerted effort to coordinate with partners to avoid duplication and make the best use of resources, but the payoff in both effectiveness and efficiency is high.

(Link: https://www.cia.gov/news-information/featured-story-archive/2010-featured-story-archive/open-source-intelligence.html)

Deputy Director of CIA for Digital Innovation

The Directorate of Digital Innovation is the Agency’s newest Directorate focused on accelerating innovation across the Agency’s mission activities with cutting-edge digital and cyber tradecraft and IT infrastructure.

Open Source Exploitation Officers (OSEOs) are experts in the discovery, collection, and assessment of foreign-based publicly available information, also known as Open Source Intelligence (OSINT) in a dynamic, ever-expanding digital environment. OSINT spans traditional media, academic papers, geospatial data and technology, datasets, social media, and plays an important role in supporting all aspects of Agency work. OSEOs work closely with Open Source Collection Officers developing plans that use effective tools and methodologies to research and collect open source information. Using their understanding of the open source information environment, language proficiency, and knowledge of a geographic or subject area, OSEOs convey their findings in informational and contextual responses and products.

Executive Order 12333 United States Intelligence Activities December 4, 1981

Part 2 Conduct of Intelligence Activities

2.1 Need. Timely, accurate, and insightful information about the activities, capabilities, plans, and intentions of foreign powers, organizations, and persons, and their agents, is essential to informed decisionmaking in the areas of national security, national defense, and foreign relations. Collection of such information is a priority objective and will be pursued in a vigorous, innovative, and responsible manner that is consistent with the Constitution and applicable law and respectful of the principles upon which the United States was founded.

2.2 Purpose. This Order is intended to enhance human and technical collection techniques, especially those undertaken abroad, and the acquisition of significant foreign intelligence, as well as the detection and countering of international terrorist activities, the spread of weapons of mass destruction, and espionage conducted by foreign powers. Set forth below are certain general principles that, in addition to and consistent with applicable laws, are intended to achieve the proper balance between the acquisition of essential information and protection of individual interests. Nothing in this Order shall be construed to apply to or interfere with any authorized civil or criminal law enforcement responsibility of any department or agency.

2.3 Collection of information. Elements of the Intelligence Community are authorized to collect, retain, or disseminate information concerning United States persons only in accordance with procedures established by the head of the Intelligence Community element concerned or by the head of a department containing such element and approved by the Attorney General, consistent with the authorities provided by Part 1 of this Order, after consultation with the Director. Those procedures shall permit collection, retention, and dissemination of the following types of information:

(a) Information that is publicly available or collected with the consent of the person concerned;

(b) Information constituting foreign intelligence or counterintelligence, including such information concerning corporations or other commercial organizations. Collection within the United States of foreign intelligence not otherwise obtainable shall be undertaken by the Federal Bureau of Investigation (FBI) or, when significant foreign intelligence is sought, by other authorized elements of the Intelligence Community, provided that no foreign intelligence collection by such elements may be undertaken for the purpose of acquiring information concerning the domestic activities of United States persons;

(c) Information obtained in the course of a lawful foreign intelligence, counterintelligence, international drug or international terrorism investigation;

(d) Information needed to protect the safety of any persons or organizations, including those who are targets, victims, or hostages of international terrorist organizations;

(e) Information needed to protect foreign intelligence or counterintelligence sources, methods, and activities from unauthorized disclosure. Collection within the United States shall be undertaken by the FBI except that other elements of the Intelligence Community may also collect such information concerning present or former employees, present or former intelligence element contractors or their present or former employees, or applicants for such employment or contracting;

(f) Information concerning persons who are reasonably believed to be potential sources or contacts for the purpose of determining their suitability or credibility;

(g) Information arising out of a lawful personnel, physical, or communications security investigation;

(h) Information acquired by overhead reconnaissance not directed at specific United States persons;

(i) Incidentally obtained information that may indicate involvement in activities that may violate Federal, state, local, or foreign laws; and

(j) Information necessary for administrative purposes. In addition, elements of the Intelligence Community may disseminate information to each appropriate element within the Intelligence Community for purposes of allowing the recipient element to determine whether the information is relevant to its responsibilities and can be retained by it, except that information derived from signals intelligence may only be disseminated or made available to Intelligence Community elements in accordance with procedures established by the Director in coordination with the Secretary of Defense and approved by the Attorney General.

Intelligence Community and elements of the Intelligence Community referred to:

(1) The Office of the Director of National Intelligence;

(2) The Central Intelligence Agency;

(3) The National Security Agency;

(4) The Defense Intelligence Agency;

(5) The National Geospatial-Intelligence Agency;

(6) The National Reconnaissance Office;

(7) The other offices within the Department of Defense for the collection of specialized national foreign intelligence through reconnaissance programs;

(8) The intelligence and counterintelligence elements of the Army, the Navy, the Air Force, and the Marine Corps;

(9) The intelligence elements of the Federal Bureau of Investigation;

(10) The Office of National Security Intelligence of the Drug Enforcement Administration;

(11) The Office of Intelligence and Counterintelligence of the Department of Energy;

(12) The Bureau of Intelligence and Research of the Department of State;

(13) The Office of Intelligence and Analysis of the Department of the Treasury;

(14) The Office of Intelligence and Analysis of the Department of Homeland Security;

(15) The intelligence and counterintelligence elements of the Coast Guard; and

(16) Such other elements of any department or agency as may be designated by the President, or designated jointly by the Director and the head of the department or agency concerned, as an element of the Intelligence Community.

Department of States Bureau of Intelligence and Research

INR is a bureau of the Department of State and a member of the Intelligence Community (IC). The Bureau of Intelligence and Research's (INR) primary mission is to harness intelligence to serve U.S. diplomacy. Secretary of State George Marshall established INR in 1947. INR is a direct descendant of the Office of Strategic Services Research Department and the oldest civilian intelligence element in the U.S. Government. Ambassador Daniel B. Smith is INR’s Assistant Secretary.

Drawing on all-source intelligence, INR provides value-added independent analysis of events to U.S. State Department policymakers; ensures that intelligence activities support foreign policy and national security purposes; and serves as the focal point in the State Department for ensuring policy review of sensitive counterintelligence and law enforcement activities around the world. The bureau directs the Department’s program of intelligence analysis and research, conducts liaison with the Intelligence Community, and represents the Department on committees and in interagency intelligence groups. The Bureau of Intelligence and Research also analyzes geographical and international boundary issues.

INR fulfills its mission through three key activities:

All-Source Analysis: INR is one of three all-source analytical units in the IC. INR analysts focus primarily on supporting diplomats and diplomacy with a wide range of information and analyses. INR participates in the production of joint IC products, usually under the auspices of the National Intelligence Council (NIC), and in the drafting and coordinating of articles for the President’s Daily Briefing (PDB). INR is also the U.S. Government (USG) leader for foreign public opinion research and analysis.

Intelligence Policy and Coordination: INR coordinates between the Department of State and the IC to ensure that intelligence activities—collection and operations—support and are informed by foreign policy. Within the State Department, INR coordinates policy review of sensitive intelligence, counterintelligence, and law enforcement activities to ensure that they are consistent with foreign policy interests. INR also represents the State Department’s interests in the formulation of intelligence policy by the Office of the Director of National Intelligence and by other elements of the IC.

Analytic Outreach: INR leads the IC in analytic outreach. INR’s Analytic Exchange Program provides analysts and policymakers with perspectives from hundreds of outside experts from the private sector, academia, and non-governmental organizations on the most challenging foreign policy and intelligence issues. INR also runs INRtalks: an audio-video program that directly engages these experts at the request of policymakers and analysts. INR co-chairs the National Intelligence Analysis Board’s Analytic Outreach Committee, the IC’s principal coordinating body for analytic outreach. INR co-manages the IC Associates Program with the NIC.

Offices within the Bureau of Intelligence and Research relevant to a review of data mining activities of the U.S. Government are:

• Humanitarian Information Unit (INR/GGI/HIU): The Humanitarian Information Unit (INR/GGI/ HIU) serves as a U.S. Government interagency center to collect, analyze, and disseminate all-source information in preparation for, and in response to, humanitarian crises worldwide.
• Office of Opinion Research (INR/OPN): The Office of Opinion Research (INR/OPN) is responsible for open-source analysis of foreign opinion in all regions of the world. INR/OPN conducts public opinion polls and media analysis to gauge foreign attitudes toward key issues, including international security, trade and economy, ethnic relations, and democratic transitions.
• Office of Analytic Outreach (INR/AO): The Office of Analytic Outreach (INR/AO) facilitates the exchange of expertise between diverse private sector specialists and government officials through four outreach programs—conferences, the congressionally mandated Title VIII grant program on Eurasia, the Global Futures Forum, and the IC Associates Program.
• Office of Cyber Affairs (INR/CYBER): The Office of Cyber Affairs (INR/CYBER) integrates all-source intelligence analysis and coordination of policy review of cyber-related intelligence operations.

(Link: https://www.state.gov/s/inr/)

Metrics for Open Source Use

One of the key challenges to managing the use of open source is the absence of widely accepted measurements or metrics. Intelligence Community managers seek quantifiable measures for day-to-day administration. Counts are made of the occasions in which open source analyses have been included in the President’s Daily Brief, one of the Intelligence Community’s most important products. Other products are published by the Open Source Center based solely on open source information and disseminated to intelligence analysts and outside experts. Use of the website opensource.gov is also monitored.

Inasmuch as open source information is used by all-source analysts in connection with information from classified sources, it is difficult to measure how much open source information contributes to a specific intelligence product. It is anticipated that open source information will increasingly be relied upon given its greater availability, the nature of issues that today’s analysts must cover, and the heavier emphasis placed on it by senior intelligence leaders. The ultimate metric for the Intelligence Community is, however, the quality of analysis. Today’s analysts work with the awareness that products reflecting ignorance of information contained in open sources will discredit the entire intelligence effort. This will be especially the case when intelligence products are made public and are scrutinized by knowledgeable outside experts.

(Link: http://www.dtic.mil/dtic/tr/fulltext/u2/a475067.pdf)