Office of the Director of National Intelligence
The Office of the Director of National Intelligence (ODNI) report several data mining efforts for 2015.
National Counterterrorism Center Threat Analysis
The NCTC continues to conduct “threat analyses” as described in the 2013 Data Mining Report. As noted in the 2013 report, this is an analytic technique to narrow the pool of information within NCTC databases that analysts will assess in response to specific threat reports. This technique does not meet all of the statutorily defined criteria for data mining, but has been reported in the interest of transparency.
Intelligence Advanced Research Project Activity (IARPA)
IARPA continues to invest in high-risk, high-payoff research programs that have the potential to provide the United States with an overwhelming intelligence advantage over future adversaries. As a scientific research funding organization, IARPA does not use, nor does it expect to make use of, data mining technology. IARPA programs are experimental by nature, and are designed to produce new capabilities, such as those described in previous reports and recapped here.
Knowledge Discovery and Dissemination (KDD) Program
The KDD program began in 2009 and completed its fourth and final period in December 2014. The KDD program created and evaluated a variety of technologies, a number of which have been determined useful in settings across the IC. Transition partners continue to evaluate prototype systems developed under the KDD program.
Aladdin Video Program
The Aladdin Video scientific research program is in its final year. The research will be complete in December 2016. The objective of the Aladdin Video program is to enable analysts to query large video data sets to quickly and reliably locate those video clips that show a specific type of event, thus automating a triage process that is currently performed manually. Although not “data mining,” technologies that result from Aladdin Video research could potentially be applied by operational organizations to support capabilities that involve pattern recognition.
Aladdin Video research addresses three significant technical areas: (1) high-speed processing of large numbers of video clips to extract information needed to support queries about each clip’s contents; (2) generation of effective queries from small sets of example video clips and a textual description; and (3) robust query processing that identifies the clips of interest and summarizes the rationale for their selection. Aladdin Video research results will be evaluated by IARPA and the National Institute for Standards and Technology.
Security and Privacy Assurance (SPAR) Program
The SPAR program completed its second and final phase of research on distributed private information retrieval (PIR) in June 2014. As discussed in the 2009 and 2010 ODNI Data Mining Reports, PIR protocols permit an entity to query a cooperating data provider and retrieve only the records that match the query without the provider learning what query was posed or what results were returned.
The final phase of SPAR involved research in three technical areas: 1) development of protocols ensuring privacy and security compliance even in the context of a “private” query; 2) implementation of fully homomorphic encryption (FHE) in the absence of any third parties; and (3) application of PIR to publish or subscribe systems.
In September 2014, IARPA began a SPAR pilot project to demonstrate the utility of the protocols in real use cases within the IC. The pilot project implemented SPAR database query protocols in a classified network with real data and real users. The pilot project was completed in December 2015, and additional work is now being considered to address some of the issues that came to light during the pilot.
As noted, SPAR protocols have the potential to enable the IC to access specific records without having to disclose classified data and without accessing, learning, ingesting, or retaining any private information about non-relevant persons. By satisfying security and privacy concerns, the technology could enable enhanced cooperative information-sharing across the IC and with other parts of the Federal Government and the private sector.
(Link: https://www.dni.gov/files/documents/CLPO/CY2015DataMiningReport.pdf)