The Rise of Private Intelligence Companies PART 3

 Limitations of Private-Sector Intelligence

While public data can answer many questions that once required secret intelligence, there are limits to what it can achieve. Open sources, for example, provided valuable insights into Russia’s military buildup prior to the 2022 invasion of Ukraine. However, only state agencies had access to the most sensitive evidence, such as intercepted Russian war plans and the movement of medical supplies like blood plasma in early 2022, which signaled imminent military action. No commercial or open-source data revealed details such as Russia’s development of an orbital nuclear weapon or Iran’s supply of ballistic missiles to Russia, all of which were obtained through secret state intelligence.

The Overlap Between Open and Secret Intelligence

It is misleading to think of open and secret intelligence as completely separate realms. In some cases, open-source data can substitute for classified intelligence, though often it works best when combined with secret information. For example, publicly available estimates of Russian military losses in Ukraine have been fairly accurate, but these assessments gain even more value when merged with classified data. However, integrating open and classified sources is technically and institutionally challenging. Government agencies may face difficulties in reconciling public data (referred to as the 'low side') with classified intelligence ('high side'). A spy agency, for instance, might track Russian intelligence officers using phone data and attempt to correlate it with leaked visa or travel records from the dark web. However, the mere act of querying public datasets could expose sensitive questions about ongoing operations, tipping off adversaries.

Legal and Ethical Challenges in Data Exploitation

The competition between states over data access presents significant legal and ethical challenges. China views data acquisition as a key element of its strategic competition with the West. Notable examples include the 2015 theft of over 22 million American government security clearance records and the 2017 acquisition of 148 million American and 15 million British records from Equifax. More recently, a leak from a Shanghai-based firm, iSoon, revealed the scale of Chinese hacking operations, which target data from various countries including immigration records from India and phone logs from South Korea. While some of this activity aligns with traditional intelligence gathering, it also enables China to track and counter Western spies, a practice similar to what Western intelligence agencies do in reverse.

Chinese intelligence has a long history of building extensive databases on individuals, filtering them to target specific groups for influence and intelligence operations. Some argue that the West could learn from this approach by merging the capabilities of governments and the private sector to defend against adversaries. However, achieving this balance is challenging. Democracies like Britain impose strict requirements on the collection and use of bulk personal data, requiring warrants and demonstrating that the collection serves a specific purpose. This makes accessing and utilizing data easier for private sector entities than for government agencies.

Privacy Laws and Technological Advances in Encryption

In the United States, intelligence agencies face similar restrictions. Emily Harding, a former CIA analyst, notes that it is difficult for American agencies to comply with laws that require the exclusion of American data from large datasets. This puts them at a disadvantage compared to private sector companies, which are not bound by the same restrictions. The famous VENONA project, which decrypted Soviet intelligence transmissions during World War II, would not have been possible under modern privacy laws in some European countries.

Since Edward Snowden's 2013 revelations about the National Security Agency’s surveillance activities, there has been significant public debate about intelligence agencies' ability to collect large amounts of data. The rise of end-to-end encryption in online communication and data storage has further limited the ability of government agencies to intercept information. Major tech companies, including Apple, Google, and Meta, have adopted stronger encryption measures, prioritizing user privacy despite resistance from law enforcement agencies worldwide.

The Paradox of Data Security in the Digital Age

Despite advancements in data security and encryption, the increasing reliance on digital technology has created a paradox. While individuals have more tools to protect their data, the sheer volume of data being generated, and the number of connected devices collecting and transmitting information, make it more challenging to maintain privacy. This ever-growing ecosystem of data creates opportunities for private intelligence firms to collect, analyze, and exploit vast amounts of information, often without the legal constraints that govern state actors.

In this environment, governments must balance the need for intelligence gathering with the protection of individual privacy, while private-sector intelligence continues to grow in influence and capability