When people think about data security in government, they immediately think about encryption. And rightfully so: Encrypting data at rest and in motion has been a best practice for the past decade. In recent years, however, the data security arsenal has expanded to include what is becoming known as “deep data inspection.”  

Deep data inspection goes one step deeper into data security and looks inside packaged data for threats and quality defects.

We’ve been trained to believe that security threats — malicious or unintentional exploits  –emerge as data is first created. What has been overlooked in many instances, however, is that data quality issues are actually an intrinsic part of data security. 

Deep data inspection is analogous in many ways to network-based deep packet inspection. In the earliest days of the internet, information crossed the internet in clear text. As hacking became more common, IT managers concluded they needed to look inside individual network packets to determine whether the data contained in those packets was legitimate.

Today, data security is beginning to conduct deep data inspection on data files – especially those that fuel artificial intelligence and machine learning products that make sense of today’s enormous data warehouses. 

A comprehensive data security strategy now must include both inspection and encryption – and, in fact, it makes the most sense to start with inspection. After all, if data is encrypted before it is inspected, it’s akin to locking the criminal inside the house, from a security perspective.

Consider the example of a comma-separated-value (CSV) file, similar to a spreadsheet. In the world of big data, these files can contain millions of rows and columns. Data files like these are typically encrypted because they must be protected as they move across the internet and are shared from one authorized user to the next. All that’s needed is an intentional or unintentional exploit in a single cell in one file for systems to be corrupted, crashed or taken over. 

It’s essential, therefore, to be able to scan all those rows and columns to validate that not only are there no threats hidden in the data, but that the data…