Enterprises lack detections for more than three-quarters of all MITRE ATT&CK techniques, while 12% of SIEM rules are broken and will never fire due to data quality issues including misconfigured data sources and missing fields.

These were among the results of a CardinalOps report which analyzed real-world data from production SIEMs including from Splunk, Microsoft Sentinel, IBM QRadar and Sumo Logic.

The data covered more than 4,000 detection rules, nearly one million log sources and hundreds of unique log source types, spanning industry verticals ranging from banking and financial services to manufacturing and energy. 

The study also indicated that while organizations are implementing “detection-in-depth”—collecting data from multiple security layers including Windows endpoints and email—monitoring of containers lags behind.

Broken Rules

Mike Parkin, senior technical engineer at Vulcan Cyber, said the biggest issue he sees is the number of “broken rules” that will never trigger an event.
“While some of them are undoubtedly edge cases that would have been unlikely to trigger an event in any case, many are almost certainly the result of misconfiguration or broken logic,” he said. 

John Gallagher, vice president of Viakoo Labs at Viakoo, said two study findings were particularly concerning.

“While it is encouraging to see there is already sufficient data to detect 94% of potential MITRE ATT&CK techniques, it raises the question of what the missing 6% is and how impactful such attacks might be,” he said.

For example, if the missing 6% resulted in catastrophic damage (e.g., an IoT attack vector that is highly damaging) it might put more focus on achieving higher than 94% coverage. 

He added that “security layers” is a term defined by CardinalOps and is useful for organizations to plan resources and strategies based on their specific organization. “However, it includes containers but not IoT/OT, which seems like a significant oversight,” Gallagher noted.

For example, IoT/OT is used by almost all organizations (more than the 68% who reported using containers) and is less covered by a security layer within their SIEM than containers are.

“Lack of high-fidelity data…