The Truth About False Positives in Security


False Positives in Security

TL;DR: As weird as it might sound, seeing a few false positives reported by a security scanner is probably a good sign and certainly better than seeing none. Let’s explain why.

Introduction

False positives have made a somewhat unexpected appearance in our lives in recent years. I am, of course, referring to the COVID-19 pandemic, which required massive testing campaigns in order to control the spread of the virus. For the record, a false positive is a result that appears positive (for COVID-19 in our case), where it is actually negative (the person is not infected). More commonly, we speak of false alarms.

In computer security, we are also often confronted with false positives. Ask the security team behind any SIEM what their biggest operational challenge is, and chances are that false positives will be mentioned. A recent report estimates that as much as 20% of all the alerts received by security professionals are false positives, making it a big source of fatigue.

Yet the story behind false positives is not as simple as it might appear at first. In this article, we will advocate that when evaluating an analysis tool, seeing a moderate rate of false positives is a rather good sign of efficiency.

What are we talking about exactly?

With static analysis in application security, our primary concern is to catch all the true vulnerabilities by analyzing source code.

False Positives in Security

Here is a visualization to better grasp the distinction between two fundamental concepts of static analysis: precision and recall. The magnifying glass represents the sample that was identified or selected by the detection tool. You can learn more about how to assess the performance of a statistical process here.

False Positives in Security

Let’s see what that means from an engineering point of view:

  • by reducing false positives, we improve precision (all vulnerabilities detected actually represent a security issue).
  • by reducing false negatives, we improve recall (all vulnerabilities present are correctly identified).
  • at 100% recall, the detection tool would never miss a vulnerability.
  • at 100% precision, the detection tool would never raise a false alert.

Put another way, a vulnerability scanner’s objective is to fit the circle (in the…

Source…