Privacy vs. Security: Is Your Bot Mitigation Solution Effective in the Wake of Web Privacy Trends?

Bad Bots Disguise as Humans to Bypass Detection

Bot mitigation providers place significant emphasis on stopping bots with the highest degree of accuracy. After all, it only takes a small number of bad bots to get through your defenses to wreak havoc on your online businesses. One challenge of stopping bad bots is keeping false positives to a minimum (where a human is incorrectly categorized as a bot).

The more aggressively rules are tuned within a bot mitigation solution, the more susceptible the solution becomes to false positives because it needs to decide whether to grant requests for indeterminate risk scores. As a result, real users are inadvertently blocked from websites and/or being served CAPTCHAs to validate they are indeed humans. This inevitably creates a poor user experience and lowers online conversions.

Much of the ongoing innovation in modern bot mitigation solutions has been a reaction to increasing sophistication of the adversary. The fact that bad bots increasingly look like humans and act like humans in an attempt to evade detection makes it more difficult to rely on rules, behaviors, and risk scores for decisioning – making false positives more pronounced.

Humans Now Disguising Themselves for Privacy

A more recent trend is exacerbating false positives, and without proper innovation, it renders legacy rule and risk-score dependent bot mitigation solutions inadequate. It results from the accelerating trends related to humans taking action towards more privacy on the Internet. Ironically, the move towards more privacy on the web can actually compromise security by making it even more difficult to distinguish between humans and bots. 

To understand why it’s essential to know how the majority of bot detection techniques work. They rely heavily on device fingerprinting to analyze device attributes and bad behavior. Device fingerprinting is performed client-side and collects information such as IP address, user agent header, advanced device attributes (e.g. hardware imperfections), and cookie identifiers. Over the years, the information collected from the device fingerprint has become a major determinant for analytics engines used to whether the request is bot…