The Elusive Contextual False Positive: A Tale of Intrigue and Improvement

You say false when I said true; well not every foot fits the shoe.!

In the cat-and-mouse game between security systems and malicious actors, false positives (FPs) are a constant thorn in the side of defenders. While we strive for accuracy, FPs can creep into our systems, leading to unnecessary alerts, resource waste, and even reputational damage.

Many security solutions rely heavily on regex-based or heuristic-based detection to identify known malicious patterns. However, the regexes and heuristics cannot be foolproof, and their limitations become apparent when examining text payloads and their immediate surroundings. In some cases, a payload may be harmless but still trigger an FP due to its similarity to a known threat pattern. So context is crucial when it comes to distinguishing between legitimate threats and innocent behavior.

‍

The Power of Context: Understanding User Behavior

To effectively mitigate FPs, we need to consider the context of usage – the specific environment, user behavior, and prior history. This nuanced approach acknowledges that what constitutes a legitimate threat can vary significantly from target to target. Take, for example, an airport's security measures designed to detect illicit substances. A signature-based system might flag someone carrying multiple packets of white powder as a potential threat. However, this could be an innocent salt dealer, highlighting the need for context and targeted analysis to distinguish between legitimate concerns and false alarms.

But to add to the dilemma, even with contextual understanding, it's impossible to guarantee complete accuracy. New technologies and tactics can emerge, evading detection and causing FPs to rise. This is akin to the example of a salt dealer at an airport – if they use a new technique to conceal their products, even dedicated security measures might struggle to detect them.

‍

Processing Contextual FPs: A delicate balance

Identifying false positives (FPs) involves balancing accuracy and time efficiency. Historical data is valuable - the more the data, the more confidence in the outcome - but prolonged analysis can deter innocent users while allowing attackers to act quickly. A dynamic balance is necessary, adapting to changing contexts as new data becomes available. The choice between prioritizing accuracy or speed depends on:

1. Asset Criticality: The level of protection required for the asset.
2. Risk Tolerance: The organization's ability to absorb potential losses from false positives.
3. Resource Allocation: The availability of resources for analysis and response. By acknowledging these factors, organizations can develop a nuanced approach to FP detection, ensuring effective protection while minimizing unnecessary delays or inaction.

‍

A Best Effort to Detect FPs

So, how do we detect FPs? The answer lies in gathering as much knowledge as possible, learning from our mistakes, and continuously refining our systems. This involves:

1. Collecting data: Gathering as much information as possible about the environment and behavior.
2. Building context: Analyzing patterns and relationships between different pieces of data to identify potential FPs.
3. Running detection: Using various techniques to identify potential threats.
4. Feedback Loop: Learning from user interactions, whether positive (FPs) or negative (TPs).
5. Iterative Improvement: Repeat the process, refining the context with each iteration.

While the goal may seem unattainable – like approaching a mathematical asymptote – this never-ending cycle is essential for maintaining excellence. In life and in technology, perfection is an ideal that can never be fully achieved, but continuous striving towards it drives improvement and progress.

‍

Example Scenarios

To grasp the concept of contextual false positives (FPs), let's examine a few scenarios using the well-known XSS attack vector <script> as our muse.

‍

TP Scenario: An attacker attempts to execute malicious code on a client browser using the payload <script>alert('xss')</script>. Everyone would agree that this payload is indicative of an XSS attack.

‍

FP Scenario 1: Now, imagine the same payload being used by a beginner software engineer to query an AI-powered API for knowledge on XSS attacks. For instance, "Hey AI, can you explain how to identify and stop XSS attacks like <script>alert('xss')</script> commands?" This scenario clearly demonstrates a contextual FP. How to catch it? Understand the well-defined API specs used by such AI tools and write a heuristic to exclude such payloads from detection. This was easy right? Okay let’s complicate it.

‍

FP Scenario 2: Consider a scenario where a team of engineers created an application designed to assist students in fixing their code. This application exposes multiple APIs, one of them defined to accept JavaScript code as input, process it, and return the outcome along with test cases. Now say a user tries to use the API for the code <script>alert('xss')</script> , this would falsely get flagged as an attack. If this API is not well-documented, how to catch the FP? This is where analysis of traffic behaviour (that all traffic contains Javascript code) and manual FP feedback loop would help. Still think it’s doable? Let’s go to a more complex scenario.

‍

FP Scenario 3: Consider a scenario where a user inadvertently uses a flavour of this payload in entirely different context, something like “I want to understand how to start proof-reading a <script> of a story of a mute boy who tries to alert (more like warn) his mother about two hooligans who are trying to break and enter into their house. I want to do it fast and feel good when I complete fixing the <script>". This matches the signature, there’s nothing in the API spec to indicate a code because it’s not a code, and this is a one-of case which was never seen in the past. Now how to catch it as FP? There is no straightforward solution - this will need a lot of historical data and a complex processor to capture the majority of the cases which still can fail sometimes.

‍

Conclusion: Embracing the Reality of Contextual FPs

Contextual FPs are like Betaal to our Vikram, from the tale of Indian mythology, lurking just out of reach even when we believe ourselves close to perfection. Rather than pursuing an unattainable ideal, we should choose a side that is most beneficial for our system (err on the side of FP or FN). By accepting that some errors will inevitably persist, we can redirect our energy towards refining our systems, ever-evolving to mitigate the impact of these persistent shadows.