True Positive vs. True Negative vs False Positive vs. False Negative For Cybersecurity.

The Challenging Concept of True Positive and True Negative Measurement Modeling.

If something is true, could it be wrong? If something were negative, could it be more accurate? Modeling results can often be very simplistic or overly complex. How CIOs, CISOs, and CFOs measure risk has a lot to do with the data they capture within their enterprise environment before, during, and after malicious activity.

Specifically, to cybersecurity, the result of monitoring the various cyber-attacks, the root cause, and post-impact on the organisation from a cost, loss of productivity, and brand weigh heavily on the modeling measurement strategy. 

Within each of the four measurement components used for modeling, true positive, true negative, false positive, and false negative all have a critical role in determining if any actual cyber-attack impacted the organisation or had no effect. Users will open help desk tickets reporting a "possible" attack, yet often do not remember the details. A frustration felt by many who have worked in IT support. When contacting the internal employee, "Why do you think this is a cyber-attack?" Often, the answers could range from "something weird is happening." A brief moment, "Okay, please let me know if it happens again."

As organisations roll out security adaptive controls, they often need to put more thought into measurement modeling strategy when dealing with false alarms, faulty security rules, and lower-than-expected favorable rates.

Security operations (SecOps) teams rely on security information event management (SIEM) and extended detection and response (XDR) artificial intelligence to classify the attack based on their learning engines and process the security telemetry over some time. Due to the amount of security telemetry messages received within SecOps, alert fatigue affects the entire monitoring team.

Organisations should develop a strategy based on the four principles of classification models and the proper workflow for cybersecurity.

Cybersecurity-managed security service providers like LinearStack assist clients in creating this measurement workflow. As a global partner of LogRhythm, LinearStack hosted a cloud-based multi-tenant SIEM to host their client's security telemetry and a 24 x 7 SOC-as-a-Service to help manage the various measurement events.

What is the Between the Measurement Modeling Events?

Knowing the difference between the four primary management modeling events is critical for an organisation. Depending on which modeling event is associated with a specific cybersecurity could determine if the organisation mobilizes escalation teams, contact the press and customers, and possibly contact their cyber insurance carrier or do nothing. False classification could result in a lawsuit, damage to the organisation's brand, or even fines for failure to comply with privacy laws and mandates.

What are the fundamental differences between the four measurement models?

True positive:

“The prediction is correct, and the actual value is positive, indicating a potential customer for the product.” In other words, a true positive is a successful identification of an attack.

False positive:

“The prediction did not match the actual value, which was positive.” In other words, an event has been flagged as an alarm/alert, when in fact, there was no threat.

True negative:

“The prediction is accurate, and the actual value is negative.” In other words, an event marked as safe, is safe i.e., there is no threat.

False negative:

The prediction does not align with the actual value, which is harmful. In other words, tooling designed to detect threats did not identify an event/activity as a threat.

Breaking Down the Data to Generate More Predictable Results.

Artificial intelligence (AI) and machine learning (ML) within the scope of cybersecurity is about moving organisations from reactive to proactive in cyber defence. Processing security telemetry data into Large Learning Models (LLM) into datasets, then feeding this into an ML engine, helps develop trends and measurement values from false to positive to false.

If, for example, the AI and ML report a specific trend that a hacker is using persistent email phishing attacks against a random set of users within the organisation, what would be an ideal measurement model to help the organisation reduce the risk?

Based on the processed data, a true positive would confirm that the hacker is using an email phishing attack method, possibly spear phishing generated by ChatGPT as their attack tool. The attack impacted the expected users. The victims click on the malicious link, so they download a PDF containing ransomware malware. SecOps operations can take this valuable, insightful data and make the needed corrections to the email security layer to prevent this email from propagating to other users.

However, if the AI and ML did not even detect this attack pattern because they were very little data telemetry to process, the measurement model could report a true negative. Under these circumstances, the AL and ML reported a possible email phishing. However, they ended up writing it as a hostile attack. This outcome often happens when attacks attempt to use whaling email phishing against a specific CEO or high-level executive within the organisation. Due to the small sampling, AI and ML may be unable to pick up the attack, resulting in considerable risk to the organisation.

What is the Role of the MSSP Regarding Developing a Strategy for Measurement Modeling?

Organisations collect vast amounts of security telemetry data. There is value in this data if organisations can classify and rationalize the content in a timely matter. Applying accurate measurement models will help determine the level of response. Mis-categorized security events often will cost the organisation money and human capital resources.

Working with MSSPs like LinearStack, the organisation can develop an accurate measurement model and a cost-effective incident response. Organisations lacking in-house talent should partner with LinearStack to help create a measurement model combined with the correct security adaptive control layers.

Measurement models must have the appropriate security detection and response capabilities to report correctly. Within a next-generation SIEM solution with XDR and other security adaptive controls, organisations will have the ideal cyber-attack visibility to make the correct decisions regarding the response to an attack.

LinearStack's experts have the expertise and managed services available to your organisation with this very challenging yet necessary technical and business requirement.

About LinearStack

LinearStack is a leading Managed Security Service Provider (MSSP) and security systems integrator based in New Zealand. Since our establishment in 2013, we have built a reputation for providing world-class 24x7 security services to businesses of all sizes. We are proud to partner with some of the top technology companies in the industry, such as Palo Alto Networks, Cisco Systems, Imperva, and LogRhythm. Our excellent operational capabilities, as well as our fulfillment of business requirements and completion of rigorous technical, sales enablement, and specialization examinations, have earned us a distinguished reputation in the industry.

At LinearStack, we take pride in providing top-notch security solutions tailored to our client's needs. We aim to help businesses reduce cyber-attack risks, strengthen security posture, and maintain regulatory compliance. Our clients rely on us for our exceptional security solutions, outstanding customer service, and industry expertise.

Culture

We’re 100% privately held, grown with a family mindset. When working with clients, we’re well-integrated within their teams and act as an extension of their operations. Augmenting existing teams is a transition we manage smoothly, empowering our customers to prioritize cybersecurity strategy while we protect their business from cyber threats 24x7.

Maintaining thriving IT systems and assuring data protection are fundamental needs that all businesses deserve.