In recent months, the NCSC has intensified its commitment to data-driven cyber (DDC). Our aim is to promote an evidence-based methodology in cyber security decisions, both in our advice to external organizations and in our internal security measures.
We recognize that enterprise cyber security is growing increasingly complicated, with many teams hesitant to incorporate an additional ‘data layer’ due to fears of being inundated. This blog aims to illustrate how focusing on manageable, actionable insights can empower teams to engage with data-driven cyber security.
This example highlights collaboration between two teams within the NCSC:
- The Vulnerability Reporting Service (VRS)
- The Data Campaigns and Mission Analytics (DCMA) team
The Vulnerability Management Team leads the NCSC’s response to vulnerabilities, while DCMA leverages its expertise in data science and analysis to provide security insights based on evidence for the NCSC Government Team.
Small actionable insights drive behavior
Numerous government teams, including the VRS, collect and manage extensive volumes of valuable data. The key challenge they face is determining how best to analyze this data, particularly given the misconception that deriving any useful insights necessitates a complete overhaul of current workflows.
This misconception arises from the belief that implementing DDC means channeling all data into a complicated ‘master formula’ to discover hidden insights. It is vital to understand that, particularly in the initial stages, DDC should be approached as a means of generating ‘small yet actionable insights’ that enhance decision-making. This streamlined and focused method can offer substantial benefits.
Vulnerability Avoidability Assessment
For the VRS, we took this exact approach, initiating our analysis with the available datasets and concentrating on a single insight that could facilitate a meaningful, evidence-based security discussion.
Thus, we developed the Vulnerability Avoidability Assessment (VAA), an analysis that utilizes two internal data sources and one public source to ascertain what proportion of vulnerability reports stemmed from outdated software. The data sources consisted of:
- The number of vulnerability reports received by VRS
- The number of reports indicating outdated software as a reason
- A public vulnerability disclosure database
We devised this analysis knowing that patch management is a category of vulnerabilities that can be influenced, with a focus on exploring the relationship between patch management and the vulnerabilities reported through the VRS. This analysis provides a relevant discussion point on how vulnerabilities may potentially be avoided or minimized.
Our findings
We deepened our understanding of the impact of unpatched software on government systems by comparing the number of vulnerability reports linked to outdated software with data from an open-source database. This database provided insights into how long these vulnerabilities had been publicly recognized and when updates became available.
Through this analysis, we defined an ‘avoidable vulnerability’ as one that has been publicly known for a significant duration, such that a responsible organization would be expected to have taken the necessary actions to implement the necessary updates and patches.
Our examination of data from 2022 revealed that each month VRS received a significant number of vulnerability reports directly associated with outdated software, ranging from 1.6% to a peak of 30.7% in any single month throughout the year.
| Month (2022) | Total Vulnerability Reports | Reports due to Unpatched Software | Percentage of Avoidable Vulnerabilities (%) |
|---|---|---|---|
| January | 64 | 3 | 4.7 |
| February | 58 | 9 | 15.5 |
| March | 128 | 36 | 28.1 |
| April | 101 | 31 | 30.7 |
| May | 92 | 15 | 16.3 |
| June | 141 | 34 | 24.1 |
| July | 65 | 0 | 0 |
| August | 81 | 8 | 9.9 |
| September | 58 | 4 | 6.9 |
| October | 62 | 1 | 1.6 |
| November | 88 | 8 | 9.1 |
| December | 97 | 7 | 7.2 |
Table 1. Total number of out-of-date software reports compared to the total number of vulnerability reports received for 2022.
We also examined the length of time that software vulnerabilities remained unpatched before being exploited. Based on NCSC guidance, which recommends applying all released updates for critical or high-risk vulnerabilities within 14 days (NCSC Cyber Essentials guidance on ‘Security Update Management’, Page 13), we adopted a 30-day timeframe for applying patches, regardless of their severity. By breaking down the timelines into these intervals, we discovered that 70% of outdated software vulnerabilities reported to the VRS involved software that had not been updated for over 30 days.
Chart 1. Duration that vulnerabilities remained public.
This newfound understanding equipped the VRS team with data sufficient to engage in an evidence-based discussion with stakeholders regarding their patch management strategies. This data insight supports a case for meaningfully reducing the volume of vulnerability reports that the VRS receives from government systems.
Conclusions
The journey towards adopting DDC has underscored the considerable value of utilizing data to arrive at evidence-based security decisions. The collaboration between VRS and the DCMA team serves as a tangible example of how data can influence decision-making. Organizations must recognize that embracing DDC does not demand a complete transformation of existing systems; instead, it requires the ability to focus on extracting small but actionable insights that can shape behaviors and decisions.
Joshua L
Data Scientist, NCSC
Based on an article from ncsc.gov.uk: https://www.ncsc.gov.uk/blog-post/data-driven-cyber-empowering-security-focused-insights
