Recently, the NCSC has been enhancing its strategy for data-driven cyber (DDC). Our objective is to promote the adoption of an evidence-based approach in cyber security decision-making, both in the guidance we provide to external organizations and in our internal security measures.
We recognize that enterprise cyber security is becoming more intricate, leading many teams to hesitate in integrating an additional ‘data layer’ for fear of becoming overwhelmed. In this blog, we aim to illustrate how focusing on manageable, actionable insights can enable teams to embrace data-driven cyber security.
Our case study highlights a partnership between two teams within the NCSC:
- The Vulnerability Reporting Service (VRS)
- The Data Campaigns and Mission Analytics (DCMA) team
The Vulnerability Management Team spearheads the NCSC’s response to vulnerabilities, whereas DCMA applies their expertise in data science and analysis to equip the NCSC Government Team with evidence-based security insights.
Small actionable insights drive action
Numerous government teams, including the VRS, collect and handle extensive amounts of valuable data. The challenge they encounter lies in effectively analyzing this data amid a common belief that gaining useful insights necessitates a comprehensive overhaul of existing processes.
This misconception arises from the belief that DDC requires all data to be plugged into a complex ‘master formula’ to reveal hidden insights and narratives. However, it is crucial to view DDC, especially at the outset, as a way to produce ‘small yet actionable insights’ that can significantly enhance decision-making. This streamlined and focused approach can lead to substantial benefits.
Vulnerability Avoidability Assessment
For the VRS, we implemented exactly that, beginning with the datasets available to the team and concentrating on a singular insight that could facilitate a meaningful, evidence-based security discussion.
To achieve this, we developed the Vulnerability Avoidability Assessment (VAA), an analytic that leverages two internal data sources and one public source to establish the proportion of vulnerability reports arising from outdated software. The data sources consisted of:
- The total number of vulnerability reports received by VRS
- The number of reports where outdated software was cited as a reason
- A public vulnerability disclosure database
We created this analytic with the understanding that patch management is one type of vulnerability that can be managed, and that a deeper examination of the relationship between patch management and the vulnerabilities reported through the VRS would yield a valuable discussion point on how vulnerabilities might be avoided or mitigated.
Our analysis
We gained deeper insights into the effects of unpatched software on government systems by comparing the count of vulnerability reports due to outdated software with data from an open-source database. This database offered estimates of how long these vulnerabilities had been publicly acknowledged and when patches had become accessible.
Using this method, we defined an ‘avoidable vulnerability’ as one that has been publicly known long enough that a responsible organization should reasonably have taken the required actions to implement the necessary updates and patches.
Our analysis of data from 2022 revealed that every month, the VRS received a significant number of vulnerability reports directly associated with software that was outdated. The figures fluctuated from 1.6% to a peak of 30.7% of vulnerabilities within a single month, throughout the year.
| Month (2022) | Total Vulnerability Reports | Unpatched Software Vulnerabilities | Proportion of Avoidable Vulnerabilities (%) |
|---|---|---|---|
| January | 64 | 3 | 4.7 |
| February | 58 | 9 | 15.5 |
| March | 128 | 36 | 28.1 |
| April | 101 | 31 | 30.7 |
| May | 92 | 15 | 16.3 |
| June | 141 | 34 | 24.1 |
| July | 65 | 0 | 0 |
| August | 81 | 8 | 9.9 |
| September | 58 | 4 | 6.9 |
| October | 62 | 1 | 1.6 |
| November | 88 | 8 | 9.1 |
| December | 97 | 7 | 7.2 |
Table 1. Total number of out-of-date software reports compared to the total number of vulnerability reports received for 2022.
We also explored how long the software vulnerabilities remained unpatched before they were exploited. Based on NCSC guidance, which advises that all updates for critical or high-risk vulnerabilities should be implemented within 14 days (NCSC Cyber Essentials guidance on ‘Security Update Management’, Page 13), we established a 30-day buffer as a consistent timeframe for applying patches, regardless of their severity. By categorizing the timelines into these increments, we discovered that 70% of outdated software vulnerabilities reported to the VRS had been unpatched for more than 30 days.
Chart 1. Illustrates the duration a vulnerability remained in the public domain.
This new understanding has equipped the VRS team with the necessary data to engage in evidence-based discussions with stakeholders concerning their patch management approach, using these insights to advocate for effectively decreasing the number of vulnerability reports received by the VRS regarding government systems.
Conclusions
The transition towards DDC has underscored the immense value of utilizing data to inform evidence-based security decisions. The partnership between the VRS and the DCMA team exemplifies how data can shape decision-making processes. It is vital for organizations to understand that adopting DDC does not necessitate a complete overhaul of existing systems, but rather requires focusing on extracting small yet actionable insights that can influence behaviors and decisions.
Joshua L
Data Scientist, NCSC
Based on an article from ncsc.gov.uk: https://www.ncsc.gov.uk/blog-post/data-driven-cyber-empowering-security-focused-insights
