Artificial intelligence is transforming the landscape of vulnerability research, yet much of the discussion remains theoretical, focusing on the potential capabilities of models rather than their current practical applications.
Our goal was to address a more pragmatic inquiry: with the models we currently have access to, how effectively can AI assist in identifying genuine, exploitable vulnerabilities within production software?
This article outlines how the Intruder team is leveraging large language models (LLMs) to uncover new vulnerabilities by utilizing code scanning frameworks in conjunction with existing pre-Mythos models.
We detail a remote, multi-stage SQL injection zero-day that we uncovered in a WordPress plugin boasting over 300,000 users — a process that was fully automated from discovery to exploitation, without any human intervention.
The central issue: why directing AI at an entire codebase is ineffective
The primary challenge when integrating AI with a code scanner is maintaining focus. LLMs excel at analyzing small snippets of code or specific problem descriptions to find intriguing solutions. However, when tasked with scanning a large codebase for security vulnerabilities, the model attempts to process every file in the repository.
This approach is costly in terms of tokens and detrimental to accuracy: by the time the model has processed half of the code, its context is cluttered with irrelevant information, causing the actual bug of interest to be obscured.
For more intricate bugs that necessitate linking multiple steps, one must depend on the framework to retain the appropriate context in memory or retrieve it intelligently when required. Our experience indicates that this often results in subpar output rather than genuine, compelling bugs.
Conventional code scanning frameworks already address this issue. We employ a technique we refer to as a program slice, akin to how an IDE or LSP tool utilizes features like ‘find implementation’ or a call graph to identify all functions invoked by the current function. These are established, well-tested tools that completely avoid the diluted-context problem.
Pentest with every major release. Security that evolves alongside engineering.
Intruder’s AI pentesting agents provide the thoroughness of a manual engagement on demand: no lead time, no scoping calls, and at a fraction of the cost.
Test with every release, minimize your exposure window, and receive an audit-ready report within hours.
Our pipeline: from codebase to functional exploit
We have developed a pipeline that processes a codebase, runs it through a code scanning engine (we utilize Joern), generates relevant code slices for each finding, and employs an LLM to triage and exploit the identified issues. The design was inspired by nooperator’s work on Slice, although we opted for Joern instead of CodeQL and crafted the slicing algorithm differently to cater to the specific classes of vulnerabilities we are targeting.
We directed it at the top 200 WordPress plugins — code that has already been extensively examined by bug bounty researchers, so discovering something substantial there would indicate that our process can rival skilled human efforts.
Initially, Joern scans the codebase using rules designed to flag broadly ‘interesting’ patterns — this is intentionally broad to prevent the creation of overly specific rules that might overlook bugs. Since we have the triage agent to filter later, we can afford to have some false positives.
For this experiment, we focused on the unauthenticated attack surface of WordPress plugins, so we instructed Joern to identify every instance where a script could be influenced by user input: REST routes, template hooks, nopriv AJAX calls, and so forth.
For each WordPress hook, Joern produces a slice: the function that the hook invokes, every method that function calls, and so on down the chain. Basic taint tracking eliminates obviously safe functions, such as SQL and XSS inputs that pass through a known-safe sanitizer. When we can statically verify that the code is safe to execute, we exclude those from being sent to the LLM.
Each slice is then processed by a lightweight triage model (Sonnet, in our tests) to filter out the clearly uninteresting: hooks that are intended to be public and have no side effects, for instance.
What remains is forwarded to a more robust model.
Source: Original article
