Google’s new framework enables LLMs to improve their vulnerability research efforts through code and reasoning advancement, helping to increase LLM scores on CyberSecEval 2, which tests vulnerabilities such as buffer overflow and advanced memory corruption.
What is Naptime?
Google’s Project Zero team recently unveiled the development of Naptime, an automated vulnerability research framework designed to empower large language models (LLMs). Intended to replicate the iterative, hypothesis-driven process used by human security researchers, Naptime provides LLMs with tools such as code browser, Python tool and debugger which improve their abilities to evaluate, exploit and verify vulnerabilities.
Naptime’s Architecture
AI had previously struggled to conduct vulnerability research autonomously. That changed this year when Google researchers created a framework enabling large language models (LLMs) to perform basic vulnerability research tasks successfully, according to Project Zero blog.
The Naptime framework, named in honor of LLMs’ potential to enable security researchers to take regular naps during automated processes, was found to increase performance on CyberSecEval 2 tests by as much as 20 times. This framework equips models to use toolsets such as code browser, Python tool and debugger.
These tools allow the agent to interact with a target program and observe its behavior as it evaluates inputs, exploits flaws, and verifies successful exploitation. Furthermore, structured mechanisms enable communication with human researchers regarding its status of work performed.
Advanced AI could become an integral component of cybersecurity in the near future, both aiding defenders in identifying vulnerabilities more rapidly and helping attackers build offensive capabilities more rapidly. To find out how Google’s privacy-preserving generative AI technology can be utilized in Gmail and Workspace, visit our product page.
Naptime’s Toolset
The Naptime framework equips an LLM with task-specific tools, enabling it to conduct vulnerability research that mimics the iterative, hypothesis-driven approach of human security researchers. These include tools like Code Browser that allows an agent to navigate through source code of its target program; Python tool that runs Python scripts within a sandboxed environment for fuzz testing; and Debugger which observes behavior with different inputs.
Google researchers performed numerous tests using Cyber SecEval 2 buffer overflow and advanced memory corruption benchmarks and discovered that their framework improved an LLM’s performance by up to 20x in both categories (see full report here).
The team is also developing the framework to accommodate other use cases, such as analyzing user-reported phishing attacks. This may enable security analysts to more efficiently triage and evaluate threats while freeing them up for more complex and enjoyable tasks, leading to faster response times and enhanced accuracy for threat detection.
Naptime’s Results
Naptime framework’s purpose is to improve automated vulnerability discovery; its LLMs mimic human security researchers in their iterative, hypothesis-driven approach to security research. With its architecture and toolset of debugger, code browser and Python tool allowing AI agents to navigate target program source codes with precision input fuzzing as well as observe program behavior with various inputs to verify successful exploit, the Naptime model has outshone CyberSecEval 2 benchmark performance by up to 20 times!
The team employed Google’s benchmark suite to assess how well an LLM could detect and exploit memory safety flaws such as buffer overflows or advanced memory corruption bugs, among others. They discovered GPT-4 Turbo excelled on the CYBERSECEVAL 2 buffer overflow test with an impressive new high score of 1.00; Gemini 1.5 Pro achieved top scores in both advanced memory corruption tests by triggering program crashes; these results demonstrate “current LLMs can start performing vulnerability research”, according to Google researchers.