Jun 24, 2024NewsroomVulnerability / Artificial Intelligence

Google has developed a framework called Project Naptime which, according to the company, allows a large language modeling (LLM) for vulnerability research in order to improve automated discovery methods.

The Naptime architecture revolves around the interaction between a AI agent and a targeted codebase, Google Project Zero researchers Sergei Glazunov and Mark Brand explained . The agent is given a set specialized tools that mimic the workflow of human security researchers.

The initiative was named after the fact that it allows people to “take regular naps”, while it helps with vulnerability research and automating analysis.

The core of the approach is to use advances in code understanding and general reasoning abilities of LLMs to mimic human behavior when it comes identifying and demonstrating vulnerabilities.

It includes several components, such as a Code Browser that allows the AI agent navigate through the target codebase. A Python tool is used to run Python scripts within a sandboxed fuzzing environment.

CYBERSECEVAL 2 was released by Meta researchers in April. It is an evaluation suite that quantifies LLM security risks. CYBERSECEVAL 2 was released by Meta researchers in April and is an evaluation tool that quantifies LLM security risks.

In tests conducted by the search giant in order to reproduce and exploit flaws, both vulnerability categories received new top scores of 1,00 and 0,76, up from 0,05 and 0.24 respectively for OpenAI GPT-4 turbo.

Researchers said that “Naptime allows an LLM perform vulnerability research in a way that closely mimics human security experts’ iterative, hypotheses-driven approach.” This architecture not only improves the agent’s capability to identify and analyze weaknesses, but also ensures the results are accurate.