hack::soho | January 2025 | RAGs to Reqs | Irene Michlin, Neo4j

Join us for an evening of fun at this month’s hack::soho taking place 30 January, 6pm - 9pm GMT, set up to be a loose networking environment where cyber security professionals can chat, get some complimentary food & drink, and discuss rising global trends. This month's hack::soho will feature a talk from guest speaker Irene Michlin, Application Security Lead at Neo4j. The abstract of the talk, 'RAGs to Reqs,' is below!

hack::soho is a monthly event hosted at our London, UK office for the cybersecurity and hacking community to discuss all things security over food and refreshments. We welcome you to invite others in your circle to extend our collective network.

We hope you can join us,

IOActive team

ABSTRACT
ASVS is awesome! At the same time it contains 200+ requirements. Even after localising it for your context, it’s likely to have 100+ relevant requirements. Can we - the security team - ask the developers to go through this list for every feature?
We can, but how likely it is to happen in a modern DevSecOps environment, and what will be the quality of the engagement with ASVS?
(Also, spreadsheets are boring and everyone hates them). Can we do better? Yes!
Retrieval-Augmented Generation (RAG) is the process of optimising the output of a large language model, so it references an authoritative knowledge base outside of its training data sources before generating a response. Luckily, ASVS is a very “graphy” data that lends itself well to being stored in a graph format.
Using this graph as the authoritative knowledge base, we use semantic similarity search on the feature description to look up relevant ASVS requirements, which is already very useful on its own. But passing these requirements as context to OpenAI or another LLM gives further useful results, reducing hallucinations and giving the developers specific security requirements for their feature that are based on ASVS.
Bottom line: Don’t let ASVS become a chore, let the developers have an easy initial engagement with it. They can always go deeper if needed.
The workflow we’ve achieved in our engineering org: Let the tool run on internal portal. Developers provide a feature description (a paragraph or though), the tool gives them top-10 most relevant ASVS requirements and some further recommendations based on these requirements. Lots of ideas for refining and expanding, we’ll discuss some of these.
We released as open source the prototype tool and the underlying graph database including pre-calculated vector embeddings for semantic similarity search: https://github.com/neo4j-examples/appsec-asvs-bot
The ideas in this talk will help AppSec engineers with their scaling and culture building efforts, and will help all the developers/builders to get some security impact in their features fast.

Host or Publisher IOActive