By Anton Lukin Sr. By Artem Kobrin Head of Cloud & Partner – Neurons Lab
|
NeuronsLab |
Audits are essential to a robust cybersecurity defense. Businesses must regularly assess their systems for vulnerabilities and ensure compliance with industry standards. This process is vital to identifying threats, ensuring regulatory compliance and assessing system configurations. However, it can be a burden for cybersecurity teams, as it requires them to review vast amounts of documentation.
The complexity and volume can lead to longer audit durations, delayed threats detection, and higher operational costs. According to Drata’s 2023 Compliance Trends Report organizations spend on average 4,300 hours per year to achieve or maintain regulatory compliance. This is a huge amount of time which could be saved by applying generative AI.
This post outlines how Neurons Laboratory, a AWS Advanced Tier Services partner, and AWS marketplace seller, with Competencies in Machine Learning, Generative AI and Healthcare, collaborated to automate manual compliance process accurately with Peak Defence.
Peak Defence offers information security compliance services to help clients meet standards like ISO 27001 and NIST. Neurons Lab, an AI consultancy, provides end-toend services to mid- and large-sized businesses.
Peak Defence’s security assessment process and RFP response were automated using Amazon Bedrock Amazon Sagemaker and anthropic Claude 2.
Challenges: Increased demand and competition
Peak Defence’s employees were unable to scale their business due to the increased demand. Achieving compliance, managing information systems and responding RFPs took up a lot of time and resources.
Peak Defence also sought to improve its existing offerings to stay competitive in the cybersecurity market. Neurons Lab collaborated with Peak Defence to implement a platform on Amazon Bedrock using LLMs (large-language models) to automate compliance processes.
Peak Defence’s goal was to transition from consulting to a SAAS platform. AI has allowed them to scale and transfer their knowledge to customers, allowing for better security.
AWS services were chosen because they allowed the customer to maintain control over their data and prevent it from ending up in an “uncontrolled AI learning environment”, a major concern for cyber security.
Solution: Generative AI for Cybersecurity
To achieve a robust, scalable and secure platform, several architectural considerations are essential. These considerations include selecting AI models that are based on performance, adaptability, and efficiency. They also include designing the architecture in order to support scalability, efficiency, and advanced security measures.
The document ingestion process of this solution is designed to seamlessly integrate company policy documents into a central knowledge base.
This data is then processed through LLM Embedding Models, including a href=”https://docs.aws.amazon.com/bedrock/latest/userguide/titan-embedding-models.html” rel=”noopener” target=”_blank”>Amazon Titan Embeddings G1/a>. These data are then processed using LLM Embedding Models including Amazon Titan Embeddings G1.
Peak Defence’s AI Automation Platform now uses QRANT vector search and keyword search as a new method of searching for and managing documents.
Vector search is particularly useful for understanding content written in different languages, searching through short texts that have typos or whose meaning is not clear without context, and searching text with typos. Keyword search is used to find exact phrases like security standard names and specific terms.
Overview
Peak Defence’s AI platform is based on Claude 3 and Amazon Titan. This is essential for automating compliance audits, and creating responses to RFPs. These AI models offer multilingual support and produce text that is similar to human speech. This makes the solution highly adaptable.
A serverless architecture powered AWS Lambda and AWS step functions ensures operational efficiency. This setup allows for dynamically handling fluctuating workloads. This is crucial for handling RFPs and audits in a timely fashion. AWS Step Functions simplify the orchestration of data pipelines and microservices, while AWS Lambda integrates with the LangChain Framework to offer scalability.
This process is integral to the Retrieval Augmented Generation (RAG) approach, which enhances AI output with real-time data retrieval. This process is an integral part of the Retrieval-Augmented Generation (RAG), approach. It enhances the AI’s output by providing real-time data retrieval.
Architecture
Figure 1 – AWS Reference Architecture
To operationalize the flow the following process was used:
- The customer is able to upload documents and interact via a web interface. This interface is deployed using AWS Fargate, a fully-managed serverless container service which can automatically scale up or down depending on demand.
- AWS fargate starts the process by hosting an API that orchestrates the document and data management.
- AWS stepFunction orchestrates workflow, beginning with AWS Lambda’s segmentation of audit questions into batches that are then processed separately. AWS Lambda executes LangChain, which includes answer generation, scoring and reasoning for each audit question.
- StepFunction then consolidates all the questions it has processed into a comprehensive report that covers every inquiry. The LLMs are then used to create a short audit report that outlines ISO 27001 requirements and identifies whether or not each criterion is compliant.
The journey to refine cybersecurity solutions begins with a solid foundation of scalable, AI driven architecture. It then dives deeper into a critical aspect of LLM testing and evaluation. This phase is crucial in ensuring the solution performs optimally and remains aligned to the dynamic nature of cyber threats and evolving compliance standards.
The evaluation strategy includes advanced tools such as Ragas, Langfuse. These tools are used to assess the LLM output based on relevancy, precision and accuracy, hallucinations, and faithfulness.
These metrics provide a quantitative basis for continuous improvement. They allow models to be refined with confidence and precision. Figure 2 shows an example of a Langfuse-based evaluation strategy.
Figure 2: Langfuse Interface
The collaborative development of a test data set in partnership with Peak Defence was key to optimizing LLM’s performance. This dataset was designed to test models by simulating a variety of cybersecurity scenarios.
Performance improvements are tracked through targeted experiments and learning mechanisms. This ensures that solutions evolve in line with the latest cybersecurity technologies and trends.
Figure 3: Testing dataset
After collecting feedback, the team processed and analyzed them to extract actionable insights. These insights were then incorporated in the dataset and used as a basis for further model tuning and enhancing the AI solutions.
This cycle of feedback and data integration is critical for the continuous improvement and evolution of the solution. It ensures that it evolves in accordance with user expectations and technological advances.
Conclusion
The AI automation platform launched in three months. Now, the AI solution can generate reports on compliance and responses to RFPs for security within minutes as opposed to the 2-3 week manual effort.
The new environment can be deployed in 15 minutes thanks to the automated infrastructure provisioning via GitHub Actions. Compliance consultancy that was once hampered by manual processes is now able to scale effectively to serve more clients.
- The Peak Defence team was previously faced with a growing number of repetitive tasks, which drained their resources and put them at risk of burnout. This left room for human error.
- Peak Defence has now automated some manual steps, while still applying higher level human thinking. This saves time for their team and ensures that they can offer the best quality for highly sensitive task.
- Peak Defence’s AI-based automation platform can produce audit reports within hours, or even minutes, depending on how big the company is.
- This allows customers to move through the compliance process much faster, without missing important details or overlooking vital information.
- The platform allows Peak Defence to provide compliance consulting services to their customers much more quickly, without reducing service quality.
Neurons Lab has a global team of data scientists, cloud experts, domain and user designers, business strategists, and 500+ engineers. They solve the most difficult AI challenges, such as automating cybersecurity operations with generative AI. Visit the website to learn more about Neurons and its services. You can also review its offerings on AWS marketplace. For more information, please contact info@neurons-lab.com.
Customer Feedback
“We needed to work with a flexible and professional partner as we went through a fundamental shift. Neurons Labs assisted us in adding generative AI capabilities into an existing platform. Peak Defence was able to evolve and unlock new abilities for its customers. The team at Neurons Lab brings creativity, organization and experience to the projects. This combination makes it easy and enjoyable to work together. “We recommend these guys without hesitation.”
– Roman Jasins – Co-Founder and Board Member of Peak Defence
.
NeuronsLab – AWS Partner Spotlight
NeuronsLab , an AWS Specialization partner, is a AI consultancy that offers end-to-end AI services – from identifying high impact AI applications to integrating the technology and scaling it – to empower companies to capitalize on AI’s capabilities.