How to Use ChatGPT for IT Security Audit
While developers have found many uses for GPT-4, OpenAI’s latest large language model, we’re now seeing how security teams can make tasks more efficient with its advanced natural language interface. Tasks that would typically take days, weeks or months to properly research and plan can now be accurately performed within seconds.
For example, an internal IT security audit for a DevOps environment usually happens over the course of a few days, but the process begins long before that. The auditor will likely need to speak with different engineers and team managers to take inventory of the company’s infrastructure and to learn about the teams’ IT workflows. This process can take days or weeks depending on the teams’ availability to provide the necessary information. It can be a frustrating wait.
We built a ChatGPT-like user interface within Slack that lets security teams ask questions and get answers within minutes to assess the efficacy of their DevOps infrastructure. The PromptOps chatbot turns manual IT security audits into automated workflows with prompts straight from Slack. The auditor is empowered to get the answers they need to ensure their systems are uncompromised and costs and protocols are on point. With an IT security audit assistant like PromptOps, you can cut down the manual work for yourself, your team and your company.
This article will dive into why we chose ChatGPT as a chat assistant for security audits and the steps you need to take to make chat assistants valuable for your business.
Why ChatGPT Is Necessary
Threat actors are already making headlines with their use of ChatGPT to strengthen attacks. Overwhelmed security teams must find ways to use the same technology to bolster their security posture while maximizing their time and resources. Security teams cannot waste time waiting for engineers to respond to questions like what systems are running in their project and their dependencies or which users are on a database.
Generative AI speeds up auditing and compliance. The process of auditing ensures that you are maintaining good observability into the different areas of your organization. It’s recommended that every organization conducts an audit at least once a year, but I’d advise a higher frequency to ensure your organization is keeping pace. It’s also important to perform security audits after an attack or a major update, such as new tool installation or a data migration. In both cases, an audit is a safeguard against new vulnerabilities that may have been introduced with the breach or environmental change.
We chose ChatGPT to create PromptOps, the chat assistant, because it offers the best mix of features for DevOps and security teams. When an application fails, developers and operators can simply ask questions in conversational English and trace the answer for why something happened back to the original cause. When performing IT security audits, security teams can get a full list of existing systems, tools and environments within minutes versus days or weeks. For example, we asked PromptOps to list all existing EC2 instances that have public IP addresses, their create dates and instance types.
Here is the response from PromptOps:
PromptOps provided the EC2 instances and the requested metadata within minutes. It also lets us view the script it’s about to run.
We continued the conversation with PromptOps, asking it to list all the identity and access management (IAM) roles associated with the EC2 instances.
Here is the response from PromptOps:
PromptOps connects all your tools like Prometheus, GitLab, Kubernetes and more with your knowledge base across Slack, Notion and Confluence. This saves you time when troubleshooting and auditing.
Preventing Hallucinations
As companies increasingly use generative AI-based tools, it’s reasonable to be concerned about accuracy or AI hallucinations. CtrlStack strives to avoid hallucinations by grounding it in your organization’s data. Your infrastructure data, dependencies and current DevOps workflows all provide the model with contextual understanding to avoid hallucinations.
Powered by a change intelligence platform for better observability, this new chat assistant will not only make it easier for security teams to interrogate the platform, it will also help security teams learn and understand how the underlying systems work through its question-and-answer format. PromptOps also provides a visual graph of all the deployments in the cluster to make learning and auditing much easier.
Our goal is to eliminate the work needed to constantly validate the outputs while building confidence in the tool. For example, when providing a comprehensive inventory of all your assets, PromptOps also provides a script to generate an up-to-date diagram of the infrastructure in CtrlStack with detailed information about system health, connectivity and dependencies. By using PromptOps to get the data from CtrlStack and reusing the data for follow-up conversations, users never need to start from scratch when interacting with the chat assistant.
Improving Security Posture
The complexity and variety of both cyberattacks and the IT environments they target make analyzing and improving security challenging. But it’s critical. Getting an accurate asset inventory is foundational to your security posture. The ability to track and audit your inventory is a baseline requirement for most security standards, including the 18 CIS Critical Security Controls, HIPAA and PCI.
Chat assistants like PromptOps help to significantly reduce the time needed to get an accurate comprehensive view of your infrastructure, down to the pods within a Kubernetes cluster. That means cutting preparation time for audits by up to 75%. Teams can stay informed about their security posture and proactively check the status of their enterprise assets at any time. Successfully automating real-time inventory for all your enterprise assets will allow security teams to focus their efforts on identifying technology gaps, mapping their attack surface and understanding their cyber risk.
With its capability to observe system changes and relationships across the stack via CtrlStack, PromptOps provides the first step in leveraging generative AI like ChatGPT to improve security posture at speed.
Parsing through CtrlStack, which serves as the system of record for change management, PromptOps lets you monitor and trace unauthorized system changes that could indicate suspicious activity with a simple interaction, minimizing the time and effort needed to identify the cause and scope of a security breach. Need to find an issue in a developer’s code that was committed a month back? Look back in time with a question to determine which lines of code were modified and when based on a complete history of all changes relevant to those problems.
PromptOps is currently offered as an experimental offering. You can use it with CtrlStack to avoid AI hallucinations or on its own for free.
