Understanding The Risks of App Development with AI
Bob was thrilled when he created his first application using an AI-based coding method. The application, which displayed how much of the US's tax money is allocated to tech companies, was promptly launched online. However, months after the launch, Bob discovered a hidden security vulnerability in his site that could have allowed unauthorized individuals to access or modify data.
This issue wasn't unique to Bob. Around the web, there are numerous stories of applications created using AI-based coding methods that are riddled with security vulnerabilities. People have reported instances of AI coding agents erasing their company's production database, and others have had to take down their apps due to hacker attacks.
The Double-Edged Sword of AI Coding
AI coding has introduced a new era of "personal software", where anyone can use AI to create their own private applications. However, this convenience comes with a new wave of security issues. While these applications may be easy to create, they present a significant challenge to secure, especially in a world where AI can also be exploited to attack them.
Experts argue that AI coding isn't bad because it allows amateurs to build software. The danger lies when a personal application becomes business software and starts storing shared, hosted data. This transition often happens without people realizing, and it can lead to significant security risks, especially when these applications handle sensitive information such as customer logs, medical data, financial records, or internal documents.
Security professionals agree that AI coding is excellent for lower-risk projects, but any application handling financial records or sensitive data needs more scrutiny. It's crucial to think through the potential threats and risks, and when in doubt, it's better to err on the side of caution.
High-Profile Cases Highlight the Risks
There have been several high-profile cases highlighting the risks of AI coding. In one case, a developer launched a social network exclusively for AI agents without writing a single line of code. Within days, security researchers exposed the app's entire production database, revealing tens of thousands of email addresses and private messages. In another instance, researchers discovered approximately 5,000 publicly accessible apps built with popular AI coding tools that lacked any authentication. Nearly 2,000 of these were found to be leaking sensitive data.
It's worth noting that many professionally developed, pre-AI software also have significant security flaws. However, the rise of AI coding has dramatically increased the number of applications being produced, likely leading to a corresponding increase in security risks.
Addressing Security Risks in AI Coding
Addressing security risks in AI coding requires proactive measures. During a typical AI coding session, nothing will stop to check for security vulnerabilities unless you've installed something to do so. Therefore, it's essential to prompt for security checks when you build the application and repeat the process at the end, especially when the tool has access to sensitive data.
Experts warn against having a false sense of security from AI coding agents' reviews, especially when the agent doesn't understand your threat model or hasn't been given the correct guidance.
It's also important to understand the risk of a lack of authentication. Developers may not consider this when they move an app from local to cloud storage, leading to the exposure of sensitive data. This is a significant concern, as apps that run fine locally can expose a wealth of information when moved to the cloud.
While human experts reviewing code is the ideal scenario, this is becoming increasingly difficult due to the sheer volume of code being produced. The key is to think through what data the application is storing and what could go wrong. Ask the AI to build the application with security in mind and run code reviews after each change. Pay close attention before moving the application from your device to the cloud or giving it access to any sensitive data or accounts. The difference between a successful project and a disaster often starts with knowing the right questions to ask.
Bob was thrilled when he created his first application using an AI-based coding method. The application, which displayed how much of the US's tax money is allocated to tech companies, was promptly launched online. However, months after the launch, Bob discovered a hidden security vulnerability in his site that could have allowed unauthorized individuals to access or modify data.
This issue wasn't unique to Bob. Around the web, there are numerous stories of applications created using AI-based coding methods that are riddled with security vulnerabilities. People have reported instances of AI coding agents erasing their company's production database, and others have had to take down their apps due to hacker attacks.
The Double-Edged Sword of AI Coding
AI coding has introduced a new era of "personal software", where anyone can use AI to create their own private applications. However, this convenience comes with a new wave of security issues. While these applications may be easy to create, they present a significant challenge to secure, especially in a world where AI can also be exploited to attack them.
Experts argue that AI coding isn't bad because it allows amateurs to build software. The danger lies when a personal application becomes business software and starts storing shared, hosted data. This transition often happens without people realizing, and it can lead to significant security risks, especially when these applications handle sensitive information such as customer logs, medical data, financial records, or internal documents.
Security professionals agree that AI coding is excellent for lower-risk projects, but any application handling financial records or sensitive data needs more scrutiny. It's crucial to think through the potential threats and risks, and when in doubt, it's better to err on the side of caution.
High-Profile Cases Highlight the Risks
There have been several high-profile cases highlighting the risks of AI coding. In one case, a developer launched a social network exclusively for AI agents without writing a single line of code. Within days, security researchers exposed the app's entire production database, revealing tens of thousands of email addresses and private messages. In another instance, researchers discovered approximately 5,000 publicly accessible apps built with popular AI coding tools that lacked any authentication. Nearly 2,000 of these were found to be leaking sensitive data.
It's worth noting that many professionally developed, pre-AI software also have significant security flaws. However, the rise of AI coding has dramatically increased the number of applications being produced, likely leading to a corresponding increase in security risks.
Addressing Security Risks in AI Coding
Addressing security risks in AI coding requires proactive measures. During a typical AI coding session, nothing will stop to check for security vulnerabilities unless you've installed something to do so. Therefore, it's essential to prompt for security checks when you build the application and repeat the process at the end, especially when the tool has access to sensitive data.
Experts warn against having a false sense of security from AI coding agents' reviews, especially when the agent doesn't understand your threat model or hasn't been given the correct guidance.
It's also important to understand the risk of a lack of authentication. Developers may not consider this when they move an app from local to cloud storage, leading to the exposure of sensitive data. This is a significant concern, as apps that run fine locally can expose a wealth of information when moved to the cloud.
While human experts reviewing code is the ideal scenario, this is becoming increasingly difficult due to the sheer volume of code being produced. The key is to think through what data the application is storing and what could go wrong. Ask the AI to build the application with security in mind and run code reviews after each change. Pay close attention before moving the application from your device to the cloud or giving it access to any sensitive data or accounts. The difference between a successful project and a disaster often starts with knowing the right questions to ask.