AI security testing depends on the safety setup around the AI model

A security company says the hard part of AI-driven security testing is not the AI itself. It is the safety setup around it. Pointing an AI agent at your own web apps takes little effort, and the AI model can write the attacks. But keeping those attacks safe, and making the results worth acting on, depends on what the company calls the harness: the code and rules wrapped around the model.

What the harness does

Security testing of this kind, often called pentesting, means trying to break into your own systems on purpose to find weak spots. An AI agent makes thousands of small choices about where to send its next request. A plain-language rule such as "stay in scope" is not enough, because a web app often reaches beyond one domain through redirects and outside services. The article says code must check every request before it leaves.

The same goes for passwords and keys. Testing often turns up working credentials outside the agent's assignment. Whether it may use them is a policy decision that code should apply every time, not something left to the AI's judgment.

The risk of planted instructions

The agent reads every page and response, including hidden fields and comments. Anyone who can post to the app can hide instructions there. The article says one product review is enough. OWASP, a well-known security group, ranks this "prompt injection" as the top risk for AI applications and says no reliable prevention exists today. So the question becomes how much damage a fooled agent can do. Limiting each action helps. Human sign-off helps too, but OWASP warns that reviewers tire when asked to approve everything.

Why reports must be provable

The article points to Google, which in October 2026 stopped accepting product vulnerability reports to its open-source bug bounty program after a surge of automated submissions. Google said most were not valid. A finding a developer cannot reproduce is just another ticket. The harness has to run the exploit, record what it reached, and package it so anyone can replay it.

The vendor angle

The piece comes from the viewpoint of Pentera, which sells a tool called Pentera Apps. The company says it pairs an AI that decides what to test with code that controls what runs. It says every action is logged and any test can be paused or stopped. It also says each finding includes a replayable exploit script and standard severity scores, and that only minimal, redacted data reaches the AI model, with zero retention. These are the company's own claims.

What to do

If you hire a firm or buy a tool for AI-driven security testing, ask what stops it from breaking live software. Useful questions are below.

• How is the agent kept inside the systems it may test?
• Can a test be paused or stopped, and is every action logged?
• Can your developers replay each finding themselves?
• What customer data is sent to the AI provider?