Bug Hunting With LLMs: How a Bastion Specialist Finds Vulnerabilities Without Drowning in Spam

A specialist in the incident response department at Bastion, writing under the handle Vladislav, explained on Habr how over the past six months he moved bug bounty vulnerability hunting to large language models (LLMs). He built a pipeline of two AI agents with three layers of verification to filter out false positives before a report is submitted.

Why an AI agent drowns you in spam

You can point an AI agent at a bug bounty program in five minutes, but turning its "stream of consciousness" into accepted reports is a harder problem, and it is easy to get banned for spam and lose your rating along the way. The author has been hunting bugs on bounty platforms for about a year and a half, won the Bug Zone 7.0 competition, and has spent the last six months finding bugs specifically with LLMs. In his words, AI can deliver results rather than junk submissions.

• The author is a specialist in Bastion's incident response department with ~1.5 years of bug bounty experience
• Winner of the Bug Zone 7.0 competition
• Has spent the last ~6 months finding vulnerabilities with LLMs
• The write-up covers 4 approaches to AI bug hunting and 4 ways to pay for the model
• It also features 3-layer verification, a pipeline of 2 AI agents, and a live blind SSRF walkthrough

How to avoid drowning in false positives

The key to workable AI bug hunting is layered checking: the author set up three verification layers that every model finding must pass before it is submitted. Without this filter, bug bounty platforms treat raw LLM output as spam, which risks a ban and a penalty to the researcher's rating.

The roles in the pair are split. One agent generates hypotheses about vulnerabilities; the second verifies and sorts them, taking the routine of first-pass triage off the human. According to the author, manually sifting through a stream of false positives is exactly what turns a researcher into a "free triager for their own agent", which is what he set out to avoid.

The author also looks at the economics. He lists four ways to pay for the model, because when hypotheses are run at scale, token costs become a significant expense for the researcher.

What the blind SSRF walkthrough shows

As a live example, the author walks through hunting for a blind SSRF (server-side request forgery) vulnerability, where the server makes requests to internal resources on an attacker's command but does not return the response directly. The case shows how the AI agent formulates a hypothesis and how the verification layers confirm exploitability before the report is filed.

"Launching an AI agent on a bug bounty program takes five minutes.

But turning its stream of consciousness into accepted reports, so you don't get banned for spam and take a rating hit, is a harder problem."

— Vladislav, incident response specialist, Bastion

The author does not promise a ready-made "make it good" button: even with LLMs, the result rests on disciplined verification, not on automation for its own sake.

What this means

LLMs already give ethical hackers a real edge in bug bounty, but only when paired with a human and strict verification. A bare AI agent without verification layers generates spam that costs you your rating and access to programs. The value comes from the architecture of the pipeline, not from the model itself.

Frequently asked questions

What is AI bug hunting?

It is the search for vulnerabilities in bug bounty programs using large language models (LLMs): the model generates hypotheses about vulnerabilities, while the researcher checks them and writes up the findings. The Bastion specialist describes 4 approaches to this process.

How does an AI agent avoid turning a researcher into a spammer?

Through verification: the author passes every model finding through three verification layers and a pipeline of two agents, where the second agent handles sorting the findings. Without this filter, raw LLM output leads to a spam ban and a rating penalty.