Free Intensive Nov 21–22Free 2-day hands-on Vibecoding Intensive on Nov 21–22 at 16:00 MSK (Sat–Sun) · Live Cohort 3 starts Nov 23Join Free →Cohort 3 (Nov 23)
ReleaseCode / SDK · ActionableGitHub Blog→ original

GitHub and Microsoft Cut Thousands of AI Findings Down to 99 Reports With Patches, and More Than Half Are Fixed

Open source maintainers are buried in AI-generated bug reports, and every one has to be checked by hand. In the Codename MDASH pilot, GitHub and Microsoft cut thousands of raw findings down to 99 verified private reports, each with a proposed fix, for six projects. These are not "something looks suspicious here" notes but bugs with a patch that can be accepted in one pass. More than half have already been fixed, so the value of AI-driven bug hunting lies in closed bugs, not in the number of findings.

AI-processed from GitHub Blog; edited by Hamidun News
GitHub and Microsoft Cut Thousands of AI Findings Down to 99 Reports With Patches, and More Than Half Are Fixed
Source: GitHub Blog. Collage: Hamidun News.
Listen to article
Article Fact Sheet & Key Figures
ParameterDetails
Primary sourceGitHub Blog ↗
Editor-in-Chief
Key metric #1Of thousands of raw AI findings, only 99 reached maintainers
Key metric #2The reports are private, and each comes with a proposed fix
Key metric #3The pilot covered six widely used open source projects

Imagine you maintain a popular open source library. Every morning, a dozen new bug reports land in the tracker. Many were written by a neural network, and it is unclear which ones are real.

Checking and fixing all of this falls to maintainers, the people who keep the project going. GitHub and Microsoft proposed a different approach and shared the results of a pilot called Codename MDASH. AI found thousands of raw issues in six widely used open source projects.

Only 99 reports reached maintainers. They were verified, sent privately, and include a proposed fix. Maintainers have already fixed more than half of these issues.

The idea is simple: instead of a message saying "there may be a hole in this function," a person gets a confirmed bug and a draft patch.

What the pilot produced

The numbers in the announcement are short but telling. Of the thousands of raw AI findings, 99 remained. This is a funnel: most of the output was filtered out before it ever reached anyone else's tracker. The reports themselves were not published. They were delivered privately, through a closed channel. Each came with a proposed solution.

For a maintainer, the difference is tangible. A raw report has to be read, reproduced, and then either closed or fixed by the maintainer. A verified report with a patch can be reviewed, accepted, or tweaked in one pass. The pilot's result: more than half of the issues delivered have already been fixed.

Why it matters

The burden on maintainers is growing not because of a shortage of bugs, but because of a surplus of poorly verified reports about them. When a model generates a report, it costs almost nothing to produce. Triage still costs a real person's time. So betting on "even more findings" only makes the situation worse.

In MDASH, verification and fix preparation happen before submission. This shifts part of the work to the side that sends the report. The success metric is clear: how many issues were actually closed, not how many were found.

What to take away

If you build AI-based tools for finding bugs or vulnerabilities, count the number of fixed issues, not the number of findings. That is a metric users understand, too.

If you are a maintainer or are responsible for open source at a company, demand three things from automated reports: reproducibility, a private channel for sensitive issues, and a proposed patch. A report without them is better treated as a draft.

If you are a product manager, note the pattern: "filter plus follow-through to a result" is worth more than a plain "generator." User attention is limited, and protecting it is part of the product.

Limitations

The pilot is small: six projects and 99 reports. It does not prove that the approach scales to thousands of repositories with different processes. The announcement does not say how much human time went into verification inside the GitHub and Microsoft teams, so the full economics of the approach cannot be assessed. Also, "fixed" does not necessarily mean "the proposed patch was accepted without changes": maintainers may have rewritten the solution their own way.

Still, the direction is clear. The next step for AI in open source security is moving from a stream of signals to closed tasks. For an ecosystem held up by maintainers, that is more useful than yet another hundred "potential issues."

Story

Key Questions Answered

What the pilot produced?

The numbers in the announcement are short but telling. Of the thousands of raw AI findings, 99 remained. This is a funnel: most of the output was filtered out before it ever reached anyone else's tracker. The reports themselves were not published. They were delivered privately, through a closed channel. Each came with a proposed solution. For a maintainer, the difference is tangible. A raw report has to be read, reproduced, and then either closed or fixed by the maintainer.

Why it matters?

The burden on maintainers is growing not because of a shortage of bugs, but because of a surplus of poorly verified reports about them. When a model generates a report, it costs almost nothing to produce. Triage still costs a real person's time. So betting on "even more findings" only makes the situation worse. In MDASH, verification and fix preparation happen before submission. This shifts part of the work to the side that sends the report.

What to take away?

If you build AI-based tools for finding bugs or vulnerabilities, count the number of fixed issues, not the number of findings. That is a metric users understand, too. If you are a maintainer or are responsible for open source at a company, demand three things from automated reports: reproducibility, a private channel for sensitive issues, and a proposed patch. A report without them is better treated as a draft.

ZK
Zhemal Khamidun — CPO at Alpina Digital, product manager of AlpinaGPT. Building AI products since 2019. Hamidun News is a pet project that grew into a production aggregator with 6,000+ readers.
✦ Unified AI Workspace · Nomad

Test Frontier LLMs & Autonomous Agents in One Workspace

Run Claude Opus 5.5, GPT-6, Gemini 3.1, and DeepSeek side-by-side in Nomad AI (nomadnet.ai) — unified model routing, deep research, and multimodal workflows under one subscription.

Try Nomad AI →Launch Nomad Agent
What do you think?
Loading comments…