Tutorials

GitHub AI Scan at Scale: Coverage, Cost, and Evidence Limits

What GitHub's November 2026 bulk AI Scan preview changes for repository security, and how to evaluate findings without mistaking a scan for proof of safety.

Approximately 6 min read

GitHub’s November 16, 2026 announcement makes it possible to start AI Scan for up to 20 repositories in one operation. This is more than a convenience for a security team with a large repository portfolio: it creates a new, on-demand source of vulnerability hypotheses that lands in the same code-scanning backlog as CodeQL. But a shared backlog does not mean equivalent coverage, evidence strength, or repeatability. The operational question is how to use a time-bounded AI agent without treating its findings as automatically confirmed defects—or a clean run as a security certification.

This article analyzes GitHub’s public release notes and product documentation. It does not report an independent scan, measured detection rates, vulnerability reproductions, or real-world billing outcomes.

What changed on November 16

The new bulk-start workflow launches AI Scan for as many as 20 selected repositories at a time, either during organization-level enablement or from the organization’s code-scanning alerts page. Individual repository scans can also be initiated from repository alerts. GitHub labels bulk start as a public preview. The feature is available on github.com and GitHub Enterprise Cloud with data residency; a Copilot license is not a prerequisite.

The scope is important. This announcement is about AI Scan for repositories, which examines the default branch on demand. It is not the same as AI scanning of pull requests or the specialized scanning of AI applications for agent risks. Those capabilities have different triggers, coverage, and commercial conditions. Mixing them in one metric would produce an inaccurate account of which code was examined.

How the scanner differs from CodeQL

GitHub describes AI Scan for repositories as a proprietary, multi-model-capable harness. An agent searches the repository, selects files and functions for deeper inspection, and stops when its time budget is exhausted. It does not promise to read every file. This design may find issues in languages CodeQL does not support and additional issues in languages CodeQL does support. The product documentation does not establish a general recall rate, a false-positive rate, or a guarantee of complete repository coverage.

CodeQL and AI Scan operate independently. Existing CodeQL alerts can suppress matching AI Scan results, but GitHub explicitly describes matching as best effort. Duplicate findings can remain. This makes the combined alert queue a useful worklist, not a deduplicated count of unique vulnerabilities by construction.

A practical interpretation is that AI Scan can add candidate attack paths and contextual reasoning to an established static-analysis program. It should not replace deterministic tests, dependency inventory, manual review of sensitive boundaries, or runtime controls. A finding deserves validation against reachable entry points, actual permissions, data flow, configuration and relevant deployed versions.

The hidden denominator: what was not examined

Security reporting often counts alerts but omits the denominator. An AI agent that stops on a budget may spend substantial effort on one subsystem and never inspect another. Therefore, zero findings could mean no issue was found in the selected code—not that the entire repository is free of vulnerabilities.

For each scan, retain the repository identity, default-branch commit, scan start and completion times, policy configuration, findings and analyst dispositions. Where the product exposes scope or execution details, retain those too. If it does not, record that limitation rather than invent coverage percentages. Repeat the same process on a deliberately selected set of high-risk repositories to learn whether findings are actionable before scaling to hundreds of projects.

Avoid comparing raw alert counts across repositories with different languages, sizes, threat models or scan histories. A larger count can reflect better coverage or noisier detection; a smaller count can reflect missing examination. Neither directly measures security posture.

Billing and administrative boundaries

GitHub’s documentation says each on-demand repository scan consumes Code Security AI credits, independently of active-committer licensing. Included credits may apply first; additional usage is subject to the relevant billing account and budgets. The documented detection limits are 30 minutes and 150,000 AI credits per scan. These are ceilings, not predictions of typical cost.

Eligibility depends on ownership and visibility. Public organization repositories can use the feature without an active-committer license; private or internal organization repositories require the relevant GitHub Code Security or Advanced Security entitlement. Public personal repositories are eligible, while private personal repositories are not. A Copilot license is not required to use AI Scan, although the billing arrangement can depend on how a personal account receives Copilot.

At enterprise level, policy controls whether organizations may enable AI Scan, and allowing the feature does not automatically start scans. At organization level, owners and security managers can enable it for eligible repositories; repository administrators can opt out. Users with write access trigger scans. The REST API also exposes a repository-scan setting, but enabling it is not the same as executing a scan.

For a pilot, decide the maximum total credit exposure before turning on bulk starts. Limit the first cohort, confirm billing ownership, and document who can initiate scans. Treat preview availability and policy behavior as subject to change.

A verification workflow that survives false positives

Start with a narrow cohort representing different languages and risk classes. Choose repositories with known entry points, tests and maintainers available to assess alerts. Record the exact default-branch commit before each scan. This matters because a later code change can make a valid finding appear stale, or make an invalid finding difficult to reproduce.

For each alert, first confirm the claimed source-to-sink or authorization path in the cited code. Then determine whether the vulnerable path is reachable under supported configurations. If safe, construct a minimal test or fixture that demonstrates the security property without touching production systems. Separate confirmed exploitability, plausible but unproven weakness, configuration-dependent exposure and false positive. Record why the analyst assigned the disposition.

Next, compare with CodeQL and other tools by root cause, not by alert count. Two alerts on the same unchecked trust boundary should not become two independent discoveries. Conversely, a novel AI Scan finding outside CodeQL coverage should be preserved with enough code and environmental context for another reviewer to verify.

Finally, track remediation. GitHub warns that merging a fix does not automatically run another repository AI Scan during preview. A new scan must be manually triggered to reassess it. The alert may automatically close only when the exact triggering line changes; otherwise a manual closure may be necessary. Neither closure path by itself proves that the root cause is eliminated. Add regression tests and inspect equivalent code paths before treating the remediation as complete.

What a useful pilot should measure

A useful evaluation separates finding production, validation effort, and remediation outcomes. Record how many alerts were independently confirmed, the time to disposition, duplicate root causes, new tests added, fix acceptance and the cost charged for scans. Use these as internal pilot measures, not as claims about GitHub’s universal accuracy.

For a fair comparison, pin the code revision and assessment criteria. Consider a small blind review in which analysts receive anonymized findings from AI Scan and existing tools, with a common evidence rubric. That cannot prove global recall without a known vulnerability corpus, but it can reveal whether the additional alerts are useful to the team and whether triage cost outweighs benefit.

The most defensible conclusion from GitHub’s November 16 release is narrow: bulk start reduces the administrative friction of requesting scans, and AI Scan adds a different detection approach to the existing backlog. It does not turn an on-demand, time-bounded analysis into exhaustive assurance. Security value appears only after scope, cost, findings and fixes are independently checked.

Primary sources

Sources and further reading

Continue reading

More in Tutorials ↗