Quick answer
When AI multiplies pull requests, reviews and CI become the real limit — not typing speed. OpenAI’s leaders describe auto-review, rebuilt CI and sparse checkouts after agents flooded their monorepo. You can steal the operating idea without replacing Git: measure the queue, shrink diffs, automate low-risk review, and protect high-risk paths.
Key takeaways
- Track review lag and CI wait, not lines of code generated.
- Split low-risk and high-risk changes so agents are not stuck behind payment-code reviews.
- Prefer smaller PRs and clearer owners over heroic human review marathons.
- Cache and sparsify checkouts if agents clone large repos repeatedly.
- Keep humans on auth, money, privacy and irreversible deletes.
- Audit auto-approvals weekly so speed does not hide silent regressions.
📘 Main article: This guide supports OpenAI Outgrew Git: What Happens When AI Agents Write Code Too Fast. It turns the interview’s bottleneck story into a team checklist.
In this guide
- Why do reviews and tests become the bottleneck?
- What should you measure first?
- How do you fix code review without slowing shipping?
- How do you keep CI/CD up with AI-written code?
- How do you tier risk so agents can move fast safely?
- What does a one-week improvement plan look like?
- What mistakes make the backlog worse?
- Frequently asked questions
- The bottom line
- Related posts
Cheaper code is only useful if it can be checked and released. This guide answers one question: how do you fix code reviews and testing when AI writes more code? It supports our summary of OpenAI outgrowing Git and the source interview.
Why do reviews and tests become the bottleneck?
Vijaye Raji’s point is simple: once ideas become prototypes in minutes, the conversation-before-coding step shrinks. Humans then face many more diffs, longer CI queues and agents that keep pushing. The organisation that only celebrates codegen speed will drown in unreviewed work.
That is not an argument against agents. It is an argument for treating review and CI as product infrastructure, the same way OpenAI treated the repository after throughput jumped.
What should you measure first?
| Metric | Healthy signal | Warning signal |
|---|---|---|
| Time to first human review | Hours, not days | PRs idle over a working day |
| CI wall-clock for main PRs | Stable or falling | Rising with every new agent seat |
| Flaky-test rate | Near zero on required checks | People re-run “until green” |
| PR size | Most under a readable threshold | Multi-thousand-line “AI dumps” |
| Revert rate | Stable | Climbing after AI adoption |
If you cannot see these numbers today, instrument them before you buy more coding seats. Otherwise you are scaling the wrong bottleneck.
How do you fix code review without slowing shipping?
- Define auto-review scope — style, obvious bugs, test presence on low-risk packages.
- Require human review for auth, payments, personal data and migrations.
- Cap PR size and ask agents to open stacked small changes.
- Name an owner for every area so review requests do not land in a void.
- Publish a review SLA (for example, first response within one business day).
- Sample auto-approved PRs in a weekly audit so the filter stays honest.
💡 Tip: Auto-review is a filter, not a rubber stamp. Keep a sample of auto-approved PRs for weekly human audit.
How do you keep CI/CD up with AI-written code?
- Move slow, non-blocking checks out of the critical path.
- Cache dependencies and build artefacts aggressively.
- Fail fast on deterministic lint/type checks before expensive suites.
- Give agents a separate CI pool if they starve human jobs.
- Fix flaky tests instead of letting agents retry forever.
- Record CI cost per merged PR so “more agents” has a visible bill.
How do you tier risk so agents can move fast safely?
| Tier | Examples | Gate |
|---|---|---|
| Green | Copy, docs, internal tools with no customer data | Auto-review + basic CI |
| Amber | Product features with feature flags | Human review + standard CI |
| Red | Auth, billing, privacy, production infra | Senior review + extra tests + slower deploy |
Write the tier list where agents can read it (repo rules, agent brief, CODEOWNERS). Ambiguity is how high-risk changes slip through on a Friday.
What does a one-week improvement plan look like?
- Day 1–2: export review lag, CI duration and flaky tests for the last month.
- Day 3: label packages green / amber / red with owners.
- Day 4: turn on auto-review for green only; announce the SLA.
- Day 5: kill or quarantine the top three flaky tests.
- Friday: review one day’s auto-approvals as a team; adjust the rules.
What mistakes make the backlog worse?
- Buying more coding seats while review capacity stays flat.
- Letting agents merge to main with no sparse-checkout or cache plan.
- Measuring “PRs closed” without measuring incidents and reverts.
- Skipping tests because “the model looked confident”.
- Hiding red-path changes inside giant “refactor” PRs.
Next: read how OpenAI outgrew Git, then apply the founder litmus test so you are not only accelerating a thin wrapper.
Frequently asked questions
Do we need OpenAI’s custom repository?
No. Start with review SLAs, CI caching and risk tiers. Rebuild storage only if checkout and merge tooling are clearly the limiter.
Should every AI PR get a human review?
High-risk paths yes. Low-risk paths can use auto-review plus sampling.
What is the first dashboard to build?
Review lag, CI duration, flaky rate, PR size distribution and revert rate.
How does this relate to agent cost?
Unused review capacity wastes coding spend. See also whether an AI agent’s cost is worth it.
Can juniors own amber reviews?
Yes, with a clear checklist and a senior on red paths. AI volume makes mentorship more important, not less.
The bottom line
AI coding shifts the scarce resource from typing to trust. Fix the queues that create trust — reviews, tests and release gates — and faster agents become an advantage instead of a backlog generator.
Related posts
- OpenAI Outgrew Git: What Happens When AI Agents Write Code Too Fast
- The Founder Litmus Test for New AI Models
- Is an AI Agent’s Cost Worth It?

[…] read what OpenAI changed when agents outpaced Git, and tighten reviews and testing so faster models do not just create faster […]