How to review AI agent pull requests without drowning

How to review AI agent pull requests when Claude Code, Codex or Cursor open more PRs than your team can read. Ask for proof, sort PRs by risk and cap what agents get.

If your agents open more pull requests than your team can read, the fix is not reading faster. Ask every PR for proof, sort PRs by risk, and stop handing agents more cards than your team reviews in a day. That is how I review AI agent pull requests now, and it is most of why I built Kahoona.

Writing code got cheap. Saying yes to it did not. A lot of people said that out loud on X and Reddit this past week, so I'll start with what they said.

What people are actually complaining about

Your job is now reviewing massive agent PRs and rotating production keys.

X

Ryan Lanciaux @ryanlanciaux

· 27 likes · 1 repost · 9 replies

as of

That one is a joke, but the serious version is a review queue that grows every day.

Code review bottleneck after AI, how are teams handling it?

We merged around 40 PRs a week in March and we are at 110 now with the same four people qualified to approve anything that touches billing, and those four are also the people writing the hardest changes.

Reddit

Specialist_Agent3599 u/Specialist_Agent3599 r/TechLeader

· Score 26 · 101 comments

as of

I built a Claude Code review inbox for the PRs piling up behind me

Fifteen PRs waiting for my review had become a normal morning.

Reddit

jtompl u/jtompl r/ClaudeCode

· Score 3 · 1 comment

as of

And the question under all of it, from a reply to Theo:

Merging by hand isn't the bottleneck for me... reviewing is 😅 What's your setup for trusting agent PRs without reading every line?

X

David Sandy @davidsandy

· 0 likes · 0 reposts · 1 reply

as of

More PRs, the same few reviewers, and nobody wants to rubber stamp code they didn't read. This is what I'd do about it.

Review AI agent pull requests by proof first

The best advice I saw this week was one line.

Stop reviewing agent PRs without verification! If there isn't a screenshot or screen recording, it's not worth your time.

10x more important than optimizing your CI checks.

X

Ben Holmes @bholmesdev

· 213 likes · 9 reposts · 15 replies

as of

Yup. An agent will happily explain why its code works. The explanation costs it nothing. A screenshot of the page actually rendering, a recording of the flow, the test output, that costs it something, and it saves you the first ten minutes of every review.

What I want on every agent PR before I open the diff:

  • The card it came from, so I know what was asked and can check the PR did that and nothing else.
  • Proof it works. A screenshot or recording for anything a user sees, the command and its output for anything they don't.
  • Green checks. If CI is red, it's not ready, and it shouldn't be in my queue at all.
  • A small diff. If the agent "improved" three files nobody asked about, it goes back.

Put those rules in the file your agent already reads, your CLAUDE.md or AGENTS.md or whatever your setup uses. Agents follow written rules better than people do. They just need them written down.

Sort PRs by risk, not by who opened them

Not every PR needs the same review. Treating a dependency bump like a billing change is how the queue gets to 110.

Three review lanes for AI agent pull requests: boring changes get a quick look, normal changes need a screenshot or recording plus the card, risky changes in billing, auth, migrations and CI need a named senior
How I sort agent PRs. A starting point, not a standard, so adjust the lanes to your repo.

Boring. Dependency bumps, flaky test fixes, copy changes. Checks pass, someone gives it a quick look, done. Anyone on the team can say yes.

Normal. Features and bug fixes that came from a card. These need the proof above, and the person who owns the card reviews them, because they know what the card meant.

Risky. Billing, auth, migrations, CI config and anything near secrets. Proof, a small diff and a named senior every time. This is the lane where the four people in that Reddit thread should be spending their hours, not on dependency bumps.

The rule that doesn't change between lanes: the agent opens the PR and a person merges it. Some people argue agents should merge their own PRs. On a small team I don't think the risk is worth it, and you don't need it once the boring lane is quick.

Hand agents only as many cards as you can review

This is the part people skip. If your team can review five PRs a day and your agents open thirty, the queue doesn't care how good your process is.

The review queue is the real limit. If the team can review 5 agent PRs a day and the loop opens 30, the other 25 rot and become the story. I'd start with one boring task class, like dependency bumps or flaky test fixes, and widen only once the merge rate holds.

X

Vivek Yaligar @vkyaligar

· 0 likes · 0 reposts · 0 replies

as of

Run his numbers for one work week and it gets ugly fast.

Example arithmetic for an agent PR review queue: if agents open 30 PRs a day and the team reviews 5, 125 PRs are waiting by Friday, and if agents are handed 5 cards a day, nothing is waiting
Example arithmetic using the 5 and 30 from @vkyaligar's post, assuming the same numbers every day. Not measured data.

That's simple subtraction, not data from a real team, but the shape is right. Twenty five extra PRs a day is 125 by Friday. Nobody reviews 125 PRs carefully. They start approving without reading, which is the exact thing you were trying to avoid.

So count what your team actually reviews on a normal day and hand agents about that many cards. Start with the boring lane, like he says, and add more once things merge cleanly. Running more agents than your reviewers can keep up with doesn't make you faster. It just makes the queue longer.

Counting pull requests opened is a different number, and I don't use it: Stop counting pull requests now that agents open them.

Let tools take the first pass, not the yes

AI reviewers help. The r/TechLeader poster said CodeRabbit on their repos "genuinely helps with the volume, but somebody senior still has to be the one who says yes." That matches what I see.

Claude Code has a Code Review service that comments on pull requests, and its docs say a review never approves or blocks the pull request, so the decision to merge stays with your team. I think that's the right line. Let a bot catch the small stuff and summarise the diff. Don't let it be the approval.

Checking the pull request against the card is a different job from that first pass. The split is in AI code review vs verification.

How Kahoona handles this

I built Kahoona because I wanted the board and the review queue to be the same thing. I make it, so weigh this part with that in mind.

A card handed to an agent comes back as a GitHub pull request. The team reviews it and merges it.
  • You hand a card to an agent and it comes back as a GitHub pull request. The card is the context, so the reviewer always knows what was asked.
  • Agents never merge. Your team reviews and merges, every time, in every lane.
  • The standup and the Friday update keep themselves current from GitHub, so nobody spends Monday morning writing up what the agents did over the weekend.
  • Clients see only the snapshots you publish, through the Client Inbox. They don't see the live board or the PRs that are still waiting.
  • Claude, Cursor and other agents connect through MCP.
  • Kahoona is free for one person. On a team the first chair stays free and each extra chair is $5 a month, so five people pay $20 and ten pay $45. No card needed.

What Kahoona doesn't do for you is decide how many cards to hand out, or write your review rules. That's still your call, and it should be.

If you're on another tool

You don't need Kahoona to do any of the above. The proof rule, the lanes and the card limit work anywhere.

On Linear you can run a full agent workflow with tickets, comments and reviews. Anurag Bhagsain described his on X, with agents planning, coding and leaving comments on Linear tickets while he stays the final check. GitHub Projects sits right next to your PRs already. Trello and ClickUp both link PRs to cards and tasks. If your board already works for your team, keep it and add the rules.

Where Kahoona is different is that the card and the pull request are one loop, and the status updates come from GitHub instead of from someone remembering to write them. I compared it fairly with the others here:

Proof first, lanes by risk, and no more cards than you can review. That's the whole system tbh.

Questions

Should AI agents be allowed to merge their own pull requests?

Not on a team, in my view. Let the agent open the PR and run the checks, and have a person merge it. In Kahoona agents never merge. Claude Code's own Code Review service also never approves or blocks a PR, so the merge decision stays with your team.

How do I review agent PRs without reading every line?

Ask for proof first. A screenshot or screen recording of the change working, the card it came from, and passing checks. Then sort PRs by risk so boring ones get a quick look and billing, auth, migrations and CI changes always get a named senior.

How many agent PRs can my team handle?

Count how many your team actually reviews in a normal day and hand agents about that many cards. Anything above that just waits in the queue.

Does an AI reviewer count as an approval?

Treat it as a first pass that cuts the small stuff, not as the person who says yes. A human still merges.

What does Kahoona cost?

Kahoona is free for one person. On a team the first chair stays free and each extra chair is $5 a month, so five people pay $20 a month and ten pay $45. No card needed.