Case study · August 5, 2026
An AI Reviewer Found Two Bugs. A Developer Merged Anyway.
By Justin Gordon. Adapted from the ShakaCode blog. The original article also covers a separate audit of 30 commits.
A developer merged a pull request after an AI reviewer had identified two bugs. Neither comment had a reply explaining a fix or a reason to leave the code as it was.
The PR contained six commits, changes to 17 files, and 83 conversation items. Two important comments were easy to miss among the updates. The useful lesson for the Shaka skill is simple: make those comments easy to find, then help the developer decide what to do about them.
What the reviewer found
Hours before the merge, the Claude reviewer reported two problems:
- A temporary failure from an external service could be mistaken for a removed listing, causing the application to delete a valid provider record.
- After the last failed attempt to contact the service, the code could wait another 60 seconds even though it would not try again.
The reviewer had done something useful. But the PR was merged without either finding being addressed. The problem was following through on the review.
Make the two important comments easy to find
A long PR discussion can contain automated summaries, test updates, review comments, and replies. Asking a person to reread the whole page after every change wastes attention and makes missed comments more likely.
The agent should bring back a short account of the findings that matter. For each one, explain whether the problem was fixed, why the reviewer was mistaken, or what decision the developer needs to make. If a fix can wait, say who will do it and when. A bug that could delete valid records should not be postponed just to finish the PR.
Passing tests alone do not answer a review comment. Before merging, check that the important findings have answers and that the tests still pass after the fixes.
Keep useful detail without making everyone read it
The separate audit found another drain on attention: a six-line code change with a 154-line commit message. It included a decision log, test results, and review records. All of that text would remain in the project's history for future readers.
Keep the explanation where it helps the reader:
| Where | What a reader needs |
|---|---|
| Commit message | What changed and why |
| PR description | The result, checks, and decisions needed before merging |
| Code walkthrough | Why the implementation works this way, with links to the code |
| Review replies | What happened to each finding |
| Expandable PR details | Longer test and review results, usage, and notes for resuming unfinished work |
The summary should be enough to decide what to read next. Detailed records should be available without crowding out the decision.
What this means for using Shaka
The Shaka skill asks the agent to test and review changes locally, fix problems, and then push the PR. Automatic code checks such as ESLint and RuboCop can catch routine mistakes early. Trying the changed behavior can reveal problems those checks miss.
When GitHub reviewers leave comments, the agent should assess them and explain the outcome. A developer should be able to see what is ready and what needs attention without sorting through dozens of status updates. Shaka gives the agent that procedure; it does not guarantee that every bug will be found or every instruction followed.
If reviews keep producing noise, improve the review instructions and project settings. The point is to help trusted developers spend their attention on better decisions. Requirements that must apply to every merge belong in GitHub's controls.
Read how to give Shaka feedback or customize a local review. The original article has the wider audit results. ShakaCode can help teams improve how they use coding agents.