AI Code Review Without the Comment Spam

Generating code is now a solved problem with AI coding agents. However, the amount of code they are generating is causing new problems. The enormous volume of new code has shifted the bottleneck further right in the Software Development Lifecycle, putting pressure on developers to review it. Developers' roles are shifting from writing code to being reviewers and decision makers. This means they need to be focused on high-value signals to best utilize their time.

At Gitar, we have been experiencing this problem first-hand. However, existing AI code review tools were spamming our Pull Requests with lots of low-signal comments. Developers today are dealing with a cluttered, overwhelming PR experience. And when a PR is flooded with comments from bots, developers are less likely to read them.

This PR is a great example of bots commenting with a lot of noise and not a lot of value.

Our Approach

With decades of experience working with thousands of engineers at Uber, Meta, and Google, our team at Gitar has experienced first-hand how noisy PRs can be detrimental to the code review process. We have created a joyful developer experience for AI code review on GitHub and GitLab by sticking to some core design principles for how Gitar should interact and post comments.

We have three main principles:

1. Focus on High Value Issues

2. Keep Developers in their Flow

3. Clean up After Ourselves

Focus on High-Value Issues

Gitar consolidates all the important information in one living “Dashboard” comment.

When we say the comment is “living”, we mean that it continually stays up-to-date, acting as a central hub for all of Gitar’s insights. It only shows up if there is something meaningful to convey, and it moves down in the activity timeline as changes are made. We believe this moving comment behavior is an industry first.

This comment has four sections, one for each of Gitar’s primary features. They are sorted by importance, with the most critical information at the top. Each section shows a summary header line, with the details folded into a collapsible beneath it.

1. CI Failure Analysis: If the PR fails a CI check, Gitar will analyze the cause and surface that at the top. Gitar’s CI Failure Analysis saves you from needing to dig through obscure failure logs, and it will de-dupe failures across multiple jobs or pipelines. All of this is kept up to date in real time. CI systems as they are today are broken, and we are rethinking how devs should pay attention to them in the AI era.

2. Code Review: Code Review findings show up next, along with whether changes are being requested, and a summary of how many findings from previous reviews have been resolved. Gitar’s Code Review is designed to stay focussed on the most important issues, and critical bugs are surfaced at the top level.

3. Checks & Automations: Gitar allows you to define custom checks and automations in natural language that will run on every push. For example, “Enhance the PR Description” or “Link relevant Jira tickets”. There will be more on this in a future blog post.

4. Tips & Options: Interacting with Gitar is key to getting the most out of it, and the last section shows you how to interact. In addition to natural language, Gitar understands some specific commands. And in some situations Gitar will surface a checkbox to make interaction as easy as one click.

Keep Developers in their Flow

Of course we could have built our own UI for Code Review, and eventually we may do that, but we felt that the best decision right now is to meet developers where they already review code - on GitHub and GitLab. Whether Gitar is helping to address code review feedback or fix a CI failure, we keep developers in their flow. A checkbox to apply a suggested fix with a single click, followed by a live status indicator, is one example of that.

Clean Up After Ourselves

We needed a way to provide updates without creating a long, confusing trail of outdated comments. Here’s how we do it:

1. Keep the comment updated and in focus: To maintain the developer's focus, Gitar intelligently repositions the Dashboard comment within the discussion thread when major events occur, such as a new commit push. This prevents the comment from being lost as activity accumulates on the Pull Request. We carefully monitor various events to determine whether to update the comment in place or move it down, striving to achieve an ideal balance between providing the most up to date information and minimizing unnecessary noise.

2. Judicious use of inline comments: Inline comments are powerful for contextual feedback, but they can be abused. We adhere to a threshold in order to prevent too many inline comments, ensuring they are reserved only for the most critical or actionable lines of code. And inline comments can be turned off entirely in the Settings if you choose. Even without any inline comments, you still get the most high value signal in the Dashboard comment. This means that you can configure Gitar to only ever add a single comment per PR. No other tool on the market has this capability.

3. Automatically resolve comments as issues are addressed: Gitar cleans up its comments as issues are addressed.

4. Don’t post empty content: If there is no meaningful or actionable content to report, we post nothing. In fact, once everything is done, Gitar will remove its own Dashboard comment if you configure it to do so.

The Result

We have to do better as an industry. By minimizing noise and maximizing signal, we’ve created a tool that developers love and makes the code review process faster and more effective for the next generation of software development with AI.

Ready to get more signal and cut noise on your PRs? Try Gitar for free, or read the Docs.