Inside Cobrainer's Engineering Workflow with CodeRabbit

Context-aware reviews, shared engineering conventions, and a collaboration that improved the integration.

TL;DR

• Cobrainer's engineering conventions are made explicit through shared CodeRabbit instructions and repository-specific lint configuration.
• CodeRabbit applied that context at an appropriate stage to recommend a shared query abstraction introduced in the same merge request, making its feedback directly actionable by the developer.
• Our teams found and traced unwanted lint findings to a configuration-precedence defect, which CodeRabbit fixed in under two weeks. In this way, authors assess the recommendations and human reviewers retain the merge decision.

The reviewer goes first. The merge decision stays with a colleague.

A code review asks more than whether a piece of code is correct. It asks whether the change belongs in this codebase, whether it follows the project's decisions, and whether the proposed remedy uses what the team has already built. Much of what makes an experienced reviewer useful lies in answering those questions together.

At Cobrainer, we build a Skills Platform that helps organisations understand their jobs, tasks, and skills. We introduced CodeRabbit into our engineering workflow to raise issues before a colleague began reviewing, leaving more of that colleague's attention available for behaviour, design, and maintainability. To make its recommendations useful, we had to give it access to the knowledge behind our own reviews.

Some of that knowledge was explicit: a linter configuration, for example, records which rules a project has chosen. Other expectations were less formal. Engineers knew which abstraction to reach for because they had helped build it, or because they had discussed a similar change before. Our work with CodeRabbit involved expressing those expectations and observing what happened when the reviewer applied them. Two findings made the problem concrete.

Making our conventions available to the reviewer

We put CodeRabbit in front of every merge request. The author considers its findings, makes changes where appropriate, and presents the result for a colleague's review. A human reviewer retains the approval decision. In this arrangement, the practical value of an early review depends on the relevance of the work it asks the author to do.

We kept our shared CodeRabbit configuration in a central repository that projects in the group inherited. This gave review-profile settings and shared instructions a common source. A change to those instructions could itself receive a review, and we could apply it without maintaining separate copies across repositories.

Writing the instructions required us to examine what we expected from code review. Which patterns were deliberate? Which apparent violations were acceptable in a particular context? When should a developer use a shared abstraction? One path instruction expressed a recurring expectation:

Do flag changed code that duplicates an existing shared capability instead of reusing it.

Applying this instruction requires knowledge of both the change and the surrounding code. The reviewer must find the shared capability, determine whether it applies, and explain the connection to the author. The instruction makes an engineering preference available for use in a specific review.

Native tool configuration provides another kind of guidance. A repository's pyproject.toml records its Ruff rule selection. These settings express decisions the team has already made, and the integrated tool needs to respect them.

We also compared review profiles using a probe containing nine deliberate defects. The comparison profile reported four; assertive reported seven and left five additional negative cases unflagged. We chose assertive. The small test gave us a reason for that choice, though it could not tell us how accurately either profile would perform across production changes.

Applying an abstraction within the change that introduced it

Three weeks into the rollout, a merge request introduced a typed expression API for storage queries. Two predicates in the same change still used string interpolation. One had the following shape, shown here in a simplified example:

async def urls_for_platform(self, platform: str) -> AsyncIterator[list[str]]: async for batch in self.stream( where=f"platform = '{platform}'", columns=["url"], ): yield batch 

The interpolation is easy to recognise. Deciding what to replace it with requires more context. CodeRabbit's comment connected the platform and source values to a registry populated through an ingest CLI and HTTP API. The query code therefore could not assume that they were quote-free constants.

The comment also identified store.expressions.Columns, introduced in the same merge request, and its literal-escaping path through lit(...).to_sql(). This gave the author a remedy that already belonged to the project. CodeRabbit recommended using it for both predicates:

_columns = Columns(CorpusRecord) async def urls_for_platform(self, platform: str) -> AsyncIterator[list[str]]: async for batch in self.stream( where=sql(_columns.platform == platform), columns=["url"], ): yield batch 

The finding as it appeared on the merge request. Identifiers and code are redacted.

The recommendation applied our reuse instruction to an incomplete adoption of a new abstraction. The expression layer existed, but these call sites had not yet adopted it. Bringing them into the same design reduced the need for each caller to handle literal construction independently.

Our engineers supplied the abstraction and the convention; CodeRabbit connected them at the point where the author could act. The author assessed the finding and made the change. The review comment records path instructions as its source and marks the finding as addressed in a subsequent commit.

Respecting repository policy and the scope of a refactor

A second finding showed how easily the meaning of a review comment can depend on facts outside the line it highlights.

In app/logging/__init__.py, a merge request extracted log-level selection into a new helper, _configured_level() -> int. The existing _setup_handlers() signature appeared further down as an added line in the diff. CodeRabbit's Ruff-backed finding requested a return annotation on that existing function, citing ANN202.

The engineer pointed out that the signature predated the refactor, while the new helper already carried a return annotation. The repository also selected a specific set of Ruff rules:

[tool.ruff.lint] select = ["E4", "E7", "E9", "F", "I"] ignore = [] 

Under Ruff's rule-selection semantics, this selection does not enable the ANN family. The engineer reported that the repository's Ruff check passed on the file.

CodeRabbit inspected pyproject.toml, quoted the selection, and withdrew its suggestion. Its reply also acknowledged that the function signature was unchanged.

Challenged on the finding, the reviewer reads the repository configuration and withdraws it.

There were two distinct questions here: had the change introduced the issue, and did the reported rule apply to this project? The diff alone did not settle the first; knowledge of the rule alone did not settle the second. Examining the refactor and the repository configuration changed the conclusion. That exchange gave us a concrete example of how an author could challenge a finding with evidence.

Making project policy hold across the integration

Resolving the comment left a further question. Why had a rule outside the repository's selection reached the review at all?

The findings identified their sources. The SQL recommendation came from path instructions, while the annotation finding came from the linter integration. This distinction helped us investigate the unwanted comments at the level where they originated: the configuration supplied to the integrated tool.

We collected inline findings from the rollout, grouped them by the analysis tool cited, and extracted their rule codes. Eleven findings named a specific rule:

The three expected findings corrected an assumption on our side. We had expected a project policy that the repository did not contain. The other eight provided concrete cases for CodeRabbit's engineering team.

They reproduced the issue and identified a precedence defect: a fallback configuration could take priority over the repository's checked-in configuration. They explained the cause, shipped a fix, and told us when it was available. The interval from our detailed report to the delivered fix was under two weeks.

The work depended on both teams. We could compare the comments with the policies and code they were meant to reflect. CodeRabbit's engineers could identify and correct the integration behaviour that produced the mismatch.

We then observed five days of real merge requests:

The unwanted findings did not recur during this period, while the broader review workflow continued to report findings.

Interpreting that absence required care. Our first attempt had used two merge requests without Python. Their quiet results could not tell us anything about the affected Python analysis. Even with relevant input, the table supports a limited conclusion: we observed no recurrence. Activity elsewhere in CodeRabbit does not establish successful Ruff execution, and patterns present in reviewed files are not necessarily eligible for inline reporting.

Bringing engineering conventions into everyday review

The two examples show different uses of project knowledge. In the query change, the shared abstraction and reuse instruction supported a specific recommendation. In the logging refactor, the rule selection and the history of the function explained why a recommendation did not apply. The collaboration with CodeRabbit's engineers addressed how that rule selection reached the integrated tool.

For our team, this made explicit conventions a practical part of adopting AI review. Instructions under version control can be revised. Repository policy can be inspected. A finding can be accepted or challenged against those sources. CodeRabbit contributes recommendations and context, while authors and human reviewers remain responsible for the changes that merge.

A useful starting point for another team is to examine the comments its reviewers repeatedly make. Some may express knowledge that could be written down and supplied to the reviewer. Others may expose decisions the team has never made explicit. Observing how those instructions affect real reviews provides a way to refine both the guidance and the workflow.

Our work with CodeRabbit gave us examples of that process and a direct collaboration when the integration needed correction. We thank its engineering team for their part in making our project's conventions more consistently available to the review.