# Can production incidents prevent the next bug? Designing an incident-aware review loop

> How Incident Knowledge turns production detections into patterns, and how Code Review checks each pull request against them without adding noise.

Source: https://entelligence.ai/blogs/incident-aware-review-loop
Updated: 2026-11-11

- Author: Aditya Jha
- Published: 2026-11-11
- Tags: Code Quality & Technical Debt, Code Review Best Practices

Ben Treynor Sloss, who founded SRE at Google, is quoted in the SRE Workbook: “To our users, a postmortem without subsequent action is indistinguishable from no postmortem.” The chapter’s own example of a flawed postmortem was published four months after its incident, which had already recurred by then. On action items the chapter is blunt: without a formal tracking process, they **“are often forgotten, resulting in outages.”**

Many teams now track action items until the ticket closes. The gap shows up months later, when a different engineer writes the same mistake, often in a file the incident never touched, and the reviewer wasn’t around for the incident. The assumption that failed is back in a diff, and nothing in the review knows about it.

**The short version**

- Translate: an incident has to become a pattern, with a failure shape and an owning repo. Some, like an expired certificate, belong in a runbook instead.
- Cite: every incident-based finding links the incident and the line it is about, so a reviewer can check it in a minute.
- Match: review checks every pattern the repo owns against the diff, whatever file the incident came from, and comments only when a new line brings the failure back with nothing protecting it.
- Forget: fixes get verified and patterns go dormant, but review doesn’t read either state. A rule nobody retires keeps commenting on PRs long after the code it was about is gone.

### What is an incident-aware review loop?

**An incident-aware review loop** is code review that checks each pull request against the failure patterns your production history has already shown you, and cites the incident when it flags something. Search over postmortems finds a similar incident. The loop also decides whether this change recreates its conditions, and drops the incident when it doesn’t.

It has four parts: turning incidents into patterns, finding the patterns that might apply to a change, deciding which ones apply, and retiring patterns when the code or the fix moves on. In Entelligence, Incident Knowledge builds the patterns, the world model links them to repos, pull requests and deploys, and Code Review reads that graph on every pull request.

_Image: Illustration: detection spikes from Sentry, Datadog and Grafana collapse into one pattern, which attaches to its owning repo in the world model and surfaces as an incident comment on the changed line of createCharge.ts. Review checks every pattern the repo owns, whatever file the incident came from. (Incidents become patterns, each pattern belongs to a repo, and review checks all of them on every pull request in that repo.)_

### An alert is not a rule

An alert says something was slow. A rule says what the code must do. Root cause analysis connects the two, and **it is mostly human judgment**.

Take a checkout 5xx spike. The alert is p99 latency and error rate on checkout. The root cause is an outbound call to the payment provider with no timeout: the upstream hung, and requests waited until the gateway killed them. Only the last step, the rule, can be checked against a diff:

_Image: Illustration: a p99 latency spike over the SLO, then a trace where charge() hangs until the gateway kills it at 30 seconds, then the rule as a diff: fetch without a timeout fails, fetch with a timeout signal passes. Expired certs, capacity misses and config pushes branch off to runbooks and alerts. (Three levels of one incident, plus the incidents that never become a code rule. Review can only check the rule.)_

Incident Knowledge handles the grouping. It turns repeated production detections into patterns, which the CLI calls Error Knowledge Units, or EKUs. For each mapped repository, ingestion looks at merged pull requests, incidents within a lookback window, and how code changes line up with incident timelines. Patterns are built from **Sentry, Datadog, New Relic, BetterStack, Grafana, ELK, CloudWatch, Sumo Logic, GCP, Azure and Axiom.** In the example pull request later on, the timeout rule is a Guideline the platform team set for the `payments/` directory, and INC-2231 is the incident behind it.

_Image: Screenshot: the Incident Knowledge overview for our backend repo, with 9,234 live detections in the past 24 hours, 561 patterns learned, 417 of them active, and a 14-day production detection chart. (Incident Knowledge on our own backend repo: about 9,000 production detections in a day, grouped into 561 patterns. Entelligence’s own workspace.)_

Patterns also depend on mapping. A Sentry project maps to a repository, and a Datadog service or monitor maps to the repo that owns it. Matching is scoped to that repository, not to a file or directory: review loads every pattern the repo owns, so a pattern with no owning repository never reaches a review. PagerDuty is separate. Its incidents come in through their own sync and don’t become patterns, so **PagerDuty incidents don’t reach review matching**.

_Image: Screenshot: the Uncaught Network Error In Token Validation pattern in Incident Knowledge, with 21,400 detections, its detection location in auth_decorators.py, the impact list, Ellie Insights and a 14-day detection trend. (One pattern from that list: 21,400 detections, attributed to one function, with what it breaks and why. Entelligence’s own workspace; hostname redacted.)_

### Every warning needs a paper trail

A model that has read a hundred postmortems can make almost any diff sound dangerous. So each finding has to name what it is based on, and a reviewer has to be able to check that before acting on it.

The sample on our [Code Review](/code-review) page shows the evidence a finding should carry: the caller count, the rule and the incident:

_Image: Illustration: the createCharge.ts diff next to one critical finding that cites 14 callers, the timeout rule and INC-2231. (Each line can be checked on its own. Sample data from the Code Review page.)_

If the reviewer opens INC-2231 and it was about something else, the finding is wrong, and they can see that in a minute. A warning that says “this looks like a past outage” with no link is unverifiable, and easy to start ignoring.

Those sources need limits too. Production logs carry customer data and internal details, so incident evidence should come only from integrations your team approved. Our [context docs](https://docs.entelligence.ai/code-review/context) list it as one of the checks to run when you verify review context.

_Image: Screenshot: the detected error for one pattern and its Flagged in PRs list, 13 pull requests with the file each one touched and the PR number. (The trail runs the other way too: each pattern lists the pull requests it was flagged on, here 13. Entelligence’s own workspace; hostname and authors redacted.)_

### Deciding which past incident applies

Because review checks every pattern in the repo, plenty of past incidents will look like a given change. The check that matters is whether the change brings one back. That works in both directions: a new file the incident never touched can still match, and a change to the very file it came from can stay quiet. Three kinds of false match need designing for, and each one fails a different question:

_Image: Flowchart: matching a pull request against the INC-2231 pattern, scoped to the whole repo. Three yes/no questions in turn: does the diff show the same failure shape, does a new line introduce it, is there no existing protection. A no at any step stays quiet: a change to the same file with a different bug, a change that only renames a log field, a call where the shared client already sets a timeout. Three yeses post a comment citing INC-2231. (Each “no” is one of the false matches. Only a change that passes all three questions gets a comment, in whichever changed file the risk shows up, and only if no other review comment already sits within a few lines.)_

Our [Incident Knowledge docs](https://docs.entelligence.ai/code-reliability/incidents) make the same point about clusters: a cluster groups related patterns and is not automatically one root cause. Matching runs in two stages. First, a model picks the pattern groups whose underlying failure shape shows up in the diff, **regardless of technology, library or variable names**. Second, it has to trace proof: **a new line introduces the risk or calls into vulnerable code, and no existing protection covers it**. The only file check comes last: the comment has to land on a file the pull request changed.

### One pull request, followed through review

This trace uses the sample walkthrough from our Code Review page. A real finding from our own backend repo comes after it.

_Image: Illustration: PR 5103 through review. Three changed files and 14 callers feed five checks. Correctness, security and performance find nothing; the team rule and INC-2231 both flag line 26, the rule’s critical comment is posted, and the INC-2231 comment is dropped because a comment already sits within a few lines. Re-reviews per push move the score from 3/5 to 4/5 to 5/5. (The rule and the incident point at the same line. The rule’s comment is posted and the incident comment is dropped, so the PR gets one comment. Sample data.)_

In step 4, the rules check and the incident check flagged the same line. When a review comment already sits within a few lines, the incident comment is dropped instead of being posted next to it. The PR gets one comment about the missing timeout, not two.

Each repository also sets a bar for what gets posted. A separate illustration on the same page shows **100 findings considered and 14 posted** to the pull request, with the other 86 suppressed but still readable. Teams pick critical only, medium and above, or everything, and can cap inline comments.

_Image: Screenshot: a critical, incident-tagged review comment on drain.py in one of our pull requests, citing the Unbounded GitLab API Timeout pattern with a link to the Incident Dashboard. (An incident-tagged finding on one of our pull requests, citing a GitLab-timeout pattern months after it was last seen in production. The comment links back to the pattern. Entelligence’s own workspace.)_

_Image: Screenshot: the Risky PRs tab grouping open pull requests under the Execution Safeguards pattern, with 5,832 log hits and 20 open PRs. One PR is expanded to show its finding and a recommended action. (Risky PRs groups open pull requests by the production pattern they touch, with the finding and the recommended action on each. Entelligence’s own workspace; authors redacted.)_

### Patterns have to be able to die

A rule about a function deleted in June still costs reviewer attention in November if nothing retires it. Here is what exists today, and what review does with it:

- **Pattern state.** Incident Knowledge marks each pattern active, growing, dormant or regressing. Review doesn’t use that state: a dormant pattern is matched exactly like an active one.
- **Fix verification.** Fix Now opens a fix pull request that goes through normal review and branch protection. After deploy, Monitored Fixes follows it until the pattern is verified resolved or regresses.
- **Feedback and muting.** Learnings shape other parts of review, but they aren’t used in incident matching. Muting a pattern silences its alerts and, by design, keeps its PR comments.
- **Owners.** Our context docs say it about context files: “stale instructions create stale reviews.” A Guideline built from an incident is the same, so give it an owner who revisits it when the architecture changes.

_Image: Illustration: a timeline of one pattern. Detections spike after the incident, drop to zero after the fix PR merges, and return later. Below it the pattern goes growing, active, dormant, regressing; the fix goes verification active, verified resolved, regression detected; and review keeps matching the pattern in every state, including while it is dormant. One finding marked unhelpful doesn’t filter later matches. (Incident Knowledge tracks the pattern and the fix. Review doesn’t read either state, so a diff that shows the failure shape gets matched in any phase.)_

Our docs define done strictly: **the change is deployed and the production pattern stays resolved**. Review doesn’t read that state, so a verified fix doesn’t stop its pattern from being matched.

Postmortems also get revised. When a root cause turns out to be something else, the pattern built on the first analysis should be marked superseded instead of firing next to the new one. That part is still open.

_Image: Screenshot: the Monitored Fixes tab for our backend repo showing 41 fixes deployed, 37 verified resolved, 0 monitoring active and 4 regressions detected. (Monitored Fixes for our backend repo: 37 of 41 deployed fixes verified resolved, 4 flagged as regressions. Entelligence’s own workspace.)_

### How to test whether incident context helps

The question is narrow: on the same pull requests, does an incident-aware reviewer find real issues the baseline misses, without adding false positives? A test any team can run on one incident:

1. Pick one incident with a written root cause and a corrective action you can check on a diff.
1. Write two replay PRs that reintroduce the original mistake: one in the file the incident came from, one in a different file in the same repo. Add the three decoys from the figure above.
1. Run the baseline reviewer, with no incident sources, and the incident-aware one on all of them, plus a sample of ordinary PRs.
1. Record each measure in the table below, and **publish the misses**.

| Measure | What it tells you |
| --- | --- |
| Replays caught | The lesson is available at review time, including in a file the incident never touched. |
| Right incident cited | Whether the citation holds up. A catch that cites the wrong incident was luck. |
| Decoys flagged | Each decoy that fires is a false positive engineers will learn to skip. |
| New comments, ordinary PRs | The noise cost of turning incident context on. |
| Marked unhelpful | Where engineers disagree with the incident check. These marks don’t filter incident matching, so someone has to read them. |

A replay test can’t show that the next incident was prevented, because that depends on whether people act on the finding. That evidence builds up over months, in Monitored Fixes and in how often engineers act on review comments. What a replay can show is smaller: when the next similar change comes in, the lesson from the last incident is in the review, with its evidence, and you have tested that it fires.

### How this differs from a postmortem folder or a lint rule

Most teams already keep one of these. The biggest difference is when the lesson gets read:

|  | Incident-aware review | Postmortem folder | Lint rule |
| --- | --- | --- | --- |
| Read | Every PR in the repo | When someone searches | Every build |
| Matches | Failure shape | What the reader finds | Exact syntax |
| Evidence | Incident, rule, line | The document | A rule name |
| Noise | Proof step and a cap | Rarely raised | Suppressions |
| Retirement | Not automatic | Never | When deleted |

A lint rule is the right tool when the lesson fits a pattern match, like “never call this deprecated function,” and you should use one there. Incident-aware review is for lessons that need context: the same failure written with a different library or different names, whether a new line actually introduces it, and whether existing protection already covers it.

### Questions people ask

#### Which incident sources does Incident Knowledge read?

Patterns are built from Sentry, Datadog, New Relic, BetterStack, Grafana, ELK, CloudWatch, Sumo Logic, GCP, Azure and Axiom. PagerDuty incidents come in through a separate sync and don’t become patterns, so they don’t reach review matching.

#### Will every PR in the repo get an incident comment?

No. Review checks every pattern the repo owns, but a pattern only comments when the diff shows its failure shape, a new line introduces the risk, and nothing already protects against it. If another review comment already sits within a few lines, the incident comment is dropped. Each repository also sets its own severity bar and comment cap.

#### What happens when an incident-based finding is wrong?

Mark it unhelpful and say why, so the team can see where the incident check misfires. Those marks don’t filter incident matching, and muting a pattern silences its alerts but keeps its PR comments.

#### Does it fix incidents on its own?

Fix Now opens a pull request. That PR goes through normal review, required checks and branch protection. Entelligence never merges on its own.

#### What about sensitive data in incident logs?

Logs can hold credentials, personal data and customer content. Redact at the source, give each connector the narrowest credential scope, and confirm retention, region and support access before you connect. Code is used only for your organization and never to train models.

### Your first week with an incident-aware reviewer

**Start with one service.** By the end of the week you should have one measured result:

1. Pick a service with a recent, written-up incident. Connect its source in Code Reliability, Incident Knowledge, with the narrowest credential.
1. Map only that project or service to its repository. Leave everything else unmapped.
1. When ingestion shows `completed`, check one pattern against the source: timestamps, evidence, detection count.
1. Make sure the corrective action exists as a Guideline on that path, with an owner.
1. Open the two replay PRs and the decoys. Both replays should get a finding that links the incident, including the one in a different file. The decoys should stay quiet.
1. Leave it on for a week of real pull requests, then read every finding marked unhelpful.

**Catch the next incident while it is still a pull request** Every pull request, checked against your codebase, your standards, and the incidents you already had. [Set up Code Review](/code-review)

Want to try it on one of your own incidents first? [Book a demo](https://calendly.com/aiswarya-qlnt/1-1-aiswarya-1) and we will run the replay test with you.

Sources: [Google SRE Workbook, Postmortem Culture](https://sre.google/workbook/postmortem-culture/); [Google SRE Book, Postmortem Culture: Learning from Failure](https://sre.google/sre-book/postmortem-culture/); [Entelligence Code Review](/code-review); [Incident Knowledge patterns and reports](https://docs.entelligence.ai/code-reliability/incidents); [Connect incident sources](https://docs.entelligence.ai/incident-knowledge); [Investigate, fix, and monitor](https://docs.entelligence.ai/code-reliability/fixes); [Guidelines, Learnings, and codebase context](https://docs.entelligence.ai/code-review/context).

More: [Blog](https://entelligence.ai/blogs)