The idea behind automated postmortem drafts is sound: incidents generate a rich data trail, and most of the tedious work in a postmortem is reconstructing the timeline from that trail. Correlating PagerDuty alert timestamps with Slack threads with deployment logs with metric changes is genuinely mechanical work. If a tool can do it in 30 seconds, that's 30 to 60 minutes of on-call time recovered after every significant incident.
We built postmortem auto-drafting into Devtract because we believed this, and we still believe it. But after watching teams use it, we've gotten clearer about what the automation actually delivers and what it structurally cannot deliver, regardless of how well the system works.
What automated drafts actually get right
Timeline accuracy. This is the strongest value proposition and the one that holds up without qualification. An automated system with access to alert timestamps, deployment records, metric anomalies, and communication logs will produce a more accurate chronology than any human reconstructing the same incident from memory four hours later. Human memory of incident timelines is systematically compressed: the subjective experience of 45 minutes of frantic diagnosis often feels like 10 minutes, and the sequence of actions gets reordered to match the narrative that made sense, not the sequence that actually happened.
Devtract's postmortem drafts pull the objective event chain: alert fired at 03:17:42 UTC, first acknowledgment at 03:19:08 UTC, deploy of payment-service v2.1.1 at 02:58:30 UTC (18 minutes before alert), first log anomalies at 03:10:15 UTC, Slack war room opened at 03:20:45 UTC. These timestamps can't be reconstructed accurately from memory. The automated draft gets them right.
Service impact scope is another area where automation adds genuine value. During an active incident, the team focusing on the immediate problem often doesn't have a complete picture of what else was affected. A downstream service with degraded performance that didn't fire its own alert may not make it into the manual postmortem. A dependency that experienced elevated latency but stayed within SLO thresholds will usually be omitted. The automated draft can include all of this because it's not limited by what the on-call team was paying attention to.
Evidence preservation is the third thing automation gets right. By the time the postmortem is written, some signals have decayed or been overwritten. Log retention windows close. Metric queries against hour-old time ranges get slower. The automated draft captures the evidence at resolution time, when it's freshest and most complete.
What automated drafts structurally cannot do
Here's where the honest assessment gets harder.
Automated drafts cannot distinguish between the incident's contributing causes and its root cause. They can correlate events in time. They can rank probable causal relationships by statistical association. But the judgment of "this is why it happened, at the level that matters for preventing it from happening again" is not a temporal correlation problem. It's a reasoning problem that requires understanding the system, the team's operational habits, and the historical context of similar failures.
Consider an incident where a payment service sees elevated error rates triggered by a misconfigured database connection pool. The automated draft correctly identifies: a configuration change deployed 20 minutes before the incident, followed by connection pool exhaustion metrics, followed by error rate increase. The root cause section might read "Configuration change in payment-service introduced connection pool misconfiguration."
But the actual root cause, at the level useful for preventing recurrence, might be: "Our deploy pipeline has no automated validation of database configuration parameters, and the manual review checklist doesn't include connection pool settings because it was added as an afterthought." That's a different statement with different action items. The automated system can give you the first statement. The second statement requires a human who understands why the configuration change wasn't caught and what in the process made it possible.
The nuance problem
Postmortems serve two purposes that are in tension with each other. The first purpose is documentation: a record of what happened. The second purpose is learning: a structured process for extracting actionable insight that prevents recurrence. The first purpose is well-served by accurate timelines and correlated evidence. The second purpose requires nuanced judgment that automation flattens.
When we look at automated postmortem drafts that teams have edited and finalized, the pattern is consistent: the timeline and impact sections are kept almost verbatim, maybe with minor corrections. The contributing factors and root cause sections are substantially rewritten by the human. The action items section is almost entirely human-authored, with the automated draft providing at most a starting point for the discussion.
This is the right use pattern. The mistake we've seen is treating automated drafts as final drafts, especially for lower-severity incidents where the pressure to close the postmortem quickly is high. A Sev-3 that affected a small subset of users, resolved in 40 minutes, might get an automated postmortem that's technically accurate and completely misses the interesting thing: that the same service has had three similar incidents in the past two months, each triggered by a slightly different configuration path, and the underlying code pattern that makes these failures possible has never been fixed.
An experienced SRE writing the postmortem would notice the pattern. The automated system, working on this incident in isolation, does not.
The template problem
Most automated postmortem systems, including our own first attempt, generate from a template: timeline, impact, root cause, contributing factors, action items. This structure is well-established in SRE practice and there are good reasons for it. But the template creates a gravitational pull toward filling in fields rather than answering questions.
A manual postmortem can depart from the template when the incident demands it. An incident that was entirely caused by a third-party provider outage might not have meaningful "action items" except "add a fallback." An incident caused by a known flaky integration that the team has chosen to accept might generate a postmortem that honestly says: "We know about this failure mode, we've accepted it, here's why, and here are the conditions under which we'd revisit that decision." Template-filling automation struggles with these departures.
We've addressed this in Devtract's postmortem feature by treating the automated draft as a structured data export rather than a narrative. The system generates: the correlated event chain, the services involved, the confidence-ranked causal hypotheses, and the supporting evidence for each. The human writes the postmortem using this as a reference. We don't generate the template sections automatically; we give the engineer the data that makes filling them accurately much faster.
When to trust the automation more
There are incident types where automated postmortems can be trusted to a higher degree without extensive human editing.
Infrastructure incidents with clear causal chains benefit most: node failure, disk exhaustion, network partition, database failover. These have well-understood failure modes, clear signal chains, and relatively standard action items (improve monitoring, add capacity, adjust failover policy). The automated draft for these is usually good enough to stand with light review.
Repeat incidents, where the pattern has been documented before, can be largely handled by automation: "This is incident type X, which we've seen 3 times. The previous postmortems are linked. The open action item from the last occurrence is [link]. This incident occurred because that action item was not completed." That's a useful postmortem that saves significant time and accurately represents the situation.
The incidents where automation should be trusted least are novel incidents, high-severity incidents with multi-team blast radius, and incidents where the causal chain has organizational or process dimensions rather than purely technical ones. These need real human postmortems. The automation can help with the data, but the analysis belongs with the people who understand why the team made the decisions it did.
The right frame
Automated postmortems should be measured by whether they make the postmortem process faster and more complete without making the learning worse. If teams are shipping accurate timelines faster, capturing more service impact data, and doing postmortems consistently (because the barrier to starting is lower), automation is working. If teams are shipping automated postmortems as final documents and skipping the analysis work because the template looks complete, automation is replacing the valuable part of the process with something that looks similar but isn't.
The test we apply internally: after reading the postmortem, does a person who wasn't on-call during the incident understand what happened and why in a way that would help them prevent it? If yes, the postmortem is doing its job. If they understand the timeline but not the reasoning, the automation did its part and the human work wasn't done.