The LLM should explain the incident, not declare it.

Why incident declaration is a decision

Calling something an incident changes the work. It may notify a team, interrupt planned activity, trigger a customer response, or start a recovery procedure. The operator should be able to inspect the measured signal, condition, time window, sample size, severity, and owner.

A language model can write a persuasive explanation without providing a stable answer to those questions. Its fluency becomes useful after the evidence exists. It is not a substitute for the incident condition.

What the system separates

FlowSentry is an open source Python project for reviewing n8n execution records. Its detector layer uses visible rules to find defined patterns. Its explanation layer receives those findings and asks a language model to describe the likely cause and one practical next step.

The current implementation checks three patterns. A failure spike fires when at least 30 percent of a workflow group fails across at least five executions, with critical severity at 50 percent. A slow workflow fires when recent successful runs exceed 2.5 standard deviations above the workflow mean. A repeated error fires when the same message appears at least three times.

These are project defaults. They should not be copied into production without a baseline, a noise review, and a named owner.

A clear authority path

Execution records provide the signal. Deterministic rules decide whether a documented condition is met. The language model explains the detector evidence. A person chooses the response or handoff. The system records the incident, decision, and any threshold change.

This split keeps the consequential decision inspectable while giving the model a bounded evidence package. It also makes later review possible because the original signal, detector result, model explanation, and human response remain distinct.

Worked example from the seeded fixture

The bundled FlowSentry sample contains 120 executions and three seeded incidents. One sample workflow records 17 failures across 30 runs. The failure rule calculates a 57 percent failure rate. Because that exceeds the documented 50 percent critical threshold, the detector creates a critical finding.

The language model does not decide that the workflow is critical. It receives the structured finding and available error evidence, then turns that evidence into a readable explanation and one suggested check.

This proves that the implemented path runs from sample executions through detection and explanation. It does not prove production reliability, customer impact, alert precision, or reduced recovery time.

What the language model should and should not own

The model can summarize a finding, group related evidence, suggest an investigation step, and translate technical detail for a service owner.

It should not own the incident threshold, severity rule, notification audience, recovery approval, or final statement of customer impact. People should define the policy, approve thresholds, review false alarms and missed incidents, own the response, and decide when the rule changes.

Opportunities, risks, and limitations

The split can reduce argument about why an alert fired. An operator can reproduce the calculation and tune the rule. The model can also make incident communication faster because it receives bounded evidence instead of an open request to search for anything unusual.

A visible rule can still be wrong. It may be copied from another environment, based on too little data, or left unchanged after the workflow evolves. A successful execution can also produce the wrong customer outcome, which requires a separate outcome check.

The explanation remains advisory. Minimize the evidence sent to a model, redact sensitive values, control access, and document retention. FlowSentry still lists baseline persistence and recovery time tracking on its roadmap, so production adoption needs additional operating controls and measured alert quality.

A practical 30, 60, and 90 day framework

Days 1 to 30: choose one workflow. Define the business outcome, one material failure condition, sample size, time window, severity, owner, and stop condition. Collect a baseline before notifications begin.

Days 31 to 60: run the detector in observation mode. Review every trigger and a sample of non triggers. Record false alarms, missed incidents, customer impact, and operator effort. Version every rule change.

Days 61 to 90: add a bounded model explanation using only the evidence required for the finding. Connect the incident to a named response path and measure time to acknowledge, time to recover, false alarm rate, and repeated incident rate.