Conventional software degrades visibly. A slow query is slow. A failed request returns an error. The user sees something is wrong, and so does the operator.

An AI feature degrades invisibly. The provider is slower than usual, so a timeout truncates the retrieval step, so the model answers from less evidence than it should have — fluently, in the same tone, at the same length, with the same absence of hedging. Nothing in the response says I answered this one with half the context. The system did not fail. It quietly produced a worse answer, and the only place that shows up is in a metric nobody is looking at.

This is why we write the degradation path before the feature.

The question that comes first

Before designing what the feature does when it works, we answer a narrower question: what does this return when it cannot be confident?

There are only a few honest answers, and choosing among them is a product decision, not an engineering one:

What is not on the list is answer anyway. That is the default behaviour of every AI system that has not been told otherwise, and it is the behaviour that costs the most trust per incident.

Why it has to come first

Written afterwards, a degradation path is a patch. Written first, it constrains the design — and the constraint is productive.

A feature that must be able to say “I don’t know” needs a definition of insufficient evidence, which forces the retrieval contract to be explicit. A feature that must be able to fall back to a deterministic path needs that path to exist, which caps how much of the product can depend on the model being right. A feature that must show the age of a cached answer needs to track provenance through the pipeline, which is the same plumbing that makes incidents debuggable.

Each of those is work we would have wanted anyway. Ordered this way, we do it before it is urgent.

What we build into the pipeline

Every stage has a defined behaviour when its input is missing or stale. Not an exception, a behaviour. Retrieval returning zero documents is an ordinary state with a defined answer, not an edge case discovered in production.

Tool failures are visible to the model as failures. An error payload passed into the context as though it were data is the single most reliable way to produce a confident wrong answer. The model must be able to tell the difference, and the system must be able to stop it from continuing when the difference matters.

Confidence has to come from the system, not from the prose. Asking a model how sure it is produces a number in the right range and the wrong distribution. We derive it from things we control: how much evidence was retrieved, how old it was, whether the tools that should have answered did, whether repeated runs agree.

Degraded responses are labelled in the payload, not only in the logs. If the caller cannot tell a full answer from a degraded one, every downstream consumer will treat both as full, and the degradation becomes invisible again one layer up.

The operational half

A degradation path that never runs is not a degradation path; it is an untested branch that will meet production for the first time during an incident. So it gets exercised on purpose: fault injection in the eval suite, and a switch that forces the degraded path in staging so that what users see in the worst case is something the team has looked at.

The number we watch afterwards is not uptime. It is the share of responses served from a degraded path, and its trend. Uptime says the system answered. That share says what it was answering with.

What this is really about

Reliability in an AI system is not the absence of bad days. The provider will be slow, the index will be stale, the tool will return something malformed. Reliability is that the system tells the truth about which day it is having.

Design that path first and the feature is shaped by it. Design it last and you will ship a system whose worst behaviour is its most confident.