5 Ways LLM Features Fail Silently (And How to Catch Them)

LLM features fail silently because a wrong answer returns HTTP 200. The five failures below account for most of what we find in AI quality audits, none of them show up in error monitoring, and all of them are testable.

1. Prompt regression

Someone rewords a system prompt to fix one complaint and quietly degrades ten other behaviors. Catch it with a prompt regression suite: a fixed battery of inputs with expected-property assertions that runs on every prompt edit, exactly like unit tests run on every commit.

2. Retrieval drift

New documents get indexed, embeddings shift, and last month's perfect answer now cites the wrong policy. Catch it by re-running retrieval accuracy against your golden dataset on every reindex.

3. Confident hallucination

The model fills gaps with plausible inventions, dates, prices, feature claims. Catch it with grounding checks that verify every claim against source context, and a hard rule that missing context must produce 'I don't know', not improvisation.

4. Format contract breaks

Downstream code expects JSON with six fields; the model returns five, or wraps it in prose, and a parser silently defaults. Catch it with schema validation on every response in test AND production, with alerting on parse-repair rates.

5. Tone and safety drift

A model upgrade subtly changes voice, verbosity, or refusal behavior. Catch it with a style battery, the same 30 prompts scored across model versions, before any upgrade reaches users.

Choosing the eval tooling (without a committee)

Teams stall for quarters picking evaluation tooling; the decision is smaller than it looks. If your Engineers live in pytest, DeepEval gives you LLM metrics as ordinary assertions. If you want config-over-code and PR-visible diffs, promptfoo. If your failures are retrieval-shaped, Ragas. They compose, promptfoo gating prompt changes while DeepEval guards output properties is a common, sane stack, and all three are open source, so the cost of choosing 'wrong' is an afternoon of porting YAML, not a procurement cycle. The only wrong choice is another quarter of choosing.

The model-upgrade drill

Provider model updates are the highest-volume source of silent drift, and you don't control their timing. The drill that makes them boring: pin model versions in production; when a new version ships, run the full battery, regression cases, format contracts, tone battery, cost and latency distributions, against both versions side by side; diff the failures; and promote only after the diff is reviewed by a human who owns the feature. Half a day of ceremony, and 'the model changed under us' stops being a root cause in your incident reports.

A sixth failure mode: silent cost and latency drift

Not wrong answers, wrong economics. A prompt edit doubles average output length; a retrieval change stuffs 30% more tokens into every context; a provider model update shifts the latency distribution. Nothing errors, users wait, margins shrink. Track tokens-per-request and p95 latency per feature version alongside the quality battery, they're one dashboard line each, and they've paid for the whole monitoring effort at more than one client.

What we actually find in audits

The pattern across AI quality audits is remarkably consistent. Almost every team has decent prompts and no regression suite, quality lives in one Engineer's memory of what they tried. About half have a format contract; almost none validate it in production, where the parse-repair rate turns out to be 3–8%. Retrieval is universally under-measured: teams know their vector store's benchmark numbers and not their own recall@5. And in nearly every audit, the highest-severity finding is the same: no defined behavior for 'the context doesn't contain the answer,' which means the model improvises, confidently, politely, and wrong.

Building the baseline battery in one week

Teams overestimate this. A useful first battery is a week of part-time work:

• Day 1: collect 50 real inputs from logs, actual user phrasing, not what the spec imagined.
• Day 2: write expected properties per input, not exact outputs; properties ('mentions the refund window', 'refuses out-of-scope asks', 'valid JSON with six fields').
• Day 3: wire promptfoo or a pytest harness to run the battery against the live prompt in CI.
• Day 4: add the format contract: schema validation on every response, test and production both.
• Day 5: baseline the numbers, put them on a dashboard, and gate the next prompt change on them. Congratulations, 'it seems fine' is now a number with a trend line.

A regression check you can copy

import json, pytest
from app.llm import answer

CASES = json.load(open("golden/support_cases.json"))

@pytest.mark.parametrize("case", CASES, ids=lambda c: c["id"])
def test_prompt_regression(case):
 out = answer(case["input"])

 data = json.loads(out) # format contract
 assert set(data) == set(case["required_fields"])

 for phrase in case["must_mention"]: # content properties
 assert phrase.lower() in data["reply"].lower()
 for phrase in case["must_not_claim"]: # hallucination tripwires
 assert phrase.lower() not in data["reply"].lower()

The production signals worth alerting on

• Parse-repair rate: how often downstream code fixes malformed output, rising rate means the format contract is quietly failing.
• Refusal rate: sudden drops mean the model stopped saying 'I don't know'; sudden spikes mean it stopped answering things it should.
• Answer length and latency distributions: drift here is the earliest visible symptom of provider-side model changes.
• Retrieval overlap: fraction of answers whose cited chunks match the golden dataset's expectations, your live hallucination early-warning.

The pattern

Every silent failure becomes loud once you have a baseline and run it on every change. That's what an AI Quality Engineering practice actually is: turning 'it seems fine' into a measured, versioned number.