Diagnosing and Fixing Prompt Failures: Hallucinations, Refusals, and Format Drift

The Four Prompt Failure Modes

1. Hallucination

The model generates confident but false information.

Symptoms: Incorrect facts, made-up citations, wrong numbers.

Fixes:

• Add: “Only use information from the provided context. If the answer isn’t there, say ‘I don’t have enough information.’”
• Use RAG to ground answers in real documents
• Ask the model to cite its sources

2. Format Drift

The model ignores your format instructions for complex inputs.

Symptoms: Prose instead of JSON, wrong number of bullets, missing sections.

Fixes:

• Repeat format instructions at the end of the prompt
• Add a few-shot example showing the exact format
• Use schema validation and retry on failure

3. Scope Creep

The model adds unrequested content, caveats, or disclaimers.

Symptoms: Unsolicited warnings, extra sections, hedge phrases.

Fixes:

• “Do not add caveats, disclaimers, or unsolicited advice”
• “Respond with exactly the requested output and nothing else”

4. Ambiguous Interpretation

The model interprets your task differently than you intended.

Symptoms: Off-topic responses, wrong level of detail, wrong audience.

Fixes:

• Add a role: “You are a [expert] writing for [audience]”
• Add a negative constraint: “Do not include [X]”
• Add an example of what you DON’T want

Debugging Workflow

1. Identify which failure mode occurred
2. Apply the targeted fix
3. Test with 5+ diverse inputs
4. If still failing, add a few-shot example