Hallucination

Hallucination is a phenomenon where a large language model generates text that is fluent and confident but factually incorrect, fabricated, or unsupported by any source material. The term draws an analogy to human perception errors, though in LLMs it arises from statistical pattern completion rather than sensory malfunction.

Key characteristics of hallucination include:

1. Confident Fabrication: Hallucinated outputs often read as authoritative and well-structured, making them difficult for non-experts to detect. The model presents invented facts, citations, or statistics with the same fluency as accurate information.

2. Types of Hallucination: Intrinsic hallucinations contradict the provided source material, while extrinsic hallucinations introduce claims that cannot be verified against any given context. Both are problematic in production systems.

3. Root Causes: Hallucinations stem from the model's training objective of predicting likely next tokens rather than verifying truth. Gaps in training data, distributional biases, and ambiguous prompts all increase hallucination rates.

4. Mitigation Strategies: Common approaches include retrieval-augmented generation (RAG) to ground responses in real documents, chain-of-thought prompting, confidence calibration, and citation requirements that force the model to reference sources.

5. Evaluation Challenge: Detecting hallucinations at scale remains an open research problem. Automated detection tools exist but are imperfect, often requiring human review for high-stakes applications.