Complete article on Researchgate
Transformer language models generate text through autoregressive next-token prediction, whereas hallucination is a semantic and evidence-relative failure that generally becomes meaningful only at the level of propositions or claims. This mismatch makes it difficult to treat hallucination as an ordinary token-classification problem.
Existing studies show that uncertainty measures, hidden activations, attention patterns and intervention directions can contain useful signals about factuality. Nevertheless, the literature does not establish the existence of a single universal internal variable corresponding to hallucination.
This paper proposes a falsifiable theoretical abstraction in which hallucination is modelled as a latent, token-conditioned hazard process. The proposed Latent Hallucination Hazard Model, or LHHM, defines a local hazard λi over generation steps, conditioned on:
- predictive uncertainty;
- generation commitment;
- hidden-state trajectory;
- evidence support;
- contextual conflict;
- optional factuality probes.
Local hazards are aggregated into claim-level risk through a survival-style formulation.
The model explains why token entropy or attention mass alone cannot be sufficient. It accommodates both uncertain confabulation and confident falsehood and supports an architectural extension called the Hallucination State Head.
A training objective, an inference-time control policy, testable hypotheses, falsification criteria and an empirical validation protocol are specified.
The paper does not claim that current Transformers possess a unique “hallucination neuron”. Rather, it offers a disciplined mathematical framework for testing whether hallucination risk can be represented, calibrated and controlled during autoregressive generation.
Keywords: large language models, Transformer, hallucination, factuality, uncertainty, hidden states, attention, token-level risk, selective generation, retrieval-augmented generation.