Probe of Internal Recognition (PIR) is a reference-free method for detecting when a language model internally ‘knows’ an answer it does not report — e.g. sandbagging on an eval. It adapts the forensic Concealed Information Test, which identifies guilty knowledge by measuring a stronger response to a true detail hidden among plausible decoys: PIR presents a model with a question and candidate answers and reads from the model’s internal states which candidate it recognizes as correct, needing no honest reference model or labeled ground truth. The authors validate the method across eight models spanning five model families (Gemma, Qwen, Llama, Mistral, Phi).
