A Hugging Face blog post presents a reproducible method for determining whether a language model was trained from scratch or derived from an existing base model. The approach compares architecture configuration fields such as hidden size and layer count, tokenizer vocabulary overlap, and embedding weights using a rotation-invariant similarity metric called Linear CKA. The authors find that raw cosine similarity between embeddings is unreliable due to rotational invariance, and that configuration and tokenizer analysis provide the strongest signal for establishing model lineage.
