DiScoFormer is a transformer-based architecture that estimates both the density and score of a data distribution in a single forward pass, without retraining for new datasets. It uses cross-attention with shared backbone components and two output heads, exploiting the mathematical relationship between score and density as a consistency loss. The model reports roughly 6.5x better score error and 37x better density error than kernel density estimation in 100-dimensional spaces, holding up across non-Gaussian distributions and unseen mixture configurations.
