DiScoFormer: One transformer for density and score, across distributions

The Hugging Face Blog has detailed DiSCoFormer, a transformer architecture that jointly models density and score functions across multiple distributions. Unlike traditional approaches requiring separate models for density estimation and score-based generation, DiSCoFormer uses a single auto-regressive transformer conditioned on distribution embeddings. This allows it to predict both the log-density and its gradient for any given distribution at inference time. The model is evaluated on synthetic datasets and demonstrates strong generalization to unseen distributions. For developers building AI workflows, this research points toward more efficient generative modeling pipelines, as it could reduce the need for training separate networks for tasks like anomaly detection, sampling, and likelihood evaluation. However, the model is still experimental and not yet packaged for production use.