Nemotron Labs Diffusion Image
NL-Diffusion-Image generates high-resolution images via masked discrete diffusion over tokenized image patches. Each image is encoded into discrete tokens (131K codebook), and generation proceeds through iterative parallel unmasking—similar to diffusion LLMs. The model is fine-tuned from Nemotron-Labs-Diffusion with two key additions:
- Token editing — revise already-unmasked tokens during inference.
- Grouped Cross-Entropy (GCE) — efficient training with large vocabularies.
This aligns image generation with LLM training and inference infrastructure, making it highly scalable.
| GenEval | DPG | HPSv3 | Speed vs EMU3.5 |
|---|---|---|---|
| 0.90 | 86.9 | 10.76 | 42.4× faster |
Generate an image
Safety filters (Nemotron 3.5 Content Safety) run on both the prompt and the generated image and cannot be disabled.
Masked Discrete Diffusion · Text-to-Image Synthesis · Token Editing · Grouped Cross-Entropy (GCE) · High-Resolution Image Generation
Sample outputs
Generation speed
Side-by-side at 1024×1024. Left: NL-Diffusion-Image. Right: EMU3.5 (autoregressive). NL-Diffusion-Image is 42.4× faster while scoring higher on GenEval.
Architecture
16×16 image patches are encoded with a pretrained discrete tokenizer from EMU3.5 (128K codebook). The Nemotron-Labs-Diffusion vocabulary is expanded with randomly initialized embeddings and fine-tuned on image/caption pairs.
Benchmarks
State-of-the-art among discrete image generators at 1024px text-to-image, surpassing prior masked image generators on quality while remaining dramatically faster than autoregressive baselines.
Key findings
Token editing for self-correction
Token editing lets the model iteratively refine outputs during inference, correcting artifacts and improving texture detail.
Grouped Cross-Entropy (GCE)
GCE alleviates codebook sparsity by supervising semantically close non-top-1 tokens in embedding space.
A fused GCE operator cuts peak VRAM from 25.2 GB to 16.1 GB and latency from 44.14 ms to 20.04 ms versus an eager implementation.
Few-step generation
Unlike continuous flow-matching models that predict blurry mean fields at low step counts, NL-Diffusion-Image produces reasonable quality in as few as 4 steps without distillation.
Future work: extend the model to unified vision generation and understanding.