PiD (Pixel diffusion Decoder) is a plug-and-play decoder that replaces traditional VAE or RAE decoders in latent diffusion models. Instead of reconstructing pixels from latents, it uses conditional pixel-space diffusion to generate super-resolved 2K-4K images directly.

How fast is PiD compared to cascaded super-resolution?

PiD decodes 512×512 latents to 2048×2048 pixels in under 1 second on an RTX 5090 (13GB VRAM) and 210ms on GB200 — about 6× faster than cascaded diffusion-based super-resolution pipelines.

Which models does PiD support?

PiD has checkpoints for FLUX, FLUX.2, FLUX.2-Klein (4B/9B), SD3, SDXL, Z-Image, Z-Image-Turbo, Qwen-Image, DINOv2, and SigLIP. It works with both VAE latents and semantic latents from RAE-based models.

Can PiD decode partially denoised latents?

Yes — PiD's sigma-aware adapter lets it decode noise-corrupted latents, enabling early termination of the latent diffusion process. You can exit at step 24/28 on FLUX and still get quality output.

What's the advantage of PiD over standard VAE decoding?

VAE decoders are reconstruction-oriented — they invert the encoder but don't synthesize new detail. PiD is generative — it adds coherent high-frequency detail during upsampling, producing sharper 4K images without the artifacts of traditional upscalers.

How many inference steps does PiD need?

PiD is distilled with DMD2 to just 4 inference steps. Combined with early LDM termination, this makes the full pipeline significantly faster than running the base model to completion plus a separate super-resolution pass.

Yes, PiD is released under the Apache 2.0 license. The code is available on GitHub at github.com/nv-tlabs/PiD, and pretrained checkpoints are hosted on HuggingFace at huggingface.co/nvidia/PiD.

Does PiD work with ComfyUI?

Yes, PiD was merged into ComfyUI on May 27, 2026. You can use the PiD decoder node as a drop-in replacement for the standard VAE decode node in your workflows.

What GPU memory does PiD require?

PiD uses approximately 13GB of VRAM at peak when decoding to 2048×2048 resolution on a consumer RTX 5090. For 4K output, you'll need more memory or can use model offloading techniques.

How does PiD compare to Real-ESRGAN for upscaling?

Unlike Real-ESRGAN which is a separate post-processing step, PiD is integrated into the generation pipeline and conditions on the latent representation. This produces more coherent detail synthesis rather than generic sharpening, and runs faster by combining decode + upscale into one pass.

NVIDIA PiD: Replace Your VAE Decoder with Pixel Diffusion for 4K Images

By Prahlad Menon Published 2026-06-30 4 min read

Every latent diffusion model — FLUX, Stable Diffusion, SDXL — has the same bottleneck at the end of the pipeline: the VAE decoder. It takes your beautiful 64×64 or 128×128 latent and reconstructs pixels. The problem? VAE decoders are reconstruction-oriented. They’re trained to invert the encoder, not to synthesize detail. At 2K or 4K resolution, this shows: soft textures, lost fine detail, and the need for a separate super-resolution pass.

NVIDIA’s PiD (Pixel diffusion Decoder) fixes this by replacing the VAE decoder entirely. Instead of reconstruction, it treats latent-to-pixel conversion as a conditional diffusion problem — denoising directly in high-resolution pixel space to produce sharp 2K-4K images in a single pass.

The core idea

Traditional pipeline:

Text → LDM (28 steps) → Latent → VAE Decode → 1024px → Super-Resolution → 4096px

PiD pipeline:

Text → LDM (24 steps) → Latent → PiD (4 steps) → 4096px

PiD unifies decoding and upsampling into one generative module. The latent conditions the pixel diffusion process, and you get 4× upscaled output directly — no cascaded models, no quality loss from compression artifacts.

Why this matters practically

1. Speed: PiD decodes 512×512 latents to 2048×2048 in under 1 second on a consumer RTX 5090 (13GB peak memory). On a GB200, it’s 210ms. That’s 6× faster than running a separate super-resolution diffusion model.

2. Early exit: PiD’s sigma-aware adapter can decode partially denoised latents. You don’t need to run FLUX to step 28 — exit at step 24 and let PiD handle the rest. Fewer LDM steps + 4 PiD steps = faster end-to-end generation.

3. Quality: Because PiD is generative, it synthesizes coherent high-frequency detail rather than hallucinating or smearing. The project page shows clear wins over VAE decode + Real-ESRGAN, InvSR, and even SeedVR2 on Gemini-3-Flash judge ratings.

4. Universal compatibility: PiD has checkpoints for FLUX, FLUX.2, FLUX.2-Klein (4B/9B), SD3, SDXL, Z-Image, Z-Image-Turbo, Qwen-Image, and semantic latent models (DINOv2, SigLIP). Drop it into your existing pipeline.

How PiD works under the hood

PiD builds on PixelDiT, a pixel-space diffusion backbone. The key innovations:

Sigma-aware latent conditioning

The latent from your LDM isn’t clean — especially if you’re doing early exit. PiD’s adapter takes the noise level (sigma) as input alongside the latent:

data_batch = {
    "caption": [prompt],
    "LQ_latent": latent.to(dtype=torch.bfloat16),
    "degrade_sigma": torch.tensor([sigma], device="cuda"),
}

This lets PiD adapt its denoising based on how noisy the input latent is. Partially denoised at step 24? PiD knows and compensates.

Flow-matching velocity prediction

PiD uses flow-matching (like FLUX) rather than epsilon prediction. The model predicts a velocity field that transforms noise to pixels:

# Forward noising (flow-matching form)
x_t = (1.0 - sigma) * clean_latent + sigma * noise

# SDXL uses variance-preserving form instead
x_t = sqrt(1 - σ²) * clean_latent + σ * noise

DMD2 distillation

The full PiD model is distilled using DMD2 (Distribution Matching Distillation) to just 4 inference steps. This is critical for the speed advantage — you’re not trading 28 VAE-decode microseconds for 50 diffusion steps.

Quick start

# Clone and install
git clone https://github.com/nv-tlabs/PiD
cd PiD
pip install hydra-core omegaconf einops loguru safetensors -e .

# Download checkpoints
huggingface-cli download nvidia/PiD --local-dir . --include "checkpoints/*"

# Generate 2K image with FLUX + PiD
PYTHONPATH=. python -m pid._src.inference.from_ldm --backbone flux \
  --prompt "A photorealistic portrait of a brown tabby cat, soft morning light, ultra-detailed fur texture" \
  --ldm_inference_steps 28 --save_xt_steps 24 \
  --output_dir ./results/flux_2k \
  --pid_inference_steps 4

4K generation

# 4K output (4096×3072, 4:3 aspect ratio)
PYTHONPATH=. python -m pid._src.inference.from_ldm --backbone flux \
  --prompt "A close photograph of a cat looking through frosted glass" \
  --resolution 4096,3072 --pid_ckpt_type 2kto4k \
  --ldm_inference_steps 28 --save_xt_steps 24 26 \
  --output_dir ./results/flux_4k

Multi-GPU batch processing

# 4 GPUs, prompt file, torch.compile for speed
PYTHONPATH=. torchrun --nproc_per_node=4 \
  -m pid._src.inference.from_ldm --backbone zimage \
  --prompt_file prompts.txt \
  --ldm_inference_steps 50 --save_xt_steps 46 \
  --compile \
  --output_dir ./results/batch

Checkpoint variants

PiD offers two decoder variants per backbone:

Variant	Resolution	Use case
`2k`	Up to 2048px	Best quality at 2K, multiple aspect ratios
`2kto4k`	2K to 4K	Variable resolution, slightly lower 2K quality

Supported aspect ratios for 2K: 1:1 (2048×2048), 4:3 (2304×1728), 3:4 (1728×2304), 16:9 (2688×1536), 9:16 (1536×2688).

Recommended LDM exit points

Not all steps are equal. NVIDIA tested early exit points for each backbone:

Backbone	Default steps	Exit at	Notes
FLUX	28	24	Sweet spot for speed/quality
FLUX.2	50	46
SD3	28	24
SDXL	30	26
Z-Image	50	46
Z-Image-Turbo	9	x0 (final)	Already distilled

ComfyUI integration

PiD is already merged into ComfyUI as of May 27, 2026. Look for the PiD decoder node — drop it in place of your VAE decode and set the upstream LDM to exit early.

When to use PiD

Good fit:

Production pipelines generating 2K-4K images
Batch processing where speed matters
Replacing VAE + separate super-resolution
FLUX/SD3/SDXL workflows

Maybe not:

1024px output where VAE is fast enough
Memory-constrained environments (PiD adds ~13GB VRAM)
Models without PiD checkpoints yet

The bigger picture

VAE decoders have been the unchanged endpoint of latent diffusion for years. PiD is the first serious challenge — treating decoding as generation rather than reconstruction. The 6× speedup over cascaded SR is nice, but the real win is quality: coherent detail synthesis instead of reconstruction artifacts.

With checkpoints already available for every major backbone and ComfyUI integration shipped, this is ready for production use today.

If you’re building AI-powered applications, check out PageBot — an embeddable AI chatbot widget that answers questions about any webpage. Drop a single script tag and your users get instant, context-aware answers powered by GPT-5.

Links: