SANA (Efficient High-Resolution Image Synthesis)¶

Efficient DiT from NVlabs/MIT Han Lab. 600M-4.8B params with competitive quality at 1024-4096px. Uses linear attention O(n), DC-AE 32× compression, and Gemma-2-2B text encoder. ICLR 2025 Oral.

Current Status (Verified 2026-08-27)¶

The SANA name now covers several related but non-interchangeable branches. Select the artifact before applying performance, memory, or licensing claims.

The August 4, 2026 report that Video2 had no public weights was accurate at the time and is now superseded for the 5B model. It remains accurate for the 14B checkpoint as of this verification date.

The official Video2 page reports a 5-second, 720p, 40-step 5B benchmark on one H100, including a 13.06-second Sol Engine result. Treat this as a pinned benchmark context, not a consumer-GPU runtime estimate.

Architecture — Full Detail¶

Model Variants¶

0.6B = deeper but narrower. 1.6B = shallower but wider. 4.8B scales ONLY depth (20→60), width same as 1.6B.

SanaBlock Structure (AdaLN-Zero)¶

Input x → LayerNorm → modulate(shift1, scale1) → Linear Self-Attention (LiteLA)
        → LayerNorm → modulate(shift2, scale2) → Cross-Attention (standard, with text)
        → LayerNorm → modulate(shift3, scale3) → Mix-FFN (GLUMBConv)
Output x (residual at each stage, 6 modulation params per block via scale_shift_table)

Linear Attention (LiteLA with ReLU Kernel)¶

# Standard quadratic: O(N^2)
# Attention = softmax(QK^T / sqrt(d)) * V

# SANA linear: O(N * d^2)
# phi(x) = ReLU(x)
# Shared terms: S = sum_j phi(K_j)^T * V_j   (shape d×d, computed ONCE)
#               Z = sum_j phi(K_j)^T          (shape d×1, computed ONCE)
# Output_i = phi(Q_i) * S / (phi(Q_i) * Z + eps)

Trade-off: linear attention alone degrades quality. Compensated by Mix-FFN with 3×3 depthwise convolution that captures local spatial info lost by ReLU kernel (no softmax locality bias).

Triton kernel fusion: ReLU activation + precision conversions + padding + division fused into matmul → ~10% speed acceleration.

Position encoding: "NoPE" (No Positional Embeddings) — 3×3 depthwise conv in Mix-FFN implicitly encodes position. Alternatively supports RoPE theta=10000, axes_dim=[0,16,16].

Mix-FFN (GLUMBConv)¶

Replaces standard MLP:

Linear(d → d×2.5) → DW-Conv3×3 → SiLU → Gate (GLU) → Linear(d×1.25 → d)

DC-AE (Deep Compression Autoencoder) — F32C32P1¶

32× spatial compression, 32 latent channels, patch size 1.

4× fewer tokens + O(n) linear attention = orders of magnitude faster at high res.

Reconstruction quality (ImageNet): rFID 0.34, PSNR 29.29, SSIM 0.84, LPIPS 0.05.

Tiling supported: pipe.vae.enable_tiling(tile_sample_min_height=1024, tile_sample_min_width=1024) — enables 4K within 22 GB VRAM.

Two versions: dc-ae-f32c32-sana-1.0, dc-ae-f32c32-sana-1.1 (improved), dc-ae-lite-f32c32 (faster/smaller).

Text Encoder: Gemma-2-2B-IT¶

Decoder-only LLM (not T5). 6× faster than T5-XXL. Max 300 tokens.

Critical: decoder-only outputs have variance orders of magnitude larger than T5. Solution: RMSNorm after encoder + learnable scale 0.01 (y_norm: true, y_norm_scale: 0.01).

Complex Human Instruction (CHI): leverages Gemma's in-context learning → +2.2 GenEval points.

Training¶

Loss & Scheduler¶

flow matching velocity prediction: v_theta(x_t, t) = epsilon - x_0. Timestep sampling: logit-normal (mean=0.0, std=1.0). Flow shift: 3.0.

Optimizer: CAME¶

optimizer: CAMEWrapper
lr: 1e-4
betas: [0.9, 0.999, 0.9999]
epsilon: [1e-30, 1e-16]
weight_decay: 0.0
grad_clip: 0.1
warmup: 2000 steps, constant after

SANA 1.5 uses CAME-8bit — block-wise 8-bit first-order moments, 32-bit second-order. 25% memory reduction vs AdamW.

Resolution Schedule¶

Skip 256px entirely. Start at 512px → finetune to 1024 → 2K → 4K.

Multi-Caption Labeling¶

4 VLMs generate captions (VILA-3B, VILA-13B, InternVL2-8B, InternVL2-26B). CLIP-score sampler selects per iteration.

SFT: 3M samples filtered by CLIP > 25 from 50M pre-training set.¶

SANA 1.5 Key Improvements¶

1. Depth-growth paradigm (1.6B → 4.8B): remove last 2 blocks of trained 1.6B → add 40 new blocks with Partial Preservation Init (identity mappings) → 60% fewer training steps
2. QK-normalization: RMSNorm on Q,K for stable large-model training
3. Depth pruning: block importance metric → prune middle blocks, keep head/tail → quick recovery with ~100 fine-tune steps
4. Inference-time scaling: generate N candidates, select best via VILA-Judge (fine-tuned NVILA-2B). GenEval: 0.81 → 0.96 with 2048 candidates. 1.6B + scaling outperforms 4.8B without

Benchmarks¶

Sana-1.6B = 23× faster than FLUX-dev. At 4K: 106× faster (9.6s vs 469s).

SANA-Sprint (Distilled, 1-4 Steps)¶

Hybrid distillation: sCM (continuous consistency) + LADD (latent adversarial).

Outperforms FLUX-schnell (7.94 FID) while 10× faster. ICCV 2025 Highlight.

SANA Video and World-Model Branches¶

SANA-Video (2025 generation)¶

The original video branch introduced Block Causal Linear Attention and Causal Mix-FFN. Its reported 2B/720p/16 FPS results belong to that release and must not be copied onto Video2 or SANA-WM.

Key for temporal tiling: causal attention is structurally relevant to tiles-as-frames, but this is an architectural analogy rather than a supported SANA workflow.

SANA-WM (2026 world model)¶

SANA-WM is a 2.6B interactive world model rather than a rename of SANA-Video. The official project describes 720p generation, sequences up to one minute, and 6-DoF camera control. Validate controller format and checkpoint license against the exact release before integration.

SANA-Video2 (2026 generation)¶

Video2 is the current generation branch. The official repository added training code, inference code, architecture details, and a 5B 720p checkpoint on 2026-08-19. A 14B variant is described but is not a runnable public checkpoint in the repository.

The project-page paper benchmark and the released-checkpoint guide use different run shapes:

For a reproducible released-checkpoint evaluation, start from the public guide and record:

artifact: SANA-Video2-5B
revision: <immutable-checkpoint-or-repository-revision>
resolution: 720p
frames: 193
fps: 24
duration_seconds: 8.04
steps: 50
gpu: <exact-model>
engine: <official-or-baseline-runtime>
seed: <integer>
refiner_enabled: <true-or-false>

Do not substitute the 14B label into this recipe. Use 5 seconds/40 steps only when reproducing the paper benchmark, not as the public-checkpoint default. Do not compare H100 timing to a local GPU without measuring the same frames/duration, resolution, steps, runtime, and refiner state.

VRAM¶

Fine-Tuning / LoRA¶

Official support via diffusers train_dreambooth_lora_sana.py. See diffusion lora training for full training pipeline details.

LoRA targets: attn.to_k, attn.to_q, attn.to_v, attn.to_out.0 + optionally FFN/MLP.

Settings: LR 1e-4, 500 steps, batch 1, grad accum 4, bf16. Requires peft >= 0.14.0.

Dataset requirements: - DreamBooth (subject): 3-5 images minimum - Style/domain LoRA: 20-30 images recommended - Format: 1024x1024+ resolution, detailed captions with rare trigger token ("sks")

Memory optimization flags: --offload (CPU offload text encoder + VAE), --cache_latents (precompute VAE latents), --use_8bit_adam.

ControlNet also supported — ControlNet-Transformer architecture for SANA backbone.

Self-Refinement (img2img)¶

SANA-Sprint supports img2img via SanaSprintImg2ImgPipeline:

from diffusers import SanaSprintImg2ImgPipeline
import torch

pipe = SanaSprintImg2ImgPipeline.from_pretrained(
    "Efficient-Large-Model/Sana_Sprint_1.6B_1024px_diffusers",
    torch_dtype=torch.bfloat16
)
pipe.to("cuda")

# Multi-pass refinement (replaces SDXL refiner pattern)
refined = pipe(
    prompt="a high quality detailed photo",
    image=initial_image,
    strength=0.3,            # 0.3 = mild refinement, 0.7 = heavy change
    num_inference_steps=4    # Sprint is 1-4 steps
).images[0]

Flow matching img2img mechanics: SANA uses flow matching, not DDPM. strength parameter interpolates between the encoded input image and pure noise: x_t = (1-strength)*image_latent + strength*noise. This differs from DDPM's noise scheduling - there is no "add noise then denoise" step, it's direct interpolation.

Multi-pass recipe:

txt2img 1024px (strength=1.0) → img2img strength=0.3-0.4 → img2img strength=0.2

SDXL refiner has no flow matching equivalent: SDXL refiner uses high-timestep sampling which is DDPM-specific. SANA's img2img is the functional replacement but works differently under the hood.

DemoFusion incompatible with SANA: DemoFusion relies on UNet skip connections for multi-scale global context. SANA's transformer architecture doesn't have these. Use FreeScale or APT for high-res tiling instead.

See flow matching for full details on flow matching img2img.

Development History¶

Practical Selection¶

• Use SANA/SANA 1.5 for efficient image generation and documented image LoRA workflows.
• Use Sprint only when its one-to-four-step distilled behavior is explicitly desired; do not reuse standard step counts.
• Use SANA-Video2 5B for the current official runnable Video2 path.
• Use SANA-WM only for interactive/world-model experiments with its own controller contract.
• Do not plan a 14B Video2 deployment until an official checkpoint and license are present.

Research Coverage and Open Gaps¶

• English primary evidence: official repository, official Video2 project page, and papers.
• Chinese query lane: searched, but no qualified first-party Chinese source was retained in the bounded pass.
• Open proof gap: a fully reproducible consumer-GPU matrix pinned to checkpoint hashes, runtime versions, and refiner state.
• Community performance reports remain secondary until reproduced with the same settings.

Community reports¶

• Issue #156 reports a community 1.6B training path on 24 GB using offload. The contributor explicitly called the patch messy; an official contributor pointed to the project's CAME-8bit configuration as a better memory-saving direction. Treat the issue as a technique lead, not a drop-in recipe.
• Issue #297 records an unresolved MacBook Pro M3/MPS case where Sana_600M_512px_diffusers with float16 produced a uniform gray image without an exception. Test float32 as a diagnostic, but no maintainer-confirmed fix was present.
• Neither issue concerns the released Video2 5B checkpoint. Do not transfer their image-model settings to Video2.

Gotchas¶

• Issue: Treating “SANA-Video” as one continuously versioned checkpoint -> Fix: record the branch (Video, WM, or Video2), model size, checkpoint revision, and release date.
• Issue: Selecting Video2 14B because it appears in architecture tables -> Fix: use the released 5B checkpoint; 14B is not a public runnable artifact as of 2026-08-27.
• Issue: Repeating the August 4 “no weights” statement as current -> Fix: mark it superseded by the August 19 5B release while retaining it as historical context.
• Issue: Generalizing H100 latency or old 2B video benchmarks -> Fix: preserve hardware, engine, duration, resolution, steps, and branch in every comparison.
• Issue: Using the paper's 5-second/40-step benchmark as the released-checkpoint recipe -> Fix: start from the public guide's 193 frames, 24 FPS, and 50 steps unless intentionally reproducing the benchmark.
• Issue: Applying an image-model offload/MPS workaround to Video2 -> Fix: require the exact branch, checkpoint, issue URL, and runtime before reuse.

Agent Brief¶

When answering a SANA question, first identify image, Sprint, Video, WM, or Video2. Prefer the official repository and project page over news summaries. Return historical claims with a temporal label (current, superseded, or unknown). Never recommend Video2 14B as downloadable unless a newer first-party checkpoint is verified. For deployment advice, request or report the exact artifact revision, GPU, runtime, resolution, duration, steps, and refiner state.

License¶

Code: Apache 2.0. Weights: NSCL v2-custom (check specific terms for commercial use).

Key Links¶

• GitHub: https://github.com/NVlabs/Sana
• SANA-Video2: https://nvlabs.github.io/Sana/Video2/
• Released Video2 checkpoint guide: https://github.com/NVlabs/Sana/blob/main/docs/sana_video2.md
• Community offload report: https://github.com/NVlabs/Sana/issues/156
• Open MPS gray-output report: https://github.com/NVlabs/Sana/issues/297
• HF: https://huggingface.co/Efficient-Large-Model/
• Papers: https://arxiv.org/abs/2410.10629 (SANA), https://arxiv.org/abs/2501.18427 (1.5), https://arxiv.org/abs/2503.09641 (Sprint)
• SANA-Video: https://arxiv.org/abs/2509.24695
• DC-AE: https://github.com/mit-han-lab/efficientvit
• Training framework: happyin-research/sana-fm/ (local)