SANA-Video 实战指南基于 Block Linear Diffusion Transformer 的高效视频生成推理、训练与部署【免费下载链接】SanaSANA: Efficient High-Resolution Image Synthesis with Linear Diffusion Transformer项目地址: https://gitcode.com/GitHub_Trending/sana/Sana导读本文围绕 Sana 仓库中的 SANA-Video 文档 展开系统讲解 SANA-Video 这一基于 Block Linear Diffusion Transformer块线性扩散 Transformer的高效视频生成模型它用仅约 2B 参数即可合成最高 720×1280 分辨率、分钟级时长的视频并能在 RTX 5090 等消费级 GPU 上部署。读完本文你将掌握三种实战能力通过diffusers的SanaVideoPipeline/SanaImageToVideoPipeline完成文生视频T2V与图生视频I2V推理通过仓库自带的 Shell 脚本与配置文件批量生成视频理解模型的线性注意力、常驻内存 KV Cache 等核心机制并复现训练与 pth 到 diffusers 格式的转换流程。SANA-Video 是什么SANA-Video是 NVIDIA 提出的面向高效视频生成的小型扩散模型定位是低成本、高质量、高速度的视频合成。它的核心能力包括支持高达720×1280分辨率的视频合成以及分钟级时长的长视频生成并具备较强的文-视频语义对齐能力可以在 RTX 5090 这类消费级 GPU 上以 NVFP4 精度部署将生成一段 5 秒 720p 视频的推理耗时从 71 秒压缩到 29 秒约2.4× 加速。模型的三个核心贡献构成了它的技术骨架贡献点说明Efficient ArchitectureLinear DiT以线性注意力为核心里算。视频生成需要处理海量 token线性注意力相比传统注意力vanilla attention显著更高效Long-Sequence CapabilityConstant-Memory KV Cache为 Block Linear Attention 引入常驻内存 KV Cache利用线性注意力的累积特性用固定大小的状态代替传统 KV Cache采用块级自回归block-wise autoregressive方式推进从而支撑分钟级长视频生成Low Training Cost通过有效的数据过滤与训练策略训练成本仅约64 块 H100 GPU 上 12 天约为 MovieGen 训练成本的1%State-of-the-Art Speed and Performance在评测中与 Wan 2.1-1.3B 等现代小型扩散模型性能相当但实测延迟快 16×文档中对 480p 场景的概括性表述具体对比数据见下文 VBench 章节从仓库源码结构看与视频生成直接相关的实现分散在多个模块中模型主体位于 diffusion/model/nets/sana_multi_scale_video.py 与 sana_video2.py采样器位于 diffusion/scheduler/数据管线位于 diffusion/data/datasets/video/推理与训练入口则分别位于 inference_video_scripts/ 和 train_video_scripts/。核心机制Block Causal Linear Attention 与 Causal Mix-FFN文档重点强调的两个机制是Block Causal Linear Attention块因果线性注意力与Causal Mix-FFN因果混合 FFNBlock Causal Linear Attention线性注意力以累积形式维护一个固定大小的状态state从而摆脱传统上随序列长度线性增长的 KV Cache。SANA-Video 将其组织为块因果形式——视频帧按块顺序处理每个块只关注自身及其之前的块既保证了因果一致性又让长视频生成的内存开销保持恒定。这正是文档所述minute-length duration长视频得以高效生成的根本原因。Causal Mix-FFN在 FFN 中引入沿时间维的因果卷积混合对应配置文件中的ffn_type: GLUMBConvTemp、t_kernel_size: 3在保持因果性的同时增强时序建模能力。在仓库配置中可以看到这两种机制的开关与形态Sana_2000M_480px_AdamW_fsdp.yaml 使用attn_type: LiteLAReLURope线性注意力 RoPE与ffn_type: GLUMBConvTemp而分块变体 Sana_2000M_480px_AdamW_fsdp_chunk.yaml 使用attn_type: chunkcausal、ffn_type: ChunkGLUMBConvTemp并显式声明chunk_index: [0, 11]每个 chunk 的起始层索引进一步体现了块级因果处理的实现思路。如何推理diffusers Pipeline 一行接入环境要求文档明确指出使用diffusers中的 SANA-Video Pipeline 需要安装最新开发版的 diffusers包含SanaVideoPipeline支持pip install githttps://github.com/huggingface/diffusers文生视频SanaVideoPipeline以下完整示例来自原文档可直接运行。关键点VAE 保持float32精度、文本编码器使用bfloat16并通过在 prompt 尾部追加motion score: N.来控制运动幅度import torch from diffusers import SanaVideoPipeline from diffusers import AutoencoderKLWan from diffusers.utils import export_to_video model_id Efficient-Large-Model/SANA-Video_2B_480p_diffusers pipe SanaVideoPipeline.from_pretrained(model_id, torch_dtypetorch.bfloat16) pipe.vae.to(torch.float32) pipe.text_encoder.to(torch.bfloat16) pipe.to(cuda) motion_score 30 prompt Evening, backlight, side lighting, soft light, high contrast, mid-shot, centered composition, clean solo shot, warm color. A young Caucasian man stands in a forest, golden light glimmers on his hair as sunlight filters through the leaves. He wears a light shirt, wind gently blowing his hair and collar, light dances across his face with his movements. The background is blurred, with dappled light and soft tree shadows in the distance. The camera focuses on his lifted gaze, clear and emotional. negative_prompt A chaotic sequence with misshapen, deformed limbs in heavy motion blur, sudden disappearance, jump cuts, jerky movements, rapid shot changes, frames out of sync, inconsistent character shapes, temporal artifacts, jitter, and ghosting effects, creating a disorienting visual experience. motion_prompt f motion score: {motion_score}. prompt prompt motion_prompt video pipe( promptprompt, negative_promptnegative_prompt, height480, width832, frames81, guidance_scale6, num_inference_steps50, generatortorch.Generator(devicecuda).manual_seed(42), ).frames[0] export_to_video(video, sana_video.mp4, fps16)参数要点frames81总帧数配合fps16输出约 5 秒视频guidance_scale6无分类器引导强度CFG对应仓库 CLI 推理中的--cfg_scale 6num_inference_steps50采样步数motion score: 30.后缀运动分数提示用于控制画面运动幅度30 为文档推荐值仓库推理脚本默认值为 10。图生视频SanaImageToVideoPipeline图生视频使用SanaImageToVideoPipeline只需额外传入一张起始帧图片。仓库的样例输入图片位于 asset/samples/i2v-1.png 与 asset/samples/i2v-2.png832×480你也可以换成自己的图片import torch from diffusers import SanaImageToVideoPipeline, FlowMatchEulerDiscreteScheduler from diffusers.utils import export_to_video, load_image pipe SanaImageToVideoPipeline.from_pretrained(Efficient-Large-Model/SANA-Video_2B_480p_diffusers) # pipe.scheduler FlowMatchEulerDiscreteScheduler(shiftpipe.scheduler.config.flow_shift) pipe.transformer.to(torch.bfloat16) pipe.text_encoder.to(torch.bfloat16) pipe.vae.to(torch.float32) pipe.to(cuda) motion_score 30 prompt A woman stands against a stunning sunset backdrop, her long, wavy brown hair gently blowing in the breeze. She wears a sleeveless, light-colored blouse with a deep V-neckline, which accentuates her graceful posture. The warm hues of the setting sun cast a golden glow across her face and hair, creating a serene and ethereal atmosphere. The background features a blurred landscape with soft, rolling hills and scattered clouds, adding depth to the scene. The camera remains steady, capturing the tranquil moment from a medium close-up angle. negative_prompt A chaotic sequence with misshapen, deformed limbs in heavy motion blur, sudden disappearance, jump cuts, jerky movements, rapid shot changes, frames out of sync, inconsistent character shapes, temporal artifacts, jitter, and ghosting effects, creating a disorienting visual experience. motion_prompt f motion score: {motion_score}. prompt prompt motion_prompt image load_image(asset/samples/i2v-1.png) output pipe( imageimage, promptprompt, negative_promptnegative_prompt, height480, width832, frames81, guidance_scale6, num_inference_steps50, generatortorch.Generator(devicecuda).manual_seed(42), ).frames[0] export_to_video(output, sana-ti2v-output.mp4, fps16)注释中保留了一行可选的调度器替换代码pipe.scheduler FlowMatchEulerDiscreteScheduler(shiftpipe.scheduler.config.flow_shift)用于显式控制 flow matching 的 shift 参数仓库配置中的对应值为flow_shift: 3.0训练与inference_flow_shift: 7.0推理见下节。通过 TXT 文件批量推理仓库原生脚本除了 diffusers仓库提供了基于原生训练框架的批量推理入口 inference_video_scripts/inference_sana_video.sh。脚本内部会调用accelerate launch运行 inference_sana_video.py支持多进程--np、多 GPU 分布式推理并使用--mixed_precisionbf16。文生视频Text-to-Videobash inference_video_scripts/inference_sana_video.sh \ --np 1 \ --config configs/sana_video_config/Sana_2000M_480px_AdamW_fsdp.yaml \ --model_path hf://Efficient-Large-Model/SANA-Video_2B_480p/checkpoints/SANA_Video_2B_480p.pth \ --txt_fileasset/samples/video_prompts_samples.txt \ --cfg_scale 6 \ --motion_score 30 \ --flow_shift 8 \ --work_dir output/sana_t2v_video_results图生视频Image-to-Videobash inference_video_scripts/inference_sana_video.sh \ --np 1 \ --config configs/sana_video_config/Sana_2000M_480px_AdamW_fsdp.yaml \ --model_path hf://Efficient-Large-Model/SANA-Video_2B_480p/checkpoints/SANA_Video_2B_480p.pth \ --txt_fileasset/samples/sample_i2v.txt \ --taskltx \ --cfg_scale 6 \ --motion_score 30 \ --flow_shift 8 \ --work_dir output/sana_ti2v_video_results关键参数与底层逻辑对照 inference_sana_video.py 中的SanaInference参数类可以把命令行参数与底层行为对应起来参数默认值说明--np8accelerate launch的进程数并行 GPU 数--configconfigs/sana_video_config/Sana_2000M_480px_AdamW_fsdp.yaml推理配置覆盖模型、VAE、文本编码器、调度器等全部细节--model_pathhf://Efficient-Large-Model/SANA-Video_2B_480p/checkpoints/SANA_Video_2B_480p.pth模型权重支持hf://前缀从 HuggingFace 自动下载--txt_fileasset/samples/video_prompts_samples.txt提示词文件每行一条 prompt--cfg_scale6.0无分类器引导强度--flow_shift由配置决定推理时的 flow shift未指定时优先取scheduler.inference_flow_shift否则取scheduler.flow_shift--motion_score10运动分数追加到 prompt 尾部--fps16输出视频帧率--bs1每批生成数量--num_frames-1取配置值帧数配置中data.num_frames: 81--sampling_algo取scheduler.vis_sampler采样器flow_dpm-solver、flow_euler、flow_euler_ltx、longlive_flow_euler、fastvideo_dmd_4step等--guidance_typeclassifier-free引导方式可选adaptive_projected_guidance、classifier-free_STG--seed0随机种子--negative_prompt默认的chaotic sequence文本负面提示词几个值得注意的实现细节来自 inference_sana_video.pymotion score 的三种形态当--motion_score 0时脚本构造 motion score: N.追加到 prompt 末尾当 0时不追加任何运动提示当等于 0 时则追加 high motion或 low motion文本DMD 4 步采样器除外。I2V 的 prompt 格式图生视频的 prompt 文件如 asset/samples/sample_i2v.txt在文本后用image分隔符拼接图片路径例如A woman stands against a stunning sunset backdrop...imageasset/samples/i2v-1.png。脚本通过image_split_token默认image切分文本与图片并将起始帧的 VAE 潜变量写入第 0 帧作为条件。I2V 强制使用flow_euler_ltx采样器当任务为ltx/ti2v时脚本会把采样器切换为flow_euler_ltx即使传入了其他采样算法。长视频限制若num_frames超过基础模型帧数81脚本断言要求使用longlive_flow_euler采样器这正是配合 LongSANA / LongLive 长视频方案的使用方式。分辨率策略默认开启use_resolution_binning从ASPECT_RATIO_VIDEO_480_MS中随机选择宽高比也可用--custom_height_width固定输出分辨率默认 480×832。如何训练5 秒视频模型预训练仓库提供了视频联合训练视频图像脚本 train_video_scripts/train_video_ivjoint.sh内部通过torchrun启动 train_video_ivjoint.py配置TRITON_PRINT_AUTOTUNING1、DISABLE_XFORMERS1等环境变量。文档给出的预训练命令# 5s Video Model Pre-Training bash train_video_scripts/train_video_ivjoint.sh \ configs/sana_video_config/Sana_2000M_480px_AdamW_fsdp.yaml \ --data.data_dir[data/toy_data] \ --train.train_batch_size1 \ --work_diroutput/sana_video \ --train.num_workers10 \ --train.visualizetrue注意脚本默认参数为--np 1、--train.log_interval1、--train.num_workers0、--train.visualizeFalse、--debugtrue并可通过--np N指定单机多卡训练规模。训练配置逐段解读以 Sana_2000M_480px_AdamW_fsdp.yaml 为例训练/推理配置可分为六大部分1.data视频数据data_dir视频数据集路径示例data/video_toy_dataexternal_caption_suffixes: [_Qwen2.5-VL]使用 Qwen2.5-VL 生成的外部 caption 后缀caption_proportionprompt 原文本与外部 caption 的采样比例5 : 95external_data_filter._unimatch: {min: 1.0, max: 30}按 UniMatch 运动分数过滤样本motion_score_cal_type: average运动分数计算方式可选average/maximage_size: 480、aspect_ratio_type: ASPECT_RATIO_VIDEO_480_MS480p 多尺度宽高比num_frames: 81每个视频样本 81 帧对应约 5 秒 16fpstype: SanaZipDatasetZIP 打包格式的数据集。2.image_data联合训练的图像数据与视频并行使用SanaWebDatasetMS图像数据集通过clip_thr: 25.0、del_img_clip_thr: 22.0做 CLIP 分数过滤实现图文-视频联合训练joint_training_interval: 10控制交替频率。3.model扩散主干model: SanaMSVideo_2000M_P2_D20约 2000M 参数的视频多尺度模型P2 阶段、20 层 DiTattn_type: LiteLAReLURope线性注意力 ReLU RoPE分块变体为chunkcausallinear_head_dim: 112线性注意力头维度ffn_type: GLUMBConvTemp、t_kernel_size: 3带时间维因果卷积的 Gated Linear FFN分块变体为ChunkGLUMBConvTemppos_embed_type: wan_ropeWan 风格 RoPE 位置编码qk_norm: true、cross_norm: trueQK 归一化与交叉归一化mixed_precision: bf16、fp32_attention: true注意力以 fp32 计算保证数值稳定。4.vae潜空间自编码器vae_type: WanVAE复用 Wan2.1 的 VAE权重hf://Efficient-Large-Model/SANA-Video_2B_480p/vae/Wan2.1_VAE.pthvae_latent_dim: 16、vae_downsample_rate: 8vae_stride: [4, 8, 8]时间维 4 倍、空间维 8×8 的下采样weight_dtype: float32VAE 保持 fp32。5.text_encoder文本编码器text_encoder_name: gemma-2-2b-it使用 Gemma-2-2B 作为文本编码器y_norm: true、y_norm_scale_factor: 0.01文本特征归一化model_max_length: 300最大文本 token 数chi_prompt一段多轮prompt 增强指令CHICaption Harmonization Instruction推理时会与用户 prompt 拼接把简单描述扩充为更丰富的视觉描述。6.scheduler与train调度与训练策略predict_flow_v: true、noise_schedule: linear_flow预测 flow velocity 的线性 flow matching 调度flow_shift: 3.0、inference_flow_shift: 7.0训练/推理分开的 flow shiftweighting_scheme: logit_normal、logit_mean: 0.0、logit_std: 1.0logit-normal 时间步加权vis_sampler: flow_dpm-solver可视化采样器分块变体为chunk_flow_eulertrainFSDPuse_fsdp: true、train_batch_size: 1视频train_batch_size_image: 4图像、梯度累积 1、grad_checkpointing: true、梯度裁剪 0.1、AdamWlr 5e-5、weight_decay 0、constant 学习率 500 步 warmup、save_model_steps: 500等。权重转换pth 到 diffusers safetensor原生训练产出的.pth权重可通过转换脚本导出为 diffusers 格式便于用SanaVideoPipeline直接加载。文档给出的命令python scripts/convert_scripts/convert_sana_video_to_diffusers.py --dump_path output/SANA_Video_2B_480p_diffusers --save_full_pipeline对照 convert_sana_video_to_diffusers.py 的 CLI 定义注意实际脚本位于tools/convert_scripts/下可补充说明--dump_path必填输出目录即导出的完整 diffusers pipeline 保存位置--save_full_pipeline是否同时保存完整 pipelinetransformer、VAE、文本编码器等全量组件不带该标志时默认只导出 transformer 子目录并以max_shard_size5GB分片保存--dtype导出权重精度可选fp32默认、fp16、bf16对应文档示例中的 bf16 推理需求。性能表现VBench 评测以下是文档给出的 VBench 评测数据表格与原文保持一致用于客观了解模型在质量与速度之间的平衡。所有数据均来自原文档未做任何加工。480p 分辨率 · 文生视频MethodsLatency (s)Speedup#Params (B)Total ↑Quality ↑Semantic / I2V ↑Open-Sora-2.04651.0×1484.3485.480.72Wan2.1-14B4841.0×1483.6985.5976.11Wan2.1-1.3B1034.7×1.383.3185.2375.65SANA-Video608.0×284.1784.8581.46展开的完整对比还包括MAGI-14.5B79.18、Step-Video30B81.83、CogVideoX1.55B82.17、SkyReels-V21.3B82.67等SANA-Video 在 Total 指标上仍保持第一梯队84.17且延迟最低60s。480p 分辨率 · 图生视频MethodsLatency (s)Speedup#Params (B)Total ↑Quality ↑Semantic / I2V ↑MAGI-14351.1×4.589.2882.4496.12Step-Video-TI2V2462.0×3088.3681.2295.50CogVideoX-5b-I2V1114.4×586.7078.6194.79HunyuanVideo-I2V2102.3×1386.8278.5495.10Wan2.1-14B4931.0×1486.8680.8292.90SANA-Video608.2×288.0279.6596.40720p 分辨率ModelsLatency (s)Total ↑Quality ↑Semantic ↑Wan-2.1-14B189783.7385.7775.58Wan-2.1-1.3B40083.3885.6774.22Wan-2.2-5B11683.2885.0376.28SANA-Video-2B3684.0584.6381.7330s 长视频 VBenchModelsFPSTotal ↑Quality ↑Semantic ↑SkyReels-V20.4975.2980.7753.37FramePack0.9281.9583.6175.32Self-Forcing17.081.5983.8272.70LongSANA-2B27.582.2983.1079.04小结引用原文档与当前的 SOTA 小型视频模型相比SANA 的性能极具竞争力且速度更快——仅凭 2B 参数即可取得 83.71 的综合 VBench 分数480p原文整体表述480p 延迟仅 60s720p 分辨率下以 36s 延迟拿到 84.05 的总分。长视频方面LongSANASANA-Video LongLive在 H100 上达到 27.5 FPS 的生成速度使实时长视频生成成为可能。关联资源长视频与视频二代模型围绕 SANA-Video仓库还提供了两条延伸技术路线可作为进一步深入的方向LongSANASANA-Video LongLive 的长视频方案支持分钟级甚至更长序列采样器 longlive_flow_euler_sampler.py 与配置 configs/longsana/ 均已就绪对应推理脚本中的sampling_algolonglive_flow_euler分支SANA-Video2视频生成的下一代模型5B、720p配置见 configs/sana_video2/模型实现见 sana_video2.py 与 sana_video2_blocks.py长视频训练与推理train_longsana.py、sana_video_inference.md 等提供了配套的脚本与文档。引用如果你在研究中使用了 SANA-Video请引用以下论文来自原文档 BibTeXmisc{chen2025sana, title{SANA-Video: Efficient Video Generation with Block Linear Diffusion Transformer}, author{Chen, Junsong and Zhao, Yuyang and Yu, Jincheng and Chu, Ruihang and Chen, Junyu and Yang, Shuai and Wang, Xianbang and Pan, Yicheng and Zhou, Daquan and Ling, Huan and others}, year{2025}, eprint{2509.24695}, archivePrefix{arXiv}, primaryClass{cs.CV}, }【免费下载链接】SanaSANA: Efficient High-Resolution Image Synthesis with Linear Diffusion Transformer项目地址: https://gitcode.com/GitHub_Trending/sana/Sana创作声明：本文部分内容由AI辅助生成（AIGC），仅供参考