模块化 Diffusers 完全指南用可复用管道块构建灵活定制的工作流【免费下载链接】diffusers Diffusers: State-of-the-art diffusion models for image, video, and audio generation in PyTorch.项目地址: https://gitcode.com/GitHub_Trending/di/diffusers模块化 DiffusersModular Diffusers是当前仓库在传统DiffusionPipeline之外提供的一套统一管道系统它以管道块Pipeline Blocks为基本单元把扩散管道拆解成可重用、可混合搭配的积木帮助开发者针对特定工作流快速组装管道而无需为每个新功能重写整条链路。读完本文你将掌握四类管道块的定义与组合方式、PipelineState/BlockState状态通信机制、ModularPipeline的创建与组件加载流程以及如何基于现有块扩展 IP-Adapter、ControlNet 等新工作流。[!WARNING] 模块化 Diffusers 正在积极开发中其 API 可能会发生变化。本文所有代码与路径以当前仓库实际内容为准。模块化 Diffusers 的设计动机与核心思想在 docs/source/zh/modular_diffusers/overview.md 中模块化 Diffusers 被定义为一个统一的管道系统通过管道块简化您的工作流程其两条核心设计原则是块是可重用的你只需要为自己的管道创建独特的块其余全部复用现成实现块可以混合搭配块之间自由组合可以适配特定工作流也可以在一个管道中同时支持多个工作流。从源码结构看这套系统的实现集中在 src/diffusers/modular_pipelines/ 目录其中 modular_pipeline.py 定义了全部核心类PipelineState、BlockState、ModularPipelineBlocks、AutoPipelineBlocks、SequentialPipelineBlocks、LoopSequentialPipelineBlocks、ModularPipelinemodular_pipeline_utils.py 定义了InsertableDict、ComponentSpec、ConfigSpec、InputParam、OutputParam等辅助数据结构。仓库还内置了大量现成的模块化实现例如 stable_diffusion_xl/SDXL 文本到图像/图像到图像/修复、flux/、wan/、cogvideo/ 等说明这套框架已被广泛用于仓库内多类模型的管道实现。四类管道块从单步到多工作流管道块是模块化 Diffusers 的定义它指定管道中单个步骤的组件、输入、输出和计算逻辑。框架共有四类块见 pipeline_block.md 与 quickstart.md块类型角色典型用途ModularPipelineBlocks最基本的单一步骤块定义一步计算如编码、去噪、解码SequentialPipelineBlocks多块线性组合其他块一个块的输出作为下一个块的输入LoopSequentialPipelineBlocks多块迭代运行默认就是迭代的去噪循环AutoPipelineBlocks针对不同工作流的块集合根据输入自动选择运行哪个块ModularPipelineBlocks管道的最小单元单个ModularPipelineBlocks无法独立执行它是管道中步骤应执行操作的蓝图要真正运行必须把块转换为ModularPipeline。在源码中ModularPipelineBlocks继承自ConfigMixin与PushToHubMixinmodular_pipeline.py每个实例维护一个sub_blocks的InsertableDict第 350 行用于承载嵌套的子块。一个块需要声明以下几类成员inputs由用户提供、从PipelineState检索的值。使用InputParam定义可指定name、type_hint、description与required。intermediate_inputs通常由前一个块产生的中间值与inputs不同它可被当前块修改。intermediate_outputs由本块创建并写入PipelineState的新值可被后续块作为intermediate_inputs消费也可作为管道最终输出。使用OutputParam定义。expected_components/expected_configs通过ComponentSpec预训练组件default_creation_methodfrom_pretrained与ConfigSpec管道级配置声明块所需组件与配置。__call__计算逻辑本体。计算逻辑遵循固定四步结构pipeline_block.mddef __call__(self, components, state): # 1. 获取该块需要的状态变量的局部视图 block_state self.get_block_state(state) # 2. 你的计算逻辑在这里通过 block_state.image 等方式访问输入 # 3. 将更新后的 block_state 推回全局 PipelineState self.set_block_state(state, block_state) # 4. 返回对下一个块可用的组件和状态 return components, state组件通过点号访问components.unet、components.vae、components.scheduler。SequentialPipelineBlocks线性串联块SequentialPipelineBlocks将多个块按注册顺序线性组合数据通过intermediate_inputs/intermediate_outputs依次流动。每个块通常代表管道中的一个步骤。官方文档给出了一个将两个块串联的完整示例sequential_pipeline_blocks.mdInputBlock输出batch_sizeImageEncoderBlock将其作为输入消费。两者通过InsertableDict连接块名映射到块实例执行顺序即注册顺序from diffusers.modular_pipelines import SequentialPipelineBlocks, InsertableDict blocks_dict InsertableDict() blocks_dict[input] input_block blocks_dict[image_encoder] image_encoder_block blocks SequentialPipelineBlocks.from_blocks_dict(blocks_dict) print(blocks) # 查看子块 print(blocks.doc) # 查看输入/输出详细信息在源码中SequentialPipelineBlocks.from_blocks_dict位于 modular_pipeline.py是创建顺序块的统一入口。LoopSequentialPipelineBlocks迭代循环块去噪过程天然是迭代的LoopSequentialPipelineBlocks也称循环包装器正是为此设计。它由两部分组成循环包装器wrapper定义循环结构、迭代变量与配置。需要声明loop_inputs等同inputs、loop_intermediate_inputs、loop_intermediate_outputs以及实现循环逻辑的__call__循环体内调用self.loop_step(components, block_state, ii)依次执行所有注册的子块。循环块loop block一种特殊的ModularPipelineBlocks其__call__签名不同——它从包装器接收当前迭代索引i直接与BlockState协作而不再检索/更新PipelineState。循环块共享同一个BlockState因此值可以在每次迭代中累积变化。import torch from diffusers.modular_pipelines import LoopSequentialPipelineBlocks, ModularPipelineBlocks, InputParam, OutputParam class LoopWrapper(LoopSequentialPipelineBlocks): model_name test property def loop_inputs(self): return [InputParam(namenum_steps)] torch.no_grad() def __call__(self, components, state): block_state self.get_block_state(state) for i in range(block_state.num_steps): components, block_state self.loop_step(components, block_state, ii) self.set_block_state(state, block_state) return components, state class LoopBlock(ModularPipelineBlocks): property def inputs(self): return [InputParam(namex)] property def intermediate_outputs(self): return [OutputParam(namex)] def __call__(self, components, block_state, i: int): block_state.x 1 return components, block_state loop LoopWrapper.from_blocks_dict({block1: LoopBlock, block2: LoopBlock})从仓库实现看SDXL 的去噪循环正是这种结构StableDiffusionXLDenoiseLoopWrapper继承自LoopSequentialPipelineBlocksdenoise.py内部由before_denoiser、denoiser、after_denoiser三个子块组成其中StableDiffusionXLLoopBeforeDenoiser位于同一文件的第 40 行。AutoPipelineBlocks按输入自动选择工作流AutoPipelineBlocks将多个工作流如文本到图像、图像到图像、修复打包进一个块集合根据运行时提供的输入自动选择执行哪个子块。定义时需要三个成员block_classes候选子块类列表block_names与block_classes一一对应的名称block_trigger_inputs触发输入名称列表——运行时若提供了某个触发输入就运行对应块用None指定未检测到任何触发输入时运行的默认块。官方文档的示例将修复、图生图、文生图三种工作流合并auto_pipeline_blocks.mdfrom diffusers.modular_pipelines import AutoPipelineBlocks class AutoImageBlocks(AutoPipelineBlocks): block_classes [block_inpaint_cls, block_i2i_cls, block_t2i_cls] block_names [inpaint, img2img, text2img] # mask 触发修复image 触发 img2img仅在未提供 mask 时否则默认 text2img block_trigger_inputs [mask, image, None] def description(self): return ( Pipeline generates images given different types of conditions!\n - inpaint workflow is run when mask is provided.\n - img2img workflow is run when image is provided (but only when mask is not provided).\n - text2img workflow is run when neither image nor mask is provided.\n )文档特别强调description对AutoPipelineBlocks极其重要——条件逻辑若不解释清楚使用者将难以理解如何触发各工作流。对于嵌套在更大管道中的AutoPipelineBlocks可用get_execution_blocks(mask)提取实际将运行的块源码实现见 modular_pipeline.py。仓库中 SDXL 的StableDiffusionXLAutoVaeEncoderStep就是实例有mask_image走修复编码、只有image走图生图编码、两者皆无则跳过modular_blocks_stable_diffusion_xl.py。状态系统PipelineState 与 BlockState块之间通过两种状态数据结构通信modular_diffusers_states.md状态描述PipelineState维护管道执行所需的整体数据允许块读取和更新BlockState允许每个块使用来自inputs的必要数据执行计算PipelineState是所有块的全局状态容器内部用字典结构化数据values字典是可变状态包含用户输入的副本与块产生的中间输出。例如values{prompt: a cat, guidance_scale: 7.0, num_inference_steps: 25, prompt_embeds: Tensor(...), negative_prompt_embeds: None}。如果一个块修改了某个input调用set_block_state后会反映到values中。BlockState是PipelineState中相关变量的局部视图直接以属性方式访问如block_state.image。两者的交互由块的inputs与intermediate_outputs定义inputs可被修改并通过set_block_state全局传播intermediate_outputs是新创建的变量会被加入values字典供后续块使用或作为管道最终输出。在源码中PipelineState与BlockState分别定义于 modular_pipeline.py 与第 255 行。ModularPipeline把块变成可执行管道ModularPipeline将块转换为可执行管道加载模型并执行块中定义的计算步骤。它是运行管道的主要接口与DiffusionPipelineAPI 非常相似主要区别是管道中包含了预期的output参数modular_pipeline.md。从预设块创建管道SDXL 模块提供了现成的块预设如TEXT2IMAGE_BLOCKS、IMAGE2IMAGE_BLOCKS、INPAINT_BLOCKS。例如一个完整的 SDXL 文生图管道只需import torch from diffusers.modular_pipelines import SequentialPipelineBlocks from diffusers.modular_pipelines.stable_diffusion_xl import TEXT2IMAGE_BLOCKS blocks SequentialPipelineBlocks.from_blocks_dict(TEXT2IMAGE_BLOCKS) modular_repo_id YiYiXu/modular-loader-t2i-0704 pipeline blocks.init_pipeline(modular_repo_id) pipeline.load_components(dtypetorch.float16) pipeline.to(cuda) image pipeline(promptAstronaut in a jungle, cold color palette, muted colors, detailed, 8k, outputimages)[0] image.save(modular_t2i_out.png)IMAGE2IMAGE_BLOCKS预设的完整结构来自 quickstart.md展示了编码 → 输入 → 设置时间步 → 准备潜变量 → 附加条件 → 去噪 → 解码的典型流水线from diffusers.modular_pipelines.stable_diffusion_xl import IMAGE2IMAGE_BLOCKS IMAGE2IMAGE_BLOCKS InsertableDict([ (text_encoder, StableDiffusionXLTextEncoderStep), (image_encoder, StableDiffusionXLVaeEncoderStep), (input, StableDiffusionXLInputStep), (set_timesteps, StableDiffusionXLImg2ImgSetTimestepsStep), (prepare_latents, StableDiffusionXLImg2ImgPrepareLatentsStep), (prepare_add_cond, StableDiffusionXLImg2ImgPrepareAdditionalConditioningStep), (denoise, StableDiffusionXLDenoiseStep), (decode, StableDiffusionXLDecodeStep) ])灵活增删改块块是InsertableDict对象源码见 modular_pipeline_utils.py支持按位置插入、弹出与替换# 在块类字典或 sub_blocks 属性上插入 BLOCKS.insert(block_name, BlockClass, index) t2i_blocks.sub_blocks.insert(block_name, block_instance, index) # 移除 BLOCKS.pop(text_encoder) text_encoder_block t2i_blocks.sub_blocks.pop(text_encoder) # 替换 BLOCKS[prepare_latents] CustomPrepareLatents t2i_blocks.sub_blocks[prepare_latents] CustomPrepareLatents()两种创建管道的方式从块组装调用ModularPipelineBlocks.init_pipeline(repo_id, components_managercomponents)源码见 modular_pipeline.py。此方法从模块化仓库的modular_model_index.json加载规范但尚未加载模型。直接加载ModularPipeline.from_pretrained(repo_id, components_managercomponents)加载自定义管道时需加trust_remote_codeTrue。加载与检查组件ModularPipeline不会自动实例化组件需显式调用load_componentst2i_pipeline.load_components(dtypetorch.float16) # 加载全部 t2i_pipeline.load_components(names[unet, vae], dtypetorch.float16) # 只加载部分打印管道可检查已加载组件应与其初始化自的modular_model_index.json匹配若管道某步骤不需要某组件即使仓库中存在也不会加载。此外有四个属性反映组件加载状态component_names所有预期组件如[text_encoder, text_encoder_2, tokenizer, tokenizer_2, guider, scheduler, unet, vae, image_processor]null_component_names尚未加载的组件pretrained_component_names将从预训练模型加载的组件config_component_names使用默认配置创建的组件如guider、image_processor因此不出现在null_component_names中。修改组件加载来源可编辑模块化仓库中的modular_model_index.json例如将 UNet 来源从stabilityai/stable-diffusion-xl-base-1.0换成RunDiffusion/Juggernaut-XL-v9unet: [ null, null, { repo: RunDiffusion/Juggernaut-XL-v9, subfolder: unet, variant: fp16 } ]组件更新方面预训练组件通过ComponentSpec更新ComponentSpec(nameunet, type_hintUNet2DConditionModel, repo..., subfolderunet, variantfp16)然后spec.load(dtypetorch.float16)配置组件可直接传对象update_components(unetunet2)替换组件并同步更新加载规范get_component_spec(unet)可获取规范副本进行修改。模块化仓库结构模块化仓库与标准仓库不同它包含modular_model_index.json其中的library/class指明组件从哪个库加载为null表示未加载loading_specs_dict包含加载所需信息仓库、子文件夹等。组件可以按需从不同仓库分别获取不要求存在于同一仓库。仓库还可能包含自定义代码如block.pyconfig.json中的auto_map键指向自定义块的实现位置modular-diffdiff-0704/ ├── block.py # 自定义管道块实现 ├── config.json # 管道配置和 auto_map └── modular_model_index.json # 组件加载规范{ _class_name: DiffDiffBlocks, auto_map: { ModularPipelineBlocks: block.DiffDiffBlocks } }ComponentsManager跨管道管理组件与显存ComponentsManager是模块化 Diffusers 的模型注册与管理中枢添加和跟踪模型、存储元数据模型大小、设备放置、适配器、防止重复模型实例、支持卸载components_manager.py。它应与ModularPipeline一起创建在from_pretrained或init_pipeline中传入collection参数可选但有助于组织组件。组件注册与复用组件仅在调用load_components时加载并注册。利用这一点可以跨管道共享组件——第二个管道直接复用第一个管道的所有组件comp ComponentsManager() pipe ModularPipeline.from_pretrained(YiYiXu/modular-demo-auto, components_managercomp, collectiontest1) pipe.load_components() pipe2 ModularPipeline.from_pretrained(YiYiXu/modular-demo-auto, components_managercomp, collectiontest2)对缺失组件用pipe2.null_component_names找出、comp.get_components_by_names(names...)检索、pipe2.update_components(**comp_dict)补全。单个组件可用comp.add(text_encoder, text_encoder)注册返回带唯一 id 的名称用comp.remove(text_encoder_139917733042864)移除。检索方法get_one返回单个组件支持模式匹配多个匹配会报错。模式示例描述精确comp.get_one(nameunet)精确名称匹配通配comp.get_one(nameunet*)名称以 unet 开头排除comp.get_one(name!unet)排除名为 unet 的组件或comp.get_one(nameunet|vae)名称为 unet 或 vae还支持collection与load_id过滤comp.get_one(nameunet, collectionsdxl)。get_components_by_names接收名称列表返回名称→组件字典可直接喂给update_components。去重与集合去重检测推荐用ComponentSpec加载模型——其唯一 id 编码了加载参数管理器可自动识别不同对象代表同一底层检查点的重复实例并发出警告附带components_manager.remove(component_id)的移除指引。若将同一组件手动加载到不同对象再add则无法检测重复此时应回归ComponentSpec方式。集合collection是分配给组件的标签。每个集合中每个名称只允许一个组件添加同名的第二个会自动替换第一个。这非常适合基于节点的系统用collection标记某节点加载的全部模型、新检查点同名加载时自动替换、节点移除时批量删除。卸载comp.enable_auto_cpu_offload(devicecuda)提供全局卸载策略——所有模型初始在 CPU管理器按需移到目标设备GPU 显存不足时把其他模型移回 CPU并可自定义卸载规则。实战用块组装一个 Differential Diffusion 管道quickstart.md 以Differential Diffusion差分扩散为例完整演示了从预设块出发、定制两个块、组装成ModularPipeline并扩展工作流的全流程。该工作流是图像到图像任务与标准图生图仅在两处不同prepare_latents与denoise块其余块全部可复用。定制 prepare_latents 块需要三项改动新增处理变化图change map的处理器、新增inputs用户提供的变化图diffdiff_map、用于预计算潜变量的timesteps、用于创建更新区域掩码的num_inference_steps、在__call__中处理变化图并生成掩码存入BlockStateclass SDXLDiffDiffPrepareLatentsStep(ModularPipelineBlocks): property def expected_components(self) - List[ComponentSpec]: return [ ComponentSpec(vae, AutoencoderKL), ComponentSpec(scheduler, EulerDiscreteScheduler), ComponentSpec(mask_processor, VaeImageProcessor, configFrozenDict({do_normalize: False, do_convert_grayscale: True})) ] property def inputs(self) - List[Tuple[str, Any]]: return [ InputParam(generator), InputParam(diffdiff_map, requiredTrue), - InputParam(latent_timestep, requiredTrue, type_hinttorch.Tensor), InputParam(timesteps, type_hinttorch.Tensor), InputParam(num_inference_steps, type_hintint), ] property def intermediate_outputs(self) - List[OutputParam]: return [ OutputParam(original_latents, type_hinttorch.Tensor), OutputParam(diffdiff_masks, type_hinttorch.Tensor), ] def __call__(self, components, state: PipelineState): # ... existing logic ... diffdiff_map components.mask_processor.preprocess(block_state.diffdiff_map, heightlatent_height, widthlatent_width) thresholds torch.arange(block_state.num_inference_steps, dtypediffdiff_map.dtype) / block_state.num_inference_steps block_state.diffdiff_masks diffdiff_map (thresholds (block_state.denoising_start or 0)) block_state.original_latents block_state.latents定制 denoise 块打印denoise块可知它由LoopSequentialPipelineBlocks组成含before_denoiser、denoiser、after_denoiser三个子块只需修改before_denoiser子块按变化图为去噪器准备潜变量输入class SDXLDiffDiffLoopBeforeDenoiser(ModularPipelineBlocks): property def inputs(self) - List[str]: return [ InputParam(latents, requiredTrue, type_hinttorch.Tensor), InputParam(denoising_start), InputParam(original_latents, type_hinttorch.Tensor), InputParam(diffdiff_masks, type_hinttorch.Tensor), ] def __call__(self, components, block_state, i, t): if i 0 and block_state.denoising_start is None: block_state.latents block_state.original_latents[:1] else: block_state.mask block_state.diffdiff_masks[i].unsqueeze(0).unsqueeze(1) block_state.latents block_state.original_latents[i] * block_state.mask block_state.latents * (1 - block_state.mask) # ... rest of existing logic ...然后构造新的去噪包装器仅替换before_denoiser子块class SDXLDiffDiffDenoiseStep(StableDiffusionXLDenoiseLoopWrapper): block_classes [SDXLDiffDiffLoopBeforeDenoiser, StableDiffusionXLLoopDenoiser, StableDiffusionXLLoopAfterDenoiser] block_names [before_denoiser, denoiser, after_denoiser]组装并初始化管道复制IMAGE2IMAGE_BLOCKS预设替换set_timesteps差分扩散不需要strength因此改用TEXT2IMAGE_BLOCKS的版本、prepare_latents与denoise再用SequentialPipelineBlocks.from_blocks_dict组装DIFFDIFF_BLOCKS IMAGE2IMAGE_BLOCKS.copy() DIFFDIFF_BLOCKS[set_timesteps] TEXT2IMAGE_BLOCKS[set_timesteps] DIFFDIFF_BLOCKS[prepare_latents] SDXLDiffDiffPrepareLatentsStep DIFFDIFF_BLOCKS[denoise] SDXLDiffDiffDenoiseStep dd_blocks SequentialPipelineBlocks.from_blocks_dict(DIFFDIFF_BLOCKS) print(dd_blocks)用init_pipeline转换为ModularPipeline从modular_model_index.json加载预期组件同时传入ComponentsManager以共享组件from diffusers.modular_pipelines import ComponentsManager components ComponentsManager() dd_pipeline dd_blocks.init_pipeline(YiYiXu/modular-demo-auto, components_managercomponents, collectiondiffdiff) dd_pipeline.load_default_componenets(dtypetorch.float16) dd_pipeline.to(cuda)扩展工作流 1插入 IP-AdapterSDXL 已有预设的 IP-Adapter 块无需改动现有差分扩散管道直接插入即可from diffusers.modular_pipelines.stable_diffusion_xl.encoders import StableDiffusionXLAutoIPAdapterStep ip_adapter_block StableDiffusionXLAutoIPAdapterStep() dd_blocks.sub_blocks.insert(ip_adapter, ip_adapter_block, 0) # 位置 0 插入 dd_pipeline dd_blocks.init_pipeline(YiYiXu/modular-demo-auto, collectiondiffdiff) dd_pipeline.load_components(dtypetorch.float16) dd_pipeline.loader.load_ip_adapter(h94/IP-Adapter, subfoldersdxl_models, weight_nameip-adapter_sdxl.bin) dd_pipeline.loader.set_ip_adapter_scale(0.6) dd_pipeline dd_pipeline.to(device) image dd_pipeline( prompta green pear, negative_promptblurry, num_inference_steps25, generatorgenerator, ip_adapter_imageip_adapter_image, diffdiff_mapmask, imageimage, outputimages )[0]插入后管道会多出image_encoder与feature_extractor两个组件并新增ip_adapter_image输入。扩展工作流 2接入 ControlNetControlNet 需要在denoise块中注入控制信息因此要替换denoiser子块为StableDiffusionXLControlNetLoopDenoiserfrom diffusers.modular_pipelines.stable_diffusion_xl.modular_blocks import StableDiffusionXLAutoControlNetInputStep control_input_block StableDiffusionXLAutoControlNetInputStep() class SDXLDiffDiffControlNetDenoiseStep(StableDiffusionXLDenoiseLoopWrapper): block_classes [SDXLDiffDiffLoopBeforeDenoiser, StableDiffusionXLControlNetLoopDenoiser, StableDiffusionXLDenoiseLoopAfterDenoiser] block_names [before_denoiser, denoiser, after_denoiser] controlnet_denoise_block SDXLDiffDiffControlNetDenoiseStep() dd_blocks.sub_blocks.insert(controlnet_input, control_input_block, 7) dd_blocks.sub_blocks[denoise] controlnet_denoise_block dd_pipeline dd_blocks.init_pipeline(YiYiXu/modular-demo-auto, collectiondiffdiff) dd_pipeline.load_components(dtypetorch.float16) dd_pipeline dd_pipeline.to(device) image dd_pipeline( prompta green pear, negative_promptblurry, num_inference_steps25, generatorgenerator, control_imagecontrol_image, controlnet_conditioning_scale0.5, diffdiff_mapmask, imageimage, outputimages )[0]扩展工作流 3用 AutoPipelineBlocks 打包所有工作流差分扩散、IP-Adapter、ControlNet 三种工作流可通过AutoPipelineBlocks捆绑到一个ModularPipeline中根据输入control_image或ip_adapter_image自动选择均未提供时默认差分扩散class SDXLDiffDiffAutoDenoiseStep(AutoPipelineBlocks): block_classes [SDXLDiffDiffControlNetDenoiseStep, SDXLDiffDiffDenoiseStep] block_names [controlnet_denoise, denoise] block_trigger_inputs [controlnet_cond, None] DIFFDIFF_AUTO_BLOCKS IMAGE2IMAGE_BLOCKS.copy() DIFFDIFF_AUTO_BLOCKS[prepare_latents] SDXLDiffDiffPrepareLatentsStep DIFFDIFF_AUTO_BLOCKS[set_timesteps] TEXT2IMAGE_BLOCKS[set_timesteps] DIFFDIFF_AUTO_BLOCKS[denoise] SDXLDiffDiffAutoDenoiseStep DIFFDIFF_AUTO_BLOCKS.insert(ip_adapter, StableDiffusionXLAutoIPAdapterStep, 0) DIFFDIFF_AUTO_BLOCKS.insert(controlnet_input, StableDiffusionXLControlNetAutoInput, 7) dd_auto_blocks SequentialPipelineBlocks.from_blocks_dict(DIFFDIFF_AUTO_BLOCKS) dd_pipeline dd_auto_blocks.init_pipeline(YiYiXu/modular-demo-auto, collectiondiffdiff) dd_pipeline.load_components(dtypetorch.float16)分享与加载用save_pretrained将管道发布到 Hubpush_to_hubTrue其他用户用from_pretrained加载自定义管道需trust_remote_codeTruedd_pipeline.save_pretrained(YiYiXu/test_modular_doc, push_to_hubTrue)import torch from diffusers.modular_pipelines import ModularPipeline, ComponentsManager components ComponentsManager() diffdiff_pipeline ModularPipeline.from_pretrained(YiYiXu/modular-diffdiff-0704, trust_remote_codeTrue, components_managercomponents, collectiondiffdiff) diffdiff_pipeline.load_components(dtypetorch.float16)自定义块的工程化实践除了复制现有块做修改官方还提供了从模板起步的自定义块工作流见英文文档 custom_blocks.md项目结构只需block.pymodular_config.json。可用ModularPipelineBlocks.from_pretrained(diffusers/custom-block-template, trust_remote_codeTrue, local_dir...)下载模板编辑block.py实现expected_components、inputs、intermediate_outputs与__call__后init_pipeline()即可运行测试save_pretrained(..., push_to_hubTrue)发布。自定义块还支持通过_requirements属性声明依赖如{transformers: 4.44.0}保存时写入modular_config.json加载时 diffusers 会校验环境并给出缺失/版本不匹配警告多个块的依赖会自动合并。文档脉络与后续深入模块化 Diffusers 的完整文档按快速开始 → 块类型 → 管道与组件组织入口见 overview.md本文对应内容均可在以下仓库路径找到原文与实现快速开始docs/source/zh/modular_diffusers/quickstart.mdDifferential Diffusion 完整实战状态机制docs/source/zh/modular_diffusers/modular_diffusers_states.md四种块类型pipeline_block.md、sequential_pipeline_blocks.md、loop_sequential_pipeline_blocks.md、auto_pipeline_blocks.md管道与组件管理modular_pipeline.md、components_manager.md核心源码src/diffusers/modular_pipelines/modular_pipeline.py、modular_pipeline_utils.py、components_manager.py以及各模型的模块化实现目录如 stable_diffusion_xl/、flux/测试佐证tests/modular_pipelines/ 下的组件管理与加载测试如 test_components_manager.py、test_modular_pipeline_loading.py通过预设块 少量定制 自由组合模块化 Diffusers 让为每个新需求重写整条管道成为过去式——这正是其作为统一管道系统的价值所在。【免费下载链接】diffusers Diffusers: State-of-the-art diffusion models for image, video, and audio generation in PyTorch.项目地址: https://gitcode.com/GitHub_Trending/di/diffusers创作声明：本文部分内容由AI辅助生成（AIGC），仅供参考