这篇文章主要讲Windows下利用unsloth对大语言模型进行微调这样可以针对特定的小行业小场景做定制服务。训练用到的python版本是3.10.6 64位由于我本地系统盘小所以我将python的临时下载目录设置到了E:/pip_cache里空间够的朋友可不用加。创建虚拟环境python -m venv E:/newtrain_env Set-ExecutionPolicy -ExecutionPolicy RemoteSigned -Scope CurrentUser #执行策略改为Y,然后确定 #然后激活环境 E:/newtrain_env/Scripts/activate.ps1设置临时目录环境变量#(仅对当前窗口有效建议每次安装前执行) mkdir E:/pip_temp mkdir E:/pip_cache $env:TMP E:/pip_temp $env:TEMP E:/pip_temp如何需求使用清华镜像源加速 可选pip config set global.index-url https://pypi.tuna.tsinghua.edu.cn/simple升级 Python 的核心安装工具链python -m pip install --upgrade pip setuptools wheel --cache-dir E:/pip_cache安装依赖库需要先安装一些必要依赖库如triton-windowsxformers等。pip install triton-windows3.6.0.post25 --cache-dir E:/pip_cache pip install bitsandbytes xformers accelerate datasets peft sentencepiece protobuf --cache-dir E:/pip_cache pip install torch2.5.1 torchvision0.20.1 torchaudio2.5.1 --index-url https://download.pytorch.org/whl/cu121 --cache-dir E:/pip_cache pip install trl0.24.0 --no-deps --cache-dir E:/pip_cache安装unsloth 和 unsloth_zoopip install unsloth githttps://github.com/unslothai/unsloth.git --cache-dir E:/pip_cache pip install unsloth_zoo githttps://github.com/unslothai/unsloth-zoo.git --no-deps --cache-dir E:/pip_cache如何出现这个问题from .rl import PatchFastRL File E:/newtrain_env/lib/site-packages/unsloth_zoo/temporary_patches/deepseek_v3_moe.py, line 271, in patched_import module original_import(name, globals, locals, fromlist, level) File E:/newtrain_env/lib/site-packages/unsloth/models/rl.py, line 289, in module sanitize_logprob RL_REPLACEMENTS[sanitize_logprob] KeyError: sanitize_logprob重新安装unsloth-zoo采用–no-deps不然会导致他升级torch一些依赖库导致版本冲突异常。(newtrain_env) PS E:/newtrain_env pip uninstall unsloth-zoo (newtrain_env) PS E:/newtrain_env pip install unsloth_zoo githttps://github.com/unslothai/unsloth-zoo.git --no-deps --cache-dir E:/pip_cache本地可运行依赖库完整介绍安装结束后我本地依赖库版本信息执行命令pip list可获得accelerate1.12.0 aiohappyeyeballs2.6.1 aiohttp3.13.3 aiosignal1.4.0 anyio4.12.1 async-timeout5.0.1 attrs25.4.0 bitsandbytes0.49.2 certifi2026.2.25 charset-normalizer3.4.4 colorama0.4.6 cut-cross-entropy25.1.1 datasets4.3.0 dill0.4.0 docstring_parser0.17.0 exceptiongroup1.3.1 filelock3.20.0 frozenlist1.8.0 fsspec2025.12.0 h110.16.0 hf_transfer0.1.9 httpcore1.0.9 httpx0.28.1 huggingface_hub0.36.2 idna3.11 Jinja23.1.6 joblib1.5.3 MarkupSafe3.0.2 mpmath1.3.0 msgspec0.20.0 multidict6.7.1 multiprocess0.70.16 networkx3.4.2 numpy2.2.6 packaging26.0 pandas2.3.3 peft0.18.1 pillow12.0.0 propcache0.4.1 protobuf7.34.0 psutil7.2.2 pyarrow23.0.1 python-dateutil2.9.0.post0 pytz2025.2 PyYAML6.0.3 regex2026.2.28 requests2.32.5 safetensors0.7.0 scikit-learn1.7.2 scipy1.15.3 sentence-transformers5.2.3 sentencepiece0.2.1 six1.17.0 sympy1.13.1 threadpoolctl3.6.0 tokenizers0.22.2 torch2.5.1cu121 torchaudio2.5.1cu121 torchvision0.20.1cu121 tqdm4.67.3 transformers4.57.6 triton-windows3.6.0.post25 trl0.24.0 typeguard4.5.1 typing_extensions4.15.0 tyro1.0.8 tzdata2025.3 unsloth githttps://github.com/unslothai/unsloth.git9b56d63337c01e9c778a8f6dff9349ce60bfda98 unsloth_zoo githttps://github.com/unslothai/unsloth-zoo.git5b80d552999687023063562271b101484f1cc5f8 urllib32.6.3 xformers0.0.35 xxhash3.6.0 yarl1.23.0准备训练数据集在训练之前要先训练数据集准备好训练集结构格式如下{instruction: 你是一名资深的神经科医生。请根据提供的患者主诉进行症状分析、病变定位并给出可能的诊断方向。, input: 患者男65岁。突发右侧肢体无力伴言语蹇涩2小时既往有高血压病史10年。, output: 【症状分析】患者表现为急性起病的局灶性神经功能缺损。右侧肢体无力偏瘫提示皮层或内囊受累言语蹇涩运动性失语提示优势半球通常为左侧额叶Broca区受累。/n/n【定位诊断】左侧大脑中动脉供血区皮层或深部白质。/n/n【定性诊断】急性起病老年男性伴高血压危险因素考虑缺血性脑卒中可能性大。/n/n【建议检查】紧急头颅CT平扫排除出血必要时行MRI(DWI)及血管评估。} {instruction: 你是一名资深的神经科医生。请根据提供的患者主诉进行症状分析、病变定位并给出可能的诊断方向。, input: 患者女45岁。双下肢麻木、无力3天伴排尿困难。1周前曾有“感冒”史。查体双下肢肌力3级胸4平面对称性痛觉减退双侧Babinski征()。, output: 【症状分析】双下肢瘫痪伴感觉平面T4及尿潴留提示脊髓横贯性损害。Babinski征()提示病变累及锥体束。/n/n【定位诊断】胸髓T4水平。/n/n【定性诊断】青年女性急性/亚急性起病有前驱感染史表现为脊髓横贯损害首选急性脊髓炎。/n/n【建议检查】脊髓MRI增强、脑脊液检查压力、细胞、蛋白、寡克隆带、AQP4抗体检测。} {instruction: 你是一名资深的神经科医生。请根据提供的患者主诉进行症状分析、病变定位并给出可能的诊断方向。, input: 患者男52岁。发作性右上肢抽搐伴意识丧失1次。家属描述患者先是右手不自主抖动随后波及全身持续约2分钟缓解事后不能回忆。, output: 【症状分析】右上肢起始抽搐提示左侧运动皮层异常放电。继发全身抽搐及意识丧失属于局灶性发作演变为双侧强直-阵挛发作GTCS。/n/n【定位诊断】左侧额叶中央前回运动区。/n/n【定性诊断】成年新发癫痫发作需警惕继发性因素如肿瘤、血管畸形。/n/n【建议检查】头颅MRI增强、长程脑电图EEG、血生化排除电解质紊乱。}为什么要这样设计字段呢Instruction指令定义角色为“神经科医生”。这会强制模型在 1.5B 的参数库里调用医学相关的权重避免口语化。Input输入放置病史和症状。这是模型分析的基础。Output输出必须包含逻辑链。不要直接给诊断结论如脑梗死,要让模型输出“分析”和“定位”的过程。这种“思维链CoT”微调能极大地提高 1.5B 小模型的逻辑准确度。形象的比喻场景相当于指令 (Instruction)输入 (Input)输出 (Output)随笔微调写随笔“请写一篇日志”(留空随便你写什么)“【标题】冬至来了…”神经科微调做数学题“请计算这道题”“1 1 ?”“等于 2”训练训练模型至少6g以上的显存不然跑不太动这里我也只是以最小模型做测试这里选择模型Qwen2.5-1.5B进行微调代码参考如下import os # --- 1. 彻底禁用导致报错的编译与优化特性 --- os.environ[UNSLOTH_USE_FUSED_CE] 0 # 禁用融合损失避开 triton 调用 os.environ[TORCH_COMPILE_DISABLE] 1 # 强制禁用 torch 编译解决 triton_key 报错 os.environ[HF_HOME] E:/huggingface_cache os.environ[HF_ENDPOINT] https://hf-mirror.com from unsloth import FastLanguageModel import torch from datasets import load_dataset from trl import SFTTrainer from transformers import TrainingArguments # 2. 配置 1.5B 模型 (非常适合 6GB 显存测试) model, tokenizer FastLanguageModel.from_pretrained( model_name unsloth/Qwen2.5-1.5B-Instruct-bnb-4bit, max_seq_length 1024, # 1.5B 模型我们可以把长度放宽到 1024 load_in_4bit True, ) # 3. 添加 LoRA 适配器 model FastLanguageModel.get_peft_model( model, r 16, target_modules [q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj,], lora_alpha 16, lora_dropout 0, bias none, use_gradient_checkpointing unsloth, random_state 3407, ) # --- 4. 数据集处理 (Alpaca 格式) --- alpaca_prompt Below is an instruction that describes a task. ### Instruction: {} ### Input: {} ### Response: {} EOS_TOKEN tokenizer.eos_token def formatting_prompts_func(examples): instructions examples[instruction] inputs examples[input] outputs examples[output] texts [] for instruction, input, output in zip(instructions, inputs, outputs): text alpaca_prompt.format(instruction, input, output) EOS_TOKEN texts.append(text) return { text : texts, } dataset load_dataset(json, data_files {train: rE:/train_data_creative.jsonl}, split train) dataset dataset.map(formatting_prompts_func, batched True) # --- 5. 设置训练参数 (稳健配置) --- trainer SFTTrainer( model model, tokenizer tokenizer, train_dataset dataset, dataset_text_field text, max_seq_length 1024, args TrainingArguments( per_device_train_batch_size 2, # 1.5B 模型可以尝试 batch_size2 gradient_accumulation_steps 4, warmup_steps 5, max_steps 30, # 跑 30 步快速验证 learning_rate 2e-4, fp16 True, logging_steps 1, optim adamw_8bit, weight_decay 0.01, torch_compile False, # 禁用编译以防 Windows 报错 gradient_checkpointing True, output_dir E:/outputs, ), ) # 6. 开始验证训练 print( 正在启动 1.5B 模型测试训练...) trainer.train() print( 训练成功完成恭喜你打通了本地训练全流程)看到如下输出就说明正在训练了 Unsloth Zoo will now patch everything to make training faster! [unsloth_zoo.log|WARNING]Unsloth: Could not patch trl.trainer.grpo_trainer: Direct module loading failed for UnslothGRPOTrainer: Unexpected optimization option triton.enable_persistent_tma_matmul, known options are [TYPE_CHECKING, enable_auto_functionalized_v2, debug, disable_progress, verbose_progress, fx_graph_cache, fx_graph_remote_cache, autotune_local_cache, autotune_remote_cache, force_disable_caches, sleep_sec_TESTING_ONLY, custom_op_default_layout_constraint, cpp_wrapper, abi_compatible, c_shim_version, dce, static_weight_shapes, size_asserts, nan_asserts, pick_loop_orders, inplace_buffers, allow_buffer_reuse, memory_planning, memory_pool, benchmark_harness, epilogue_fusion, epilogue_fusion_first, pattern_matcher, b2b_gemm_pass, post_grad_custom_pre_pass, post_grad_custom_post_pass, joint_custom_pre_pass, joint_custom_post_pass, pre_grad_custom_pass, _pre_fusion_custom_pass, split_cat_fx_passes, efficient_conv_bn_eval_fx_passes, is_predispatch, group_fusion, batch_fusion, pre_grad_fusion_options, post_grad_fusion_options, reorder_for_locality, dynamic_scale_rblock, force_fuse_int_mm_with_mul, use_mixed_mm, fx_passes_numeric_check, mixed_mm_choice, reorder_for_compute_comm_overlap, reorder_for_compute_comm_overlap_passes, estimate_op_runtime, intra_node_bw, inter_node_bw, max_autotune, max_autotune_pointwise, max_autotune_gemm, force_same_precision, max_autotune_gemm_backends, max_autotune_conv_backends, max_autotune_gemm_search_space, autotune_fallback_to_aten, unbacked_symint_fallback, search_autotune_cache, save_args, autotune_in_subproc, max_autotune_subproc_result_timeout_seconds, max_autotune_subproc_graceful_timeout_seconds, max_autotune_subproc_terminate_timeout_seconds, autotune_multi_device, coordinate_descent_tuning, coordinate_descent_check_all_directions, coordinate_descent_search_radius, autoheuristic_collect, autoheuristic_use, autoheuristic_log_path, layout_opt_default, layout_optimization, force_layout_optimization, keep_output_stride, warn_mix_layout, realize_reads_threshold, realize_opcount_threshold, realize_acc_reads_threshold, fallback_random, implicit_fallbacks, aggressive_fusion, debug_fusion, benchmark_fusion, enabled_metric_tables, loop_ordering_after_fusion, benchmark_epilogue_fusion, max_epilogue_benchmarked_choices, max_fusion_size, max_pointwise_cat_inputs, unroll_reductions_threshold, comment_origin, conv_1x1_as_mm, split_reductions, benchmark_kernel, constant_and_index_propagation, always_keep_tensor_constants, assert_indirect_indexing, compute_all_bounds, combo_kernels, benchmark_combo_kernel, combo_kernels_autotune, combo_kernel_allow_mixed_sizes, combo_kernel_foreach_dynamic_shapes, joint_graph_constant_folding, debug_index_asserts, emulate_precision_casts, is_nightly_or_source, developer_warnings, optimize_scatter_upon_const_tensor, worker_start_method, _fuse_ddp_communication, _fuse_ddp_bucket_size, _fuse_ddp_communication_passes, _micro_pipeline_tp, compile_threads, global_cache_dir, kernel_name_max_ops, shape_padding, comprehensive_padding, pad_channels_last, disable_padding_cpu, padding_alignment_bytes, padding_stride_threshold, pad_outputs, bw_outputs_user_visible, force_shape_pad, permute_fusion, profiler_mark_wrapper_call, generate_intermediate_hooks, debug_ir_traceback, _raise_error_for_testing, _profile_var, profile_bandwidth, profile_bandwidth_regex, profile_bandwidth_output, profile_bandwidth_with_do_bench_using_profiling, disable_cpp_codegen, freezing, freezing_discard_parameters, allow_stack_allocation, use_minimal_arrayref_interface, decompose_mem_bound_mm, assume_aligned_inputs, unsafe_ignore_unsupported_triton_autotune_args, check_stack_no_cycles_TESTING_ONLY, cpp.threads, cpp.no_redundant_loops, cpp.dynamic_threads, cpp.simdlen, cpp.min_chunk_size, cpp.cxx, cpp.enable_kernel_profile, cpp.weight_prepack, cpp.inject_relu_bug_TESTING_ONLY, cpp.inject_log1p_bug_TESTING_ONLY, cpp.vec_isa_ok, cpp.descriptive_names, cpp.max_horizontal_fusion_size, cpp.fallback_scatter_reduce_sum, cpp.enable_unsafe_math_opt_flag, cpp.enable_floating_point_contract_flag, cpp.enable_tiling_heuristics, cpp.gemm_max_k_slices, cpp.gemm_cache_blocking, cpp.gemm_thread_factors, cpp.enable_loop_tail_vec, triton.cudagraphs, triton.cudagraph_trees, triton.cudagraph_skip_dynamic_graphs, triton.slow_path_cudagraph_asserts, triton.cudagraph_trees_history_recording, triton.cudagraph_support_input_mutation, triton.cudagraph_unexpected_rerecord_limit, triton.cudagraph_dynamic_shape_warn_limit, triton.force_cudagraph_sync, triton.force_cudagraphs_warmup, triton.fast_path_cudagraph_asserts, triton.skip_cudagraph_warmup, triton.debug_sync_graph, triton.debug_sync_kernel, triton.dense_indexing, triton.max_tiles, triton.prefer_nd_tiling, triton.autotune_pointwise, triton.autotune_cublasLt, triton.autotune_at_compile_time, triton.tiling_prevents_pointwise_fusion, triton.tiling_prevents_reduction_fusion, triton.unique_kernel_names, triton.descriptive_names, triton.persistent_reductions, triton.multi_kernel, triton.divisible_by_16, triton.min_split_scan_rblock, triton.store_cubin, triton.spill_threshold, triton.use_block_ptr, triton.inject_relu_bug_TESTING_ONLY, triton.codegen_upcast_to_fp32, aot_inductor.output_path, aot_inductor.debug_compile, aot_inductor.debug_dump_consts_bin, aot_inductor.debug_intermediate_value_printer, aot_inductor.filtered_kernel_names, aot_inductor.serialized_in_spec, aot_inductor.serialized_out_spec, aot_inductor.use_runtime_constant_folding, aot_inductor.force_mmap_weights, aot_inductor.package, cuda.arch, cuda.version, cuda.compile_opt_level, cuda.enable_cuda_lto, cuda.enable_ptxas_info, cuda.enable_debug_info, cuda.use_fast_math, cuda.cutlass_dir, cuda.cutlass_max_profiling_configs, cuda.cuda_cxx, cuda.cutlass_backend_min_gemm_size, cuda.generate_test_runner, cuda.cutlass_op_allowlist_regex, cuda.cutlass_op_denylist_regex, rocm.arch, rocm.ck_supported_arch, rocm.compile_opt_level, rocm.is_debug, rocm.save_temps, rocm.use_fast_math, rocm.flush_denormals, rocm.print_kernel_resource_usage, rocm.rocm_home, rocm.ck_dir, rocm.n_max_profiling_configs, rocm.use_preselected_instances, cpu_backend, cuda_backend, halide.cpu_target, halide.gpu_target, halide.scheduler_cuda, halide.scheduler_cpu, halide.asserts, halide.debug, halide.scan_kernels, trace.enabled, trace.debug_dir, trace.debug_log, trace.info_log, trace.fx_graph, trace.fx_graph_transformed, trace.ir_pre_fusion, trace.ir_post_fusion, trace.output_code, trace.graph_diagram, trace.draw_orig_fx_graph, trace.dot_graph_shape, trace.log_url_for_graph_xform, trace.compile_profile, trace.upload_tar, trace.log_autotuning_results, _save_config_ignore, _cache_config_ignore_prefix] (()) Unsloth 2026.2.1: Fast Qwen2 patching. Transformers: 4.57.6. // /| Num GPUs 1. Max memory: 6.0 GB. Platform: Windows. O^O/ /_/ / Torch: 2.5.1cu121. CUDA: 7.5. CUDA Toolkit: 12.1. Triton: 3.6.0 / / Bfloat16 FALSE. FA [Xformers None. FA2 False] -____- Free license: http://github.com/unslothai/unsloth Unsloth: Fast downloading is enabled - ignore downloading bars which are red colored! unsloth/qwen2.5-1.5b-instruct-bnb-4bit does not have a padding token! Will use pad_token |PAD_TOKEN|. Unsloth 2026.2.1 patched 28 layers with 28 QKV layers, 28 O layers and 28 MLP layers. Generating train split: 390 examples [00:00, 31617.80 examples/s] Map: 100%|███████████████████████████████████████████████████ ███████| 390/390 [00:0000:00, 20143.32 examples/s] Unsloth: Tokenizing [text]: 100%|███████████████████████████████████| 390/39Unsloth: Tokenizing [text]: 100%|███████████████████████████████████| 390/390 [00:0000:00, 449.94 examples/s] 正在启动 1.5B 模型测试训练... (()) Unsloth - 2x faster free finetuning | Num GPUs used 1 // /| Num examples 390 | Num Epochs 1 | Total steps 30 O^O/ /_/ / Batch size per device 2 | Gradient accumulation steps 4 / / Data Parallel GPUs 1 | Total batch size (2 x 4 x 1) 8 -____- Trainable parameters 18,464,768 of 1,562,179,072 (1.18% trained) 0%| | 0/30 [00:00?, ?it/s]Unsloth: Will smartly offload gradients to save VRAM! 3%|██▍ | 1/30 [02:221:08:54, 142.57s/i{loss: 3.1134, grad_norm: 0.26568782329559326, learning_rate: 0.0, epoch: 0.02} 3%|██▍ | 1/30测试然后耐心等微调模型训练结束之后会在E:/outputs 文件夹里生成微调后的模型然后可以与原来的模型做下测试。测试代码也一起提供如下from unsloth import FastLanguageModel import torch # 准备相同的问题和模板 #alpaca_prompt ### Instruction:/n{}/n/n### Input:/n{}/n/n### Response:/n{} #question 你是一个专业的神经科医生请对如下症状*做下分析总结。 #inputs_text alpaca_prompt.format(question, , ) instruction 你是一名资深的神经科医生。请根据提供的患者主诉进行症状分析、病变定位并给出可能的诊断方向。 test_input 患者男60岁。突发左侧肢体麻木伴口角歪斜1小时既往有糖尿病史。 # 构造完整的推理输入 inputs_text alpaca_prompt.format(instruction, test_input) # --- 第一部分用【新脑袋】测试 --- print(/n *20 【新模型 (已微调)】 *20) model, tokenizer FastLanguageModel.from_pretrained( model_name E:/outputs/checkpoint-30, max_seq_length 512, load_in_4bit True, ) FastLanguageModel.for_inference(model) inputs tokenizer([inputs_text], return_tensors pt).to(cuda) outputs model.generate(inputs, max_new_tokens 1000) print(tokenizer.batch_decode(outputs)[0].split(### Response:)[1]) # 释放显存准备加载下一个 del model torch.cuda.empty_cache() # --- 第二部分用【旧脑袋】对比 --- print(/n *20 【原模型 (底座)】 *20) model, tokenizer FastLanguageModel.from_pretrained( model_name unsloth/Qwen2.5-1.5B-Instruct-bnb-4bit, max_seq_length 512, load_in_4bit True, ) FastLanguageModel.for_inference(model) inputs tokenizer([inputs_text], return_tensors pt).to(cuda) outputs model.generate(inputs, max_new_tokens 1000) print(tokenizer.batch_decode(outputs)[0].split(### Response:)[1])写在最后当整个流程跑通之后你就可以针对自己的行业、自己的业务场景去做定制化微调。不管是医疗问答、企业内部知识库还是某个垂直领域的专业助手都可以通过这种方式不断“调教”模型让它更贴近实际需求。模型不一定要多大但只要它足够懂你的场景就已经是一种非常高效的辅助工具了。这里给大家精心整理了一份全面的AI大模型学习资源包括AI大模型全套学习路线图从入门到实战、精品AI大模型学习书籍手册、视频教程、实战学习、面试题等资料免费分享扫码免费领取全部内容1. 成长路线图学习规划要学习一门新的技术作为新手一定要先学习成长路线图方向不对努力白费。这里我们为新手和想要进一步提升的专业人士准备了一份详细的学习成长路线图和规划。可以说是最科学最系统的学习成长路线。2. 大模型经典PDF书籍书籍和学习文档资料是学习大模型过程中必不可少的我们精选了一系列深入探讨大模型技术的书籍和学习文档它们由领域内的顶尖专家撰写内容全面、深入、详尽为你学习大模型提供坚实的理论基础。书籍含电子版PDF3. 大模型视频教程对于很多自学或者没有基础的同学来说书籍这些纯文字类的学习教材会觉得比较晦涩难以理解因此我们提供了丰富的大模型视频教程以动态、形象的方式展示技术概念帮助你更快、更轻松地掌握核心知识。4. 2026行业报告行业分析主要包括对不同行业的现状、趋势、问题、机会等进行系统地调研和评估以了解哪些行业更适合引入大模型的技术和应用以及在哪些方面可以发挥大模型的优势。5. 大模型项目实战学以致用当你的理论知识积累到一定程度就需要通过项目实战在实际操作中检验和巩固你所学到的知识同时为你找工作和职业发展打下坚实的基础。6. 大模型面试题面试不仅是技术的较量更需要充分的准备。在你已经掌握了大模型技术之后就需要开始准备面试我们将提供精心整理的大模型面试题库涵盖当前面试中可能遇到的各种技术问题让你在面试中游刃有余。7. 资料领取全套内容免费抱走学 AI 不用再找第二份不管你是 0 基础想入门 AI 大模型还是有基础想冲刺大厂、了解行业趋势这份资料都能满足你现在只需按照提示操作就能免费领取扫码免费领取全部内容