Sample Usage Guide【免费下载链接】geGEGraph Engine是面向昇腾的图编译器和执行器提供了计算图优化、多流并行、内存复用和模型下沉等技术手段加速模型执行效率减少模型内存占用。 GE 提供对 PyTorch、TensorFlow 前端的友好接入能力并同时支持 onnx、pb 等主流模型格式的解析与编译。项目地址: https://gitcode.com/cann/geFeature DescriptionThis sample demonstrates a custom pass for removing add-zero operations, providing two verification methods: online inference and ATC offline model compilation. The sample uses eager style API and fusion interface.Directory Structure├── src │ ├──add_zero_pass.cpp // pass implementation file ├── CMakeLists.txt // build script ├── data | ├──torch_gen_onnx.py // torch script for exporting onnx | ├──torch_forward.py // torch script for online inference |—— gen_es_api | |——CMakeLists.txt // build script for generating eager style apiEnvironment RequirementsCompiler: GCC 7.3.xPython and dependencies: python3.9, pytorch2.1Environment preparation completed.Implementation StepsDefine classAddZeroPassinheriting fromPatternFusionPass.Override three functions from base classPatternFusionPass:Patternsdefines matching templates for identifying topologies matching the template in the graph.MeetRequirementsfilters topologies matched by template.Replacementdefines replacement part.RegisterAddZeroPassas custom fusion pass with execution phase BeforeInferShape.Program CompilationAssume CANN software package installation directory is INSTALL_PATH, e.g.,/home/HwHiAiUser/Ascend/.Configure environment variables.Run environment setup script from software package:source ${ASCEND_PATH}/set_env.sh${ASCEND_PATH}is cann path under CANN software package installation directory. Replace with actual installation path, e.g.,${INSTALL_PATH}/cann.ModifyCMakeLists.txtas needed.ASCEND_PATH: Default software package path. If$ASCEND_HOME_PATHset via set_env.sh, no modification needed.PASS_SO_DIR: Custom fusion pass dynamic library installation directory name, defaultpass_so_dir.target_include_directories: Required header files. For this sample, no modification needed. For custom development, add header files below the example without deleting existing items. If network has custom operators, add custom operator prototype definition headers.target_link_libraries: Required libraries. For this sample, no modification needed. For custom development, add libraries below the example without deleting existing items.Do not link other SOs from software package to avoid compatibility issues during future upgrades.Execute sequentially:mkdir build cd build cmake ..Run make to compile custom pass so, then install dynamic library libadd_zero_pass.so to custom fusion pass directory via make install. Optional parameter-j$(nproc)can be added after make for parallel build tasks,$(nproc)dynamically gets CPU core count.make -j$(nproc) add_zero_pass make installAfter sample verification, run the following command to clean custom pass so installed under CANN package to avoid affecting subsequent UT/ST:make clean_custom_passProgram ExecutionConfigure environment variables (if already done, skip).Run environment setup script:source ${ASCEND_PATH}/set_env.shReplace${ASCEND_PATH}with actual software package installation path.Use ATC offline inference.Set environment variable to dump model graph during compilation:export DUMP_GE_GRAPH1Enter data directory and execute .py file to export onnx (uses torch onnx exporter, depends on additional Python package onnx, ensure installed before running. ATC tool currently supports onnx opset_version up to 18, if torch exports higher version by default, specify explicitly, see script comments):python torch_gen_onnx.pyAfter execution, .onnx format model file named model.onnx generated in data directory.Execute ATC tool command (for detailed ATC tool instructions, visit Ascend Documentation and search for ATC Offline Model Compilation Tool), modifysoc_versionbased on actual environment:atc --model./model.onnx --framework5 --soc_versionxxx --output./modelLog shows:Define pattern for AddZeroPass Define MeetRequirements for AddZeroPass Define replacement for AddZeroPassOnline inferenceSet environment variable to dump model graph during compilation:export DUMP_GE_GRAPH1Enter data directory and execute .py file for online inference (ensure torch_npu plugin installed for online inference):python torch_forward.pyLog shows:Define pattern for AddZeroPass Define MeetRequirements for AddZeroPass Define replacement for AddZeroPassView resultsAfter execution, series of .pbtxt files generated in directory. Compare the following dump graphs:ge_onnx_xxxxx_PreRunBegin.pbtxtdump graph before executionge_onnx_xxxxx_RunCustomPassBeforeInferShape.pbtxtcustom pass dump graph before InferShape executionModel optimized as expected, i.e., add-zero nodes removed.If results not as expected, set following environment variables (for atc command, also add parameter--logdebug) to print logs to screen for troubleshooting.export ASCEND_SLOG_PRINT_TO_STDOUT1 #print logs to screen export ASCEND_GLOBAL_LOG_LEVEL0 #log level debug【免费下载链接】geGEGraph Engine是面向昇腾的图编译器和执行器提供了计算图优化、多流并行、内存复用和模型下沉等技术手段加速模型执行效率减少模型内存占用。 GE 提供对 PyTorch、TensorFlow 前端的友好接入能力并同时支持 onnx、pb 等主流模型格式的解析与编译。项目地址: https://gitcode.com/cann/ge创作声明：本文部分内容由AI辅助生成（AIGC），仅供参考