GE Local Operator Feature Analysis【免费下载链接】geGEGraph Engine是面向昇腾的图编译器和执行器提供了计算图优化、多流并行、内存复用和模型下沉等技术手段加速模型执行效率减少模型内存占用。 GE 提供对 PyTorch、TensorFlow 前端的友好接入能力并同时支持 onnx、pb 等主流模型格式的解析与编译。项目地址: https://gitcode.com/cann/ge1 Feature OverviewGE Local Operator (abbreviated as GE Local operator) is a class of local operators built into the GE graph engine. It handles operator nodes thatdo not require actual computation on the Ascend NPU. These operators serve as skeleton nodes in the graph—handling data transfer, control flow orchestration, constant storage, and shape inference.Unlike engines designed for actual computation tasks such as FE (Fusion Engine) and AICPU, the GE Local engine (engine nameDNN_VM_GE_LOCAL) is a zero-computation engine. Operators managed by this engine complete parameter calculation or memory layout planning during compilation. At runtime, they only perform lightweight data movement or reference operations without generating any device-side kernel calls.Core PositioningThe core problem that GE Local engine solves is:How to elegantly handle a large number of non-computational nodes in a graph compilation system designed for heterogeneous accelerators?A typical deep learning computation graph, after being converted to AscendIR by a framework adapter (such as TorchAir), contains many non-computational nodes: data entry (Data), model output (NetOutput), constants (Constant/Const), control flow (If/While/Case), shape operations (Shape/Reshape/Squeeze), and so on. These nodes should not occupy NPU computational resources, but they still need to participate in the graph compilation, scheduling, and execution processes.The design philosophy of GE Local is to consolidate these nodes into a dedicated engine that fulfills their placeholder responsibilities with minimal overhead, ensuring completeness of the compilation process and correctness of the execution flow.2 Architecture DesignThe core logic of the GE Local feature concentrates in the compilation (compiler) phase. The overall architecture is as follows:2.1 Compilation PhaseThe core code for the compilation phase is located incompiler/engines/local_engine/, producing two dynamic libraries:Dynamic LibraryResponsibilityRegistration Macrolibge_local_engine.soEngine registration entry, provides four C interfaces externally (Initialize/GetOpsKernelInfoStores/GetGraphOptimizerObjs/Finalize), loaded as a plugin by the GE frameworkEngine pluginlibge_local_opskernel_builder.soOperator builder, responsible for calculating running parameters (CalcOpRunningParam) and generating tasks (GenerateTask), registered asDNN_VM_GE_LOCAL_OP_STOREREGISTER_OPS_KERNEL_BUILDER2.1.1 Engine Entry (GeLocalEngine)TheGeLocalEngineclass undercompiler/engines/local_engine/engine/adopts the singleton pattern and is loaded by the GE engine manager as a dynamic library plugin. It exposes four C-style interfaces:Initialize: CreatesGeLocalOpsKernelInfoStoreandGeLocalGraphOptimizerinstancesGetOpsKernelInfoStores: Registers the operator information registry to the GE framework withDNN_VM_GE_LOCAL_OP_STOREas the keyGetGraphOptimizerObjs: Registers the graph optimizer to the GE frameworkFinalize: Releases resourcesThe engine is loaded during GE initialization, following GEs plugin-based engine registration protocol—each engine dynamic library exports four unified C symbols, and the GE framework loads them viadlopenand binds by symbol name.2.1.2 Operator Information Registration (GeLocalOpsKernelInfoStore)GeLocalOpsKernelInfoStoreis responsible for declaring to the GE framework which operators I support. During initialization, it retrieves the list of all registered operator types fromOpFactoryand creates a defaultOpInfostructure for each operator:engine DNN_VM_GE_LOCAL: Owning engine nameopKernelLib DNN_VM_GE_LOCAL_OP_STORE: Owning operator librarycomputeCost 0: Computation cost is zero, indicating the scheduler need not perform special scheduling for these operatorsflagAsync false,flagPartial false,isAtomic false: Synchronous execution, does not support partial support, non-atomic operationTheCheckSupportedmethod implementation is extremely concise—it directly searches for matches in the registered operator name table. For GE Local operators, there is no concept of partial support; a type match means full support.2.1.3 Operator Factory (OpFactory)TheOpFactoryundercompiler/engines/local_engine/ops_kernel_store/op/adopts the registration-based factory pattern. It binds operator types with creation functions at compile time through theREGISTER_OP_CREATORmacro. The factory manages two types of operator implementations:NoOp (Null Operation Operator)TheRun()method ofNoOpreturns success directly without performing any operation. It covers the following categories of operators:Operator CategoryIncluded Operator TypesDesign IntentData EntryData, RefData, QueueData, AippDataData nodes are managed directly by runtime, no processing needed during compilationConstant StorageConstant, Const, FileConstant, ConstPlaceHolderConstants have completed data preparation during compilationControl FlowIf, Case, While, For, PartitionedCall, and so onControl flow is handled by the runtime subgraph mechanismShape OperationsReshape, Bitcast, Flatten, ExpandDims, ReFormat, Squeeze/Unsqueeze seriesThese operators complete memory reuse marking during compilation, directly reference input at runtimeAuxiliary NodesNoOp, ControlTrigger, Merge, Variable, OpTilingOnly participate in graph structure, no actual computationData FlowStack, StackPush, StackPop, StackCloseHandled by the runtime DataFlow mechanismVirtual ConcatenationPhonyConcat, PhonySplitMarked as NoTask after offset calculation completes during compilationGeDeletedOp (Operators to be Deleted)TheRun()method ofGeDeletedOpintentionally returns FAILEDwith detailed diagnostic information. These operators (such as Identity, Shape, Size, Rank, Placeholder, and so on)should not existin a correctly compiled graph—they should be eliminated by graph optimization passes. If these operators reach the GE Local engine, it indicates a problem with the graph optimization process.This is a carefully designed defensive approach: instead of silently skipping or throwing vague errors, it explicitly tells the user which optimization pass should have deleted this operator, and whether that pass is currently enabled. For example, for theShapeoperator, it checks whether constant folding (OO_CONSTANT_FOLDING) is enabled and provides targeted suggestions.2.1.4 Graph Optimizer (GeLocalGraphOptimizer)GeLocalGraphOptimizercurrently has substantial logic only in theOptimizeOriginalGraphJudgeInsertphase, specifically handling two virtual operators:PhonyConcatandPhonySplit:ForPhonyConcat: SetsNOTASK(no execution task generated),NOPADDING_CONTINUOUS_INPUT(input continuous without padding),OUTPUT_REUSE_INPUT(output reuses input memory)ForPhonySplit: Sets similar attributes, with the difference beingNOPADDING_CONTINUOUS_OUTPUT(output continuous without padding)These attribute settings enable PhonyConcat/PhonySplit to be recognized as zero-copy concatenation/split during the memory planning phase—the memory planner knows these nodes do not need independent output buffers and only need to reference at appropriate offsets in the input buffer.2.1.5 Operator Builder (GeLocalOpsKernelBuilder)GeLocalOpsKernelBuilderis the core working component during compilation, implementing theOpsKernelBuilderinterface and responsible for two key tasks:CalcOpRunningParam—Calculate Operator Running ParametersThe core work of this method is calculating the memory size of each output tensor. For GE Local operators, memory calculation has some special handling:Data/RefData and other data nodes: UsesGetTensorMemorySizeInBytesWithAutoPaddingto calculate aligned memory sizeConstant/Const with type DT_STRING: Uses specialized string memory calculation logicGetConstantStrMemSizeFileConstant: Directly reads preset length fromATTR_NAME_LENGTHattributePhonyConcat/PartitionedCall: Performs additional 32-byte alignment (AlignOutputMemSize)Unknown shape nodes: Skips calculation, determined dynamically at runtimeFor specific operator types, specialized offset calculation functions are also called:PhonyConcat:CalcPhonyConcatNodeOffset—calculates offset positions of multiple inputs in continuous memoryPhonySplit:CalcPhonySplitNodeOffset—calculates offset positions of multiple outputs in continuous memoryBitcast/Flatten/ExpandDims/ReFormat/Squeeze/Unsqueeze:CalcNodeOffsetByReuseInput—marks output to reuse input memoryPhonyConcat Offset Calculation DetailsCalcPhonyConcatNodeOffset(defined in theGeLocalOpsKernelBuilderCalcOpParamclass) supports offset calculation for multi-axis concatenation. It calculates the offset position of each input node in its output buffer through theconcat_dim(concatenation axis list) andN(concatenation count list) attributes.The calculation process uses a hierarchical slice_id approach: decomposes the operator index into position indices on each axis layer by layer, then accumulates offsets from inner to outer axes. It supports negative axis indexing (automatically converted to positive), and performs strict validity checks: input shape consistency check, 32-byte alignment check, axis attribute and tensor dimension matching check, and so on.GenerateTask—Task GenerationThe logic ofGenerateTaskis relatively simple:For operators likeStackPopthat depend on computation, sets theDEPEND_COMPUTEattribute to indicate shape depends on computation resultsFor unknown shape nodes, sets theNOTASKattribute to skip task generationFor other nodes, creates the corresponding Op object throughOpFactoryand callsRun()3 User Scenarios3.1 Scenario 1: Basic Skeleton Construction of Computation GraphAny model compiled through GE naturally uses GE Local operators. When framework adapters (TorchAir/TFA) convert models to AscendIR, they automatically insert nodes such as Data (input nodes), NetOutput (output nodes), and Constant (weight constants). These nodes are automatically assigned to the GE Local engine by the engine scheduler, without user awareness.3.2 Scenario 2: Shape Inference and Constant FoldingIn dynamic shape scenarios, operators like Shape, Rank, and Size need to compute shape information at runtime based on actual inputs. The GE Local engine executes these computations on the Host side through the Host Kernel mechanism and copies the results to the device side for use by subsequent operators.If the user enables constant folding optimization (OO_CONSTANT_FOLDING), these shape-related operators are folded into constants during compilation and do not enter the runtime phase.3.3 Scenario 3: Zero-Copy Memory ReuseShape transformation operators such as Reshape, Bitcast, Flatten, ExpandDims, Squeeze, and Unsqueeze do not change the underlying data, only the shape description. The GE Local engine marksReuseInputduring compilation throughCalcNodeOffsetByReuseInput, and at runtime directly references the input memory, achieving zero-copy.3.4 Scenario 4: Virtual Concatenation/Splitting (PhonyConcat/PhonySplit)PhonyConcat and PhonySplit are virtual operators used internally by GE to represent the concatenation and splitting relationships of multiple tensors in continuous memory. During the graph optimization phase, GeLocalGraphOptimizer sets theNOTASKattribute for them. During the compilation phase,CalcPhonyConcatNodeOffset/CalcPhonySplitNodeOffsetcalculates the memory offsets for each input/output. At runtime, these nodes do not execute any operations; the actual memory sharing is coordinated by the memory planner and execution framework through offset attributes.3.5 Scenario 5: Control Flow and Data FlowControl flow operators such as If/While/Case/For and data flow operators such as Stack/StackPush/StackPop/StackClose are handled by the GE Local engine. Control flow operators are processed through the runtime subgraph execution mechanism, and data flow operators manage cross-node data transfer through theDataFlowResourcemechanism.4 Operator Classification OverviewNoOp Class Operators (No task generated during compilation, null operation at runtime)Operator TypePurposeData, RefData, QueueData, AippDataData entry nodesConstant, Const, FileConstant, ConstPlaceHolderConstant storageNoOp, ControlTriggerPure control flow signalsMergeMulti-way mergeVariableVariable referenceIf, Case, While, For, PartitionedCall and their Stateful/Stateless variantsControl flowOpTiling, ConditionCalc, UnfedDataCompilation assistanceStack, StackPush, StackPop, StackCloseData flowReshape, BitcastShape transformation (zero-copy)PhonyConcat, PhonySplitVirtual concatenation/splittingFlatten, FlattenV2, ExpandDims, ReFormat, Squeeze/Unsqueeze seriesShape transformation (zero-copy)GeDeletedOp Class Operators (Should not exist in normal compilation flow, error if present)Identity, IdentityN, Shape, ShapeN, Size, Rank, Placeholder, Switch, Snapshot, ReadVariableOp, VarHandleOp, TemporaryVariable, DestroyTemporaryVariable, GatherShapes, TransShape, and so on.5 Key Design Decisions5.1 Separation of Responsibilities Between Compilation and RuntimeA core design of GE Local is to push as much work as possible to the compilation phase:Compilation Phase: Calculate output memory size (CalcOpRunningParam), set memory reuse markers (ReuseInput), calculate PhonyConcat/PhonySplit offsets, setNOTASKattributesRuntime Phase: Only perform lightweight operations—reference setting, constant value output, Host shape calculation, and so onThis design enables the compilation phase to complete the vast majority of work, making the runtime execution path extremely short with negligible impact on overall inference performance.5.2 Defensive Design of GeDeletedOpExplicitly registering operators that should be optimized away asGeDeletedOpand returning an error at runtime is a strongly constrained design choice. An alternative approach could be silent skipping (like NoOp), but this would mask graph optimization issues. The current implementation exposes compilation flow anomalies at the earliest opportunity and helps users identify problems by associating optimization option names.5.3 Zero-Copy Strategy for PhonyConcat/PhonySplitThe design of PhonyConcat/PhonySplit embodies the philosophy of plan at compilation, zero overhead at runtime. By calculating all participants memory offsets during compilation, these nodes execute nothing at runtime. Actual memory continuity is guaranteed by the memory planner based onCONTINUOUS_INPUT/OUTPUTand offset attributes.6 Key Files InvolvedFile PathResponsibilitycompiler/engines/local_engine/engine/ge_local_engine.h/.ccEngine entry, singleton pattern, plugin-based registrationcompiler/engines/local_engine/engine/ge_local_graph_optimizer.h/.ccGraph optimizer, handles PhonyConcat/PhonySplit attribute settingcompiler/engines/local_engine/ops_kernel_store/ge_local_ops_kernel_info_store.h/.ccOperator information registry, declares supported operator typescompiler/engines/local_engine/ops_kernel_store/ge_local_ops_kernel_builder.h/.ccOperator builder, calculates running parameters and generates taskscompiler/engines/local_engine/ops_kernel_store/ge_local_ops_kernel_calc_op_param.h/.ccPhonyConcat/Split offset calculation and ReuseInput markingcompiler/engines/local_engine/ops_kernel_store/op/op_factory.h/.ccOperator factory, registration-based operator instance creationcompiler/engines/local_engine/ops_kernel_store/op/op.h/.ccOperator base classcompiler/engines/local_engine/ops_kernel_store/op/no_op.h/.ccNoOp null operation operator, registers all NoOp class operatorscompiler/engines/local_engine/ops_kernel_store/op/ge_deleted_op.h/.ccOperators to be deleted, registers all operators that should be eliminated during optimization phasecompiler/engines/local_engine/common/constant/constant.hEngine name and operator library name constant definitionscompiler/host_kernels/kernel.hHost Kernel base class interfacecompiler/host_kernels/kernel_factory.hHost Kernel factory, used by DependInputShapeTaskcompiler/host_kernels/array_ops/shape_kernel.h/.ccand othersVarious Host Kernel implementationsinc/graph_metadef/graph/ge_local_context.hThread-local context (not directly related to GE Local engine, part of common infrastructure)【免费下载链接】geGEGraph Engine是面向昇腾的图编译器和执行器提供了计算图优化、多流并行、内存复用和模型下沉等技术手段加速模型执行效率减少模型内存占用。 GE 提供对 PyTorch、TensorFlow 前端的友好接入能力并同时支持 onnx、pb 等主流模型格式的解析与编译。项目地址: https://gitcode.com/cann/ge创作声明：本文部分内容由AI辅助生成（AIGC），仅供参考