Building Custom AI Operators with CANN TIK and TVM Training Course

Introduction to Custom Operator Development

• The rationale for building custom operators: use cases and constraints.
• CANN runtime structure and points for operator integration.
• An overview of TBE, TIK, and TVM within the Huawei AI ecosystem.

Programming Low-Level Operators with TIK

• Understanding the TIK programming model and its supported APIs.
• Memory management and tiling strategies within TIK.
• Creating, compiling, and registering custom operations with CANN.

Testing and Validating Custom Operations

• Performing unit and integration testing of operations within the graph.
• Diagnosing kernel-level performance issues.
• Visualising operation execution and buffer behaviour.

Scheduling and Optimisation via TVM

• An introduction to TVM as a compiler for tensor operations.
• Drafting schedules for custom operations in TVM.
• TVM tuning, benchmarking, and code generation tailored for Ascend.

Integration with Frameworks and Models

• Registering custom operations for MindSpore and ONNX.
• Verifying model integrity and fallback mechanisms.
• Supporting multi-operator graphs involving mixed precision.

Case Studies and Specialised Optimisations

• Case study: Achieving high-efficiency convolution for small input shapes.
• Case study: Optimising attention operators with memory awareness.
• Best practices for deploying custom operations across various devices.

Summary and Next Steps