Huawei’s CloudMatrix 384: A Game Changer in AI Computing

Huawei has made significant strides in the AI landscape with the introduction of its CloudMatrix 384 AI chip cluster. This innovative system, leveraging clusters of Ascend 910C processors, is engineered for advanced AI learning and promises to outperform traditional GPU setups. Despite the individual Ascend processors being less powerful than their competitors, the distributed architecture enhances resource utilization and on-chip efficiency.

Positioning Against Nvidia: A Formidable Challenge

Huawei’s new framework positions the company as a formidable challenger to Nvidia’s market-leading position, as asserted by the tech giant. This is especially noteworthy given the ongoing US sanctions. Huawei’s strategy aims to shift the balance in the AI computing market, encouraging developers to consider its technologies over established players.

Transitioning to the Huawei Framework

Data engineers looking to leverage Huawei’s AI capabilities will need to adapt their workflows. This involves utilizing frameworks that support Huawei’s Ascend processors, particularly MindSpore, Huawei’s proprietary deep learning framework.

Framework Transition: From PyTorch/TensorFlow to MindSpore

Unlike NVIDIA’s ecosystem, which predominantly utilizes frameworks like PyTorch and TensorFlow optimized for CUDA, Huawei’s Ascend processors excel with MindSpore. Engineers with existing models in PyTorch or TensorFlow will need to convert these models or retrain them using the MindSpore API.

It’s essential to note that MindSpore features different syntax and training pipelines, requiring a degree of re-engineering to replicate results. Variances in operator behavior and default weight initialization methods are notable, particularly in convolution and pooling layers.

Utilizing MindIR for Model Deployment

MindSpore introduces MindIR (MindSpore Intermediate Representation), akin to Nvidia’s NIM. According to official documentation, once a model is trained in MindSpore, it can be exported using the mindspore.export utility, converting the trained network into the MindIR format.

Deploying a model for inference involves loading the exported MindIR model and running predictions through MindSpore’s inference APIs for Ascend chips. This process efficiently manages model de-serialization, allocation, and execution.

Unlike PyTorch or TensorFlow, MindSpore distinctly separates training and inference logic, necessitating that all preprocessing aligns with training inputs. Employing MindSpore Lite or the Ascend Model Zoo for hardware-specific tuning is highly recommended.

Adapting to CANN: Compute Architecture for Neural Networks

Huawei’s CANN (Compute Architecture for Neural Networks) provides a suite of tools and libraries tailored for Ascend software, similar in functionality to NVIDIA’s CUDA. Huawei advocates for utilizing CANN’s profiling and debugging tools to monitor and enhance model performance on Ascend hardware.

Execution Modes: GRAPH_MODE vs. PYNATIVE_MODE

MindSpore offers two execution modes:

• GRAPH_MODE – Compiles the computation graph before execution, which can enhance speed and performance optimization.
• PYNATIVE_MODE – Executes operations immediately, simplifying the debugging process and is better suited for early model development.

For initial development, PYNATIVE_MODE is advisable for its straightforward testing and debugging capabilities. Once models are ready for deployment, transitioning to GRAPH_MODE can maximize efficiency on Ascend hardware. Engineers should adjust code accordingly, avoiding Python-native control flow in GRAPH_MODE for optimal performance.

Deployment Environment: Huawei ModelArts

Huawei’s ModelArts is a cloud-based AI development and deployment platform tightly integrated with Ascend hardware and the MindSpore framework. Comparable to platforms like AWS SageMaker and Google Vertex AI, ModelArts is specifically optimized for Huawei’s AI processors.

ModelArts supports a full pipeline from data labeling and preprocessing to model training, deployment, and monitoring, with each stage accessible via API or web interface.

Conclusion: Navigating the Transition

Adapting to MindSpore and CANN will require training and time, particularly for teams accustomed to NVIDIA’s ecosystem. Data engineers must familiarize themselves with new processes, including CANN’s model compilation and optimization techniques for Ascend hardware, and learn the unique APIs and workflows associated with MindSpore.

While Huawei’s tools are evolving, they currently lack the maturity and ecosystem support that frameworks like PyTorch with CUDA provide. However, Huawei aims to demonstrate that migrating to its processes will yield beneficial results and reduce reliance on US-based Nvidia. While Huawei’s Ascend processors are designed for AI workloads, their limited distribution outside core markets poses challenges for teams looking to test or deploy models on Ascend hardware, unless utilizing partner platforms like ModelArts for remote access.

Fortunately, Huawei offers extensive migration guides, support, and resources to facilitate any transition.

Engagement Questions

1. What are the key differences between Huawei’s Ascend processors and Nvidia’s GPUs?
2. How does the transition to MindSpore impact existing AI workflows?
3. What advantages does the CANN architecture provide for AI model optimization?
4. In what ways does Huawei’s ModelArts enhance the AI development process?
5. What challenges might developers face when migrating from NVIDIA frameworks to Huawei’s ecosystem?

source