iFLYTEK: Cooperating with Huawei to realize large-scale cross-node expert parallel cluster reasoning with domestic computing power
On March 11, iFlytek announced that iFlytek and Huawei have recently taken the lead in realizing large-scale cross-node expert parallel cluster reasoning with domestic computing power. According to its introduction, through distributed architecture innovation and algorithm co-optimization, the static memory usage of a single card is reduced to 1/4 of that of a dual-machine deployment, the efficiency is improved by 75%, the expert computing density is increased by 4 times, the reasoning throughput is increased by 3.2 times, and the end-to-end latency is reduced by 50%. This solution will also be applied to the training acceleration of iFlytek's Spark deep reasoning model, and it is expected that the reasoning efficiency during training will increase by 200%.