What Jiu-Jitsu Taught Me About Fixing AI's Slow-Motion Problem

From the Mats to the Motherboard

There's a saying in jiu-jitsu: you don't win by overpowering your opponent; you win by finding the right angle at the right moment. That same philosophy is playing out in artificial intelligence, where engineers wrestle with bottlenecks that slow down large language models. At the upcoming AICon conference in Shenzhen, Huawei's Li Baichao will share how his team applied this mindset to optimize communication in the Pangu model.

Li, a systems engineering expert at Huawei's 2012 Lab, has spent years studying communication protocols. His talk focuses on a deceptively simple goal: make data move faster between chips. But as anyone who has ever rolled on the mats knows, simple goals often hide complex challenges.

The AllToAll Bottleneck: A Guard Pass You Can't Force

In jiu-jitsu, if you try to muscle through a strong guard, you'll gas out fast. Similarly, in Mixture-of-Experts (MoE) models, the AllToAll communication operation can consume over 30% of total end-to-end time. It's the guard that every grappler has to pass, and it's been a stubborn problem for years.

The Huawei team didn't just throw more compute at it. Instead, they studied the hardware's topology—specifically the Ascend 950's network layout—and paired it with a dedicated communication accelerator called CCU. The result? A 10% performance improvement on the EP communication domain's AllToAll. That's like finding a new grip that lets you pass guard with half the energy.

Why Generic Solutions Fail

One key insight from Li's preparation is that a one-size-fits-all approach doesn't work. The team tried using DeepEP, a popular communication library, but it simply didn't fit the Ascend 910A3's architecture. It's like trying to use a spider guard when your opponent is standing—you need to adapt to the situation.

This is a lesson that resonates beyond AI. In jiu-jitsu, the best techniques are the ones that work for your body type and your opponent's reactions. The same goes for hardware optimization: you have to be intimately familiar with the platform's strengths and weaknesses.

KV Cache Transfers: The Newest Submission Threat

As models push toward 1-million-token contexts, a new bottleneck has emerged: the time it takes to transfer KV cache from host memory to device memory. This Host-to-Device (H2D) communication can seriously delay the time to first token (TTFT), making users wait longer for a response.

Think of it like this: you've drilled a perfect armbar from mount, but if you take too long to set it up, your opponent escapes. In long-context inference, every millisecond of delay in KV cache transfer gives the user a chance to lose patience.

Huawei's solution is twofold. On the hardware side, the Ascend 950 provides a dedicated H2D pathway for each NPU, reducing contention. On the software side, they developed Omni Cache to handle efficient cache offloading. Together, these improvements cut TTFT by over 10%. That's the difference between a smooth transition and a scramble.

Coordinating Communication and Computation: The Flow State

In jiu-jitsu, the best grapplers don't just execute moves; they flow. They blend transitions so seamlessly that it looks like one continuous motion. Li and his team are aiming for the same effect in AI systems—they want to hide communication time behind computation, so the model never sits idle waiting for data.

The approach involves fusion operators and multi-stream parallel scheduling, all tailored to the Ascend hardware. It's a delicate dance, but when it works, the system feels effortless. The user just sees fast responses, without realizing the complex choreography happening underneath.

Trade-Offs: The Cost of Specialization

But there's a catch. All this optimization is deeply tied to the Ascend 950 hardware. What works beautifully on that platform might fail—or even backfire—on other chips like the Ascend 910A2/A3 or NVIDIA H20.

This is a familiar dilemma in martial arts. A technique that works against a taller opponent might be useless against a shorter one. You have to choose your battles. For Huawei, the choice is clear: they're betting on Ascend's future, and they're willing to sacrifice some portability for raw performance.

What Practitioners Can Learn

Li's talk isn't just for Huawei insiders. The principles he's sharing have universal applications. If you're training or deploying large models, you need to understand your hardware's characteristics—its topology, its memory hierarchy, its communication channels. Only then can you make informed decisions about how to optimize.

For jiu-jitsu practitioners, the lesson is even simpler: adapt and flow. Don't force a technique that doesn't suit the situation. Study your opponent, understand your own body, and find the path of least resistance.

The Road Ahead

As AI models grow larger and more complex, communication will only become more critical. Li's team is already looking at ways to further reduce exposed communication time through even tighter integration with Ascend's hardware. The goal is to make communication so efficient that it's practically invisible—like a good guard pass that your opponent never sees coming.

In both AI and jiu-jitsu, the future belongs to those who can think strategically, adapt quickly, and execute with precision. Huawei's work on the Pangu model is a reminder that sometimes the biggest gains come not from brute force, but from smart, targeted adjustments.

If you're attending AICon Shenzhen, don't miss Li's session. And if you're on the mats, remember: the next time you're stuck in a bad position, think about how you can optimize your own communication—with your body, your partner, and the ground. The answer might just be a subtle shift in angle.