Pushing the Frontier of AMD Inference: Optimizing MegaMoE with Ave and ROCm Hyperloom
Pushing the Frontier of Inference Optimization with AI Agents
Our first post showed that data-flow invariants make recursive self-improvement efficient: Ave optimizes GPU kernels on par with human experts at about 100x the token efficiency of other agentic approaches. This post asks a harder question. Can an AI agent push GPU optimization into territory with no reference kernel to match, and still deliver production quality?
We answer with a case study. We integrated Ave with ROCm Hyperloom, AMD's multi-agent harness for inference optimization, and used the pair to build a state-of-the-art MegaMoE kernel for the AMD Instinct MI355X. On eight MI355X GPUs it reaches nearly 1.5x the effective throughput of vLLM's AITER all-to-all backend at 1,024 tokens per rank, and cuts time per output token by about 21% at concurrency 128 serving DeepSeek V4 Pro on GSM8K. The kernel is the product of a multi-step campaign in which Ave and Hyperloom act as one experienced, automated kernel engineer: apply the knowledge base, profile, understand what each change did, iterate. The whole campaign took about 75 minutes and roughly 8 million tokens, for under $6 in API cost.
Mixture-of-Experts (MoE) layers are how large language models scale toward trillions of parameters, and they account for more than 30% of critical-path latency in models such as DeepSeek V4 Pro. Serving partitions the experts across GPUs, so each token must travel to the ranks that hold its experts and its results must travel back. At serving batch sizes, the communication, synchronization, and launch overhead around those steps outweighs the matrix multiplications. A MegaMoE kernel fuses all of it into one persistent kernel, overlapping communication with compute and eliminating the launches.
The hard part is not the GEMMs. It is synchronization that stays correct across eight GPUs without a grid-wide barrier, scheduling tuned to the hardware and the model, and containing load skew so that no GPU idles waiting for a straggler. The rest of the post describes how Hyperloom and Ave do this automatically: what the agent starts from, the invariants that keep the kernel correct while it changes, and the benchmark that keeps the speedup real.
Knowledge and Invariants for In-Kernel Synchronization
We brought the Ave knowledge base into Hyperloom in three categories. Hardware primitives: the LDS layout, asynchronous loads into LDS, and the cache-coherence scope of buffer instructions. Optimization recipes: persistent scheduling, kernel fusion, synchronization primitives for mega kernels, and cache-coherence policy. MoE-specific optimizations: token shuffling across ranks, push versus pull shuffle protocols, and ticket-based work stealing. Each entry states when it applies, what it changes, and the invariants it must preserve.
Recall the two constructs from the first post. A tag function labels data with user-defined tuples, and a tag assertion is a compile-time check that two tags are equal, or that one is at least the other. When it fails, the compiler returns a concrete witness. We apply both to MegaMoE's in-kernel synchronization.
On MI355X, the scope bits of a buffer instruction set its memory-consistency scope, wave, workgroup, device, or system, by bypassing the L1 or L2 of the compute unit or the GPU. This is what makes signals across GPUs possible, and it is where they go wrong. A signal must be visible across GPUs, and it must become visible only after the data it announces: the notification that a token is ready must not be observable before the token's bytes have landed on the remote GPU. The wrong scope or the wrong order means wrong results or a hang, and the guarantee has to hold down to the assembly, where the compiler is free to reorder.
We enforce two invariants, both checked statically.
Scope invariant. Every memory instruction executes at a consistency scope at least as wide as the scope its target bytes require, and every poll executes at system scope. Scope compares a tag on the data with a tag on the instruction.
Ordering invariant. A signal is published only after every byte it announces has been stored, completed, and joined by the publishing workgroup, and a consumer reads announced bytes only after its poll on that signal returns. Order treats the tags on a handoff buffer as a small state machine, advanced where the protocol says the state of the data changes.
The scope invariant takes two tag functions and one assertion to state. The workspace heap is partitioned into a rank-local region and shared regions holding peer-visible data and barriers, and the first tag function labels every byte with the scope its readers need: system for the shared regions, device for rank-local data read by other workgroups on the same GPU, wave for the rank's own inputs. The second labels every buffer load, store, and atomic with the scope in its aux bits, a compile-time constant. The assertion at every memory operation requires the instruction's tag to be at least the tag of the bytes it touches. A poll, which is how in-kernel synchronization waits, must be at system whatever it reads, since a poll served from L1 never observes the store it waits for. Both tags are static, so the compiler decides every instance, and a failure names the instruction, the bytes, and the two tags. In a run, the same mistake is eight GPUs at 100% utilization with no output and one wrong aux among dozens to find.
The ordering invariant takes one tag function and three assertions. Take pushing token rows into a peer's heap. A payload store labels the bytes it writes stored. The publication point, the wait for outstanding stores followed by the workgroup join, is where the protocol says the data becomes complete, so it carries an assertion and a tag function: the assertion requires every byte the signal will announce to be stored, and the tag function relabels them synced. The signal store asserts that the bytes it announces are synced. On the consumer, the poll's return relabels the region visible, every load from it asserts visible, and the tags are cleared at the end of the epoch. Each step is an ordinary equality assertion on data, and the tags propagate through the lowered code like any other, so the check survives the compiler. A row store placed after the join, by the programmer or by the compiler, leaves bytes stored where the signal expects synced, and the witness is that store and those bytes.
The tags ride with the data. When a later step moves five buffers from the shared region into rank-local memory, every byte in them is relabeled and every instruction touching them is re-checked without anyone listing them. The placement rule is the first tag function, written once.
Finding the Benchmark That Measures Real Traffic
An invariant catches a wrong kernel. It takes a benchmark to catch a wrong speedup, and our first benchmark would have led the campaign to the wrong kernel.
The obvious microbenchmark routes tokens uniformly: every rank gets 12.5% of the token-expert pairs and finishes at the same time. Under that workload the natural design has each destination rank pull its inputs from the sources, which keeps the asynchronous remote-to-LDS load path and is fast when every link carries the same traffic. Our early kernels did this, and the uniform benchmark rewarded them.
Real routing is skewed. Profiling DeepSeek V4 Pro on all 1,319 GSM8K problems under eight-way expert parallelism, some layers sent 44% of the token-expert pairs to one rank, worst during decode, where too few tokens smooth the distribution. A pull design then funnels most of the layer's traffic through the overloaded rank's inbound links while the rest of the fabric idles. The eight GPUs form a full xGMI mesh with up to 64 GB/s per direction on every link. The bandwidth is there, but a pull cannot use it. Pushing turns it around: each source spreads its tokens over its own outbound links, the overloaded rank's traffic arrives over seven links at once, and the receive path loses a wait. We gave up the remote-to-LDS path for it, and for the small transfers of decode the trade is clearly worth it. Readiness bits per 32-row group, so an overloaded rank starts computing before all its tokens arrive, and compute-unit and tile allocation that adapt to per-rank load follow from the same observation.
The campaign ran through Hyperloom's kernel phase with Claude Sonnet 5.5, with a budget of up to one hour per step, on one server with eight MI355X GPUs under eight-way expert parallelism. It took about 75 minutes end to end and roughly 8 million tokens, most of them prompt-cache reads, for under $6 in API cost. The kernel it produced is the one evaluated in our MegaMoE post: nearly 1.5x the effective throughput of vLLM's AITER all-to-all backend at 1,024 tokens per rank, and about 21% lower time per output token at concurrency 128 while serving DeepSeek V4 Pro on GSM8K.
Conclusion
MegaMoE was a different kind of target from attention and GEMM. There was no library to match, the hard problems were synchronization and skew rather than tiling, and the kernel had to be correct across eight GPUs before it could be fast on one. Two things made the campaign work.
The first is using invariants in creative ways. Tags and assertions were built for data layout, but a handoff protocol is data flow too. A few tags on a buffer, advanced where the protocol changes the state of the data, turn memory ordering across eight GPUs into a compile-time check with a witness. The invariants that pay off most are the ones that turn a hang into an error message.
The second is choosing the right benchmark. A uniform-routing benchmark would have had the campaign perfect a pull-based kernel that falls over on real decode traffic. Measuring the slowest rank under skew is what makes push, per-group readiness, and adaptive scheduling look like what they are. A driver that measures the wrong thing lets an agent optimize the wrong thing with perfect discipline, so the driver is written before the first step and the agent never touches it.
The broader lesson is the first post's, one level up. Recursive self-improvement on inference needs a harness that owns policy and verification, an agent that implements efficiently, and invariants that make each step checkable. With those in place, every round gets cheaper as the knowledge base grows, which is the direction the scaling law has to point for self-improvement to pay for itself. MegaMoE shows what that buys in practice: with the right harness, AI agents can push the frontier of inference performance on any platform, and do it quickly.