Archive on matrix-multiply

Benchmarking Trillion-Parameter Open Models Without Fooling Yourself

Sun, 30 Aug 2026 00:00:00 +0000

We ran Kimi K3 and DeepSeek V4 (Pro and Flash) head-to-head on a hard-benchmark suite — same harness, same serving stack, same hardware, back to back. The scores are below, but the scores are honestly the less useful half of this post. The useful half is the list of ways the numbers were nearly wrong.

The setup

Everything ran on one 8×B300 node (tensor-parallel 8) with vLLM, driven by lm-eval plus custom harnesses for AIME-25 and BBEH. Identical prompts, budgets, and scoring for all three models. We deliberately dropped saturated benchmarks (GSM8K, MMLU, ARC — everything scores in the high 90s or clusters within noise) and kept rows with headroom:

896 Experts, None Dead: What MoE Load Actually Looks Like at Token Level

Fri, 28 Aug 2026 00:00:00 +0000

There’s a seductive idea floating around MoE deployment discussions: with huge scale-up domains coming (NVL72 and beyond), why not give every expert its own accelerator? One expert, one chip, perfect disaggregation.

We measured what that would actually do, using Kimi K3 — 896 routed experts per MoE layer, 92 such layers, top-16 routing — served on an 8×B300 node. A small patch in the serving stack recorded a per-layer histogram of every expert selection across a mixed workload (math, knowledge, and code benchmarks). Here’s what the router’s traffic really looks like.

The Free Lunch That Wasn't (Part 2: Ten Seeds Later)

Wed, 26 Aug 2026 00:00:00 +0000

This is part 2 of two. Part 1 set up the experiment and ended on a +5.5 point HumanEval gain from routing to 32 experts instead of 16, flagged as single-seed and unbelieved. This post is the replication.

To recap the premise: Kimi K3 is a 1.56 TB, 896-expert mixture-of-experts model. For every token, at every one of its 61 MoE layers, a router picks 16 experts out of 896 and blends their outputs. That top-k = 16 is a training-time hyperparameter, and you can override it at inference time with a ~10-line patch to the serving stack. No retraining, no fine-tuning.

Does Kimi K3 Want More Experts? (Part 1: The Tempting Result)

Sat, 15 Aug 2026 00:00:00 +0000

This is part 1 of two. It ends with a result that looks like a free lunch. Part 2 spends ten seeds and a router instrumentation pass finding out whether it was real. It wasn’t.

Kimi K3 activates 16 of its 896 experts per token. That number comes from training; nothing forces you to keep it at inference time. The router scores experts, and how many of those scores you act on is a runtime choice.