LLM Routing2026Shipped
TinyRouter
A 10K-parameter head that picks which LLM answers, and in what role.
A from-scratch replication of the TRINITY coordinator. A frozen 0.6B encoder maps each question to a 1024-dim vector, and a ~10K-parameter head picks which of three open-source LLMs answers and whether it acts as Thinker, Worker or Verifier, over up to 5 turns. The choice is discrete, so the head is trained with separable CMA-ES, an evolution strategy, instead of gradients.
livelast push
~/tinyrouter
Project Telemetry
Avg accuracy
0.858
Best single LLM
0.835
MMLU vs random
0.925 / 0.875
API spend
$62
How it works
~/tinyrouter/architecture · step 1/4
01
question
02
frozen 0.6B encoder → 1024-d
03
10K-param head · CMA-ES
04
LLM × role: thinker / worker / verifier
05
verifier accepts → answer
↺ loops to 03
01Evaluated on 120 held-out questions per task, with every single-model baseline averaged over 3 runs and random routing as a null model.
02An oracle-ceiling diagnostic with 95% CIs showed math had +0.049 [0.005, 0.085] of reachable gain the router missed. That pinned the flat result on the router, not the model pool.
03Supervised warm-start from per-(question, model) labels, with a shaped training fitness and a strictly binary eval metric.
04Every token of the $62 spend tracked from ledgers; training and grading harness covered by 54 offline tests.
Project Info
DomainAI Systems
RoleAuthor
Year2026
StatusShipped
Tech Stack
PythonPyTorchsep-CMA-ESFireworksNVIDIA H200