Model Routing Is Now A Core Product Skill
The durable product-builder skill is no longer choosing one “best model” and wiring the whole product around it. It is owning a routing layer and an eval harness that can match task shape, cost, latency, and tool behavior to the right model at run time.
The strongest July 16 signal came from Digg Tech clustering around Moonshot AI’s Kimi K3 release and the immediate expert reaction to it. Moonshot positioned K3 as a 2.8T open 3T-class model for long-horizon coding and knowledge work, while Simon Willison’s same-day hands-on analysis translated the launch into builder terms: frontier-level capability, notable pricing, strong agentic performance, and real tradeoffs around verbosity and reasoning cost. Artificial Analysis reinforced that this was not just launch theater: Kimi K3 landed near the top of the intelligence rankings with a 1M-token context window, but remained slower and relatively expensive on output. The practical lesson is that the market no longer cleanly separates “closed frontier” from “usable open alternative.”
Once several models are good enough for serious coding, research, and agentic workflows, product advantage shifts away from vendor allegiance and toward task allocation. Teams that can test and route well will ship cheaper and faster than teams that hard-code one premium model into every interaction. The July 16 shift is that model selection becomes an operational capability: a system for deciding when to pay for maximum reasoning, when to prefer context length, when to favor lower latency, and when an open or proxy-accessible model is sufficient.
Turn one workflow into a routing policy. Trigger: a request enters a high-cost or high-variance workflow such as coding, support research, document synthesis, or agentic task execution. Context: classify the task by required tool use, context length, latency tolerance, output format, and acceptable error cost. Tools: expose at least two model options plus a lightweight eval harness, prompt templates, and logging. Verifier: compare outputs against a small scored task set or production acceptance checks such as tests passing, sources cited, or artifact structure preserved. Budget: set per-run ceilings for tokens, wall-clock time, and fallback count. Artifacts: persist the chosen model, reasoning effort, run outcome, and cost for each class of task. Stop condition: the workflow falls back or escalates when the first model misses the verifier or exceeds budget, rather than silently continuing to burn tokens.
Routing policy for one multi-model workflow
Choose one expensive AI workflow and write a routing table with three rows: cheap/default, deep-reasoning, and long-context. For each row, define the trigger, verifier, budget, and fallback. If you cannot explain why a request lands in one row instead of another, you do not yet own the routing layer.
- Separates task classes by cost, latency, context, or tool-use needs.
- Pairs each route with a verifier and a fallback path.
- Defines a measurable budget instead of an informal preference.
- Captures run outcomes so the routing policy can improve over time.
If your workflow has a tiny task surface and one model is already clearly dominant on your acceptance criteria, adding routing too early can create unnecessary complexity and wider failure modes.
A first-party model release, a same-day hands-on operator analysis, and third-party benchmark data all point in the same direction: Kimi K3 is credible enough that builders should treat routing and evaluation as infrastructure, not an optimization for later.
Did adding an explicit routing policy reduce cost or latency without harming acceptance quality?
Watch: cost per successful run · fallback frequency · acceptance rate by model route · latency by task classKeep Going