Babysitter · developer tool · Node.js · open source
Deciding which model to use is not the same as switching to it
Babysitter wraps Claude Code and Codex. It classifies every prompt and runs it on the smallest model tier that fits, so small prompts stop paying big-model prices. The careful part is knowing when a switch is not worth the warm cache it throws away.
One model for everything is wrong both ways
In a coding agent, most prompts are small: rename this, run the tests, fix that typo. Some are not: rework the auth layer, migrate a schema across the repo. Running everything on the biggest model wastes money and time on the small ones. Running everything on a small one gets the big ones wrong.
So pick per prompt. Except that switching model has a cost nobody sees in a pricing table. A long session has a warm prompt cache, tied to the model that built it. Switch, and the next turn pays to read the whole context again, cold.
A router that is right about every single prompt can still cost more than no router at all, if it keeps throwing the cache away to be right.
How I got there
Three passes, in the order they happened.
Explore
Pick the best model for each prompt, every time
Realise
Every switch throws away a warm cache
Insight
Split the deciding from the switching
Classify every turn. Switch only when it pays.
What happens to a prompt
Two separate steps. The first decides what this prompt needs. The second decides whether changing model is worth it right now.
Ask five questions about it
Each prompt goes to Jev, an external classifier, with five structured questions: how complex, how risky, how broad, what kind of task, and how likely it is to need a clarifying question.
src/jev.js
Start from complexity
The tier begins at the complexity score, capped below the top. Four tiers: Haiku, Sonnet, Opus and Fable on Claude; the matching four on Codex.
src/policy.js
Risk and breadth push it up
High risk or work across the whole repo means at least Opus. The top tier needs both: the hardest complexity and a high-impact signal. Low confidence is noted and never spends more.
Then the cache guard
Moving up is always allowed. Moving down is blocked until the current model has had three turns, and blocked outright once the context passes 12,000 tokens. Effort is never lowered on a warm model either, since that resets the cache too.
src/claude-session-policy.js
Run it, or fall back
If the classifier is down or slow after three tries, the prompt runs on the middle tier at medium effort. A session never stops because the router did.
The tier, in plain code
The routing rules are ordinary code with tests, not a prompt to another model. Anyone can read why a prompt went where it did, and the reasons are printed beside every answer.
let level = Math.min(complexity, 2);
if (risk >= 2 && level < 2) {
level = 2;
reasons.push("high-risk work requires at least Sol");
}
if (breadth >= 2 && level < 2) {
level = 2;
reasons.push("repository-wide scope requires at least Sol");
}
const astraEligible = complexity === 3 && (risk >= 2 || breadth >= 2);
if (astraEligible) {
level = 3;
} else if (complexity === 3) {
reasons.push("Astra gate not met; capped at Sol");
}
if (classifierConfidence < 0.45) {
reasons.push("low confidence noted; tier not increased");
}The other choices
Decision
Why
The provider's own control APIs, not CLI hooks
Hooks cannot change the model for the prompt being submitted, so Babysitter drives Codex through its app-server and Claude through the Agent SDK. /native hands the same session to the official CLI whenever you want it.
Its own /compact
The current model writes a short handoff (goal, decisions, files, next steps) and the next prompt starts a fresh routed session. The new model never reloads the old transcript. The context stays small and focused.
Unknown slash commands are never sent as text
A typo'd command would otherwise go to the model as a prompt and get an earnest answer. It says the command is not supported and points to /native.
A likely clarifying question adds an instruction
When the classifier thinks a prompt is ambiguous, the model is told to ask before doing anything it cannot undo.
Updates only notify
It checks for a new version at most once a day and never installs one. bbs-update fast-forwards the clone and refuses if there are local changes. No surprise code on anyone's machine.
The official commands are untouched
It installs its own launchers next to claude and codex instead of wrapping them, so turning it off is just not typing bbs.
What Babysitter came down to:
01
Deciding is not switching
Every prompt is classified, but a separate guard decides whether changing model is worth the cache it throws away.
02
Rules you can read
The routing policy is plain, tested code, and it prints its reasons next to every answer.
03
Never in the way
A fallback when the classifier fails, /native for anything missing, and the official commands left exactly as they were.