Cost optimisation
Cheaper tokens, smaller models, smarter routing — practical patterns that actually move the bill.
CleverRouter bills provider cost + 5%. Most cost reduction therefore comes from what and how you send, not from us. Here's what actually moves the needle.
At a glance
| Lever | Typical saving |
|---|---|
| Drop one size in model class | 30 – 70% |
| Use rerank instead of top-50 chat | 60 – 90% |
| Batch embeddings | 5 – 20% |
Set max_tokens aggressively | 10 – 30% |
reasoning_effort: 'low' | 50 – 80% |
Strategy cheapest | 0 – 25% |
Pick a smaller model when you can
A common over-spend pattern: defaulting to mistral/mistral-large-2
for everything. Two-thirds of typical traffic doesn't need it.
| Use case | Right-size model |
|---|---|
| Quick chat replies | mistral/mistral-small-3.2 |
| Classification, extraction | mistral/mistral-small-3.2 |
| Long-form drafting | mistral/mistral-large-2 |
| Code generation | qwen/qwen3-coder |
| Reasoning-heavy problems | deepseek/deepseek-r1-distill-llama-70b |
| Embeddings | cohere/embed-v4 (smallest dims you can stomach) |
Rerank instead of cramming chat with documents
The expensive way: stuff 50 docs into a chat prompt with a 200k-token model. The cheap way: rerank to top-5, send the top-5.
// Cheap path
const reranked = await cr.rerank({ query, documents: candidates, top_n: 5 });
const top5 = reranked.results.map((r) => candidates[r.index]);
const answer = await cr.chat({
model: 'mistral/mistral-small-3.2',
messages: [
{ role: 'system', content: 'Answer from context only.' },
{ role: 'user', content: `Context:\n${top5.join('\n\n')}\n\nQ: ${query}` },
],
});Full pattern in RAG pipeline.
Batch embeddings
// Slow + expensive
for (const doc of docs) {
await cr.embed(doc, { model: 'cohere/embed-v4' });
}
// Fast + cheap
await cr.embed(docs, { model: 'cohere/embed-v4' });One batched call beats N serial calls on both latency and overhead.
Cap max_tokens
Models will happily ramble. Cap them:
await cr.chat({
model: 'mistral/mistral-small-3.2',
messages,
max_tokens: 200, // a short answer is usually enough
});The cap doesn't replace good prompting — but it stops the rare "write me a 3000-word answer" surprise from doubling your bill.
Set reasoning_effort: 'low' by default
Reasoning tokens cost real money
'high' reasoning effort can quietly triple your bill on long
questions. Default to 'low'; upgrade per request when the user
selects "think harder" or the task obviously needs it.
Use strategy: 'cheapest' when latency doesn't matter
const cr = createCleverRouter({
apiKey: process.env.CLEVERROUTER_API_KEY!,
strategy: 'cheapest',
});For background jobs and batch pipelines where P95 latency doesn't matter, this picks the cheapest of the providers that host the model.
Set per-key budgets
Budgets are the safety net. Even if every other lever fails, a
daily budget cap returns 402 before the runaway loop drains your
balance. See Get your API key.
Monitor before you optimise
The dashboard's Usage view shows daily spend per model, per key, per day. Before tuning anything, find the top three lines — that's where 80% of the bill lives. Optimising the small ones wastes engineering time.
What's not worth doing
- Aggressively low
max_tokens— when answers get cut mid- sentence, users prompt again and you pay twice. Find a cap that fits 95% of answers, not 50%. - Switching models mid-conversation — context length and tokeniser differences make this painful. Pick once per use case.
- Caching prompts client-side — fine, but watch out for stale answers when the underlying data changes.