Function calling

OpenAI-shaped tools mapped onto every EU provider — Mistral, Anthropic-via-Bedrock, Llama, Qwen.

CleverRouter accepts the OpenAI tools and tool_choice shape and translates it to whatever the underlying provider expects (Mistral tool format, Bedrock Converse tools, etc). Your loop stays the same.

At a glance


Wire format	OpenAI `tools` + `tool_choice`
Streaming tools	Yes — accumulate deltas, see Streaming
Parallel tools	Common on Mistral and Anthropic — handle in parallel
Recommended models	`mistral/mistral-large-2`, `mistral/mistral-small-3.2`, `qwen/qwen3-235b`

Tool loop

tool-loop.ts

import { createCleverRouter } from '@cleverrouter/sdk';

const cr = createCleverRouter({ apiKey: process.env.CLEVERROUTER_API_KEY! });

// 1. Declare the tools
const tools = [
  {
    type: 'function' as const,
    function: {
      name: 'get_weather',
      description: 'Current weather for a city.',
      parameters: {
        type: 'object',
        properties: {
          city: { type: 'string' },
          unit: { type: 'string', enum: ['c', 'f'], default: 'c' },
        },
        required: ['city'],
      },
    },
  },
];

// 2. Start the conversation
const messages: any[] = [
  { role: 'user', content: 'Weather in Paris and Berlin right now?' },
];

const first = await cr.chat({
  model: 'mistral/mistral-large-2',
  messages,
  tools,
});

const toolCalls = first.choices[0]?.message.tool_calls ?? [];

if (toolCalls.length) {
  // 3. Push the assistant message that requested the tool calls
  messages.push(first.choices[0]!.message);

  // 4. Run all tools in parallel, then push their results
  const results = await Promise.all(
    toolCalls.map(async (call) => {
      const args = JSON.parse(call.function.arguments);
      const result = await getWeather(args.city, args.unit);
      return {
        role: 'tool' as const,
        tool_call_id: call.id,
        content: JSON.stringify(result),
      };
    }),
  );
  messages.push(...results);

  // 5. Ask the model to produce the final answer
  const final = await cr.chat({
    model: 'mistral/mistral-large-2',
    messages,
    tools,
  });

  console.log(final.choices[0]?.message.content);
}

async function getWeather(city: string, unit: 'c' | 'f'): Promise<unknown> {
  // Your own data source.
  return { city, temp: 19, unit };
}

Parallel tool calls are common

Mistral and Anthropic frequently emit several tool calls in one assistant turn. Collect them all, run them in parallel and push every result back together. Sequential dispatch wastes round-trips and confuses the model.

Streaming + tools

Tool arguments stream as delta.tool_calls[].function.arguments fragments. Accumulate until finish_reason: 'tool_calls', then dispatch:

const acc: Record<number, { id?: string; name?: string; args: string }> = {};

for await (const chunk of cr.stream({
  model: 'mistral/mistral-large-2',
  messages,
  tools,
})) {
  for (const tc of chunk.choices[0]?.delta?.tool_calls ?? []) {
    const slot = (acc[tc.index!] ??= { args: '' });
    if (tc.id) slot.id = tc.id;
    if (tc.function?.name) slot.name = tc.function.name;
    if (tc.function?.arguments) slot.args += tc.function.arguments;
  }
  if (chunk.choices[0]?.finish_reason === 'tool_calls') {
    // All accumulated calls are ready to run.
  }
}

Forcing or disabling tools

// Force a specific tool
await cr.chat({
  model: 'mistral/mistral-large-2',
  messages,
  tools,
  tool_choice: { type: 'function', function: { name: 'get_weather' } },
});

// Disable tools for this turn
await cr.chat({
  model: 'mistral/mistral-large-2',
  messages,
  tools,
  tool_choice: 'none',
});

Provider notes

Provider	Tool format used internally
Scaleway	Mistral tool API (OpenAI-compatible passthrough)
Tensorix	OpenAI-compatible tool API
Bedrock	Converse `toolConfig` — mapped on the gateway

You never see those details — the gateway converts back and forth behind the OpenAI-shaped surface.

Common gotchas

Stale tool_call_ids. Each tool result message must reference the id from the assistant turn that requested it. Mixing them up produces nonsense answers.
Schema must be JSON Schema, not Zod. If you use the Vercel AI SDK's tool() helper, it converts Zod → JSON Schema for you.
Don't strip description. Models lean on tool and parameter descriptions to pick the right call. Generic names are a common reason for "model never called the tool".

Streaming — pattern for tool deltas.
Chat completions — request shape.
Mastra — tool wiring via the Mastra agent helper.