Streaming

Server-Sent Events in OpenAI chunk format, with typed AsyncIterable and reconnect-on-drop in the SDK.

CleverRouter streams chat completions as Server-Sent Events in the exact OpenAI chunk format. Every OpenAI-compatible client consumes it unmodified.

At a glance


Wire format	`text/event-stream` SSE — same chunks as OpenAI
SDK type	`AsyncIterable<ChatStreamChunk>`
Reconnect	Up to 2 retries with `Last-Event-ID` (SDK only)
Cancellation	`AbortSignal` on every helper

With the SDK

stream.ts

import { createCleverRouter } from '@cleverrouter/sdk';

const cr = createCleverRouter({ apiKey: process.env.CLEVERROUTER_API_KEY! });

const stream = cr.stream({
  model: 'mistral/mistral-small-3.2',
  messages: [{ role: 'user', content: 'Write a haiku about Berlin.' }],
});

for await (const chunk of stream) {
  process.stdout.write(chunk.choices[0]?.delta?.content ?? '');
}

For just-the-text-deltas:

import { streamTextDeltas } from '@cleverrouter/sdk';

for await (const delta of streamTextDeltas(stream)) {
  process.stdout.write(delta);
}

With the OpenAI SDK

import OpenAI from 'openai';

const client = new OpenAI({
  apiKey: process.env.CLEVERROUTER_API_KEY!,
  baseURL: 'https://api.cleverouter.eu/v1',
});

const stream = await client.chat.completions.create({
  model: 'mistral/mistral-small-3.2',
  messages: [{ role: 'user', content: 'Stream a haiku.' }],
  stream: true,
  stream_options: { include_usage: true },
});

let usage;
for await (const chunk of stream) {
  if (chunk.usage) usage = chunk.usage;
  process.stdout.write(chunk.choices[0]?.delta.content ?? '');
}

Raw wire format

If you're not using a client SDK (Edge runtime, raw fetch, Hono):

data: {"id":"chatcmpl-x","object":"chat.completion.chunk","choices":[{"delta":{"content":"Hello"},"index":0}]}

data: {"id":"chatcmpl-x","object":"chat.completion.chunk","choices":[{"delta":{"content":", "},"index":0}]}

data: [DONE]

Response headers:

content-type: text/event-stream; charset=utf-8
cache-control: no-cache, no-transform
x-accel-buffering: no

Reverse proxies

The gateway sets X-Accel-Buffering: no so nginx and Cloudflare don't buffer chunks. Pass the same header through any reverse proxy you put in front of your own app.

Cancel a stream

const ctrl = new AbortController();

const stream = cr.stream(
  { model: 'mistral/mistral-small-3.2', messages: [{ role: 'user', content: 'Long answer' }] },
  { signal: ctrl.signal },
);

// User clicked "stop"
setTimeout(() => ctrl.abort(), 5_000);

try {
  for await (const chunk of stream) {
    process.stdout.write(chunk.choices[0]?.delta?.content ?? '');
  }
} catch (err) {
  if ((err as Error).name === 'AbortError') {
    console.log('user cancelled');
  } else {
    throw err;
  }
}

Reconnect on drop

If the upstream connection drops before [DONE], cr.stream() retries up to twice. It sends Last-Event-ID with the most recent chunk id so a supporting upstream can resume rather than restart.

const stream = cr.stream({ /* ... */ }); // reconnect built-in

Disable reconnect by switching to cr.openai() (the OpenAI SDK's streaming has no reconnect) or by calling the lower-level helper with maxReconnects: 0.

Reasoning content in the stream

Models like DeepSeek-R1 emit a reasoning_content delta before the final answer. The SDK types it explicitly:

for await (const chunk of stream) {
  const delta = chunk.choices[0]?.delta;
  if (delta?.reasoning_content) {
    // Chain-of-thought tokens (typically hidden in UI)
  }
  if (delta?.content) {
    process.stdout.write(delta.content);
  }
}

The gateway can't replay a stream on a different provider once the first bytes have left. Streaming failover happens only before the first chunk is sent to the client. For non-stream requests, the gateway retries the next eligible EU provider automatically on a 5xx.

Chat completions — request shape, params.
Function calling — streaming tool deltas.
Edge runtime — forwarding the stream from a Worker.
Retry & timeout — reconnect vs retry.