Rerank

Cohere-spec rerank against Bedrock — `top_n`, billed search units, multilingual.

POST /v1/rerank re-orders a candidate set against a query. The wire format follows the Cohere rerank spec; the gateway translates to whichever Bedrock-resident rerank model you pick.

At a glance


Endpoint	`POST /v1/rerank`
Default model	`cohere/rerank-v3-5`
Provider	Bedrock `eu-central-1` (Cohere or Amazon Rerank)
Pricing unit	Search units (`meta.billed_units.search_units`)
Languages	Multilingual (Cohere); English-focused (Amazon)

Quick example

rerank.ts

import { createCleverRouter } from '@cleverrouter/sdk';

const cr = createCleverRouter({ apiKey: process.env.CLEVERROUTER_API_KEY! });

const res = await cr.rerank({
  query: 'EU GDPR for AI inference',
  documents: [
    'GDPR Article 28 covers data processors.',
    'OpenAI uses US-based servers by default.',
    'CleverRouter runs every model in the EU.',
    'Postgres 17 supports incremental sort.',
  ],
  top_n: 3,
});

for (const r of res.results) {
  console.log(r.index, r.relevance_score);
}

Options

interface RerankOpts {
  query: string;
  documents: string[];
  top_n?: number;                  // keep only top N by score
  model?: string;                  // default: 'cohere/rerank-v3-5'
  max_chunks_per_doc?: number;     // long-doc chunking before scoring
  return_documents?: boolean;      // include original text in results
  provider?: 'bedrock';            // currently the only option
}

Option	Default	Notes
`model`	`cohere/rerank-v3-5`	Or `amazon/rerank-v1` for the cheaper English one
`top_n`	all documents	Keep only the top N by relevance
`return_documents`	`false`	If `true`, results include the original text
`max_chunks_per_doc`	provider default	Splits long documents before scoring

Response shape

interface RerankResponse {
  id?: string;
  results: Array<{
    index: number;             // index into the input `documents` array
    relevance_score: number;   // 0 – 1
    document?: { text: string } | string; // only if return_documents=true
  }>;
  meta?: {
    billed_units?: { search_units?: number };
    api_version?: { version?: string };
  };
}

meta.billed_units.search_units tells you how many search units this call cost — useful for cost accounting alongside the token usage of your upstream embedding calls.

Provider notes

Model	Provider	Region	Notes
`cohere/rerank-v3-5`	Bedrock	`eu-central-1`	Multilingual, recommended default
`amazon/rerank-v1`	Bedrock	`eu-central-1`	English-focused, slightly cheaper

Why no Scaleway / Tensorix rerank?

Rerank is Bedrock-only today. Scaleway and Tensorix don't expose rerank models — pin attempts to those providers return 400 provider_unsupported.

cURL

curl https://api.cleverouter.eu/v1/rerank \
  -H "Authorization: Bearer $CLEVERROUTER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "cohere/rerank-v3-5",
    "query": "EU GDPR",
    "documents": ["doc1", "doc2", "doc3"],
    "top_n": 2
  }'

When to rerank

Rerank shines as the second stage of a retrieval pipeline:

Fast vector search (embeddings + ANN) → top-50 candidates.
Rerank the top-50 → keep top-5 for the LLM prompt.

Cohere rerank scores ~50 docs in under 200 ms — significantly cheaper than feeding all 50 to a chat model.

Common gotchas

Don't rerank without a query. Rerank scores pairs of (query, doc). For pure clustering, stick with cosine on raw embeddings.
Long documents. Cohere rerank truncates at the model's context window. Use max_chunks_per_doc for long-form content, or chunk before sending.

Embeddings — generate the initial candidate set.
Cookbook: RAG pipeline — full embed + rerank + chat flow.
Providers — what Bedrock does and doesn't host.

On this page