Web search
Web search lets the model look things up before it answers. It gets two tools: one runs a search query and returns the top results with their page text, and the other fetches the text of a specific URL. Use it for current events, documentation lookups, or anything past the model’s training data.
Searches go to DuckDuckGo and return up to 10 results per query. Each page is cleaned down to its main text before the model sees it.
Turning it on
Section titled “Turning it on”Start the server with --enable-search (or --agent), then turn search on for each request that should use it. A server started without it rejects search requests with a 400 error, so clients can’t make it fetch from the web unless you allow it.
mistralrs serve --enable-search -m <model>Chat Completions requests turn search on with web_search_options. An empty object {} uses the defaults:
{ "model": "default", "messages": [{"role": "user", "content": "What happened at CES this year?"}], "web_search_options": { "search_context_size": "medium" }}Responses requests use OpenAI’s hosted-tool syntax instead:
{ "model": "default", "input": "What happened at CES this year?", "tools": [ { "type": "web_search", "search_context_size": "medium", "return_token_budget": "default" } ], "tool_choice": "required"}from mistralrs import ChatCompletionRequest, Runner, WebSearchOptions, Which
runner = Runner( which=Which.Plain(model_id="Qwen/Qwen3-4B"), enable_search=True,)
response = runner.send_chat_completion_request( ChatCompletionRequest( model="default", messages=[{"role": "user", "content": "What happened at CES this year?"}], web_search_options=WebSearchOptions(), ))use mistralrs::{IsqBits, ModelBuilder, RequestBuilder, SearchEmbeddingModel, WebSearchOptions};
let model = ModelBuilder::new("Qwen/Qwen3-4B") .with_auto_isq(IsqBits::Four) .with_search(SearchEmbeddingModel::default()) .build() .await?;
let request = RequestBuilder::from(messages).with_web_search_options(WebSearchOptions::default());The search and page tools use strict tool calling, so the model’s queries and URLs always arrive as valid arguments.
Reranking
Section titled “Reranking”A search returns more page text than the model can usefully read, so mistral.rs picks the most relevant parts before passing them on. It does this one of two ways:
| How results are picked | What the model gets | |
|---|---|---|
| With a reranker | Pages are split into passages of about 500 tokens. A keyword pass keeps the best 20, and an embedding model orders those by how well they answer the query. | The most relevant passages, drawn from across all the results. |
| Without a reranker | Whole pages are ranked by keyword overlap with the query. | As many complete pages as fit in the budget. |
The reranker gives the model more relevant text, from more sources, in the same budget. The cost is an extra small model, google/embeddinggemma-300m, which is downloaded and loaded at startup.
The reranker is on by default whenever you turn on search: --enable-search or --agent on the CLI, enable_search=True in Python, and .with_search(...) in Rust. --search-embedding-model embedding-gemma selects it explicitly; it’s currently the only choice. In Rust, a request with web_search_options on a model built without .with_search(...) still searches, using keyword ranking only.
Per-request options
Section titled “Per-request options”These fields go in web_search_options for Chat Completions, or in the web_search tool entry for Responses.
| Field | Values | What it does |
|---|---|---|
search_context_size |
low, medium (default), high |
How much result text the model gets per search: about 4,096, 8,192, or 16,384 tokens. |
user_location |
Any of city, region, country, timezone, inside {"type": "approximate", "approximate": {...}} for Chat Completions or directly in {"type": "approximate", ...} for Responses |
Tells the model where the user is, so it can search for local results such as weather or opening hours. |
filters |
allowed_domains, blocked_domains (Responses allows up to 100 each) |
Keeps only results from allowed domains, or drops results from blocked ones. A domain also covers its subdomains. |
return_token_budget |
default, unlimited |
Responses web_search only. unlimited uses the high budget whatever search_context_size says. |
search_content_types |
["text"] |
Text is the only content type. Responses rejects a request for images. |
external_web_access |
true (default), false |
Responses web_search only. false turns search off for that request. |
search_description, extract_description |
Any text | Chat Completions and SDKs. Replaces the description the model sees for the search tool or the page tool, for example to steer it toward particular sources. |
Responses also accepts web_search_preview, without filters or return_token_budget.
Custom search backends
Section titled “Custom search backends”To search your own documents, an internal wiki, or a private search service instead of DuckDuckGo, pass a search_callback in the Python or Rust SDK. It receives the model’s query and returns a list of results. The results then go through the same filters and ranking as web results.
The callback returns a list of dicts with keys title, description, url, and content:
def my_search(query: str) -> list[dict]: return [ {"title": "...", "description": "...", "url": "internal://...", "content": "..."}, ]
runner = Runner( which=Which.Plain(model_id="Qwen/Qwen3-4B"), enable_search=True, search_callback=my_search,)The callback receives SearchFunctionParameters and returns Vec<SearchResult>:
use std::sync::Arc;use mistralrs::{ModelBuilder, SearchFunctionParameters, SearchResult};
let model = ModelBuilder::new("Qwen/Qwen3-4B") .with_search_callback(Arc::new(|params: &SearchFunctionParameters| { Ok(vec![SearchResult { title: "...".to_string(), description: "...".to_string(), url: "internal://...".to_string(), content: "...".to_string(), }]) })) .build() .await?;