Skip to content

Anthropic Messages API

Use Anthropic clients with http://localhost:1234 as the base URL, without /v1. The server exposes /v1/messages and /v1/messages/count_tokens alongside the OpenAI endpoints. Start it with the serving guide; see coding agents for Claude Code setup.

Terminal window
curl http://localhost:1234/v1/messages \
-H 'content-type: application/json' \
-H 'x-api-key: not-used' \
-H 'anthropic-version: 2023-06-01' \
-d '{
"model": "default",
"max_tokens": 128,
"system": "You are concise.",
"messages": [
{"role": "user", "content": "Write a haiku about local inference."}
]
}'

Response shape:

{
"id": "0",
"type": "message",
"role": "assistant",
"content": [
{"type": "text", "text": "..."}
],
"model": "Qwen/Qwen3-4B",
"stop_reason": "end_turn",
"stop_sequence": null,
"usage": {
"input_tokens": 18,
"cache_creation_input_tokens": 0,
"cache_read_input_tokens": 0,
"output_tokens": 31
}
}

id is the server’s per-request sequence number and model echoes the loaded model’s name, not the "default" alias sent in the request.

The server accepts Anthropic headers for client compatibility, but does not validate x-api-key. Put authentication in a reverse proxy when exposing the server to users.

Add stream: true to the request. The server emits Anthropic message/content-block events, including text, thinking, and tool-argument deltas, plus idle pings. See HTTP streaming semantics for event names and the streaming example for a parser.

Use POST /v1/messages/count_tokens with the same request shape to count the input tokens after chat-template formatting:

Terminal window
curl http://localhost:1234/v1/messages/count_tokens \
-H 'content-type: application/json' \
-H 'x-api-key: not-used' \
-H 'anthropic-version: 2023-06-01' \
-d '{
"model": "default",
"messages": [{"role": "user", "content": "Count these tokens."}]
}'

Response:

{"input_tokens": 14}

Request fields:

Anthropic field Support
model Supported. Use default or a loaded model id.
messages Supports user, assistant, and mid-conversation system messages with string content or content blocks.
system Supported as a top-level string or text-block array.
max_tokens Supported.
temperature, top_p, top_k, min_p Supported.
stop_sequences Supported.
stream Supported.
tools Client tools are converted to OpenAI-compatible function tools. Anthropic server tools for web_search_* and code_execution_* map to mistral.rs agentic features.
tool_choice auto, any, none, and specific client tool choices are supported. any requires a client tool call when client tools are present. Anthropic server-tool choices use local agentic auto-selection.
container.skills Supports uploaded custom Skills with {"type":"custom","skill_id":"...","version":"latest"}. Anthropic-managed built-in Skills such as pptx, xlsx, docx, and pdf are not bundled.
thinking enabled and adaptive map to thinking on; disabled maps to thinking off. budget_tokens caps reasoning before switching to the answer. display: "omitted" suppresses reasoning text while preserving the thinking block lifecycle. The loaded chat template determines the effect.
output_config Native effort and JSON Schema format are supported. Effort accepts low, medium, high, xhigh, and max; max maps to local xhigh. Do not combine these fields with the equivalent mistral.rs extensions.
enable_thinking, reasoning_effort Supported as mistral.rs extensions. Effort also accepts off; none aliases off. If all controls are omitted, thinking defaults on with no selected effort. Contradictory native and extension controls return a validation error.
logit_bias, logprobs, top_logprobs Supported as mistral.rs extensions.
presence_penalty, frequency_penalty, repetition_penalty Supported as mistral.rs extensions.
response_format, grammar Supported as mistral.rs extensions. Do not set both in one request.
dry_multiplier, dry_base, dry_allowed_length, dry_sequence_breakers Supported as mistral.rs extensions.
metadata Accepted for client compatibility.

Content blocks:

Block Support
text Supported.
image Supports base64 and URL sources. Requires a multimodal model.
tool_use Supported on assistant messages.
tool_result Supported on user messages. String and text-block results are forwarded as tool messages.
thinking, redacted_thinking Accepted in request history. Returned when the model exposes separate reasoning content.

mistral.rs agentic extensions accepted on this endpoint: session_id, web_search_options, enable_code_execution, agent_permission, code_execution_permission, files, max_tool_rounds, and truncate_sequence.

Messages, client tool loops, errors, and token counts use Anthropic wire shapes. Hosted features differ:

Feature Local behavior
Prompt caching cache_control is accepted. mistral.rs uses its own automatic prefix cache, reports detected hits as cache_read_input_tokens, and reports cache_creation_input_tokens as 0; explicit breakpoints and TTLs are not implemented.
Extended thinking thinking.budget_tokens limits reasoning tokens before the engine closes the reasoning section and continues with the answer. Zero closes reasoning immediately. Reasoning and closing-marker tokens count toward max_tokens. Unsupported reasoning formats and positive budgets with thinking disabled are rejected. Returned thinking blocks include an empty structural signature because mistral.rs cannot produce Anthropic cryptographic signatures.
Rich tool results Text results are supported. Nested image, document, search-result, and citation blocks are not translated as multimodal tool content, and is_error is not separately modeled.
Server tools Web search and code execution use mistral.rs agentic implementations. Their extra streaming events are mistral.rs extensions, not Anthropic server-tool result blocks.
Beta request controls Unknown forward-compatible fields are ignored. Anthropic context management and context editing are not implemented.

The thinking budget is shared across automatic tool-call rounds in the request. Block-diffusion models reject the budget field.

The usage.reasoning_tokens extension reports reasoning tokens, excluding delimiters. These tokens are included in usage.output_tokens.

Combining a thinking budget with output_config.format, a JSON response_format, or an explicit grammar returns HTTP 400.

max_tokens must be greater than zero.

Anthropic tool definitions:

{
"tools": [
{
"name": "get_weather",
"description": "Get weather for a city.",
"input_schema": {
"type": "object",
"properties": {"city": {"type": "string"}},
"required": ["city"]
}
}
]
}

When the model asks for a tool, the response contains a tool_use content block:

{
"type": "tool_use",
"id": "call-...",
"name": "get_weather",
"input": {"city": "Paris"}
}

Return the result in a later user message with a tool_result block:

{
"role": "user",
"content": [
{
"type": "tool_result",
"tool_use_id": "call-...",
"content": "Light rain, 12 C."
}
]
}

Enable the required executors when starting the server, for example with mistralrs serve --agent .... Then include server tools in the request:

{
"tools": [
{"type": "web_search_20250305", "name": "web_search"},
{"type": "code_execution_20250825", "name": "code_execution"}
],
"agent_permission": "auto"
}

web_search_20250305 requires search enabled; web_search_20260209 also requires code execution for dynamic filtering. Code-execution declarations require the Python executor. Search accepts approximate user_location fields such as city, country, region, and timezone.

tool_choice: {"type":"none"} disables both client and server tools. Server-tool streams include mistral.rs progress, approval, and file events; see HTTP semantics.

Upload a custom Skill through /v1/skills, then reference it in a Messages request:

{
"container": {
"skills": [{"type": "custom", "skill_id": "skill_...", "version": "latest"}]
}
}

Execution requires shell support (--enable-shell or --agent). Adding a code-execution tool also enables Python execution for that request. See the Skills guide for uploads and the Messages Skills example for a complete request.

With Anthropic headers, skill-list responses use display_title, latest_version, source, has_more, and next_page. source=custom lists uploads; source=anthropic is empty because built-in Anthropic Skills are not bundled.

Generated files appear in response files[] or streamed file_produced events. Download them from /v1/files/{file_id}/content.

Example What it shows
anthropic_chat Plain non-streaming Messages request.
anthropic_streaming Anthropic SSE parsing.
anthropic_tool_calling Client-side tool use with tool_use and tool_result.
anthropic_agentic Anthropic server-tool declarations mapped to mistral.rs web search and code execution.
anthropic_skills Upload a custom Skill and use it from container.skills.