Configure the HTTP server

mistralrs serve defaults: bind on 0.0.0.0:1234, CORS open, no authentication, 50 MB request body limit.

Host and port

mistralrs serve --host 127.0.0.1 --port 8080 -m <model>

--host controls the bind interface. 0.0.0.0 (default) accepts connections from any host on the network; 127.0.0.1 restricts to the local machine. --port is the TCP port (default 1234).

CORS and body limit

CORS allowed origins and the request body limit are not exposed as CLI flags. They can be configured programmatically through MistralRsServerRouterBuilder in mistralrs-server-core.

The default allowed origin is any. The default body limit is 50 MB. Allowed methods are GET, POST, PUT, DELETE; allowed headers include Content-Type and Authorization.

Authentication

mistral.rs does not implement authentication. The intended pattern is a reverse proxy (nginx, Caddy, Traefik) handling authentication and TLS.

OpenAI-protocol clients always send an Authorization: Bearer ... header because the OpenAI SDK requires an API key at initialization. mistral.rs does not validate the header.

Logging

mistralrs serve -v -m <model>

Use -vv for trace-level file/cache internals. Module filters are still available with RUST_LOG, for example RUST_LOG=mistralrs_core=debug,tower_http=info.

Config file versus flags

Most CLI flags have a TOML config equivalent. Run with mistralrs from-config -f config.toml.

Full schema: CLI TOML config reference. Minimal example:

command = "serve"

[server]
host = "127.0.0.1"
port = 8080

[runtime]
enable_search = true
enable_code_execution = true

[[models]]
kind = "text"
model_id = "Qwen/Qwen3-4B"

[models.quantization]
in_situ_quant = "4"