Reference

Lookup-oriented pages. For task-oriented documentation, start at the Quickstart; for internals, the Developer Guide.

• CLI: every subcommand and flag, generated from the clap definitions.
• TOML configuration: the mistralrs from-config schema, with CLI flag mapping.
• HTTP API: endpoints, request/response schemas, SSE events.
• OpenAI compatibility: what is implemented, ignored, and extended relative to OpenAI’s surface.
• Python API: generated from the package’s type stub.
• Rust SDK reference: the Model API surface; full rustdoc at docs.rs/mistralrs.
• MCP configuration schema: the MCP client config file format.
• Sandbox: isolation layers and threat model for code execution.
• Hardware support: supported GPUs, compute capabilities, and accelerators; which prebuilt binaries are published.
• Supported models: architectures, modalities, quantization compatibility, per-family notes.
• Quantization types: bit widths, hardware constraints, quality.
• UQFF format: on-disk layout of the UQFF quantization format.
• Cargo features: build-from-source feature flags.
• Environment variables: every env var the binary or build scripts read.
• Troubleshooting: symptom-to-cause index.