Skip to content

CLI reference

Subcommand Purpose
mistralrs serve Start an OpenAI-compatible HTTP server, with a web UI at /ui
mistralrs run Chat with a model interactively, or send a single prompt with -i
mistralrs completions Generate shell completions
mistralrs quantize Quantize a model and save it as UQFF files
mistralrs uqff Inspect, report, or verify UQFF artifacts
mistralrs doctor Run system diagnostics and environment checks
mistralrs tune Recommend a quantization level and device mapping that fit your hardware
mistralrs download Download the files needed to run a model without loading it
mistralrs cache Manage the Hugging Face model cache
mistralrs bench Run performance benchmarks for base or LoRA model generation
mistralrs from-config Run from a full TOML configuration file
mistralrs update Update or migrate an install using the installer
mistralrs uninstall Remove an installer-managed install
Option Default Description
--seed <SEED> Random seed for reproducibility
-l, --log <LOG> Log all requests and responses to this file
--token-source <TOKEN_SOURCE> cache Token source for Hugging Face authentication. Formats: literal:<token>, env:<var>, path:<file>, cache, none
-v, --verbose 0 Increase logging verbosity. Use -v for debug and -vv for trace-level internals