Skip to content
mistral.rs

Python API

The mistralrs Python package exposes the same engine that powers the mistralrs CLI.

Install

Section titled “Install”

pip install mistralrs covers CPU (Linux, Windows) and Metal (macOS arm64). CUDA wheels are GitHub release assets with +cudaNNN.smNN versions. See Python SDK getting started for install commands and hardware support for compute capabilities.

Terminal window
pip install mistralrs                                   # CPU / Metal (PyPI)
pip install "mistralrs==0.8.23+cuda128.sm89" \          # NVIDIA (replace version, CUDA level, and SM)
  --find-links https://github.com/EricLBuehler/mistral.rs/releases/expanded_assets/v0.8.23

Pages

Section titled “Pages”
PageCovers
RunnerThe main entry point. Load a model and send requests.
WhichVariants that select which kind of model to load.
RequestsRequest dataclasses passed to Runner methods.
ResponsesResponse and streaming types returned by the engine.
EnumsArchitecture, dtype, and option enums.
SearchTypes for web-search tool configuration.
AnyMoEAnyMoE expert and config types.
Code and shell executionConfiguration for the built-in Python and shell executors.
Agent approvalsRequest and decision types for agent action approval callbacks.
FilesInput files and first-class output files surfaced from agentic runs.
MCPMCP client configuration types.
Auto-mappingHints for automatic device mapping.

See Python getting started for a walkthrough and the Python guides for task-oriented recipes.

Generated from mistralrs-pyo3/mistralrs.pyi.