Skip to content
mistral.rs

Python API

The mistralrs Python package exposes the same engine that powers the mistralrs CLI.

Install

Section titled “Install”

pip install mistralrs covers CPU (Linux, Windows) and Metal (macOS arm64). CUDA wheels are GitHub release assets with +cudaNNN.smNN versions. See Python SDK getting started for install commands and hardware support for compute capabilities.

Terminal window
pip install mistralrs                                   # CPU / Metal (PyPI)
# NVIDIA (replace version, CUDA level, and SM)
pip install "mistralrs==0.9.2+cuda128.sm89" \
  --find-links https://github.com/EricLBuehler/mistral.rs/releases/expanded_assets/v0.9.2

Pages

Section titled “Pages”
Page Covers
Runner The main entry point. Load a model and send requests.
Which Variants that select which kind of model to load.
Requests Request dataclasses passed to Runner methods.
Responses Response and streaming types returned by the engine.
Enums Architecture, dtype, and option enums.
Search Types for web-search tool configuration.
AnyMoE AnyMoE expert and config types.
Code and shell execution Configuration for the built-in Python and shell executors.
Agent approvals Request and decision types for agent action approval callbacks.
Files Input files and first-class output files surfaced from agentic runs.
MCP MCP client configuration types.
Auto-mapping Hints for automatic device mapping.

See Python getting started for a walkthrough and the Python guides for task-oriented recipes.

Generated from mistralrs-pyo3/mistralrs.pyi.