Module mistralrs_for_server_builder

mistral.rs instance for server builder.

Modules

defaults

Provides the default values used for the mistral.rs instance for server. These defaults can be used for CLI argument fallbacks, config loading, or general initialization.

Structs

MistralRsForServerBuilder

A builder for creating a mistral.rs instance with configured options for the mistral.rs server.

ModelConfig

Configuration for a single model in a multi-model setup

Functions

configure_paged_attn_from_flags

Configures PagedAttention based on two flags.

get_bert_model

Creates a BERT embedding model configuration for search functionality.