Auto-mapping
TextAutoMapParams
Section titled “TextAutoMapParams”Auto-mapping parameters for a text model. These affect automatic device mapping but are not a hard limit.
| Field | Type | Default |
|---|---|---|
max_seq_len | int | 4 * 1024 |
max_batch_size | int | 1 |
MultimodalAutoMapParams
Section titled “MultimodalAutoMapParams”Auto-mapping parameters for a multimodal model. These affect automatic device mapping but are not a hard limit.
| Field | Type | Default |
|---|---|---|
max_seq_len | int | 4 * 1024 |
max_batch_size | int | 1 |
max_num_images | int | 1 |
max_image_length | int | 1024 |
Generated from mistralrs-pyo3/mistralrs.pyi.