Module v0_4_api

This will be the API as of v0.4.0. Other APIs will not be deprecated, but moved into a module such as this one.

Structs

• AnyMoeModelBuilder
• DiffusionModelBuilder
  Configure a text model with the various parameters for loading, running, and other inference behaviors.
• GgufLoraModelBuilder
  Wrapper of GgufModelBuilder for LoRA models.
• GgufModelBuilder
  Configure a text GGUF model with the various parameters for loading, running, and other inference behaviors.
• GgufXLoraModelBuilder
  Wrapper of GgufModelBuilder for X-LoRA models.
• LoraModelBuilder
  Wrapper of TextModelBuilder for LoRA models.
• Model
  The object used to interact with the model. This can be used with many varietes of models,
  and as such may be created with one of:
• PagedAttentionMetaBuilder
  Builder for PagedAttention metadata.
• RequestBuilder
  A way to add messages with finer control given.
• TextMessages
  Plain text (chat) messages.
• TextModelBuilder
  Configure a text model with the various parameters for loading, running, and other inference behaviors.
• VisionMessages
  Text (chat) messages with images.
• VisionModelBuilder
  Configure a vision model with the various parameters for loading, running, and other inference behaviors.
• XLoraModelBuilder
  Wrapper of TextModelBuilder for X-LoRA models.

Enums

• TextMessageRole
  A chat message role.

Traits

• RequestLike
  A type which can be used as a chat request.

Functions

• best_device
  Gets the best device, cpu, cuda if compiled with CUDA, or Metal