Expand description
This will be the API as of v0.4.0. Other APIs will not be deprecated, but moved into a module such as this one.
Structs§
- AnyMoe
Model Builder - Diffusion
Model Builder - Configure a text model with the various parameters for loading, running, and other inference behaviors.
- Gguf
Lora Model Builder - Wrapper of
GgufModelBuilderfor LoRA models. - Gguf
Model Builder - Configure a text GGUF model with the various parameters for loading, running, and other inference behaviors.
- GgufX
Lora Model Builder - Wrapper of
GgufModelBuilderfor X-LoRA models. - Lora
Model Builder - Wrapper of
TextModelBuilderfor LoRA models. - Model
- The object used to interact with the model. This can be used with many varietes of models,
and as such may be created with one of: - Paged
Attention Meta Builder - Builder for PagedAttention metadata.
- Request
Builder - A way to add messages with finer control given.
- Text
Messages - Plain text (chat) messages.
- Text
Model Builder - Configure a text model with the various parameters for loading, running, and other inference behaviors.
- Text
Speculative Builder - Uqff
Text Model Builder - Configure a UQFF text model with the various parameters for loading, running, and other inference behaviors.
This wraps and implements
DerefMutfor the TextModelBuilder, so users should take care to not call UQFF-related methods. - Uqff
Vision Model Builder - Configure a UQFF text model with the various parameters for loading, running, and other inference behaviors.
This wraps and implements
DerefMutfor the VisionModelBuilder, so users should take care to not call UQFF-related methods. - Vision
Messages - Text (chat) messages with images.
- Vision
Model Builder - Configure a vision model with the various parameters for loading, running, and other inference behaviors.
- XLora
Model Builder - Wrapper of
TextModelBuilderfor X-LoRA models.
Enums§
- Text
Message Role - A chat message role.
Traits§
- Request
Like - A type which can be used as a chat request.
Functions§
- best_
device - Gets the best device, cpu, cuda if compiled with CUDA, or Metal