List of all items

Structs

• BnbLinear
• BnbQuantParmas
• CollectedImatrixData
• DummyLayer
• FP8Linear
• GgufMatMul
• GptqLayer
• HqqConfig
• HqqLayer
• ImatrixLayerStats
• MatMul
• QuantizedConfig
• StaticLoraConfig
• UnquantLinear
• distributed::Comm
• distributed::Id
• distributed::SumAllReduce
• distributed::layers::ColumnParallelLayer
• distributed::layers::ReplicatedLayer
• distributed::layers::RowParallelLayer
• distributed::socket::Client
• distributed::socket::Server
• safetensors::MmapedSafetensors

Enums

• BnbQuantType
• HqqAxis
• HqqBits
• IsqType
• QuantMethodConfig
• QuantMethodType
• QuantizedSerdeType
• safetensors::Shard
• safetensors::ShardedSafeTensors

Traits

• QuantMethod
• QuantizedSerde
• distributed::BarrierLike
• distributed::DistributedOperation
• safetensors::Load

Functions

• distributed::get_global_tp_size_from_devices
• distributed::layers::compute_kv_shard
• distributed::layers::compute_n_kv_groups
• distributed::use_nccl
• linear
• linear_b
• linear_no_bias
• linear_no_bias_static_lora

Type Aliases

• safetensors::ShardedVarBuilder

Constants

• UQFF_QUANT_TYPE_OFFSET