Crate mistralrs_quant

Re-exports

• pub use safetensors::Shard;
• pub use safetensors::ShardedSafeTensors;
• pub use safetensors::ShardedVarBuilder;

Modules

• safetensors

Structs

• BnbLinear
• BnbQuantParmas
• CollectedImatrixData
• ColumnParallelLayer
  This layer has a weight that is parallelized along the output dimension, taking the “full” input dimension.
• Comm
• DummyLayer
• FP8Linear
• GgufMatMul
• GptqLayer
• HqqConfig
• HqqLayer
• Id
• ImatrixLayerStats
• MatMul
  Device/configurable intelligent matrix multiplication
• QuantizedConfig
• ReplicatedLayer
  This layer has no parallelization
• RowParallelLayer
  This layer has a weight that is parallelized along the input dimension, returning the “full” output dimension.
• SumAllReduce
• UnquantLinear

Enums

• BnbQuantType
• HqqAxis
• HqqBits
• IsqType
• QuantMethodConfig
• QuantMethodType
• QuantizedSerdeType

Constants

• UQFF_QUANT_TYPE_OFFSET
  Offset for the quant type. UQFF always serializes the version first.

Statics

• INHIBIT_GEMM_F16

Traits

• QuantMethod
  Quantized method for a quantized matmul.
• QuantizedSerde

Functions

• get_use_matmul_via_f16
• linear
• linear_b
• linear_no_bias
• set_use_matmul_via_f16