pub fn fp8_vector_dequantize(
weight: &Tensor,
inv_scales: &Tensor,
out_ty: DType,
) -> Result<Tensor>
Expand description
FP8 vector dequantize.
- Expects weight to be fp8
- Expects inv_scales to be f32
- weight * inv_scale = dequantized
pub fn fp8_vector_dequantize(
weight: &Tensor,
inv_scales: &Tensor,
out_ty: DType,
) -> Result<Tensor>
FP8 vector dequantize.