Function fp8_vector_dequantize

Source
pub fn fp8_vector_dequantize(
    weight: &Tensor,
    inv_scales: &Tensor,
    out_ty: DType,
) -> Result<Tensor>
Expand description

FP8 vector dequantize.

  • Expects weight to be fp8
  • Expects inv_scales to be f32
  • weight * inv_scale = dequantized