mistralrs_quant::distributed::layers

Function compute_n_kv_groups

Source
pub fn compute_n_kv_groups(
    total_num_kv_heads: usize,
    num_attention_heads: usize,
    comm: &Comm,
) -> usize
Expand description

Compute the number of KV groups, taking into account KV head replication.