Function compute_kv_shard

Source
pub fn compute_kv_shard(
    total_num_kv_heads: usize,
    head_dim: usize,
    comm: &Comm,
) -> Shard
Expand description

Compute the appropriate KV shard. This handles KV head replication. Be sure to use compute_n_kv_groups in tandem.