mistralrs::llguidance::toktrie

Type Alias StepResult

pub type StepResult = Branch<SimpleVob>;

Aliased Type§

struct StepResult {
    pub sample_mask: Option<SimpleVob>,
    pub temperature: Option<f32>,
    pub splices: Vec<Splice>,
}

Fields§

§sample_mask: Option<SimpleVob>

If None, no sampling is performed. If Some(set), only tokens from the set are allowed.

§temperature: Option<f32>

Override temperature for sampling. It may or may not be sticky.

§splices: Vec<Splice>

Describes what to do after sampling. If no sampling, there should be exactly one splice, with empty when_sampled.