List of all items

Structs

• AnyMoeConfig
• AnyMoeLoader
• AnyMoePipeline
• CalledFunction
• ChatCompletionChunkResponse
• ChatCompletionResponse
• ChatTemplate
• Choice
• ChunkChoice
• CompletionChoice
• CompletionChunkChoice
• CompletionChunkResponse
• CompletionResponse
• Delta
• DetokenizationRequest
• DeviceLayerMapMetadata
• DeviceMapMetadata
• DiffusionGenerationParams
• DiffusionLoader
• DiffusionLoaderBuilder
• DiffusionSpecificConfig
• DrySamplingParams
• Function
• GGMLLoader
• GGMLLoaderBuilder
• GGMLSpecificConfig
• GGUFLoader
• GGUFLoaderBuilder
• GGUFSpecificConfig
• GemmaLoader
• Idefics2Loader
• ImageChoice
• ImageGenerationResponse
• LLaVALoader
• LLaVANextLoader
• LayerDeviceMapper
• LayerTopology
• LlamaLoader
• LoaderBuilder
• LocalModelPaths
• Logprobs
• MemoryUsage
• MistralLoader
• MistralRs
• MistralRsBuilder
• MistralRsConfig
• MixtralLoader
• NormalLoader
• NormalLoaderBuilder
• NormalRequest
• NormalSpecificConfig
• Ordering
• PagedAttentionConfig
• Phi2Loader
• Phi3Loader
• Phi3VLoader
• Qwen2Loader
• ResponseLogprob
• ResponseMessage
• SamplingParams
• SpeculativeConfig
• SpeculativeLoader
• SpeculativePipeline
• Starcoder2Loader
• Tensor
• TokenizationRequest
• Tool
• ToolCallResponse
• TopLogprob
• Topology
• Usage
• VisionLoader
• VisionLoaderBuilder
• VisionSpecificConfig
• layers::CausalMasker
• layers::Conv3dConfig
• layers::Conv3dNoBias
• layers::F32RmsNorm
• layers::Llama3RopeConfig
• layers::MatMul
• layers::PhiRopeConfig
• layers::PhiRotaryEmbedding
• layers::QLinear
• layers::QRmsNorm
• layers::Qwen2VLRotaryEmbedding
• layers::RmsNorm
• layers::RotaryEmbedding
• layers::Sdpa
• llguidance::CommitResult
• llguidance::Constraint
• llguidance::GrammarBuilder
• llguidance::JsonCompileOptions
• llguidance::Logger
• llguidance::NodeRef
• llguidance::TokenParser
• llguidance::api::GenGrammarOptions
• llguidance::api::GenOptions
• llguidance::api::GrammarWithLexer
• llguidance::api::NodeId
• llguidance::api::NodeProps
• llguidance::api::ParserLimits
• llguidance::api::RegexId
• llguidance::api::TopLevelGrammar
• llguidance::derivre::AlphabetInfo
• llguidance::derivre::ExprRef
• llguidance::derivre::JsonQuoteOptions
• llguidance::derivre::Regex
• llguidance::derivre::RegexBuilder
• llguidance::derivre::StateID
• llguidance::derivre::raw::DerivCache
• llguidance::derivre::raw::ExprSet
• llguidance::derivre::raw::NextByteCache
• llguidance::derivre::raw::RelevanceCache
• llguidance::derivre::raw::VecHashCons
• llguidance::earley::CGrammar
• llguidance::earley::CSymIdx
• llguidance::earley::DefaultBiasComputer
• llguidance::earley::Grammar
• llguidance::earley::Parser
• llguidance::earley::ParserRecognizer
• llguidance::earley::ParserStats
• llguidance::earley::lexerspec::Lexeme
• llguidance::earley::lexerspec::LexemeClass
• llguidance::earley::lexerspec::LexemeIdx
• llguidance::earley::lexerspec::LexemeSpec
• llguidance::earley::lexerspec::LexerSpec
• llguidance::earley::regexvec::AlphabetInfo
• llguidance::earley::regexvec::ExprRef
• llguidance::earley::regexvec::RegexVec
• llguidance::earley::regexvec::StateDesc
• llguidance::earley::regexvec::StateID
• llguidance::ffi::LlgCommitResult
• llguidance::ffi::LlgConstraint
• llguidance::ffi::LlgConstraintInit
• llguidance::ffi::LlgConstraintStep
• llguidance::ffi::LlgMaskResult
• llguidance::ffi::LlgTokenizer
• llguidance::ffi::LlgTokenizerInit
• llguidance::output::BytesOutput
• llguidance::output::ParserStats
• llguidance::output::Reporter
• llguidance::toktrie::Branch
• llguidance::toktrie::InferenceCapabilities
• llguidance::toktrie::SimpleVob
• llguidance::toktrie::SimpleVobIter
• llguidance::toktrie::Splice
• llguidance::toktrie::StepArg
• llguidance::toktrie::TokEnvWithTrie
• llguidance::toktrie::TokRxInfo
• llguidance::toktrie::TokTrie
• llguidance::toktrie::TrieNode
• llguidance::toktrie::bytes::U32Pair
• llguidance::toktrie::recognizer::AnythingGoes
• llguidance::toktrie::recognizer::StackRecognizer
• llguidance::toktrie::rng::Rng
• v0_4_api::AnyMoeModelBuilder
• v0_4_api::DiffusionModelBuilder
• v0_4_api::GgufLoraModelBuilder
• v0_4_api::GgufModelBuilder
• v0_4_api::GgufXLoraModelBuilder
• v0_4_api::LoraModelBuilder
• v0_4_api::Model
• v0_4_api::PagedAttentionMetaBuilder
• v0_4_api::RequestBuilder
• v0_4_api::TextMessages
• v0_4_api::TextModelBuilder
• v0_4_api::VisionMessages
• v0_4_api::VisionModelBuilder
• v0_4_api::XLoraModelBuilder

Enums

• AnyMoeExpertType
• Constraint
• DType
• DefaultSchedulerMethod
• Device
• DiffusionLoaderType
• EngineInstruction
• GGUFArchitecture
• ImageGenerationResponseFormat
• IsqOrganization
• IsqType
• MemoryGpuConfig
• MistralRsError
• ModelCategory
• ModelDType
• ModelKind
• ModelSelected
• NormalLoaderType
• Request
• RequestMessage
• Response
• ResponseErr
• ResponseOk
• SchedulerConfig
• StopTokens
• TokenSource
• ToolCallType
• ToolChoice
• ToolType
• VisionLoaderType
• layers::Activation
• layers::Llama3RopeType
• layers::Llama3RotaryEmbedding
• layers::PhiRopeScalingConfig
• layers::ScaledRopeType
• llguidance::api::GrammarId
• llguidance::api::Node
• llguidance::api::RegexNode
• llguidance::api::RegexSpec
• llguidance::api::StopReason
• llguidance::derivre::NextByte
• llguidance::derivre::RegexAst
• llguidance::earley::ParserError
• llguidance::earley::regexvec::NextByte
• llguidance::output::ParserOutput
• llguidance::toktrie::SpecialToken
• v0_4_api::TextMessageRole

Traits

• CustomLogitsProcessor
• Loader
• ModelPaths
• Pipeline
• TryIntoDType
• VisionPromptPrefixer
• llguidance::earley::BiasComputer
• llguidance::toktrie::Recognizer
• llguidance::toktrie::TokenizerEnv
• llguidance::toktrie::recognizer::FunctionalRecognizer
• v0_4_api::RequestLike

Macros

• llguidance::infoln
• llguidance::loginfo
• llguidance::warn

Functions

• cross_entropy_loss
• get_model_dtype
• get_tgt_non_granular_index
• get_toml_selected_model_dtype
• initialize_logging
• layers::get_use_matmul_via_f16
• layers::repeat_kv
• llguidance::derivre::map_ast
• llguidance::earley::grammars_from_json
• llguidance::ffi::constraint_to_llg
• llguidance::ffi::llg_clone_constraint
• llguidance::ffi::llg_clone_tokenizer
• llguidance::ffi::llg_commit_token
• llguidance::ffi::llg_compute_mask
• llguidance::ffi::llg_constraint_init_set_defaults
• llguidance::ffi::llg_flush_logs
• llguidance::ffi::llg_free_constraint
• llguidance::ffi::llg_free_tokenizer
• llguidance::ffi::llg_get_error
• llguidance::ffi::llg_get_temperature
• llguidance::ffi::llg_is_stopped
• llguidance::ffi::llg_new_constraint
• llguidance::ffi::llg_new_constraint_any
• llguidance::ffi::llg_new_constraint_json
• llguidance::ffi::llg_new_constraint_lark
• llguidance::ffi::llg_new_constraint_regex
• llguidance::ffi::llg_new_tokenizer
• llguidance::ffi::llg_par_compute_mask
• llguidance::ffi::llg_stringify_tokens
• llguidance::ffi::llg_tokenize_bytes
• llguidance::ffi::llg_tokenize_bytes_marker
• llguidance::lark_to_llguidance
• llguidance::panic_utils::catch_unwind
• llguidance::panic_utils::mk_panic_error
• llguidance::token_bytes_from_tokenizer_json
• llguidance::toktrie::bytes::clone_vec_as_bytes
• llguidance::toktrie::bytes::from_hex_string
• llguidance::toktrie::bytes::limit_bytes
• llguidance::toktrie::bytes::limit_str
• llguidance::toktrie::bytes::to_hex_string
• llguidance::toktrie::bytes::vec_from_bytes
• paged_attn_supported
• parse_isq_value
• v0_4_api::best_device

Type Aliases

• LlguidanceGrammar
• MessageContent
• Result
• llguidance::ffi::LlgCallback
• llguidance::ffi::LlgToken
• llguidance::ffi::LlgTokenizeFn
• llguidance::toktrie::StepResult
• llguidance::toktrie::TokEnv
• llguidance::toktrie::TokenId

Statics

• ENGINE_INSTRUCTIONS
• TERMINATE_ALL_NEXT_STEP

Constants

• GGUF_MULTI_FILE_DELIMITER
• SYSTEM_FINGERPRINT
• llguidance::api::DEFAULT_CONTEXTUAL