Crate llguidance

Modules§

api
derivre
earley: This is the primary interface for llguidance – the one on which the others (FFI and LLInterpreter) are built. While not cleanest of these interfaces, it is the most inclusive.
ffi
output
panic_utils
substring
toktrie

CommitResult
Constraint
GrammarBuilder
Instant: A measurement of a monotonically nondecreasing clock. Opaque and useful only with Duration.
JsonCompileOptions
Logger
Matcher: This is meant to be used in server-side scenarios. The Constraint interface is more for usage in Python Guidance.
NodeRef
ParserFactory
StopController
TokenParser

json_merge
regex_to_lark: Make sure given regex can be used inside /…/ in Lark syntax. Also if use_ascii.contains('d') replace \d with [0-9] and \D with [^0-9]. Similarly for \w/\W ([0-9a-zA-Z_]) and \s/\S ([ \t\n\r\f\v]). For standard Unicode Python3 or Rust regex crate semantics use_ascii = "" For JavaScript or JSON Schema semantics use_ascii = "dw" For Python2 or byte patters in Python3 semantics use_ascii = "dws" More flags may be added in future.
token_bytes_from_tokenizer_json: Parse HF tokenizer.json file and return bytes for every token