Module ffi

Structs

LlgCommitResult

Represents result from llg_commit_token()

LlgConstraint

LlgConstraintInit

LlgConstraintStep

LlgMaskResult

LlgStopController

LlgTokenizer

LlgTokenizerInit

Functions

constraint_to_llg

llg_clone_constraint

Clone the constraint

llg_clone_tokenizer

Clone a tokenizer. This increments a reference count and does a small allocation.

llg_commit_token

Commit the token sampled with the mask returned from llg_compute_mask(). Can be run on the critical path of sampling (is fast). Returns 0 on success and -1 on error (use llg_get_error() to get the exact error). When 0 is returned, the result is written to *res_p.

llg_compute_mask

Compute mask for the next token sampling It typically takes up to a millisecond for a 100k tokenizer, so should be called in background. Returns 0 on success and -1 on error (use llg_get_error() to get the exact error). When 0 is returned, the result is written to *res_p.

llg_constraint_init_set_defaults

Set the default values for the ConstraintInit Disables ff_tokens and backtracking, enables warnings on stderr and all logging to the buffer (get with llg_flush_logs()). You need to set the tokenizer field manually.

llg_flush_logs

Get the logs from the constraint, since last call to this function. The logs are null-terminated. The logs are kept in the constraint until the next call to this function or until the constraint is freed.

llg_free_constraint^⚠

Free the constraint

llg_free_stop_controller^⚠

Free the stop-sequence controller

llg_free_tokenizer^⚠

Free the tokenizer. Should NOT be called while there are still constraints using it.

llg_get_error

Get the error message from the constraint or null if there is no error. After it returns a non-null value, it will always return it until the constraint is freed using llg_free_constraint() (at which point the pointer will be invalid).

llg_get_temperature

Get the current temperature of the constraint. It is updated by mask computation.

llg_is_stopped

Check if constraint is stopped (cannot be extended further).

llg_new_constraint

Create a new constraint from a grammar JSON string Always returns a non-null value. Call llg_get_error() on the result to check for errors.

llg_new_constraint_any

Create a new constraint with specified type Type can be one of “regex”, “json_schema” (or “json”), “lark”, “llguidance” (or “guidance”) Always returns a non-null value. Call llg_get_error() on the result to check for errors.

llg_new_constraint_json

Create a new constraint from a given JSON schema Always returns a non-null value. Call llg_get_error() on the result to check for errors.

llg_new_constraint_lark

Create a new constraint from a given lark grammar Always returns a non-null value. Call llg_get_error() on the result to check for errors.

llg_new_constraint_regex

Create a new constraint from a given regular expression Always returns a non-null value. Call llg_get_error() on the result to check for errors.

llg_new_stop_controller^⚠

Create a new stop-sequence controller

llg_new_tokenizer

Construct a new tokenizer from the given TokenizerInit

llg_par_compute_mask^⚠

Compute mask for several constraints in parallel.

llg_stop_commit_token

Commit a token to the stop-sequence controller. Returns a valid utf8 string to be returned to the user (which can be empty) and whether the sequence should be then finished. The string is valid until the next call to this function, or until the stop-sequence controller is freed.

llg_stringify_tokens^⚠

Return a string representation of the tokens, useful for debugging. The output is null-terminated. Returns the number of bytes that would be written to output if output_len was large enough.

llg_tokenize_bytes^⚠

Tokenize the given bytes and return the tokens. Always returns the number of tokens that would be written to output_tokens if output_tokens_len was large enough.

llg_tokenize_bytes_marker^⚠

Tokenize the given bytes and return the tokens. Special tokens will be tokenized, if they follow 0xFF byte prefix. Always returns the number of tokens that would be written to output_tokens if output_tokens_len was large enough.

Type Aliases

LlgCallback

Function which llg calls when an operation is done.

LlgToken

LlgTokenizeFn

Tokenization function Will not write more than output_tokens_len tokens (which can be 0) Returns the total number of tokens (which can be more than output_tokens_len) This function has to be thread-safe!