llg_clone_tokenizer

mistralrs::llguidance::ffi

Function llg_clone_tokenizer 

#[unsafe(no_mangle)]
pub extern "C" fn llg_clone_tokenizer(
    tok: &LlgTokenizer,
) -> *mut LlgTokenizer
Expand description

Clone a tokenizer. This increments a reference count and does a small allocation.