Constantsยง
- CHAT_
TEMPLATE - CPU
- DEVICE
- ENABLE_
SEARCH - INTERACTIVE_
MODE - IN_
SITU_ QUANT - JINJA_
EXPLICIT - LOG
- MAX_
SEQS - MODEL
- NO_
KV_ CACHE - NUM_
DEVICE_ LAYERS - PAGED_
ATTN - PAGED_
ATTN_ BLOCK_ SIZE - PAGED_
ATTN_ CPU - PAGED_
ATTN_ CUDA - PAGED_
ATTN_ GPU_ MEM - PAGED_
ATTN_ GPU_ MEM_ USAGE - PAGED_
ATTN_ METAL - PAGED_
CACHE_ TYPE - PAGED_
CTXT_ LEN - PREFIX_
CACHE_ N - SEARCH_
CALLBACK - SEARCH_
EMBEDDING_ MODEL - SEED
- TOKEN_
SOURCE