8 2 weeks ago

Generates excellent search queries, title summaries, etc. Substantial track record. IQ2_XXS - Adequate for the strengths of this AI LLM model. Compatible with ~16GB VRAM. INSTRUCT version - should NOT generate thinking tokens.

tools
{
"min_p": 0.05,
"num_ctx": 2048,
"num_gpu": 999,
"num_keep": 1792,
"num_predict": 896,
"num_thread": 14,
"repeat_last_n": 96,
"repeat_penalty": 1.14,
"temperature": 0.13,
"top_k": 15,
"top_p": 0.7
}