2 2 weeks ago

Generates excellent search queries, title summaries, etc. Substantial track record. IQ2_XXS - Adequate for the strengths of this AI LLM model. Compatible with ~16GB VRAM.

tools
{
"num_ctx": 3584,
"num_gpu": 999,
"num_keep": 3584,
"num_predict": 8192,
"num_thread": 14,
"repeat_penalty": 1.1,
"temperature": 0.6,
"top_p": 0.95
}