nsheth/ llama-3-lumimaid-8b-v0.1-iq-imatrix:latest

3,940 2 years ago

It uses this one Q4_K_M-imat (4.89 BPW) quant for up to 12288 context sizes. for less than 8gb vram

vision
... /
params
56bb8bd477a5 ยท 96B
{
"stop": [
"<|start_header_id|>",
"<|end_header_id|>",
"<|eot_id|>"
]
}