quantized versions of mlabonne/NeuralBeagle14-7B
141 Pulls Updated 10 months ago
Updated 10 months ago
10 months ago
3107cf5b688f · 5.9GB
model
archllama
·
parameters7.24B
·
quantizationQ6_K
5.9GB
template
<|im_start|>system
{{ .System }}<|im_end|>
<|im_start|>user
{{ .Prompt }}<|im_end|>
<|im_start|>assi
105B
params
{
"num_ctx": 4096,
"stop": [
"<|im_end|>",
"|im_end|>",
"<|im_start|
116B
Readme
q4_K_M, q6_K and q8_0 quantized versions of mlabonne/NeuralBeagle14-7B
Supports up to 8K context. Modelfile is configured for 4K.