eas / neuralbeagle14

quantized versions of mlabonne/NeuralBeagle14-7B

7B

119 Pulls Updated 7 months ago

7 months ago

f07bf0818961 · 4.4GB

model
llama
·
7.24B
·
Q4_K_M
template
<|im_start|>system {{ .System }}<|im_end|> <|im_start|>user {{ .Prompt }}<|im_end|> <|im_start|>assistant
params
{"num_ctx":4096,"stop":["<|im_end|>","|im_end|>","<|im_start|>","<|im_start|system"]}

Readme

q4_K_M, q6_K and q8_0 quantized versions of mlabonne/NeuralBeagle14-7B

Supports up to 8K context. Modelfile is configured for 4K.