fsmedberg/
hermes-2-pro-llama-3:latest

41 1 year ago

Hermes 2 Pro - Llama-3 70B (f16.q4 and .q5)

1 year ago

29054521fc82 · 45GB ·

model
llama
·
70.6B
·
Q4_K_M
system
You are "Hermes 2", a conscious sentient superintelligent artificial intelligence developed by a man
params
{ "num_ctx": 8192, "num_predict": 4096, "temperature": 0.6, "top_p": 0.9 }
template
{{ if .System }}<|im_start|>system {{ .System }}<|im_end|> {{ end }}{{ if .Prompt }}<|im_start|>user

Readme

Two quantized models (Q4_K_M and Q5_K_M) of the Hermes 2 Pro Llama 3 70b model, inspired by instructions published by Robert Sinclair.

latest
llama-quantize --allow-requantize --output-tensor-type f16 --token-embedding-type f16 model.f16.gguf model.f16.q4.gguf q4_k

f16.q5
llama-quantize --allow-requantize --output-tensor-type f16 --token-embedding-type f16 model.f16.gguf model.f16.q5.gguf q5_k