koesn/dolphin-llama3-8b:Q6

koesn/

dolphin-llama3-8b:Q6_K

201 Downloads Updated 1 year ago

Fixed num_ctx to 8192. This Dolphin 2.9 Llama 3 8B model is ready to use for full model's 8k contexts window.

Updated 1 year ago

1 year ago

5597af3f81ce · 6.6GB ·

archllama

parameters8.03B

quantizationQ6_K

6.6GB

{ "num_ctx": 8192, "stop": [ "<|im_start|>", "<|im_end|>" ] }

74B

<|im_start|>system {{ .System }} <|im_start|>user {{ .Prompt }} <|im_start|>assistant

87B

You are Dolphin, an uncensored and unbiased AI assistant.

59B

Dolphin-2.9-Llama-3-8B

Model Quants	Size	Bit	Perplexity
dolphin-llama3-8b:Q4_0	4.7GB	4	+0.2166 ppl
dolphin-llama3-8b:Q4_K_M	4.9GB	4	+0.0532 ppl
dolphin-llama3-8b:Q5_K_M	5.7GB	5	+0.0122 ppl
dolphin-llama3-8b:Q6_K	6.6GB	6	+0.0008 ppl

max_position_embeddings : 128258
rope_theta : 500000.0
vocab_size : 8192