q4_k_m quantization only. Now using 8k context size

7B

79 Pulls Updated 10 months ago

Readme

I’ve only uploaded the -q4_k_m quantization

2023.11.06 Updated modelfile with
PARAMETER num_ctx 8192

Original Model on HuggingFace

Quantized by TheBloke