q4_k_m quantization only. Now using 8k context size

83 12 months ago

Readme

I’ve only uploaded the -q4_k_m quantization

2023.11.06 Updated modelfile with PARAMETER num_ctx 8192

Original Model on HuggingFace

Quantized by TheBloke