Zen-5-Coder GGUF

GGUF Q4_K_M quantization of Zen-5-Coder 80B for Ollama and compatible runtimes.

The original model was released by Zen LM in Hugging Face Transformers format.

This repository provides converted and quantized version optimized for local inference.

Full documentation: https://huggingface.co/VibeManGeo/Zen-5-Coder-GGUF

---

Overview

Property	Value
Model	Zen-5-Coder
Architecture	Mixture of Experts (MoE)
Parameters	80B
Original Format	Hugging Face Transformers
GGUF Conversion	llama.cpp
Repository Maintainer	VibeManGeo

---

Available Quantizations

Quantization	Description
Q4_K_M	Recommended default

---

Conversion Pipeline

All files were generated locally using the standard llama.cpp workflow:

Hugging Face Transformers
        ↓
GGUF FP16
        ↓
GGUF Quantization
        ↓
Ollama create

Tools Used

• llama.cpp
• convert_hf_to_gguf.py
• llama-quantize
• ollama

---

Hardware Used For Conversion

The quantization in this repository war generated and tested on:

• GPU 0 NVIDIA RTX 3060 12 GB Headless
• GPU 1 NVIDIA Tesla P40 24 GB Headless
• AMD Ryzen 7 5700G
• 64 GB DDR-4 3200Mhz System RAM
• Debian Linux 13.2

Actual performance will depend on context size, quantization level, GPU offloading, and runtime configuration.

---

Credits

Original Model

Zen LM — creators of Zen-5-Coder.

GGUF Conversion & Quantization

VibeManGeo

Fun fact: these 80B quantizations were produced before the author passed CompTIA A+ Core 1.

---

Acknowledgements

Special thanks to the llama.cpp developers for providing the tools that make efficient local inference and GGUF quantization possible.

---

Disclaimer

This repository contains converted and quantized derivatives of the original model. All credit for model architecture, training, datasets, and original weights belongs to the original authors.

---

Notes

The goal of this repository is to make Zen-5-Coder immediately accessible to the local inference community without requiring users to perform the conversion process themselves.