Calme 3.2 Instruct 78B - GGUF Q8_0

This is a quantized GGUF version of the powerful MaziyarPanahi/calme-3.2-instruct-78b model, optimized for use with Ollama.

🎯 Model Details

• Original Model: MaziyarPanahi/calme-3.2-instruct-78b
• Model Creator: MaziyarPanahi
• Base Model: Qwen/Qwen2.5-72B (self-merged to 78B)
• Quantization: Q8_0 (8-bit quantization)
• Model Size: ~82.8GB
• Context Length: 32,768 tokens
• License: Qwen License

📊 Performance Benchmarks

Metric	Value
Average	52.02
IFEval (0-Shot)	80.63
BBH (3-Shot)	62.61
MATH Lvl 5 (4-Shot)	39.95
GPQA (0-shot)	20.36
MuSR (0-shot)	38.53
MMLU-PRO (5-shot)	70.03

🚀 Quick Start

ollama run richardyoung/calme-3.2:78b-q8

💬 Example Usage

# Simple query
ollama run richardyoung/calme-3.2:78b-q8 "Explain quantum computing in simple terms"

# With system prompt
ollama run richardyoung/calme-3.2:78b-q8 --system "You are a helpful coding assistant" "Write a Python function to calculate fibonacci numbers"

🛠️ Customization

You can customize the model behavior by creating a Modelfile:

FROM richardyoung/calme-3.2:78b-q8

# Adjust temperature for more creative responses
PARAMETER temperature 0.8

# Custom system prompt
SYSTEM "You are a knowledgeable AI assistant specialized in technical topics."

Then create your custom model:

ollama create my-calme -f ./Modelfile

📝 Prompt Template

This model uses the ChatML format:

<|im_start|>system
{System message}
<|im_end|>
<|im_start|>user
{User message}
<|im_end|>
<|im_start|>assistant
{Assistant response}
<|im_end|>

🔧 Parameters

• temperature: 0.7 (default)
• top_p: 0.9
• top_k: 40
• num_ctx: 32768

🙏 Credits

• Original Model: Created by MaziyarPanahi
• Base Architecture: Built on Qwen2.5-72B by Alibaba Cloud
• Quantization: This Q8_0 GGUF conversion maintains high quality while reducing memory requirements

⚠️ Important Notes

• This is an experimental model that may be sensitive to hyperparameters
• Requires significant RAM (~83GB) to run effectively
• Best performance on systems with GPU acceleration

📚 Additional Resources

• Original Model Card
• GGUF Quantizations by bartowski
• EXL2 Quantization by DavidCatalano

📄 License

This model is subject to the Qwen License. Please review the license terms before use.

---

This GGUF conversion is provided for easier use with Ollama. All credit for the model development goes to MaziyarPanahi and the Qwen team.