A tiny lil 9B model that can reliably power a conversational Hermes Agent that actually uses tools and stays on task!

This is the exact model from edtorre/qwen3.5-hermes, but with the following tweaks:

• CTX window: 96k
• Temp: 0.7
• Top_K: 20
• Top_P: 0.9
• Presence_Penalty: 1.5
• Num_Predict: 8192

Additionally, as per the original repo, this is made more persistent by passing the following environment variables before running Ollama:

Environment="OLLAMA_KV_CACHE_TYPE=q8_0"
Environment="OLLAMA_FLASH_ATTENTION=1"

I recommend adding them to your Ollama server’s systemd module if you use that. If not, it has to be a flag you pass to the Ollama binary each time.

Here’s a little script that adds the correct Ollama environment variables, assuming you’re on any system that uses systemd.

#!/bin/bash
# Deploy script: qwen3.5:4b discipline variant + KV cache quantization
# Run with: sudo bash /tmp/deploy-qwen4b-discipline.sh
set -e

echo "=== 1. Add KV cache type env vars to ollama.service ==="
# Drop-in override (cleaner than editing the main unit)
mkdir -p /etc/systemd/system/ollama.service.d
cat > /etc/systemd/system/ollama.service.d/kv-quant.conf << 'EOF'
[Service]
Environment="OLLAMA_KV_CACHE_TYPE=q8_0"
Environment="OLLAMA_FLASH_ATTENTION=1"
EOF

echo "=== 2. Reload systemd + restart ollama ==="
systemctl daemon-reload
systemctl restart ollama
sleep 3
systemctl is-active ollama

echo "=== 3. Verify env vars took ==="
systemctl show ollama -p Environment

echo "=== 4. Create discipline variant (256k context) ==="
# Uses the REST API - weights shared with base qwen3.5:9b, only params change
curl -s http://localhost:11434/api/create -d '{
  "name": "qwen3.5-hermes-discipline-96k",
  "from": "qwen3.5:9b",
  "parameters": {
    "num_ctx": 262144,
    "num_gpu": 99,
    "temperature": 0.7,
    "top_k": 20,
    "top_p": 0.9,
    "presence_penalty": 1.5,
    "num_predict": 8192
  }
}'

echo ""
echo "=== 5. Verify variant exists ==="
ollama list | grep -E "qwen3.5|NAME"

echo ""
echo "DONE. Test with:"
echo "  ollama run qwen3.5-hermes-discipline-96k \"hey\""