license: apache-2.0 base_model: Qwen/Qwen3.6-35B-A3B language: - en - mi tags: - gguf - llama.cpp - unsloth - moe - vision-language-model - conversational - endpoints_compatible - new-zealand pipeline_tag: image-text-to-text

library_name: llama.cpp

KiwiMate-Large-1.0

KiwiMate is an AI companion app for iOS and Android with a distinctly New Zealand cultural identity — everyday NZ English, te reo Māori, and Kiwi context baked into how it talks. KiwiMate-Large-1.0 is the flagship tier of the KiwiMate model family, built for the heaviest reasoning and longest-context workloads while staying deployable on modest hardware thanks to its mixture-of-experts design.

Model Details

KiwiMate-Large-1.0 was fine-tuned from Qwen3.6-35B-A3B on KiwiMate’s proprietary dataset of New Zealand English, te reo Māori, and Kiwi cultural context, tuned for the AI-companion persona used across the KiwiMate app. Because only ~3B of the ~35B total parameters activate per token, Large aims for noticeably stronger reasoning than Medium while staying lighter to serve than a dense model of equivalent total size — most of the expert weights can sit in system RAM, with only the active path needing to live on the GPU.

Licensing note: Qwen3.6-35B-A3B is released under Apache 2.0, which permits commercial use, fine-tuning, and redistribution without royalties — the same clean license position as Small and Medium, and a cleaner one than KiwiMate Mini’s Qwen2.5-VL-3B base (Qwen Research license).

Status

This tier had a rockier road than the others — the base model went through several candidates (Step-3.7-Flash, then Inkling-Small as a fallback) before settling on Qwen3.6-35B-A3B, and training this one surfaced its own share of problems. A few things worth knowing before you deploy:

• Text generation: weights are up and confirmed working.
• Vision: enabled — mmproj-F16.gguf is now published alongside the text weights.
• Quantized GGUFs: only the full-precision F16 split is published so far. Smaller quants (Q4_K_M, Q8_0) aren’t up yet.

Given the rocky training run, run a thorough eval pass — vision included — before routing production traffic to this tier.

Available Files

Because this is an MoE model, the full ~71 GB doesn’t need to sit in VRAM — llama.cpp can offload most of the expert weights to system RAM and keep only the ~3B active path on GPU (--n-cpu-moe / -ot expert-offload flags). A 12–24 GB GPU paired with enough system RAM to hold the routed experts is a realistic target once quantized builds are up; the current F16-only files are best suited to a workstation/server with substantial combined VRAM+RAM.

Usage

llama.cpp

Text-only:

llama-cli -hf KiwiMate/KiwiMate-Large-1.0 --jinja

Multimodal (text + image):

llama-mtmd-cli -hf KiwiMate/KiwiMate-Large-1.0 --jinja

Qwen3.6 doesn’t use the in-prompt /think / /no_think toggle that Qwen3.5 does — disable reasoning via chat template kwargs instead:

llama-server -hf KiwiMate/KiwiMate-Large-1.0 --jinja --port 8080 \
  --chat-template-kwargs '{"enable_thinking": false}'

Serving as an OpenAI-compatible endpoint

llama-server -hf KiwiMate/KiwiMate-Large-1.0 --jinja --port 8080

curl http://localhost:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "messages": [{"role": "user", "content": "What is the difference between a tramping track and a walking track in NZ?"}]
  }'

Intended Use

The reasoning-heavy, long-context tier of KiwiMate — for conversations, planning, and agentic tasks that need more headroom than Medium provides, now including multimodal tasks like document and screenshot understanding. Once quantized builds land, this is also the recommended tier for lower-VRAM multimodal deployments.

Limitations

• Vision understanding inherits the base model’s ViT encoder as-is (not further fine-tuned on KiwiMate’s own visual data); expect reduced accuracy on dense text-in-image (OCR-heavy) tasks.
• Only F16 weights are published; running this in production today means either a large-VRAM/RAM box or quantizing the F16 files yourself with llama-quantize.
• Fine-tuned for a specific companion persona; not intended as a general-purpose assistant.
• As with any fine-tune — and especially given the rocky training path for this tier — verify outputs before using in production-critical contexts.

Acknowledgements

Fine-tuned and converted to GGUF using Unsloth.