KiwiMate/ KiwiMate-Medium-1.0:Q8_0

4 3 weeks ago

The first version of KiwiMate that has decent intelligence

vision 
ollama run KiwiMate/KiwiMate-Medium-1.0:Q8_0

Details

3 weeks ago

6a994371a7f6 · 30GB

model
qwen35
·
27.3B
·
Q8_0
projector
clip
·
461M
·
BF16
template
{{- if .System -}} system {{ .System }} {{- end -}} {{- range $i, $_ := .Messages }} {{- $last := eq
system
You are KiwiMate a friendly ai.
params
{ "temperature": 0.0001 }

Readme

license: apache-2.0 base_model: Qwen/Qwen3.6-27B language: - en - mi tags: - gguf - llama.cpp - unsloth - vision-language-model - conversational - endpoints_compatible - new-zealand pipeline_tag: image-text-to-text

library_name: llama.cpp

KiwiMate-Medium-1.0

KiwiMate is an AI companion app for iOS and Android with a distinctly New Zealand cultural identity — everyday NZ English, te reo Māori, and Kiwi context baked into how it talks. KiwiMate-Medium-1.0 is the upper-mid tier in the KiwiMate model family, built for conversations that need stronger reasoning than Small can offer while staying self-hostable on a single high-VRAM GPU.

Model Details
Base model Qwen/Qwen3.6-27B
Parameters ~27B
Architecture Hybrid linear/full attention (48 linear-attention layers, 16 full-attention layers, 64 total)
Context length 262,144 tokens
Modality Text + image (vision-language)
Fine-tuning Unsloth, 2x faster training
Format GGUF (for llama.cpp)
License Apache 2.0

KiwiMate-Medium-1.0 was fine-tuned from Qwen3.6-27B on KiwiMate’s proprietary dataset of New Zealand English, te reo Māori, and Kiwi cultural context, tuned for the AI-companion persona used across the KiwiMate app.

Licensing note: Qwen3.6-27B is released under Apache 2.0, which permits commercial use, fine-tuning, and redistribution without royalties — a cleaner license position than KiwiMate Mini’s Qwen2.5-VL-3B base, which ships under the more restrictive Qwen Research license.

Available Files

File Type Size Use case
KiwiMate-Medium-1.0.BF16-00001-of-00002.gguf + ...00002-of-00002.gguf Full precision (split) ~54.7 GB total Evaluation, highest fidelity
KiwiMate-Medium-1.0.Q8_0.gguf 8-bit quant 29.0 GB Near-lossless, lower VRAM
KiwiMate-Medium-1.0.Q4_K_M.gguf 4-bit quant 16.8 GB Best for single high-VRAM consumer GPU
KiwiMate-Medium-1.0.BF16-mmproj.gguf Vision projector 931.1 MB Required alongside any of the above for image input

Usage

llama.cpp

Text-only:

llama-cli -hf KiwiMate/KiwiMate-Medium-1.0 --jinja

Multimodal (text + image):

llama-mtmd-cli -hf KiwiMate/KiwiMate-Medium-1.0 --jinja

Serving as an OpenAI-compatible endpoint

Intended for deployment behind a Hugging Face Inference Endpoint (llama.cpp backend) or self-hosted with llama-server, which exposes the same OpenAI-compatible API:

llama-server -hf KiwiMate/KiwiMate-Medium-1.0 --jinja --port 8080

curl http://localhost:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "messages": [{"role": "user", "content": "Explain the difference between a tramping track and a walking track in NZ."}]
  }'

Intended Use

Conversations requiring stronger reasoning, longer context, or more nuanced NZ cultural/te reo Māori handling than KiwiMate-Small-1.0. Suited to a single high-VRAM GPU deployment (e.g. L40S-class or better at Q4_K_M/Q8_0).

Limitations

  • Fine-tuned for a specific companion persona; not intended as a general-purpose assistant.
  • Vision understanding inherits the base model’s ViT encoder; expect reduced accuracy on dense text-in-image (OCR-heavy) tasks.
  • Larger footprint than Small — not suited to edge/low-VRAM deployment; use Small or Mini for that.
  • As with any fine-tune, verify outputs before using in production-critical contexts.

Acknowledgements

Fine-tuned and converted to GGUF using Unsloth.