Details

Updated 1 month ago

1 month ago

1b4469a57778 · 6.7GB

model

archqwen35

parameters9.2B

quantizationQ4_K_M

5.8GB

projector

archclip

parameters456M

quantizationBF16

922MB

template

{{- if .System -}} <|im_start|>system {{ .System }}<|im_end|> {{- end -}} {{- range $i, $_ := .Messa

487B

system

You are KiwiMate a friendly ai.

31B

params

{ "temperature": 0.0001 }

23B

license: apache-2.0 base_model: Qwen/Qwen3.5-9B language: - en - mi tags: - gguf - llama.cpp - unsloth - vision-language-model - conversational - endpoints_compatible - new-zealand pipeline_tag: image-text-to-text

library_name: llama.cpp

KiwiMate-Small-1.0

KiwiMate is an AI companion app for iOS and Android with a distinctly New Zealand cultural identity — everyday NZ English, te reo Māori, and Kiwi context baked into how it talks. KiwiMate-Small-1.0 is the mid-tier model in the KiwiMate family, sitting between Mini and Medium for everyday chat with vision support.

Model Details


Base model	Qwen/Qwen3.5-9B
Parameters	~9B
Architecture	Hybrid linear/full attention (24 linear-attention layers, 8 full-attention layers, 32 total)
Context length	262,144 tokens
Modality	Text + image (vision-language)
Fine-tuning	Unsloth, 2x faster training
Format	GGUF (for llama.cpp)
License	Apache 2.0

KiwiMate-Small-1.0 was fine-tuned from Qwen3.5-9B on KiwiMate’s proprietary dataset of New Zealand English, te reo Māori, and Kiwi cultural context, tuned for the AI-companion persona used across the KiwiMate app.

Licensing note: Qwen3.5-9B is released under Apache 2.0, which permits commercial use, fine-tuning, and redistribution without royalties. This is a cleaner license position than KiwiMate Mini’s Qwen2.5-VL-3B base, which ships under the more restrictive Qwen Research license.

Available Files

File	Type	Size	Use case
`KiwiMate-Small-1.0.F16.gguf`	Full precision	18.4 GB	Evaluation, highest fidelity
`KiwiMate-Small-1.0.Q8_0.gguf`	8-bit quant	9.8 GB	Near-lossless, lower VRAM
`KiwiMate-Small-1.0.Q4_K_M.gguf`	4-bit quant	5.8 GB	Best for constrained hardware / edge deployment
`KiwiMate-Small-1.0.BF16-mmproj.gguf`	Vision projector	921.7 MB	Required alongside any of the above for image input

Usage

llama.cpp

Text-only:

llama-cli -hf KiwiMate/KiwiMate-Small-1.0 --jinja

Multimodal (text + image):

llama-mtmd-cli -hf KiwiMate/KiwiMate-Small-1.0 --jinja

Serving as an OpenAI-compatible endpoint

This model is served in production via a Hugging Face Inference Endpoint backed by llama.cpp. If you’re self-hosting, llama-server exposes the same OpenAI-compatible API:

llama-server -hf KiwiMate/KiwiMate-Small-1.0 --jinja --port 8080

curl http://localhost:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "messages": [{"role": "user", "content": "Kia ora, what'\''s a good spot for fish and chips in Christchurch?"}]
  }'

Note: if you’re hitting an HF Inference Endpoint Playground and getting unexplained 404s, they’re usually a request-routing issue during container restarts — calling /v1/chat/completions directly tends to work even when the Playground doesn’t.

Intended Use

Casual conversation, everyday NZ-context Q&A, and lightweight image understanding within the KiwiMate app. For heavier reasoning tasks, route to KiwiMate-Medium-1.0 or KiwiMate-Large-1.0.

Limitations

Fine-tuned for a specific companion persona; not intended as a general-purpose assistant.
Vision understanding inherits the base model’s ViT encoder; expect reduced accuracy on dense text-in-image (OCR-heavy) tasks.
As with any fine-tune, verify outputs before using in production-critical contexts.

Acknowledgements

Fine-tuned and converted to GGUF using Unsloth.

The first (non-preview) version of KiwiMate Small