mix_77/
gemma3-qat-tools:4b

1,540 5 months ago

The current, most capable model that runs on a single GPU. With quantization and tools.

vision tools 4b 12b 27b

5 months ago

36fa44ea4aa4 · 4.0GB

model
gemma3
·
4.3B
·
Q4_0
template
{{- if .Messages }} {{- if or .System .Tools }}<start_of_turn>user {{- if .System}} {{ .System }} {{
params
{ "stop": [ "<end_of_turn>" ], "temperature": 1, "top_k": 64, "top_p": 0

Readme

Gemma3 with quantization and tool support created based on the 4b-it-qat, 12b-it-qat and 27b-it-qat models of the official gemma3 image (all the original licenses apply)

Tools prompt is written according to this tutorial