mix_77/
gemma3-qat-tools:27b

1,618 6 months ago

The current, most capable model that runs on a single GPU. With quantization and tools.

vision tools 4b 12b 27b

6 months ago

876e863a1290 · 18GB ·

model
gemma3
·
27.4B
·
Q4_0
template
{{- if .Messages }} {{- if or .System .Tools }}<start_of_turn>user {{- if .System}} {{ .System }} {{
params
{ "stop": [ "<end_of_turn>" ], "temperature": 1, "top_k": 64, "top_p": 0

Readme

Gemma3 with quantization and tool support created based on the 4b-it-qat, 12b-it-qat and 27b-it-qat models of the official gemma3 image (all the original licenses apply)

Tools prompt is written according to this tutorial