mannix/gemma4-98e:CD-Q6

mannix/ gemma4-98e:CD-Q6_K

35 Downloads Updated 2 weeks ago

Pruned to 98 experts gemma-4 a4b 26b

tools thinking

ollama run mannix/gemma4-98e:CD-Q6_K

curl http://localhost:11434/api/chat \
  -d '{
    "model": "mannix/gemma4-98e:CD-Q6_K",
    "messages": [{"role": "user", "content": "Hello!"}]
  }'

from ollama import chat

response = chat(
    model='mannix/gemma4-98e:CD-Q6_K',
    messages=[{'role': 'user', 'content': 'Hello!'}],
)
print(response.message.content)

import ollama from 'ollama'

const response = await ollama.chat({
  model: 'mannix/gemma4-98e:CD-Q6_K',
  messages: [{role: 'user', content: 'Hello!'}],
})
console.log(response.message.content)

Details

Updated 2 weeks ago

2 weeks ago

fe3e5a2568e7 · 15GB ·

model

archgemma4

parameters19.9B

quantizationQ6_K

15GB

template

{{- if or .System .Tools }}<bos><|turn>system {{ if .System }}{{ .System }} {{ end }}{{- if .Tools }

1.3kB

params

{ "num_ctx": 256000, "repeat_last_n": 256, "repeat_penalty": 1.15, "stop": [

150B

Readme

Template fixed for tools usage

Model on HF:

https://huggingface.co/ManniX-ITA/gemma-4-A4B-98e-v3-it

Full GGUF:

https://huggingface.co/ManniX-ITA/gemma-4-A4B-98e-v3-it-GGUF