20 3 days ago

Initially built this 7b model to fit in my RTX 4060 with 8GB

tools
ollama run michaelmontuori/Qwen2.5-7B-Instruct-GGUF:Q4_K_M-32k

Details

3 days ago

b7245a065e9a · 4.7GB

qwen2
·
7.62B
·
Q4_K_M
{{- if .Suffix }}<|fim_prefix|>{{ .Prompt }}<|fim_suffix|>{{ .Suffix }}<|fim_middle|> {{- else if .M
{ "num_ctx": 32768 }

Readme

michaelmontuori/Qwen2.5-7B-Instruct-GGUF:Q4_K_M-32k is built to fit RTX 4060 with 8GB of VRAM