34 yesterday

T-lite-it-1.0 is a model built upon the Qwen 2.5 model family and incorporates both continual pre-training and alignment techniques (quantized Q5_K_M)

tools 7b

yesterday

553f99cc2e66 · 5.4GB

qwen2
·
7.61B
·
Q5_K_M
{{- if .Messages }} {{- if or .System .Tools }}<|im_start|>system {{- if .System }} {{ .System }} {{
{ "stop": [ "<|im_start|>", "<|im_end|>" ] }

Readme

Based on https://huggingface.co/vzotov/T-lite-it-1.0-Q5_K_M-GGUF

Feature Value
vision false
thinking false
tools true
Device Speed Version
RTX 3090 24gb ~107 token/s Q5_K_M
RTX 2080ti 11gb ~78 token/s Q5_K_M
RTX 3070ti Mobile 8gb ~65 token/s Q5_K_M
M1 Max 32gb ~40 token/s Q5_K_M