sammcj/
devstral-small-24b-2505-ud:128k-q6_k_xl

901 5 months ago

Built from Unsloth's UD Q6_K_XL quant

tools

5 months ago

7cb1f6f499f0 · 21GB ·

model
llama
·
23.6B
·
Q6_K
template
{{- range $index, $_ := .Messages }} {{- if eq .Role "system" }}[SYSTEM_PROMPT]{{ .Content }}[/SYSTE
system
You are Devstral, a helpful agentic model trained by Mistral AI and using the OpenHands scaffold. Yo
params
{ "min_p": 0.01, "num_ctx": 131072, "num_gpu": 99, "num_keep": 1024, "repeat_pen

Readme

GGUF: https://huggingface.co/unsloth/Devstral-Small-2505-GGUF/blob/main/Devstral-Small-2505-UD-Q6_K_XL.gguf

  • 128k tags default to 128k context which requires around 35GB vRAM.
  • 64k tags default to 64k context which requires around 27GB vRAM, num_batch set to 1024 (up from 512) for performance, you can tune num_batch to trade off speed and vRAM requirements.
  • cline tags are optimised for Cline & Roo Code Agentic Coding.