mirage335/
Devstral-Small-2507-virtuoso:latest

25 1 week ago

Generates good programming code, command line help, etc. Compatible with opencode .

tools

1 week ago

a3a15f9d343a · 13GB ·

model
llama
·
23.6B
·
IQ4_XS
template
{{- range $index, $_ := .Messages }} {{- if eq .Role "system" }}[SYSTEM_PROMPT]{{ .Content }}[/SYSTE
system
You are Devstral, a helpful agentic model trained by Mistral AI and using the OpenHands scaffold. Yo
license
Apache License Version 2.0, January 2004 http://www.apache.org/licenses/ TERMS AND CONDITIONS FOR US
params
{ "num_ctx": 4096, "num_gpu": 41, "num_keep": 4096, "num_predict": 8096, "repeat

Readme

Apache License Version 2.0

NOTICE

Design

Generates good programming code, command line help, etc. Recommended for opencode .

Usage

ollama_pull_virtuoso() {
ollama pull mirage335/"$1"
ollama cp mirage335/"$1" "$1"
ollama rm mirage335/"$1"
}

ollama_pull_virtuoso Devstral-Small-2507-virtuoso

echo "FROM Devstral-Small-2507-virtuoso" > Modelfile-128k
echo "PARAMETER num_ctx 131072" >> Modelfile-128k
echo "PARAMETER num_keep 131072" >> Modelfile-128k
echo "PARAMETER num_predict 24576" >> Modelfile-128k
echo "PARAMETER num_gpu 999" >> Modelfile-128k
ollama create Devstral-Small-2507-128k-virtuoso -f Modelfile-128k
rm -f Modelfile-128k

Recommended environment variables. KV_CACHE quantization “q4_0” in particular RECOMMENDED, unless “q8_0” is needed (eg. by Qwen-2_5-VL-7B-Instruct-virtuoso, etc).

export OLLAMA_NUM_THREADS=18
export OLLAMA_FLASH_ATTENTION=1
export OLLAMA_KV_CACHE_TYPE="q4_0"
export OLLAMA_NEW_ENGINE=true
export OLLAMA_NOHISTORY=true
export OLLAMA_NUM_PARALLEL=1
export OLLAMA_MAX_LOADED_MODELS=1

Adjust OLLAMA_NUM_THREADS and/or disable HyperThreading, etc, to prevent crippling performance loss.

CAUTION - Preservation

Pulling the model this way relies on the ollama repository, and more generally, reliability of internet services, which has been rather significantly fragile.

If possible, you should use the “Llama-3-virtuoso” project, which automatically caches an automatically installable backup copy.

https://github.com/mirage335-colossus/Llama-3-virtuoso