sam860/
dolphin3-qwen2.5:3b

798 6 months ago

Dolphin 3.0 Qwen 2.5 🐬 - A powerful, customizable AI model for local use.

tools 1.5b 3b

6 months ago

15a6026c9e28 · 1.9GB ·

model
qwen2
·
3.09B
·
Q4_K_M
params
{ "temperature": 0.5 }
template
{{- if .Suffix }}<|fim_prefix|>{{ .Prompt }}<|fim_suffix|>{{ .Suffix }}<|fim_middle|> {{- else if .M

Readme

I’ve uploaded the quants of these models that I find most useful. I quantized them myself using the fp16 GGUF files provided by mradermacher. The default temperature has been adjusted - as I find the smaller qwen models tends to hallucinate too much at higher temps.

For 1.5b:

  • Temperature set to 0.2

  • Q4_K_M to me is the lower limit of usable

  • Q5_K_M is the default, and a fine middle ground with perfectly acceptable quality to me

  • Q8_0 has no compromises on accuracy or performance; I don’t see a reason to use a larger un-quantized fp model

For 3b:

  • Temperature set to 0.5

  • Q4_0 is as low as I’d go for quality concerns

  • Q4_K_M is the default, and a fine middle ground with perfectly acceptable quality to me

  • Q5_K_M is going to be just about perfect in my experience with this model

  • Q8_0 has no compromises on accuracy or performance; I don’t see a reason to use a larger un-quantized fp model

Dolphin 3.0 Qwen 2.5 🐬

The ultimate general-purpose local AI model, enabling coding, math, and more.

  • You’re in control: Set the system prompt, alignment, and guidelines.
  • Your data stays yours: No more worries about your data being used by others.
  • Built for your needs: Tailor Dolphin to your specific use case.

References:

Created by the cognitivecomputations team.

HuggingFace

mradermacher