jetelain/ Flint:q8

21 1 week ago

Recursive Flint with Unsloth Dynamic

tools thinking 
ollama run jetelain/Flint:q8

Details

1 week ago

5c20cb9a030c · 42GB

model
flint
·
39.8B
·
Q8_0
license
Apache License Version 2.0, January 2004 http://www.apache.org/licenses/ TERMS AND CONDITIONS FOR US
params
{ "temperature": 1, "top_k": 20, "top_p": 0.95 }
template
{{ .Prompt }}

Readme

Recursive Flint (Unsloth Dynamic Q4 & Q8)

Recursive Flint is a mixture-of-experts language model specifically designed for coding agents, featuring 39.8 billion total parameters and approximately 2.5 billion active per token (40B-A2.5B).

Engineered to investigate codebases, fix bugs, implement complex features, and write comprehensive tests with native support for reasoning, multi-turn tool use, and long-horizon agentic workflows.

🚀 Key Highlights

  • Sparse MoE Architecture (40B-A2.5B): 256 routed experts with 6 active per token plus an always-active shared expert, delivering 40B-class intelligence at 2.5B parameter inference speed and efficiency.
  • Unsloth Dynamic Quantization (UD): Selective precision keeps critical attention and routing layers at high precision for near-lossless output.
  • Hybrid Attention: 35 sliding-window layers (513 tokens) with RoPE and 14 full-attention layers with learned attention sinks, optimized for both local code cache and global project comprehension.
  • Massive Context Support: Native context length of up to 262,144 tokens.
  • Agent-First Native Parser & Renderer: Fully integrated with Ollama’s native flint renderer and parser for reasoning traces and structured tool calling.

📦 Tags & Variants

Tag Quantization Size Default Settings Target Use Case
latest / q4 UD-Q4_K_XL 23.3 GB temp 1.0, top_k 20, top_p 0.95 Recommended Sweet Spot: High quality + fast generation for coding agents
q8 Q8_0 39.4 GB temp 1.0, top_k 20, top_p 0.95 Maximum reasoning precision for complex architectures

💡 Usage Recommendations

1. Fast CLI Run

# Recommended sweet spot (quality + speed):
ollama run jetelain/Flint:q4

# Maximum precision:
ollama run jetelain/Flint:q8

2. Hardware Recommendations

  • 16 GB VRAM GPUs: Offload with host RAM (~64 GB); ideal with q4.
  • 24 GB VRAM GPUs (RTX 3090 / 4090): Fits the 23.3 GB q4 model with generous context in VRAM.
  • Unified Memory (64–96 GB / Apple Silicon / AMD Strix Halo): Flawless full offload for q4 and q8 with ultra-large context up to 256k tokens.

3. 💡 Hardware Alias Tip (Zero Disk Overhead)

curl http://localhost:11434/api/create -d '{
  "model": "Flint",
  "from": "jetelain/Flint:q4",
  "parameters": {
    "num_ctx": 163840
  }
}'

curl http://localhost:11434/api/create -d '{
  "model": "Flint",
  "from": "jetelain/Flint:q8",
  "parameters": {
    "num_ctx": 262144
  }
}'