slickwillies/ornith15-35b-a3b-apex-mtp-fixed

slickwillies/ ornith15-35b-a3b-apex-mtp-fixed:latest

16 Downloads Updated 3 hours ago

ollama run slickwillies/ornith15-35b-a3b-apex-mtp-fixed

curl http://localhost:11434/api/chat \
  -d '{
    "model": "slickwillies/ornith15-35b-a3b-apex-mtp-fixed",
    "messages": [{"role": "user", "content": "Hello!"}]
  }'

from ollama import chat

response = chat(
    model='slickwillies/ornith15-35b-a3b-apex-mtp-fixed',
    messages=[{'role': 'user', 'content': 'Hello!'}],
)
print(response.message.content)

import ollama from 'ollama'

const response = await ollama.chat({
  model: 'slickwillies/ornith15-35b-a3b-apex-mtp-fixed',
  messages: [{role: 'user', content: 'Hello!'}],
})
console.log(response.message.content)

Details

Updated 3 hours ago

3 hours ago

a4cc9d9aebde · 17GB ·

model

archqwen35moe

·

parameters35.5B

·

quantizationQ4_K_M

17GB

params

{ "min_p": 0.05, "repeat_penalty": 1.02, "temperature": 0.4, "top_k": 20 }

66B

Readme

Ornith-1.5-35B-A3B — APEX-MTP-Fixed Repaired MTP-head build. The stock Ornith-1.5 MTP (multi-token-prediction) head shipped untrained (garbage norms ~1e31), so speculative decoding never worked. This build splices in the Avifenesh continued-trained head (dequantized NVFP4→f32→Q8_0).

Quant details Routed experts • Type: Q4_K • Count: 221

Higher-precision layers • Type: Q6_K • Count: 121

Deepest-compressed experts • Type: Q3_K • Count: 90

MTP head • Type: Q8_0 • Count: 11 (kept high-precision — the spec-decode bottleneck)

Norms / embeddings • Type: F32 • Count: 310 (all verified sane)

File size • Type: 17.4 GB • Count:

MTP head fix Vendor head: untrained (norms ~1e31, garbage) — depth-²⁄₃ accuracy 0.²⁷⁄₀.13 Fixed head: Avifenesh continued-trained, spliced Q8_0 — depth-²⁄₃ accuracy 0.⁵⁸⁄₀.43 Draft acceptance: 0.43 (vendor 0.35)

Performance (RTX 4070 12GB) Decode 63.7 tok/s · 12-prompt battery 15.3s avg · ¹¹⁄₁₂ standard + ¹¹⁄₁₃ torture ~33% faster wall-clock than the broken-head factory bird; ~39% faster than Ornith1.0

Specs qwen35moe · 35.5B (3B active) · 262,144 ctx · 2048 embedding · MIT

Ornith-1.5-35B-A3B — APEX-MTP-Fixed
Repaired MTP-head build. The stock Ornith-1.5 MTP (multi-token-prediction) head shipped untrained (garbage norms ~1e31), so speculative decoding never worked. This build splices in the Avifenesh continued-trained head (dequantized NVFP4→f32→Q8_0).

Quant details
Routed experts
• Type: Q4_K
• Count: 221

Higher-precision layers
• Type: Q6_K
• Count: 121

Deepest-compressed experts
• Type: Q3_K
• Count: 90

MTP head
• Type: Q8_0
• Count: 11 (kept high-precision — the spec-decode bottleneck)

Norms / embeddings
• Type: F32
• Count: 310 (all verified sane)

File size
• Type: 17.4 GB
• Count:

MTP head fix
Vendor head: untrained (norms ~1e31, garbage) — depth-2/3 accuracy 0.27/0.13
Fixed head: Avifenesh continued-trained, spliced Q8_0 — depth-2/3 accuracy 0.58/0.43
Draft acceptance: 0.43 (vendor 0.35)

Performance (RTX 4070 12GB)
Decode 63.7 tok/s · 12-prompt battery 15.3s avg · 11/12 standard + 11/13 torture
~33% faster wall-clock than the broken-head factory bird; ~39% faster than Ornith1.0

Specs
qwen35moe · 35.5B (3B active) · 262,144 ctx · 2048 embedding · MIT

Paste, drop or click to upload images (.png, .jpeg, .jpg, .svg, .gif)