aiconjured/ Qwen3.8-27B-Uncensored-HauhauCS-Aggressive-MTP-GGUF-Q8-NVFP4

1,464 1 month ago

Text + Vision Qwen3.8-27B-Uncensored-HauhauCS-Aggressive-MTP-GGUF-Q8-NVFP4

vision 
ollama run aiconjured/Qwen3.8-27B-Uncensored-HauhauCS-Aggressive-MTP-GGUF-Q8-NVFP4

Models

View all →

1 model

Qwen3.8-27B-Uncensored-HauhauCS-Aggressive-MTP-GGUF-Q8-NVFP4:latest

18GB · 256K context window · Text, Image · 1 month ago

Readme

Request a model Quant in GGUF - Contact : AIconjured@gmail.com

Qwen3.8-27B-Uncensored-HauhauCS-Aggressive — NVFP4 Mixed Re-Quant

A tensor-level NVFP4 re-quantization of the HauhauCS Qwen3.8-27B-Uncensored-HauhauCS-Aggressive model, built for NVIDIA Blackwell (RTX 50-series) local inference.

This release takes the HauhauCS Q8_K_P text model (31.46 GB, 9.21 bpw) and re-quantizes it to a mixed NVFP4 / Q8_0 / F32 / F16 profile (17.10 GB, 5.00 bpw) — cutting VRAM by ~15 GB while preserving the model’s native MTP/NextN head, vision, tokenizer, and quality-critical tensors. It runs on standard GGUF runtimes and leverages Blackwell’s native FP4 tensor cores for the quantized compute matrices.

Credit: Based on the Qwen and HauhauCS teams’ work. This is a re-quantization of the HauhauCS Aggressive uncensored variant — no changes to datasets, weights’ values, or intended capabilities.

What is this?

The HauhauCS Aggressive release is a 27B dense causal language model with a vision encoder: direct answers, no refusal behavior, minimal preamble on hard prompts (0/465 refusals). It preserves Qwen3.8’s native NextN (MTP) head and offers the HauhauCS FastMTP acceleration sidecar.

This NVFP4 mixed re-quant keeps all of that intact and makes the model fit comfortably on a 24 GB Blackwell card (RTX 5060 Ti / 5070 / 5080) with room for a large context, while keeping the quality-sensitive tensors at high precision.

Why NVFP4?

NVFP4 (4-bit floating point, 16-element sub-block scaling) is the native precision of Blackwell’s FP4 tensor cores. Re-quantizing the large compute-heavy matrices to NVFP4: - Halves VRAM vs. the Q8_K_P source (31.46 GB → 17.10 GB). - Uses the GPU’s fastest tensor cores for the dominant matmuls (FFN, attention projections, DeltaNet output). - Stays near Q5-class quality because the quality anchors (output head, embeddings, recurrent state, norms, MTP head) are kept at Q8_0 / F32 / F16.

The re-quant was done from the Q8_K_P GGUF (9.21 bpw — high enough to produce a clean NVFP4 result) using an importance matrix, so no BF16 source was required.

Files

File Description Size BPW
Qwen3.8-27B-Uncensored-HauhauCS-Aggressive-NVFP4-mixed.gguf This release — mixed NVFP4 re-quant of Q8_K_P 17.10 GB 5.00
mmproj-Qwen3.8-27B-Uncensored-HauhauCS-Aggressive-BF16.gguf Vision projector (unchanged, from HauhauCS) 931 MB —
Qwen3.8-27B-Uncensored-HauhauCS-Aggressive-FastMTP-32K.gguf HauhauCS FastMTP 32K acceleration sidecar (unchanged) 903 MB —

The projector and FastMTP sidecar work with this NVFP4 model exactly as they do with any HauhauCS text quant. Download the projector only if you need image/video input; the FastMTP sidecar is optional and requires the HauhauCS runtime patch (see below).

The quantization recipe (tensor-level)

This is the core of the release — a per-tensor-type mixed profile. 866 tensors total, verified after quantization:

Type Count Tensors Rationale
NVFP4 371 ffn_gate / ffn_up / ffn_down (64 layers); DeltaNet attn_qkv / attn_gate / ssm_out (48); full-attention attn_q / attn_v (16) The large compute-heavy matrices — biggest VRAM + Blackwell speed win
Q8_0 115 token_embd, output (lm_head), ssm_beta / ssm_alpha / ssm_convNd (48 DeltaNet recurrent-state), blk.64.nextn.eh_proj (MTP), all attn_output Quality anchors + recurrent state (error-compounding) + MTP draft path
F32 360 all norms (attn_norm, attn_post_norm, attn_q_norm, attn_k_norm, ssm_norm, nextn.enorm / hnorm / shared_head_norm) + ssm_a / ssm_dt Tiny, error-sensitive — kept exact at negligible cost
F16 16 MTP layer-64 attn_q / attn_k / attn_v / attn_output The draft head’s attention kept exact to protect FastMTP acceptance

Design principles

  • NVFP4 every large compute matrix — that’s where VRAM and Blackwell speed live.
  • Keep quality-critical and delicate tensors high-precision — the output head (lm_head) and token embeddings anchor output quality; the DeltaNet recurrent state (ssm_beta/alpha/conv1d) compounds error across the sequence; norms are tiny and must stay exact.
  • Protect the MTP head — the MTP/NextN draft head is tied to token_embd (kept Q8_0), and nextn.eh_proj is kept Q8_0, so the FastMTP draft acceptance rate (and its 3× speed) is preserved.
  • attn_output kept Q8_0 — attention output projections stay high-precision for quality.

Method

  1. Importance matrix generated with llama-imatrix on the Q8_K_P source (45 calibration chunks spanning code, prose, reasoning, JSON, and agentic prompts).
  2. Re-quantization with llama-quantize --allow-requantize --imatrix <imatrix> --tensor-type-file <recipe> --token-embedding-type q8_0 --output-tensor-type q8_0, from the Q8_K_P GGUF.
  3. Verification — all 866 tensor types confirmed against the recipe after quantization.

How to run

Ollama

Import with the dual Modelfile (model + projector), then run:

”`dockerfile FROM /path/to/Qwen3.8-27B-Uncensored-HauhauCS-Aggressive-NVFP4-mixed.gguf FROM /path/to/mmproj-Qwen3.8-27B-Uncensored-HauhauCS-Aggressive-BF16.gguf

PARAMETER temperature 0.7 PARAMETER top_p 0.9 PARAMETER num_ctx 32768 PARAMETER num_predict 8192 PARAMETER repeat_penalty 1.0