New Report bugs or version requests.

About : Spark-X2.5-4B

GGUF: https://huggingface.co/ngquocvinh/Spark-X2.5-4B-GGUF
HF BF16: https://huggingface.co/XHToken/Spark-X2.5-4B

Actual load footprint is about ~6GB. With a 32K Ollama Context setting.

Spark-X2.5-4B is a compact, general-purpose language model developed by SparkLLM. According to the upstream authors, the model is designed for:

General-purpose capabilities: conversation, writing, translation, reasoning, and coding
Tool use and agentic workflows
Native context window of up to 1M tokens
More than 200 supported languages
Efficient hybrid attention: one full-attention layer combined with three sliding-window attention layers
Broad ecosystem support: llama.cpp, vLLM, SGLang, MLX, Ollama, and LM Studio

The upstream model is also designed for broad hardware compatibility and efficient long-context inference.

For the original model architecture, training details, benchmarks, and official usage instructions, see the official model card.

This repository is a quantization-only release for local inference. No model training or fine-tuning was performed.

Recommended quantized profiles *Kitsune note - Q6 would be better quality, but maintaining a smaller footprint than BF16.

These are three practical starting points, all excluding the BF16 reference: Profile Choose it when Spark-X2.5-4B-Q8_0.gguf You want the closest practical profile to the BF16 reference. Spark-X2.5-4B-Q4_K_M.gguf You want the main size/quality balance. Spark-X2.5-4B-Q2_K.gguf You need a smaller profile and accept a larger fidelity trade-off. Fidelity measurements

The table below compares every published GGUF file with the BF16 reference on a held-out WikiText-2 evaluation: eight chunks from wiki.test.raw and eight chunks from wiki.valid.raw, using a 4,096-token context and the same Spark llama.cpp runtime. Values are averaged across the two splits. The BF16 reference mean PPL was 20.718057 in this evaluation. Lower Mean KLD, ΔPPL, and RMS Δp, and higher Top-1 agreement, indicate closer next-token behavior to BF16. These are next-token fidelity measurements, not a direct percentage of capabilities retained and not a substitute for task-specific evaluation.

The machine-readable results are available in reproducibility/quality-summary.tsv, with corpus hashes, evaluation settings, calibration provenance, and runtime details in reproducibility/manifest.md.

Reproducibility and validation

This is a quantization-only release: no training or fine-tuning was performed. The BF16 source, runtime revision, clean calibration split, importance matrix, quantization commands, checksums, and validation profile are recorded in the reproducibility/ package. All 16 published files passed load/generate validation on an NVIDIA A10M.

License and attribution

The upstream model is released under Apache License 2.0. Preserve the upstream attribution and license when redistributing these derivative files. This is a community GGUF quantization, not an official XHToken/SparkLLM release or endorsement.