245 Downloads Updated 1 month ago
ollama run yousefburhanahmad/spark-x2.5-4b:q8
Community Q8_0 quantized conversion of Spark-X2.5-4B for local inference with Ollama and llama.cpp-compatible runtimes.
Converted to GGUF, quantized to Q8_0, tested, and packaged by Yousef Burhan Ahmad.
Model: Spark-X2.5-4B
Original Developer: XHToken
Base Model: XHToken/Spark-X2.5-4B
GGUF Conversion: Yousef Burhan Ahmad
Q8_0 Quantization: Yousef Burhan Ahmad
Runtime Preparation: Yousef Burhan Ahmad
CUDA Testing: Yousef Burhan Ahmad
Ollama Packaging: Yousef Burhan Ahmad
Quantization: Q8_0
Format: GGUF
Size: ~4.07 GiB
Ollama Download Size: ~4.4 GB
Architecture: spark2_5
Parameters: ~4B
Native Context Length: up to 1,048,576 tokens
Recommended Local Context: 16K–32K depending on available VRAM
License: Apache-2.0
Conversion Tool: llama.cpp b10828
The standard Ollama runtime may not yet support the spark2_5 architecture on all released versions.
If you run:
ollama run yousefburhanahmad/spark-x2.5-4b:q8
and receive an error similar to:
unknown model architecture: 'spark2_5'
your Ollama runtime does not yet include Spark-X2.5 architecture support.
In that case, use a Spark-X2.5-compatible Ollama build backed by a compatible llama.cpp version.
The GGUF model itself is valid and was successfully converted and quantized to Q8_0 using a Spark-X2.5-compatible llama.cpp build.
If your Ollama version already supports spark2_5, simply run:
ollama run yousefburhanahmad/spark-x2.5-4b:q8
Or download it first:
ollama pull yousefburhanahmad/spark-x2.5-4b:q8
Then run:
ollama run yousefburhanahmad/spark-x2.5-4b:q8
If your standard Ollama installation reports:
unknown model architecture: 'spark2_5'
you currently need to build Ollama against a llama.cpp version that supports Spark-X2.5.
git clone -b b10828 https://github.com/ggml-org/llama.cpp.git llama.cpp-spark
This version includes support for the:
spark2_5
architecture.
git clone https://github.com/ollama/ollama.git ollama-spark
Enter the repository:
cd ollama-spark
export OLLAMA_LLAMA_CPP_SOURCE="$(cd ../llama.cpp-spark && pwd)"
$env:OLLAMA_LLAMA_CPP_SOURCE="$(Resolve-Path ..\llama.cpp-spark)"
Verify it on Windows:
$env:OLLAMA_LLAMA_CPP_SOURCE
It should point to your local:
llama.cpp-spark
directory.
cmake -S . -B build
cmake --build build --parallel 8
For NVIDIA GPUs, build with CUDA support.
Example using CUDA 13:
cmake -S . -B build -DOLLAMA_LLAMA_BACKENDS=cuda_v13 -DCMAKE_CUDA_ARCHITECTURES=native
cmake --build build --parallel 8
On Windows with Visual Studio 2022:
cmake -S . -B build -G "Visual Studio 17 2022" -A x64 -DOLLAMA_LLAMA_BACKENDS=cuda_v13 -DCMAKE_CUDA_ARCHITECTURES=native
Then:
cmake --build build --config Release --parallel 8
A Windows build may require:
Git
Go
CMake
Visual Studio 2022
Desktop Development with C++
CUDA Toolkit for NVIDIA GPU acceleration
MSYS2 / UCRT64 GCC for CGO
If Go reports:
cgo: C compiler "gcc" not found
install GCC through MSYS2.
Example:
winget install -e --id MSYS2.MSYS2
Then install GCC:
C:\msys64\usr\bin\bash.exe -lc "pacman -S --needed --noconfirm mingw-w64-ucrt-x86_64-gcc"
Expose GCC to the current PowerShell session:
$env:Path = "C:\msys64\ucrt64\bin;$env:Path"
$env:CGO_ENABLED = "1"
$env:CC = "C:\msys64\ucrt64\bin\gcc.exe"
$env:CXX = "C:\msys64\ucrt64\bin\g++.exe"
Verify:
gcc --version
Then build the Ollama Go binary:
cmake --build build --config Release --target ollama-go --parallel 8
./ollama serve
.\ollama.exe serve
Keep this terminal open.
Open another terminal.
./ollama pull yousefburhanahmad/spark-x2.5-4b:q8
.\ollama.exe pull yousefburhanahmad/spark-x2.5-4b:q8
./ollama run yousefburhanahmad/spark-x2.5-4b:q8
.\ollama.exe run yousefburhanahmad/spark-x2.5-4b:q8
Recommended generation settings:
temperature: 1.0
top_p: 0.95
top_k: -1
repeat_penalty: 1.0
context: 32768
max output tokens: 8192
For GPUs with approximately 8 GB of VRAM, a practical context starting point is:
16384–32768
Although Spark-X2.5 supports a much larger native context window, using extremely large contexts significantly increases memory usage.
Example Modelfile:
FROM Spark-X2.5-4B-Q8_0.gguf
PARAMETER num_ctx 32768
PARAMETER num_predict 8192
PARAMETER temperature 1.0
PARAMETER top_p 0.95
PARAMETER top_k -1
PARAMETER repeat_penalty 1.0
PARAMETER repeat_last_n 64
The original Spark-X2.5-4B model weights were converted from the Hugging Face format to GGUF FP16 and then quantized to Q8_0 by Yousef Burhan Ahmad.
The quantization pipeline used for this release was:
Original Spark-X2.5-4B weights
↓
Hugging Face Safetensors
↓
GGUF FP16
↓
Q8_0 Quantization
↓
Spark-X2.5-4B-Q8_0.gguf
The Q8_0 quantization was performed using a Spark-X2.5-compatible version of:
llama.cpp b10828
Approximate sizes:
FP16 GGUF: ~7.66 GiB
Q8_0 GGUF: ~4.07 GiB
Ollama package/download: ~4.4 GB
This reduced the GGUF model size from approximately 7.66 GiB to 4.07 GiB, while keeping a higher bit precision than more aggressive quantization formats such as Q4 or Q5.
Q8_0 was intentionally selected to provide a balance between:
Weight Precision
Model Quality
Memory Usage
VRAM Requirements
Local Inference Efficiency
The goal of this conversion was not maximum compression.
The goal was to reduce the model’s memory footprint while preserving as much weight precision and inference quality as reasonably possible.
Lower-bit quantization formats such as Q4, Q5, or Q6 can reduce model size further.
However, lower bit-widths also apply stronger compression to the model weights and may introduce greater quantization error.
For this release, Q8_0 was selected because Spark-X2.5-4B is a relatively compact ~4B parameter model, making an approximately 4.07 GiB Q8_0 model practical for many consumer GPUs.
The objective was therefore:
Reduce memory requirements
+
Preserve high weight precision
+
Keep inference quality close to the higher-precision model
rather than simply producing the smallest possible model file.
Example tested configuration:
GPU: NVIDIA GeForce RTX 5070
VRAM: ~8 GB
CUDA Toolkit: 13.2
Quantization: Q8_0
Recommended Context: 16K–32K
Operating System: Windows 11
Actual memory usage depends on:
For an 8 GB GPU, Q8_0 is a practical choice for Spark-X2.5-4B.
After loading the model with Ollama:
ollama ps
On NVIDIA systems:
nvidia-smi
You should see GPU memory usage increase after the model is loaded.
This model uses:
spark2_5
A runtime with Spark-X2.5 architecture support is required.
Older versions of Ollama or llama.cpp may fail with:
unknown model architecture: 'spark2_5'
This is a runtime compatibility issue and does not indicate that the GGUF model file itself is corrupted.
This community quantized release is based on the official Spark-X2.5-4B model developed and released by XHToken.
Original model:
https://huggingface.co/XHToken/Spark-X2.5-4B
Official project:
https://github.com/XHToken/Spark-X2.5
All credit for the original:
belongs to XHToken and the original Spark-X2.5 contributors.
This community release was:
Converted to GGUF, quantized to Q8_0, tested, and packaged by Yousef Burhan Ahmad.
My contribution to this release included:
I do not claim ownership or authorship of the original Spark-X2.5 model.
The original model remains the work of XHToken and the Spark-X2.5 contributors.
The complete conversion process used for this release was:
XHToken/Spark-X2.5-4B
↓
Hugging Face Safetensors
↓
Spark-Compatible llama.cpp
↓
GGUF FP16
↓
Q8_0 Quantization
↓
Spark-X2.5-4B-Q8_0.gguf
↓
CUDA Runtime Testing
↓
Ollama Packaging
↓
yousefburhanahmad/spark-x2.5-4b:q8
Original / FP16 GGUF:
~7.66 GiB
Q8_0 GGUF:
~4.07 GiB
Ollama distribution:
~4.4 GB
The converted GGUF metadata reports a native context length of:
1,048,576 tokens
However, the maximum practical context depends heavily on hardware.
Recommended starting points:
8 GB VRAM → 16K–32K
12 GB VRAM → 32K+
16 GB VRAM → larger contexts may be practical
Do not assume that the full 1M context will fit into consumer GPU memory.
Run:
ollama run yousefburhanahmad/spark-x2.5-4b:q8
Then prompt:
Explain how retrieval-augmented generation works and provide a simple Python example.
Once official Ollama releases include native spark2_5 architecture support, users should be able to run this model directly with the standard Ollama installation:
ollama run yousefburhanahmad/spark-x2.5-4b:q8
without building a custom runtime.
Until then, use the Spark-compatible runtime instructions above if your installed Ollama reports an unsupported architecture error.
XHToken — Spark-X2.5-4B
Original source:
https://huggingface.co/XHToken/Spark-X2.5-4B
Official project:
https://github.com/XHToken/Spark-X2.5
Yousef Burhan Ahmad
yousefburhanahmad/spark-x2.5-4b:q8
Apache License 2.0
The original Spark-X2.5-4B model is distributed under the Apache License 2.0.
Please review the original model repository and model card for the authoritative license terms and usage requirements.
This is an unofficial community GGUF conversion and Q8_0 quantized release by Yousef Burhan Ahmad.
It is not an official XHToken release unless explicitly stated by the original developers.
I do not claim ownership of the original Spark-X2.5 architecture, training methodology, research, or model weights.
My contribution is specifically the:
GGUF Conversion
Q8_0 Quantization
Runtime Preparation
CUDA Testing
Ollama Packaging
Distribution Documentation
Please refer to the original Spark-X2.5 model card and repository for authoritative information regarding:
Original project:
https://github.com/XHToken/Spark-X2.5
Original model: