ukuwzi/ qweble-sol

8 13 hours ago

Fine-tuned derivative of Qwen3.5-4B, trained on GPT 5.6 Sol, Luna, and Fable 5.1 traces 

ollama run ukuwzi/qweble-sol:4b-q4_k_m

Models

View all →

1 model

qweble-sol:4b-q4_k_m

2.7GB · 256K context window · Text · 13 hours ago

Readme

Qweble-Sol 4B

Qweble-Sol logo

Qweble-Sol is a fine-tuned derivative of Qwen3.5-4B focused on systems administration, programming, troubleshooting, DevOps, and technical assistance.

Quick Start

ollama run ukuwzi/qweble-sol:4b-q4_k_m

Overview

Qweble-Sol is intended to be a compact local technical assistant capable of helping with tasks such as:

  • Linux and macOS administration
  • Shell scripting
  • Python, Go, Rust, and JavaScript
  • Debugging and code review
  • Docker and container workflows
  • Networking and troubleshooting
  • Server configuration
  • Automation
  • CI/CD and DevOps
  • Security best practices
  • General technical questions

The goal of the fine-tune is to preserve the general reasoning and conversational abilities of Qwen3.5-4B while specializing its behavior toward practical technical and systems-oriented tasks.

Qweble-Sol supports Qwen3.5’s reasoning/thinking behavior.

Model Lineage

Qwen3.5-4B
    │
    ▼
MLX fine-tuning
    │
    ▼
Qweble-Sol 16-bit MLX
    │
    ├──────────────► Native MLX inference
    │
    ▼
HF-compatible weight export
    │
    ▼
F16 GGUF
    │
    ▼
Q4_K_M GGUF
    │
    ├──────────────► llama.cpp
    │
    └──────────────► Ollama

The original Qwen3.5-4B model was developed by the Qwen Team at Alibaba.

Qweble-Sol was fine-tuned, adapted, converted, and packaged by ukuwzi.

Available Formats

MLX 16-bit (https://huggingface.co/ukuwzi/Qweble-Sol-4B-MLX)

The MLX release is the native fused version of Qweble-Sol and is recommended for Apple Silicon users who want maximum model fidelity.

Typical directory contents:

config.json
model-00001-of-00002.safetensors
model-00002-of-00002.safetensors
model.safetensors.index.json
tokenizer.json
tokenizer_config.json
...

Approximate model size:

~7.8 GB

Run with MLX

Install MLX-LM:

pip install -U mlx-lm

Then run:

mlx_lm.chat --model /path/to/Qweble-Sol-4B-MLX

For generation:

mlx_lm.generate \
  --model /path/to/Qweble-Sol-4B-MLX \
  --prompt "Explain Docker in one paragraph." \
  --max-tokens 256

GGUF Q4_K_M

The GGUF release is intended for:

  • Ollama
  • llama.cpp
  • Other GGUF-compatible runtimes

Quantization:

Q4_K_M

Model size:

2.51 GiB

This version offers substantially reduced storage and memory requirements compared with the 16-bit MLX model.

llama.cpp Usage

Run Qweble-Sol directly with llama.cpp:

./llama-cli \
  -m Qweble-Sol-4B-Q4_K_M.gguf

With a prompt:

./llama-cli \
  -m Qweble-Sol-4B-Q4_K_M.gguf \
  -p "Explain the difference between a Docker image and a container."

A moderate context length such as 8192 tokens is recommended as a starting point on machines with limited unified memory:

./llama-cli \
  -m Qweble-Sol-4B-Q4_K_M.gguf \
  -c 8192

Performance

Test System

The following results were measured locally on:

Component Configuration
Hardware Apple M4 Pro
Unified memory 24 GB
OS macOS
GGUF runtime llama.cpp
GGUF backend Metal + BLAS
GGUF quantization Q4_K_M
MLX runtime mlx-lm
llama.cpp build e613ef2c8 / b11056

These numbers should be treated as hardware-specific measurements rather than universal performance estimates.

GGUF / llama.cpp Performance

Measured with llama-bench, using five repetitions per test.

Test Throughput
Prompt processing — 512 tokens 776.14 ± 0.96 tok/s
Prompt processing — 1024 tokens 773.16 ± 1.88 tok/s
Prompt processing — 2048 tokens 766.76 ± 0.15 tok/s
Generation — 128 tokens 65.75 ± 0.12 tok/s
Generation — 256 tokens 65.63 ± 0.03 tok/s

Model information reported by llama-bench:

Parameters: 4.21B
GGUF size: 2.51 GiB
Quantization: Q4_K_M
Backend: Metal + BLAS
Threads: 10

Summary

On the tested M4 Pro:

Prompt processing: ~767–776 tokens/second
Token generation:  ~65.6 tokens/second

Native MLX Performance

Measured using the fused 16-bit MLX version:

Prompt tokens: 19
Generated tokens: 256

Results:

Metric Result
Prompt processing 83.92 tok/s
Generation 27.84 tok/s
Peak memory 8.52 GB
Precision 16-bit

Important Benchmark Note

The MLX and llama.cpp results use different benchmark methodologies.

llama-bench isolates model inference throughput, while the MLX result comes from mlx_lm.generate.

The numbers therefore should not be interpreted as a direct apples-to-apples comparison between MLX and GGUF.

Benchmark Summary

Runtime Format Model Size Prompt Generation Peak Memory
MLX / mlx-lm 16-bit ~7.8 GB 83.92 tok/s 27.84 tok/s 8.52 GB
llama.cpp Q4_K_M 2.51 GiB ~767–776 tok/s* ~65.6 tok/s* Not yet measured

*Measured with llama-bench; benchmark methodologies differ between runtimes.

Intelligence / Quality Benchmarks

Formal quality benchmarks are still being evaluated.

Planned evaluations include:

Benchmark Focus Qweble-Sol
MMLU-Pro General knowledge + reasoning TBD
AIME Mathematical reasoning TBD
MBPP / MBPP+ Python programming TBD
LiveCodeBench Code generation TBD
Sysadmin Eval Systems administration TBD

Future results should ideally be reported alongside the original Qwen3.5-4B under equivalent inference settings.

This section will be updated as evaluations are completed.

Qweble-Sol Sysadmin Evaluation

Because Qweble-Sol is specifically oriented toward technical and systems-administration work, general academic benchmarks alone may not accurately represent its intended strengths.

A dedicated evaluation is planned across categories such as:

Linux troubleshooting
macOS administration
Networking
Shell scripting
Python debugging
Docker
Server administration
DevOps
Automation
Security fundamentals

Results and evaluation prompts may be published separately for reproducibility.

Recommended Generation Settings

A reasonable starting configuration is:

temperature: 0.6
top_p: 0.95
top_k: 20
context: 8192

These are recommendations rather than strict requirements.

For deterministic benchmarking, use a temperature of 0 where supported.

Thinking / Reasoning

Qweble-Sol retains Qwen3.5’s thinking-style response behavior.

Depending on the runtime, reasoning may appear as:

Thinking...
...
done thinking.

or through <think> blocks.

Different runtimes may display or suppress this reasoning differently.

Conversion Notes

Important Qwen3.5 + MLX Compatibility Note

Qweble-Sol was fine-tuned and fused using MLX.

A direct conversion of the MLX-fused Qwen3.5 checkpoint using:

convert_hf_to_gguf.py

produced a GGUF that loaded successfully but generated corrupted output.

The MLX model itself remained fully functional.

The issue resulted from differences between the MLX-native representation of some Qwen3.5 tensors and the canonical Hugging Face representation expected by the GGUF conversion pipeline.

To produce the working GGUF release:

  1. The original MLX model was preserved unchanged.
  2. A separate copy was exported back into HF-compatible tensor semantics.
  3. The corrected copy was converted to F16 GGUF.
  4. The F16 GGUF was validated with llama.cpp.
  5. The model was quantized to Q4_K_M.
  6. The Q4_K_M release was validated with both llama.cpp and Ollama.

The native MLX checkpoint remains the canonical MLX version.

This distinction is important for anyone attempting to reproduce a Qwen3.5 MLX → GGUF conversion.

Known Working Configurations

Qweble-Sol has been locally validated with:

MLX

Apple Silicon
mlx-lm
16-bit fused checkpoint

llama.cpp

Q4_K_M GGUF
Metal backend
Apple M4 Pro

Ollama

Q4_K_M GGUF
8192-token configured context
Custom Qweble-Sol system prompt

Intended Uses

Qweble-Sol is primarily intended for:

  • Local technical assistance
  • Programming help
  • Code explanation
  • Debugging
  • Shell scripting
  • Systems troubleshooting
  • Linux/macOS administration
  • Docker help
  • Networking concepts
  • DevOps workflows
  • Automation
  • Educational technical assistance

It can also perform ordinary general-purpose conversational and reasoning tasks inherited from its base model.

Limitations

Qweble-Sol is a relatively small ~4B-parameter model.

It may:

  • Produce incorrect technical information
  • Invent commands, flags, APIs, packages, or configuration options
  • Misinterpret ambiguous infrastructure problems
  • Produce code that requires additional testing
  • Give outdated answers about rapidly changing software
  • Make mistakes in long or complicated reasoning chains
  • Incorrectly describe its own training history or provenance without an explicit system prompt
  • Behave differently across MLX, llama.cpp, and Ollama because of runtime and prompt-template differences

Always verify commands before running them against important systems.

Extra care should be taken with:

rm
dd
disk utilities
partitioning
firewall changes
permissions
production databases
deployment commands
system configuration

Back up important data before performing destructive operations.

Training Details

Qweble-Sol was fine-tuned using MLX / mlx-lm.

Base

Qwen/Qwen3.5-4B

Fine-tuning

Framework: mlx-lm
Method: Adapter/LoRA fine-tuning followed by fusion
Fine-tuner: ukuwzi

Additional dataset and hyperparameter details can be added here if/when they are publicly released.

Architecture

Qweble-Sol retains the underlying Qwen3.5-4B architecture.

High-level properties of the underlying language model include:

Parameters: ~4B
Hidden dimension: 2560
Transformer/hybrid blocks: 32
Feed-forward dimension: 9216
Hybrid linear-attention + full-attention architecture

The GGUF build reports approximately 4.21B parameters.

Model Provenance

To avoid ambiguity:

Item Developer
Qwen3.5 architecture / base model Alibaba / Qwen Team
Qwen3.5-4B base weights Alibaba / Qwen Team
Qweble-Sol fine-tune ukuwzi
Qweble-Sol MLX packaging ukuwzi
Qweble-Sol GGUF conversion ukuwzi
Qweble-Sol Ollama packaging ukuwzi

Qweble-Sol is not an official Alibaba or Qwen release.

License

Qweble-Sol is derived from Qwen3.5-4B, which is distributed under the Apache License 2.0.

Users should review the upstream Qwen3.5 license and any applicable terms before redistribution or commercial use.

See the upstream model:

Qwen/Qwen3.5-4B

Acknowledgements

Qweble-Sol would not exist without the work of:

  • Qwen Team / Alibaba — Qwen3.5-4B
  • MLX / mlx-lm contributors — Apple Silicon fine-tuning and inference
  • llama.cpp contributors — GGUF conversion and local inference
  • Ollama contributors — simple local model deployment

Citation

If you use Qweble-Sol in a project, benchmark, or derivative work, you can reference the Hugging Face repository.

A formal citation entry may be added in a future release.

Please also cite the original Qwen3.5 work when appropriate.

Status

MLX 16-bit       ✅ Working
GGUF F16         ✅ Working
GGUF Q4_K_M      ✅ Working
llama.cpp        ✅ Tested
Ollama           ✅ Tested
MMLU-Pro         ⏳ Pending
Coding benchmarks ⏳ Pending
Sysadmin eval    ⏳ Pending

Disclaimer

Qweble-Sol can generate incorrect or unsafe technical instructions. Generated commands and code should be reviewed before execution, especially on production systems or systems containing important data.

This project is an independent fine-tune and is not affiliated with or endorsed by Alibaba, Qwen, Apple, llama.cpp, or Ollama.