Muse Glimmer 30B — UD-IQ2_XXS

An ultra-low-bit GGUF build of Meta’s Muse Glimmer 30B, packaged for local inference with Ollama. This release is intended to make the dense 30B model practical on memory-constrained consumer systems by combining mixed quantization with Ollama’s CPU/GPU offloading.

##Quick start

ollama run X3S00/muse-glimmer-30b:ud-iq2_xxs

Disable the model’s visible reasoning trace for faster everyday chat and concise output:

ollama run X3S00/muse-glimmer-30b:ud-iq2_xxs --think=false

Hardware notes

• Tested: NVIDIA RTX 3070 8 GB with CUDA acceleration and CPU/RAM offloading
• Download size: approximately 11 GB
• Default configured context: 8,192 tokens
• Architecture-declared maximum: 131,072 tokens
• Quantization: UD-IQ2_XXS mixed quantization

The model does not fit entirely in 8 GB of VRAM. Ollama keeps the layers that fit on the GPU and uses system memory plus the CPU for the remainder. More VRAM and memory bandwidth should improve speed substantially. The full 128K architectural limit is shown by Ollama, but this package defaults to 8K to keep memory usage practical on the tested RTX 3070 system. Increasing context requires additional memory, particularly for the KV cache.

Reasoning modes

Muse Glimmer supports native reasoning as well as direct-response operation.

Reasoning enabled:

ollama run X3S00/muse-glimmer-30b:ud-iq2_xxs

Reasoning disabled:

ollama run X3S00/muse-glimmer-30b:ud-iq2_xxs --think=false

Reasoning can help on difficult multi-step tasks, while reasoning-off mode is usually faster and well suited to ordinary conversation, extraction, classification, and structured-data generation.

Structured output and distillation

The model can act as a local teacher for generating synthetic instruction data. For clean JSON, make the output constraint explicit:

Generate three algebra question-and-answer pairs. Return only a valid JSON array.
Do not include Markdown fences, commentary, or reasoning.

For dataset generation, validate the JSON, remove duplicates, verify answers, and retain provenance before using generated samples for training.

Ollama API

curl http://localhost:11434/api/chat -d '{
  "model": "X3S00/muse-glimmer-30b:ud-iq2_xxs",
  "messages": [
    { "role": "user", "content": "Explain why the sky is blue." }
  ],
  "think": false,
  "stream": false
}'

What this release is

• The coherent, unmodified Muse Glimmer model architecture
• Quantized for a much smaller memory footprint
• Packaged with an 8K default context for practical consumer-hardware use
• Suitable for chat, reasoning, code, structured output, and synthetic-data generation

This is not the unsuccessful experimental FFN-pruned model. No feed-forward layers or channels were removed from this release.

Credits and license

• Original model: Meta Muse Glimmer 30B
• Meta GGUF release: Muse Glimmer 30B GGUF
• Quantization source: Unsloth Muse Glimmer 30B GGUF
• Runtime: Ollama

Released under the Apache License 2.0, following the upstream model license. Meta, Muse Glimmer, Unsloth, and Ollama belong to their respective owners. This is a community packaging of the quantized model and is not an official Meta or Ollama release.