Luganda Gemma 1B IT

A fine-tuned version of Google Gemma 3 1B Instruct for English ↔ Luganda translation and Luganda conversational AI.

Highlights

• BLEU 13.85 on English→Luganda translation, up from 0.06 on the base model
• chrF++ 46.59 — strong character-level accuracy for morphologically rich Luganda
• Trained with QLoRA (4-bit quantization + LoRA adapters) — runs on consumer GPUs
• Only 52 MB adapter on top of the 1B base model

Results

Usage

Quick start

from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
from peft import PeftModel
import torch

base_model_id = "google/gemma-3-1b-it"
adapter_id = "AmplifiedAccess/Luganda-gemma-1b-it"

# Load with same quantization used during training
tokenizer = AutoTokenizer.from_pretrained(base_model_id)

model = AutoModelForCausalLM.from_pretrained(
    base_model_id,
    quantization_config=BitsAndBytesConfig(
        load_in_4bit=True,
        bnb_4bit_quant_type="nf4",
        bnb_4bit_compute_dtype=torch.bfloat16,
        bnb_4bit_use_double_quant=True,
    ),
    device_map={"": 0},
)

model = PeftModel.from_pretrained(model, adapter_id)
model.eval()

Translation (English → Luganda)

prompt = "Translate to Luganda:\nThe farmers need better seeds to improve their harvest."

messages = [{"role": "user", "content": prompt}]
inputs = tokenizer.apply_chat_template(
    messages, return_tensors="pt", return_dict=True, add_generation_prompt=True
).to(model.device)

with torch.no_grad():
    outputs = model.generate(**inputs, max_new_tokens=128, do_sample=False)

prompt_len = inputs["input_ids"].shape[1]
response = tokenizer.decode(outputs[0][prompt_len:], skip_special_tokens=True).strip()
print(response)

Supported prompt formats

The model was trained on varied prompt templates. All of these work:

English → Luganda: - Translate to Luganda:\n{text} - Convert this to Luganda:\n{text} - English to Luganda:\n{text} - How do you say this in Luganda?\n{text}

Luganda → English: - Translate to English:\n{text} - Convert this to English:\n{text} - Luganda to English:\n{text} - What does this mean in English?\n{text}

Conversational (respond in Luganda): - Respond in Luganda: {text} - Answer in Luganda:\n{text} - Yogera mu Luganda: {text}

Example translations

Training details

Data

• 94,542 training examples derived from ~24,000 clean English-Luganda parallel sentence pairs
• Primary source: Sunbird SALT dataset — professionally translated, multi-way parallel corpus covering agriculture, health, society, and other locally relevant topics
• Three task types: English→Luganda translation, Luganda→English translation, and Luganda conversation
• Split: 90% train / 5% validation / 5% test

Configuration

Training loss curve

Important notes

• Use 4-bit quantization for inference — the LoRA weights were trained against 4-bit base weights. Loading in 8-bit or full precision will produce degraded translations.
• Use do_sample=False for deterministic, highest-quality translations.
• Use the base model tokenizer (google/gemma-3-1b-it) for best results.
• The model was trained on 1 epoch. Further training (3+ epochs) and additional data sources (JW300, Kimera corpus) would likely improve scores further.

Limitations

• Trained primarily on Sunbird SALT data, which covers agriculture, health, and society topics. Performance may be weaker on highly specialized domains (legal, medical, technical).
• Luganda→English translation quality may lag behind English→Luganda since the base model already has strong English capabilities.
• Short sentences (under 5 words) may produce less accurate translations.
• The model may occasionally produce Luganda synonyms or dialectal variants that differ from a specific reference translation while still being semantically correct.

Intended use

• English ↔ Luganda machine translation
• Luganda conversational AI and chatbot applications
• Educational tools for Luganda language learning
• Research on low-resource African language NLP

Acknowledgments

• Sunbird AI for the SALT parallel corpus
• Google DeepMind for the Gemma model family
• Makerere University AI Lab for contributions to the SALT dataset
• Crane AI Labs for their pioneering work on Ugandan language models

Framework versions

• PEFT 0.18.1
• Transformers 4.x
• TRL 0.x
• PyTorch 2.x
• bitsandbytes 0.45.x

Citation

If you use this model, please cite:

@misc{luganda-gemma-1b-2026,
  title={Luganda Gemma 1B IT: Fine-tuned Gemma 3 1B for English-Luganda Translation},
  author={Amplified Access},
  year={2026},
  publisher={HuggingFace},
  url={https://huggingface.co/AmplifiedAccess/Luganda-gemma-1b-it}
}