Occitan Lengadocian — Gemma 4 E2B RS-LoRA

Fine-tune of Gemma 4 E2B Instruct on Occitan in the Lengadocian dialect (IEO grafia classica norm). Trained via QLoRA on literary, journalistic, grammar, and encyclopedic sources normalised to Lengadocian standard.

Best Gemma 4 model in the series. Holds three project records and delivers the richest literary vocabulary density of any model in the collection. Fast inference due to the 2B base.

Key results

• Literary continuation markers: 5.7 (project record)
• Journalistic continuation markers: 8.7 (project record)
• Verb paradigm metalanguage: 7.3 (project record)
• Train loss: 0.35 — the cleanest training signal in the project

Strengths

Particularly strong on sustained literary prose with rare Lengadocian vocabulary (finòca, destriava, parpalhons negrós, roginassa, grols, lentèl, remòls, teulissas pesugadas, fogals petejats) and on dense journalistic register with zero interference.

System prompt

Ès un escritor e grammarista occitan lengadocian. Respon unicament en
occitan lengadocian. Escriu dirèctament lo tèxte demandat, sens cap
d'introduccion, de comentari ni d'explicacion sus ton trabalh. Pas de
preamble. Pas de version multiplas. Pas de traduccion.

Recommended quantisation

Training

RS-LoRA · r=32 · α=32 · block_size=384 · 2535 steps · 5 epochs
RTX 3060 12GB · ~3h20m · FastVisionModel (text-only) + SFTTrainer