julienp79/ occitan-gemma-3-12b-it-rslora-sfttrainer:f16

2 yesterday

Gemma 3 12B Instruct fine-tuned on Occitan (Lengadocian) via RS-LoRA (r=16). Q2_K, Q4_K_M, Q5_K_M, Q8_0, f16 quants available. 

ollama run julienp79/occitan-gemma-3-12b-it-rslora-sfttrainer:f16

Details

yesterday

0a946a9bafde · 24GB ·

model
gemma3
·
11.8B
·
F16
template
{{- $systemPromptAdded := false }} {{- range $i, $_ := .Messages }} {{- $last := eq (len (slice $.Me
system
Ès un escritor e grammarista occitan lengadocian. Respon unicament en occitan lengadocian. Escriu d
params
{ "num_ctx": 768, "stop": [ "<end_of_turn>" ], "temperature": 0.7, "top_

Readme

Occitan Lengadocian — Gemma 3 12B RS-LoRA

Fine-tune of Gemma 3 12B Instruct on Occitan in the Lengadocian dialect (Hérault-rooted, IEO grafia classica norm). Trained on literary prose (Bodon, Lafont, Bosc), journalistic sources (Sapiència), Alibert’s Gramatica Occitana, and encyclopedic texts — all normalised to remove Provençal contamination.

Project flagship. Strongest overall balance across literary register, journalistic summary, grammatical metalanguage, and correction tasks.

Key results

  • Journalistic summary: 9.0/10 with zero interference tokens (project record)
  • Grammatical correction: 1.7⁄2 (best in 12B family)
  • Literary dialect consistency: 4.2⁄5

System prompt

For best results, use this system prompt:

Ès un escritor e grammarista occitan lengadocian. Respon unicament en
occitan lengadocian. Escriu dirèctament lo tèxte demandat, sens cap
d'introduccion, de comentari ni d'explicacion sus ton trabalh. Pas de
preamble. Pas de version multiplas. Pas de traduccion.

Recommended quantisation

Quant Size Use case
Q4_K_M 7.3 GB Best quality/size balance — recommended
Q5_K_M 8.4 GB Slightly better quality, needs 12 GB VRAM
Q8_0 13 GB Near-lossless, needs 16 GB VRAM or CPU
Q2_K 4.8 GB CPU inference on 8 GB RAM, quality reduced

Training

RS-LoRA · r=16 · α=32 · block_size=768 · 1270 steps · 5 epochs
RTX 3060 12GB · ~11h · FastLanguageModel + SFTTrainer