17 1 week ago

A GPT-style language model built entirely from scratch (no HuggingFace/nanoGPT for the model itself) and trained on 50 children's moral stories. It's an educational reproduction of the full LLM lifecycle at miniature scale.

8811797bc90d · 314B
story-llm: a 0.94M-parameter GPT built from scratch and trained on 50 children's moral stories. An educational reproduction of the full LLM pipeline (custom BPE -> pretraining -> SFT -> Ollama). Learning artifact only; output is garbled at this scale. Source: https://github.com/sppandita85/story-llm-finetuned-mac