Alieno/ ailo-340m-v4:f16

25 1 month ago

341M-parameter English chat model (32 layers, GPT-2 tokenizer) trained end-to-end on a single consumer AMD GPU (RX 6750 XT, 12 GB) with a hand-written backward engine PyTorch autograd is unstable on this hardware for deep transformers.

tools thinking
... /
params
cb825e85c66c · 180B
{
"num_predict": 200,
"repeat_penalty": 1.25,
"stop": [
"<|user|>",
"<|assistant|>",
"<|end|>",
"</|end|>"
],
"temperature": 0.3,
"top_k": 20,
"top_p": 0.9
}