You need to run it by giving it a incomplete dialogue and a prompt format: for example:

“Write theabbie’s next reply to this.\n***\nstack8156: ironically that was the reason sir\nGoatTamer: :noelthink:\n18o3: maine hagg diya\ntheabbie:”

Even then it’s pretty stupid since it’s a 3b model (llama 3.2-3b finetuned). I recommend using the FP16 model, though it is 6GB, the q4 quant can’t really do well, and the q8 is meh.

dataset used for training: https://huggingface.co/datasets/mesosan/cmc_server_abbie_sir

I don’t claim any right over anything and none of this is my intellectual property. This is pretty stupid tbh.