46 1 month ago

K2-Horizon-MoVA-36B-A4B is the sparse member of the K2-Horizon family: a Mixture-of-Experts model with Mixture-of-Values attention (MoVA) that stores 36B parameters and runs 4B per token.

36b
ollama run ahmgam/k2-horizon:36b

Details

1 month ago

15d5db55af40 · 22GB

k2-horizon
·
37.4B
·
Q4_K_M
{ "stop": [ "<|im_start|>", "<|im_end|>", "<|endoftext|>" ], "te

Readme