ahmgam/ k2-horizon:36b

46 1 month ago

K2-Horizon-MoVA-36B-A4B is the sparse member of the K2-Horizon family: a Mixture-of-Experts model with Mixture-of-Values attention (MoVA) that stores 36B parameters and runs 4B per token.

36b
... /
params
fefd3a5c0c53 ยท 114B
{
"stop": [
"<|im_start|>",
"<|im_end|>",
"<|endoftext|>"
],
"temperature": 1,
"top_p": 0.95
}