Sarvam-105B (Q4_K_M GGUF) packaged for Ollama. Multilingual Indian MoE LLM made available for local experimentation.

ollama run predictivemanish/sarvam-105b

curl http://localhost:11434/api/chat \
  -d '{
    "model": "predictivemanish/sarvam-105b",
    "messages": [{"role": "user", "content": "Hello!"}]
  }'

from ollama import chat

response = chat(
    model='predictivemanish/sarvam-105b',
    messages=[{'role': 'user', 'content': 'Hello!'}],
)
print(response.message.content)

import ollama from 'ollama'

const response = await ollama.chat({
  model: 'predictivemanish/sarvam-105b',
  messages: [{role: 'user', content: 'Hello!'}],
})
console.log(response.message.content)

Details

Updated 4 months ago

4 months ago

6d62953e4809 · 64GB ·

model

archdeepseek2

parameters11.4B

quantizationQ4_K_M

64GB

template

User: {{ .Prompt }} Assistant:

31B

params

{ "num_ctx": 4096, "temperature": 0.7 }

35B

Readme

Sarvam-105B (GGUF Q4_K_M) — Ollama

This is the Sarvam-105B model packaged in GGUF format for direct usage with Ollama.

Model Details

Model: Sarvam-105B
Format: GGUF
Quantization: Q4_K_M
Size: ~64GB
Context Length: 128K
Architecture: Mixture-of-Experts (MoE)

Usage

Run directly with:

ollama run predictivemanish/sarvam-105b

Notes

This is a very large model (~64GB)
Requires high system memory
May not run on all machines
MoE support in local runtimes is still evolving

This release is intended to make Sarvam-105B easily accessible for experimentation in the Ollama ecosystem.

Source

Original model released by Sarvam AI GGUF weights from Hugging Face

Sarvam-30B also available: predictivemanish/sarvam-30b