Sarvam-105B (Q4_K_M GGUF) packaged for Ollama. Multilingual Indian MoE LLM made available for local experimentation.

ollama run predictivemanish/sarvam-105b

curl http://localhost:11434/api/chat \
  -d '{
    "model": "predictivemanish/sarvam-105b",
    "messages": [{"role": "user", "content": "Hello!"}]
  }'

from ollama import chat

response = chat(
    model='predictivemanish/sarvam-105b',
    messages=[{'role': 'user', 'content': 'Hello!'}],
)
print(response.message.content)

import ollama from 'ollama'

const response = await ollama.chat({
  model: 'predictivemanish/sarvam-105b',
  messages: [{role: 'user', content: 'Hello!'}],
})
console.log(response.message.content)

Models

View all →

Name

1 model

Size / Usage

Context

Input

sarvam-105b:latest

64GB · 128K context window · Text · 4 months ago

sarvam-105b:latest

64GB

128K

Text

Readme

Sarvam-105B (GGUF Q4_K_M) — Ollama

This is the Sarvam-105B model packaged in GGUF format for direct usage with Ollama.

Model Details

Model: Sarvam-105B
Format: GGUF
Quantization: Q4_K_M
Size: ~64GB
Context Length: 128K
Architecture: Mixture-of-Experts (MoE)

Usage

Run directly with:

ollama run predictivemanish/sarvam-105b

Notes

This is a very large model (~64GB)
Requires high system memory
May not run on all machines
MoE support in local runtimes is still evolving

This release is intended to make Sarvam-105B easily accessible for experimentation in the Ollama ecosystem.

Source

Original model released by Sarvam AI GGUF weights from Hugging Face

Sarvam-30B also available: predictivemanish/sarvam-30b