A bitnet-style LLM that runs on consumer hardware

21b

ollama run reecdev/bytenet

curl http://localhost:11434/api/chat \
  -d '{
    "model": "reecdev/bytenet",
    "messages": [{"role": "user", "content": "Hello!"}]
  }'

from ollama import chat

response = chat(
    model='reecdev/bytenet',
    messages=[{'role': 'user', 'content': 'Hello!'}],
)
print(response.message.content)

import ollama from 'ollama'

const response = await ollama.chat({
  model: 'reecdev/bytenet',
  messages: [{role: 'user', content: 'Hello!'}],
})
console.log(response.message.content)

Models

View all →

Name

2 models

Size / Usage

Context

Input

bytenet:latest

4.5GB · 128K context window · Text · 3 weeks ago

bytenet:latest

4.5GB

128K

Text

bytenet:21b

4.5GB · 128K context window · Text · 3 weeks ago

bytenet:21b

4.5GB

128K

Text

Readme

ByteNet

ByteNet is a 21 billion parameter LLM based on ERNIE 4.5 that can run on consumer GPUs with less than 6 GB of VRAM using aggressive compression techniques.

Quick Start

Pull ByteNet and run on GPU (min vram: 5 GB)

ollama run reecdev/bytenet