freerainboxbox/phi4

freerainboxbox/ phi4

212 Downloads Updated 1 year ago

Alternative quantization levels, no fine-tuning

ollama run freerainboxbox/phi4:14b-q3_K_S

curl http://localhost:11434/api/chat \
  -d '{
    "model": "freerainboxbox/phi4:14b-q3_K_S",
    "messages": [{"role": "user", "content": "Hello!"}]
  }'

from ollama import chat

response = chat(
    model='freerainboxbox/phi4:14b-q3_K_S',
    messages=[{'role': 'user', 'content': 'Hello!'}],
)
print(response.message.content)

import ollama from 'ollama'

const response = await ollama.chat({
  model: 'freerainboxbox/phi4:14b-q3_K_S',
  messages: [{role: 'user', content: 'Hello!'}],
})
console.log(response.message.content)

Models

View all →

Name

11 models

Size

Context

Input

phi4:14b-q3_K_S

6.5GB · 16K context window · Text · 1 year ago

phi4:14b-q3_K_S

6.5GB

16K

Text

phi4:14b-q3_K_M

7.4GB · 16K context window · Text · 1 year ago

phi4:14b-q3_K_M

7.4GB

16K

Text

phi4:14b-q3_K_L

7.9GB · 16K context window · Text · 1 year ago

phi4:14b-q3_K_L

7.9GB

16K

Text

phi4:14b-q4_0

8.4GB · 16K context window · Text · 1 year ago

phi4:14b-q4_0

8.4GB

16K

Text

phi4:14b-q4_1

9.3GB · 16K context window · Text · 1 year ago

phi4:14b-q4_1

9.3GB

16K

Text

phi4:14b-q4_K_S

8.4GB · 16K context window · Text · 1 year ago

phi4:14b-q4_K_S

8.4GB

16K

Text

phi4:14b-q5_0

10GB · 16K context window · Text · 1 year ago

phi4:14b-q5_0

10GB

16K

Text

phi4:14b-q5_1

11GB · 16K context window · Text · 1 year ago

phi4:14b-q5_1

11GB

16K

Text

phi4:14b-q5_K_S

10GB · 16K context window · Text · 1 year ago

phi4:14b-q5_K_S

10GB

16K

Text

phi4:14b-q5_K_M

11GB · 16K context window · Text · 1 year ago

phi4:14b-q5_K_M

11GB

16K

Text

phi4:14b-q6_K

12GB · 16K context window · Text · 1 year ago

phi4:14b-q6_K

12GB

16K

Text

Readme

These are alternative quantization levels from Microsoft’s phi4. No fine-tuning has been done, these are purely quantized.