krith/phi-3.5-mini-instruct

krith/ phi-3.5-mini-instruct

163 Downloads Updated 2 years ago

I-quants for phi-3.5-mini-instruct

ollama run krith/phi-3.5-mini-instruct:IQ3_M

curl http://localhost:11434/api/chat \
  -d '{
    "model": "krith/phi-3.5-mini-instruct:IQ3_M",
    "messages": [{"role": "user", "content": "Hello!"}]
  }'

from ollama import chat

response = chat(
    model='krith/phi-3.5-mini-instruct:IQ3_M',
    messages=[{'role': 'user', 'content': 'Hello!'}],
)
print(response.message.content)

import ollama from 'ollama'

const response = await ollama.chat({
  model: 'krith/phi-3.5-mini-instruct:IQ3_M',
  messages: [{role: 'user', content: 'Hello!'}],
})
console.log(response.message.content)

Models

View all →

Name

4 models

Size / Usage

Context

Input

phi-3.5-mini-instruct:IQ3_M

1.9GB · 4K context window · Text · 2 years ago

phi-3.5-mini-instruct:IQ3_M

1.9GB

Text

phi-3.5-mini-instruct:IQ2_M

1.3GB · 4K context window · Text · 2 years ago

phi-3.5-mini-instruct:IQ2_M

1.3GB

Text

phi-3.5-mini-instruct:IQ3_XS

1.6GB · 4K context window · Text · 2 years ago

phi-3.5-mini-instruct:IQ3_XS

1.6GB

Text

phi-3.5-mini-instruct:IQ4_XS

2.1GB · 4K context window · Text · 2 years ago

phi-3.5-mini-instruct:IQ4_XS

2.1GB

Text

Readme

Created using GGUF files from https://huggingface.co/bartowski/Phi-3.5-mini-instruct-GGUF