neotherack/phi4_tools:14b_q2

neotherack/ phi4_tools:14b_q2_K

141 Downloads Updated 1 year ago

Tool enabled, quantized version Q2_K of Phi4:14b

tools

ollama run neotherack/phi4_tools:14b_q2_K

curl http://localhost:11434/api/chat \
  -d '{
    "model": "neotherack/phi4_tools:14b_q2_K",
    "messages": [{"role": "user", "content": "Hello!"}]
  }'

from ollama import chat

response = chat(
    model='neotherack/phi4_tools:14b_q2_K',
    messages=[{'role': 'user', 'content': 'Hello!'}],
)
print(response.message.content)

import ollama from 'ollama'

const response = await ollama.chat({
  model: 'neotherack/phi4_tools:14b_q2_K',
  messages: [{role: 'user', content: 'Hello!'}],
})
console.log(response.message.content)

Details

Updated 1 year ago

1 year ago

b062969d651e · 5.5GB ·

model

archphi3

parameters14.7B

quantizationQ2_K

5.5GB

license

Microsoft. Copyright (c) Microsoft Corporation. MIT License Permission is hereby granted, free of ch

1.1kB

template

{{- if .Messages }} {{- if or .System .Tools }}<|im_start|>system {{- if .System }} {{ .System }} {{

1.5kB

params

{ "num_ctx": 16384, "stop": [ "<|im_start|>", "<|im_end|>", "<|im_se

114B

Readme

This is my very first quantization and template for Phi4 on ollama. It seems to work on function calling on a low VRAM usage.

I recommend to use it with variables like this: OLLAMA_FLASH_ATTENTION=true OLLAMA_KV_CACHE_TYPE=q8_0

It works kind of good in my old 4Gb VRAM laptop :)