neotherack/phi4_tools:14b_q2

neotherack/

phi4_tools:14b_q2_K

108 Downloads Updated 6 months ago

Tool enabled, quantized version Q2_K of Phi4:14b

tools

Updated 6 months ago

6 months ago

b062969d651e · 5.5GB

archphi3

parameters14.7B

quantizationQ2_K

5.5GB

Microsoft. Copyright (c) Microsoft Corporation. MIT License Permission is hereby granted, free of ch

1.1kB

{{- if .Messages }} {{- if or .System .Tools }}<|im_start|>system {{- if .System }} {{ .System }} {{

1.5kB

{ "num_ctx": 16384, "stop": [ "<|im_start|>", "<|im_end|>", "<|im_se

114B

This is my very first quantization and template for Phi4 on ollama. It seems to work on function calling on a low VRAM usage.

I recommend to use it with variables like this: OLLAMA_FLASH_ATTENTION=true OLLAMA_KV_CACHE_TYPE=q8_0

It works kind of good in my old 4Gb VRAM laptop :)