anaskhedr/bonsai-27b-32k

anaskhedr/ bonsai-27b-32k:latest

42 Downloads Updated 1 week ago

a no-edit fork of MichelRosselli/bonsai-27b where context size has been shrieked to fit 8GB vRam card.

vision tools thinking

ollama run anaskhedr/bonsai-27b-32k

curl http://localhost:11434/api/chat \
  -d '{
    "model": "anaskhedr/bonsai-27b-32k",
    "messages": [{"role": "user", "content": "Hello!"}]
  }'

from ollama import chat

response = chat(
    model='anaskhedr/bonsai-27b-32k',
    messages=[{'role': 'user', 'content': 'Hello!'}],
)
print(response.message.content)

import ollama from 'ollama'

const response = await ollama.chat({
  model: 'anaskhedr/bonsai-27b-32k',
  messages: [{role: 'user', content: 'Hello!'}],
})
console.log(response.message.content)

Details

Updated 1 week ago

1 week ago

1d8c26674325 · 4.4GB

model

archqwen35

parameters26.9B

quantization(!unknown_file_type 40!)

3.8GB

projector

archclip

parameters461M

quantizationQ8_0

629MB

license

Apache License Version 2.0, January 2004 http://www.apache.org/licenses/ TERMS AND CONDITIONS FOR US

10kB

license

413B

params

{ "min_p": 0, "num_ctx": 32768, "repeat_penalty": 1, "stop": [ "<|im_start|>

173B

Readme

Referee to the readme from the original model https://ollama.com/MichelRosselli/bonsai-27b

Exact steps for recreating the model(for transparency’s sake)

Manifest:

FROM MichelRosselli/bonsai-27b:latest
PARAMETER num_ctx 32768
PARAMETER temperature 0.05

used command: ollama create bonsai-27b-32k -f Modelfile-bonsai-27b-128ctx

My Rant

I highly recommend using it with: page assist on Brave or Chrome
tested on GeForce RTX 3070 8GB card:

$ ollama ps
NAME                               ID              SIZE      PROCESSOR    CONTEXT    UNTIL              
anaskhedr/bonsai-27b-32k:latest    1d8c26674325    6.3 GB    100% GPU     32768      4 minutes from now

failed to load all bonsai on ollama/windows with AMD Radeon 780M iGpu but can load different variants from Lemonade