huihui_ai/granite3.1-dense-abliterated:8b

huihui_ai/ granite3.1-dense-abliterated:8b

1,050 Downloads Updated 1 year ago

The IBM Granite 2B and 8B models are text-only dense LLMs trained on over 12 trillion tokens of data, demonstrated significant improvements over their predecessors in performance and speed in IBM’s initial testing.

tools 2b 8b

ollama run huihui_ai/granite3.1-dense-abliterated:8b

curl http://localhost:11434/api/chat \
  -d '{
    "model": "huihui_ai/granite3.1-dense-abliterated:8b",
    "messages": [{"role": "user", "content": "Hello!"}]
  }'

from ollama import chat

response = chat(
    model='huihui_ai/granite3.1-dense-abliterated:8b',
    messages=[{'role': 'user', 'content': 'Hello!'}],
)
print(response.message.content)

import ollama from 'ollama'

const response = await ollama.chat({
  model: 'huihui_ai/granite3.1-dense-abliterated:8b',
  messages: [{role: 'user', content: 'Hello!'}],
})
console.log(response.message.content)

Details

Updated 1 year ago

1 year ago

6567faf671e9 · 4.9GB ·

model

archgranite

parameters8.17B

quantizationQ4_K_M

4.9GB

system

Knowledge Cutoff Date: April 2024. You are Granite, developed by IBM.

69B

license

Apache License Version 2.0, January 2004 http://www.apache.org/licenses/ TERMS AND CONDITIONS FOR US

11kB

template

<|start_of_role|>system<|end_of_role|> {{- if and (gt (len .Messages) 0) (eq (index .Messages 0).Rol

1.4kB

Readme

This is an uncensored version of ibm-granite/granite-31-language-models created with abliteration (see remove-refusals-with-transformers to know more about it).
This is a crude, proof-of-concept implementation to remove refusals from an LLM model without using TransformerLens.

References

HuggingFace