huihui_ai/granite3.1-dense-abliterated:8b-instruct-fp16

huihui_ai/ granite3.1-dense-abliterated:8b-instruct-fp16

1,051 Downloads Updated 1 year ago

The IBM Granite 2B and 8B models are text-only dense LLMs trained on over 12 trillion tokens of data, demonstrated significant improvements over their predecessors in performance and speed in IBM’s initial testing.

tools 2b 8b

ollama run huihui_ai/granite3.1-dense-abliterated:8b-instruct-fp16

curl http://localhost:11434/api/chat \
  -d '{
    "model": "huihui_ai/granite3.1-dense-abliterated:8b-instruct-fp16",
    "messages": [{"role": "user", "content": "Hello!"}]
  }'

from ollama import chat

response = chat(
    model='huihui_ai/granite3.1-dense-abliterated:8b-instruct-fp16',
    messages=[{'role': 'user', 'content': 'Hello!'}],
)
print(response.message.content)

import ollama from 'ollama'

const response = await ollama.chat({
  model: 'huihui_ai/granite3.1-dense-abliterated:8b-instruct-fp16',
  messages: [{role: 'user', content: 'Hello!'}],
})
console.log(response.message.content)

Details

Updated 1 year ago

1 year ago

c6ab093b8e7c · 16GB ·

model

archgranite

parameters8.17B

quantizationF16

16GB

system

Knowledge Cutoff Date: April 2024. You are Granite, developed by IBM.

69B

license

Apache License Version 2.0, January 2004 http://www.apache.org/licenses/ TERMS AND CONDITIONS FOR US

11kB

template

<|start_of_role|>system<|end_of_role|> {{- if and (gt (len .Messages) 0) (eq (index .Messages 0).Rol

1.4kB

Readme

This is an uncensored version of ibm-granite/granite-31-language-models created with abliteration (see remove-refusals-with-transformers to know more about it).
This is a crude, proof-of-concept implementation to remove refusals from an LLM model without using TransformerLens.

References

HuggingFace