huihui_ai/granite3.1-dense-abliterated:2b-instruct-fp16

huihui_ai/ granite3.1-dense-abliterated:2b-instruct-fp16

1,051 Downloads Updated 1 year ago

The IBM Granite 2B and 8B models are text-only dense LLMs trained on over 12 trillion tokens of data, demonstrated significant improvements over their predecessors in performance and speed in IBM’s initial testing.

tools 2b 8b

ollama run huihui_ai/granite3.1-dense-abliterated:2b-instruct-fp16

curl http://localhost:11434/api/chat \
  -d '{
    "model": "huihui_ai/granite3.1-dense-abliterated:2b-instruct-fp16",
    "messages": [{"role": "user", "content": "Hello!"}]
  }'

from ollama import chat

response = chat(
    model='huihui_ai/granite3.1-dense-abliterated:2b-instruct-fp16',
    messages=[{'role': 'user', 'content': 'Hello!'}],
)
print(response.message.content)

import ollama from 'ollama'

const response = await ollama.chat({
  model: 'huihui_ai/granite3.1-dense-abliterated:2b-instruct-fp16',
  messages: [{role: 'user', content: 'Hello!'}],
})
console.log(response.message.content)

Details

Updated 1 year ago

1 year ago

4d124254b2bd · 5.1GB ·

model

archgranite

parameters2.53B

quantizationF16

5.1GB

system

Knowledge Cutoff Date: April 2024. You are Granite, developed by IBM.

69B

license

Apache License Version 2.0, January 2004 http://www.apache.org/licenses/ TERMS AND CONDITIONS FOR US

11kB

template

<|start_of_role|>system<|end_of_role|> {{- if and (gt (len .Messages) 0) (eq (index .Messages 0).Rol

1.4kB

Readme

This is an uncensored version of ibm-granite/granite-31-language-models created with abliteration (see remove-refusals-with-transformers to know more about it).
This is a crude, proof-of-concept implementation to remove refusals from an LLM model without using TransformerLens.

References

HuggingFace