ibm/
granite4:small-h-q4_1

7,365 1 month ago

Granite 4 features improved instruction following (IF) and tool-calling capabilities, making them more effective in enterprise applications.

tools 350m 1b 3b

1 month ago

28a5cfc16675 · 20GB ·

model
granitehybrid
·
32.2B
·
Q4_1
template
{{- /* ------ MESSAGE PARSING ------ */}} {{- /* Declare the system prompt chunks used for different
license
Apache License Version 2.0, January 2004 http://www.apache.org/licenses/ TERMS AND CONDITIONS FOR US

Readme

Granite 4.0 models

Granite 4.0 models are finetuned from their base models using a combination of open source instruction datasets with permissive license and internally collected synthetic datasets. They feature improved instruction following (IF) and tool-calling capabilities, making them more effective in enterprise applications.

Please Note: our 3b, 1b, and 350m model sizes are alternative options for users when mamba-2 support is not yet optimized. Models denoted -h use the hybrid mamba-2 architecture.

Parameter Sizes

350m

ollama run ibm/granite4:350m

350m-h

ollama run ibm/granite4:350m-h

1b

ollama run ibm/granite4:1b

1b-h

ollama run ibm/granite4:1b-h

3b (micro)

ollama run ibm/granite4:3b
ollama run ibm/granite4:micro

3b-h (micro-h)

ollama run ibm/granite4:3b-h
ollama run ibm/granite4:micro-h

7b-a1b-h (tiny-h)

ollama run ibm/granite4:7b-a1b-h
ollama run ibm/granite4:tiny-h

32b-a9b-h (small-h)

ollama run ibm/granite4:32b-a9b-h
ollama run ibm/granite4:small-h

other quantizations Models above have a default quantization of Q4_K_M. To run other quantizations (e.g., Q8): ollama run ibm/granite4:tiny-h-q8_0

base models Base models without instruction tuning are provided for all sizes and quantizations. These can be accessed with tags such as ibm/granite4:tiny-h-base-f16.

Supported Languages

Supported Languages: English, German, Spanish, French, Japanese, Portuguese, Arabic, Czech, Italian, Korean, Dutch, and Chinese. Users may finetune Granite 4.0 models for languages beyond these languages.

Intended Use

This model is designed to handle general instruction-following tasks and can be integrated into AI assistants across various domains, including business applications.

Capabilities

  • Summarization
  • Text classification
  • Text extraction
  • Question-answering
  • Retrieval Augmented Generation (RAG)
  • Code related tasks
  • Function-calling tasks
  • Multilingual dialog use cases
  • Fill-In-the-Middle (FIM) code completions

Learn more