vedangurade/ smollm2-135m-instruct-f16

10 Downloads Updated 1 week ago

SmolLM2-135M-Instruct is a lightweight instruction-tuned language model developed by Hugging Face, converted to F16 GGUF format for local inference using Ollama.

ollama run vedangurade/smollm2-135m-instruct-f16

curl http://localhost:11434/api/chat \
  -d '{
    "model": "vedangurade/smollm2-135m-instruct-f16",
    "messages": [{"role": "user", "content": "Hello!"}]
  }'

from ollama import chat

response = chat(
    model='vedangurade/smollm2-135m-instruct-f16',
    messages=[{'role': 'user', 'content': 'Hello!'}],
)
print(response.message.content)

import ollama from 'ollama'

const response = await ollama.chat({
  model: 'vedangurade/smollm2-135m-instruct-f16',
  messages: [{role: 'user', content: 'Hello!'}],
})
console.log(response.message.content)

Models

Name

1 model

Size / Usage

Context

Input

smollm2-135m-instruct-f16:latest

271MB · 8K context window · Text · 1 week ago

smollm2-135m-instruct-f16:latest

271MB

8K

Text

Readme

SmolLM2-135M-Instruct (F16 GGUF)

SmolLM2-135M-Instruct is a compact, instruction-tuned language model developed by Hugging Face. This repository contains an F16 GGUF conversion of the original model, prepared for local inference and experimentation using Ollama.

Model Details

Property	Details
Original Developer	Hugging Face
Model	SmolLM2-135M-Instruct
Parameters	135M
Architecture	Transformer-based causal language model
Original Format	Safetensors
Published Format	GGUF (F16)
File Size	Approximately 271 MB
Context Window	8K tokens
License	Apache 2.0

Run the Model

Install Ollama and run:

ollama run vedangurade/smollm2-135m-instruct-f16

Conversion and Deployment

The original model was downloaded from Hugging Face and converted from Safetensors to F16 GGUF using llama.cpp.

The conversion preserves F16 weight precision without applying 4-bit or 8-bit quantization.

The converted model was imported into Ollama, tested locally on Windows, and published to this repository.

Intended Use

This model is suitable for experimenting with lightweight language models, basic instruction-following tasks, and local AI deployment workflows.

As a 135M-parameter model, it has limited capabilities compared with larger language models and may produce inaccurate responses.

References and Attribution

Original Model: https://huggingface.co/HuggingFaceTB/SmolLM2-135M-Instruct
Conversion Tool: https://github.com/ggml-org/llama.cpp
Deployment Platform: https://ollama.com

This is an independently converted and republished version of Hugging Face’s SmolLM2-135M-Instruct model. It is not an independently trained model or an official Hugging Face release.

# SmolLM2-135M-Instruct (F16 GGUF)

SmolLM2-135M-Instruct is a compact, instruction-tuned language model developed by Hugging Face. This repository contains an F16 GGUF conversion of the original model, prepared for local inference and experimentation using Ollama.

## Model Details

| Property | Details |
|---|---|
| Original Developer | Hugging Face |
| Model | SmolLM2-135M-Instruct |
| Parameters | 135M |
| Architecture | Transformer-based causal language model |
| Original Format | Safetensors |
| Published Format | GGUF (F16) |
| File Size | Approximately 271 MB |
| Context Window | 8K tokens |
| License | Apache 2.0 |

## Run the Model

Install Ollama and run:

`ollama run vedangurade/smollm2-135m-instruct-f16`

## Conversion and Deployment

The original model was downloaded from Hugging Face and converted from Safetensors to F16 GGUF using llama.cpp.

The conversion preserves F16 weight precision without applying 4-bit or 8-bit quantization.

The converted model was imported into Ollama, tested locally on Windows, and published to this repository.

## Intended Use

This model is suitable for experimenting with lightweight language models, basic instruction-following tasks, and local AI deployment workflows.

As a 135M-parameter model, it has limited capabilities compared with larger language models and may produce inaccurate responses.

## References and Attribution

- Original Model: https://huggingface.co/HuggingFaceTB/SmolLM2-135M-Instruct
- Conversion Tool: https://github.com/ggml-org/llama.cpp
- Deployment Platform: https://ollama.com

This is an independently converted and republished version of Hugging Face's SmolLM2-135M-Instruct model. It is not an independently trained model or an official Hugging Face release.

Paste, drop or click to upload images (.png, .jpeg, .jpg, .svg, .gif)