Hushiyar-Lite
1B Parameter Lightweight Instruction-Tuned Language Model
Hushiyar-Lite is a custom-trained 1B parameter large language model optimized for efficient inference, low resource consumption, and general-purpose reasoning. It is designed for developers and researchers who require a compact yet capable model suitable for local, edge, and embedded environments.
📌 Model Summary
| Property |
Details |
| Model Name |
Hushiyar-Lite |
| Parameter Count |
~1B |
| Model Type |
Generative Transformer |
| Architecture |
Gemma-style, custom-tuned |
| Training Objective |
Supervised fine-tuning (SFT) + instruction alignment |
| Context Length |
32768 |
| Primary Use |
Low-compute conversational tasks, reasoning |
🧬 Architecture Overview
Hushiyar-Lite uses a modified decoder-only transformer architecture with:
- Multi-head self-attention
- Rotary positional embeddings (RoPE)
- Optimized attention scaling for small models
- Low-rank feed-forward projections for efficiency
🛠️ Training Pipeline
1. Dataset Composition
Training data included:
- Instruction-tuned conversational datasets
- QA + reasoning corpora
- Lightweight code and logic datasets
- Curated general-purpose text
- Safety-aligned filtered content
Dataset size: 2–6 TB raw text
Text was cleaned using:
- Deduplication
- Normalization
- Token filtering
- Toxicity reduction
- Format standardization
2. Training Configuration
- Precision: fp16
- Hardware used: (8x NVIDIA H200, 1128GB of VRAM)
3. Instruction Tuning
Hushiyar-Lite was aligned for:
- Simple reasoning tasks
- Step-by-step explanation capability
- Safety-enhanced conversational behavior
- Deterministic response patterns suitable for local agents
🚀 Ollama Integration
Install Model
ollama pull aryanvala/hushiyar-lite
Run Model
ollama run aryanvala/hushiyar-lite
Performance Characteristics
Although Hushiyar-Lite is a 1B-parameter model, it was optimized for:
- High throughput on CPU
- Sub-1GB VRAM operation on GPU (quantized)
- Low latency on edge hardware
Below are estimated tokens/sec numbers (inference) — tuned for the fact we trained/ran on 8 × NVIDIA H200 GPUs. Numbers for H200 are extrapolated from published H200 LLM benchmarks and H200 hardware specs; actual throughput depends heavily on quantization (q4_0, q4_K, q8_0, etc.), sequence length, batch size, TensorRT / FlashAttention / kernel choices, and I/O. (NVIDIA)
| Device |
Tokens/sec (approx.) |
| 4-core CPU (typical x86, single process) |
10–20 t/s (small model, quantized; CPU inference remains slow). |
| Low-end GPU (4–6 GB, e.g., L4 / small consumer GPU, quantized) |
30–300 t/s (wide range: 30–60 t/s conservative; up to a few hundred t/s if highly optimized and using 4-bit quantization). |
| Single NVIDIA H200 (high-end GPU, TensorRT/FP8 or optimized kernels) |
~120,000 t/s (order-of-magnitude estimate for a 1B model; derived by scaling published H200 throughput on 13B models to smaller model size). (nvidia.github.io) |
| 8 × NVIDIA H200 (your training/deploy cluster) |
~960,000 t/s (near-linear scaling: ~8 × single-GPU value for large batch / throughput workloads; real scaling will be slightly sublinear due to overheads). (NVIDIA) |
Notes & caveats
- The single H200 figure is an estimate derived from NVIDIA/TensorRT published Llama-13B H200 throughput (≈11,819 t/s on Llama-13B) and the H200 hardware/Memory improvements; because model compute/memory roughly scale with parameter count, a 1B model can be expected to achieve an order of magnitude+ higher tokens/sec on the same hardware when all else (kernel, batch, seq length) is optimal. See NVIDIA H200 specs and TensorRT LLM benchmarks for the reference points. (NVIDIA)
- Quantization matters: q4_0 / q4_K_M will greatly reduce memory use and can increase throughput; combining TensorRT or optimized kernels (FlashAttention, fused kernels) yields the best numbers. Baseten / independent evaluations also note H200 should outperform H100/H100-class results for LLM inference. (Baseten)
- Scaling across 8 GPUs: near-linear for throughput (batching many requests or very large batch sizes). Latency (time-to-first-token) may not improve linearly. Networking/NVLink and serving stack (TensorRT TRTorch, Triton, etc.) affect real-world numbers. (Lenovo Press)
🧠 Capabilities
- Instruction following
- Text generation
- Summaries
- Basic reasoning
- Translation
- Code assistance (lightweight tasks)
- Conversational dialogue
- JSON/structured outputs
⚠️ Limitations
- Not suitable for deep mathematical proofs
- Limited long-context retention (1B constraint)
- May hallucinate under ambiguous prompts
- Not recommended for critical decision systems
🔐 Safety Notes
The training pipeline included:
- Toxicity filtering
- Prompt–response alignment
- Instruction safety constraints
- Removal of harmful content
However, small models may still produce imperfect outputs. Users should apply guardrails depending on deployment.
🧩 Future Work
- Hushiyar-Lite v2 (improved reasoning)
- Hushiyar-Pro 8B model
- Multimodal support (vision + text)
- RAG-augmented version
💬 Support & Feedback
If you’d like to improve this model or collaborate on future releases, feel free to connect or open an issue/discussion.