A family of efficient AI models under 10B parameters performant in science, math, and coding through innovative training techniques.
1b
3b
7b
10b
8,214 Pulls Updated 12 days ago
Updated 12 days ago
12 days ago
5449d006a9c7 · 1.8GB
model
archllama
·
parameters1.67B
·
quantizationQ8_0
1.8GB
params
{
"stop": [
"<|system|>",
"<|user|>",
"<|end|>",
"<|assistant|>"
101B
template
{{- range $i, $_ := .Messages }}
{{- $last := eq (len (slice $.Messages $i)) 1 -}}
<|{{ .Role }}|>
{
218B
license
Falcon 3 TII Falcon License
December 2024
FalconLLM.tii.ae
Introductory note
This license is, in
13kB
Readme
Falcon3 represents TII’s latest advancement in efficient language models under 10B parameters, focused on enhancing science, math, and code capabilities while maintaining training efficiency.
Key Features
- Four sizes: 1B, 3B, 7B, 10B
- Depth up-scaling technique used to create 10B model from 7B
- Knowledge distillation for smaller models (1B, 3B)
Performance Highlights
falcon3:1boutperformssmollm2:1.7b, matchesgemma2:2bfalcon3:10bachieves SOTA in under-13B category- Extended context length up to 32K tokens (8K for 1B model)