A family of efficient AI models under 10B parameters performant in science, math, and coding through innovative training techniques.
1b
3b
7b
10b
3,947 Pulls Updated 3 days ago
Updated 3 days ago
3 days ago
770fc0679654 · 15GB
model
archllama
·
parameters7.46B
·
quantizationF16
15GB
params
{
"stop": [
"<|system|>",
"<|user|>",
"<|end|>",
"<|assistant|>"
101B
template
{{- range $i, $_ := .Messages }}
{{- $last := eq (len (slice $.Messages $i)) 1 -}}
<|{{ .Role }}|>
{
218B
license
Falcon 3 TII Falcon License
December 2024
FalconLLM.tii.ae
Introductory note
This license is, in
13kB
Readme
Falcon3 represents TII’s latest advancement in efficient language models under 10B parameters, focused on enhancing science, math, and code capabilities while maintaining training efficiency.
Key Features
- Four sizes: 1B, 3B, 7B, 10B
- Depth up-scaling technique used to create 10B model from 7B
- Knowledge distillation for smaller models (1B, 3B)
Performance Highlights
falcon3:1boutperformssmollm2:1.7b, matchesgemma2:2bfalcon3:10bachieves SOTA in under-13B category- Extended context length up to 32K tokens (8K for 1B model)