A family of efficient AI models under 10B parameters performant in science, math, and coding through innovative training techniques.
1b
3b
7b
10b
8,216 Pulls Updated 12 days ago
Updated 12 days ago
12 days ago
05a32e830399 · 3.4GB
model
archllama
·
parameters3.23B
·
quantizationQ8_0
3.4GB
params
{
"stop": [
"<|system|>",
"<|user|>",
"<|end|>",
"<|assistant|>"
101B
template
{{- range $i, $_ := .Messages }}
{{- $last := eq (len (slice $.Messages $i)) 1 -}}
<|{{ .Role }}|>
{
218B
license
Falcon 3 TII Falcon License
December 2024
FalconLLM.tii.ae
Introductory note
This license is, in
13kB
Readme
Falcon3 represents TII’s latest advancement in efficient language models under 10B parameters, focused on enhancing science, math, and code capabilities while maintaining training efficiency.
Key Features
- Four sizes: 1B, 3B, 7B, 10B
- Depth up-scaling technique used to create 10B model from 7B
- Knowledge distillation for smaller models (1B, 3B)
Performance Highlights
falcon3:1boutperformssmollm2:1.7b, matchesgemma2:2bfalcon3:10bachieves SOTA in under-13B category- Extended context length up to 32K tokens (8K for 1B model)