Forge: A Meta Llama 3.1 Based Model

Overview

This model, Forge, is a powerful language model based on the well-established Meta Llama 3.1 architecture.

Key Features

• Architecture: Based on the Meta Llama 3.1 architecture
• Base Model: Meta Llama 3.1 8B Instruct
• File Type: F16
• Quantization Version: 2
• Attention Head Count: 32
• Context Length: 131072
• Embedding Length: 4096
• Feed Forward Length: 14336
• RoPE Dimension Count: 128
• Vocabulary Size: 128256