Satoshi 7B: The Spirit of Satoshi’s Bitcoin-Focused Language Model

Introduction

Satoshi 7B is a large language model fine-tuned on a Q&A dataset related to Bitcoin principles, technology, culture, Austrian economics, and ‘basedness’ (non-woke political perspectives). Developed by the Spirit of Satoshi team, this conversational model is designed to provide responses aligned with a strong bitcoin maximalist, Austro-libertarian, and ‘non-woke’ bias.

Key Features

• Excels in Bitcoin and Austrian economics topics, particularly in areas such as shitcoinery, self-custody, privacy, and censorship
• Open source and freely available for use and modification under the Apache License 2.0
• Meets or exceeds the performance of larger models like GPT3.5 and GPT4 on most Bitcoin benchmark categories

Model Specifications

• 7 billion parameters
• Fine-tuned on a custom Q&A dataset
• 32k MAX context window (theoretically - practically smaller due to fine-tuning dataset context length)
• Rope-theta = 1e6
• No Sliding-Window Attention

Bias, Risks, and Limitations

• May rely on outdated information and generate plausible but factually incorrect responses
• Intentionally presents a strong bias that may contradict traditionally held viewpoints on bitcoin, economics, and ‘hot-button’ political issues

Evaluation

Satoshi 7B was evaluated using the Bitcoin Maximalism benchmark, an internally developed open-source benchmark. GPT4 was used to assess the model’s responses against expected answers. Despite its modest size, Satoshi 7B performs exceptionally well in Bitcoin vs. Crypto, Adjacent protocols, and ‘basedness’ categories.

Getting Started

To get started with the model, Open terminal and run ollama run shreyshah/satoshi-7b-q4_k_m

Conclusion

Satoshi 7B is a powerful tool for anyone interested in exploring Bitcoin, Austrian economics, and ‘non-woke’ perspectives through a conversational AI model. Its open-source nature and impressive performance make it a valuable resource for the Bitcoin community and beyond.