Opentensor and Cerebras announce BTLM-3B-8K, a leading 3B param. language model

https://huggingface.co/cerebras/btlm-3b-8k-base

by cs-fan-101 • 3 years ago

9 2 3 years ago

cs-fan-101• 3 years ago

Cerebras and Opentensor are pleased to announce BTLM-3B-8K (Bittensor Language Model), a new state-of-the-art 3 billion parameter open-source language model that achieves breakthrough accuracy across a dozen AI benchmarks.

BTLM-3B-8K Highlights: • 7B level model performance in a 3B model • State-of-the-art 3B parameter model • Optimized for long sequence length inference 8K or more • First model trained on the SlimPajama, the largest fully deduplicated open dataset • Runs on devices with as little as 3GB of memory when quantized to 4-bit • Apache 2.0 license for commercial use.

BTLM fits on mobile and edge devices with as little as 3GB of memory, helping democratize AI access to billions of devices worldwide.

BTLM was commissioned by the Opentensor foundation for use on the Bittensor network. Bittensor is a blockchain based network that lets anyone contribute AI models for inference, providing a decentralized alternative to centralized model providers like OpenAI and Google. Bittensor serves over 4,000 AI models with over 10 trillion model parameters across the network.

BTLM was trained on the newly unveiled Condor Galaxy 1 (CG-1) supercomputer, the first public deliverable of the G42 Cerebras strategic partnership. Cerebras acknowledges the generous support of G42 Cloud and the Inception Institute of Artificial Intelligence. Cerebras also thanks Cirrascale, who first introduced OpenTensor to Cerebras and provided additional technical support. Finally, Cerebras thanks the Together AI team for the RedPajama dataset.

To learn more, check out the following: - Blog: https://hubs.li/Q01YG_yr0 - Model on Hugging Face: https://hubs.li/Q01YG_HS0

jwan584• 3 years ago

Meta announced a partnership with Qualcomm to bring LLMs to mobile. But 3B is a lot more compact than LLaMA's 7B.

Opentensor and Cerebras announce BTLM-3B-8K, a leading 3B param. language model

Related Stories