Falcon is a new family of language models created by the Technology Innovation Institute in Abu Dhabi and released under the Apache 2.0 license. The Falcon family consists of two base models, Falcon-40B and Falcon-7B, with Falcon-40B being the first «truly open» model with capabilities that rival many closed-source models. Falcon-40B requires ~90GB of GPU memory, while Falcon-7B only needs ~15GB, making inference and fine-tuning accessible even on consumer hardware. The Falcon models have been trained on 1.5 trillion and 1 trillion tokens, respectively, leveraging a novel massive web dataset based on CommonCrawl. The Falcon models also use multi-query attention, which greatly improves the scalability of inference. Hugging Face offers tools to build on top of the Falcon models, such as inference, quantization, fine-tuning, and more.
Mira el artículo original en inglés aquí o también traducido al español aquí
