A set of Mixture of Experts (MoE) model with open weights by Mistral AI in 8x7b and 8x22b parameter sizes. The Mixtral large language models (LLMs) are a set of pretrained generative Sparse Mixture of Experts (SMoE).
Sizes:
- mixtral:8x22b
- mixtral:8x7b
Mixtral 8x22B:
To run: ollama run mixtral:8x22b
Mixtral 8x22B sets a new benchmark for performance and efficiency in the AI community. As a sparse Mixture-of-Experts model, it activates only 39 billion parameters out of a total 141 billion, making it highly cost-efficient for its size.
Key Features:
- Proficiency in multiple languages: English, French, Italian, German, and Spanish
- Strong mathematical and coding abilities
- Native support for function calling
- 64K token context window, enabling accurate recall of information from large documents