One of Europe’s leading artificial intelligence companies, Mistral AI, has unveiled a new model called Mistral Small 3. It is a 24 billion parameter model, but is on par with larger models such as Llama 3.3 70B and Qwen 32B (at least in the MMLU-Pro benchmark). Not only does it operate on par with Llama 3.3 70B, but it’s also quicker.
The most commonly used model on ChatGPT is the GPT-4o mini, the fallback model when users run out of GPT-4o requests. Mistral Small 3 has better performance than this OpenAI model and is also said to experience lower latency.
“We’re releasing both a pretrained and instruction-tuned checkpoint under Apache 2.0,” Mistral AI said about the model’s license. “The checkpoints can serve as a powerful base for accelerating progress. Note that Mistral Small 3 is neither trained with RL nor synthetic data, so is earlier in the model production pipeline than models like Deepseek R1 (a great and complementary piece of open-source technology!). It can serve as a great base model for building accrued reasoning capacities. We look forward to seeing how the open-source community adopts and customizes it.”
As a model on the smaller side, it’s possible to run it locally on your own computer, if you have higher computer specs. Mistral AI said that it can be run on a single Nvidia RTX 4090 graphics card or a MacBook with 32 GB of RAM.
While the model did better against the other mentioned models on the MMLU-Pro benchmark, it wasn’t always the favorite choice of human evaluators. Mistral compared its model with other models on a set of over 1k proprietary coding and generalist prompts. It found that Mistral Small 3 was the preferred option compared to Gemma-2 27B and Qwen-32B, but was less preferred compared to Llama 3.3 70B and GPT-4o mini.
Mistral Small 3 is now available on la Plateforme as mistral-small-latest or mistral-small-2501.