Codestral is Mistral AI‘s inaugural code model, built specifically for code generation tasks, featuring a 22-billion parameter architecture.
Proficient in over 80 programming languages.
Codestral has been trained on a diverse dataset encompassing more than 80 languages, such as Python, Java, C, C++, JavaScript, Swift, Fortran, and Bash.
It can complete functions, generate tests, and fill in partial code using a fill-in-the-middle approach.
Run Codestral on Ollama
ollama run codestralPerformance
As a 22B model, Codestral sets a new standard on the performance/latency space for code generation compared to previous models used for coding.
Python
Mistral dev team used four benchmarks: HumanEval pass@1, MBPP sanitised pass@1 to evaluate Codestral’s Python code generation ability, CruxEval to evaluate Python output prediction, and RepoBench EM to evaluate Codestral’s Long-Range Repository-Level Code Completion.
SQL
To evaluate Codestral’s performance in SQL, they used the Spider benchmark.
Additional languages. Additionally, we evaluated Codestral’s performance in multiple HumanEval pass@1 across six different languages in addition to Python: C++, bash, Java, PHP, Typescript, and C#, and calculated the average of these evaluations.