Qwen2.5-Coder 32B Instruct
Zero-eval
#1BigCodeBench-Full
#1BigCodeBench-Hard
#2MBPP
+1 more
by Alibaba Cloud / Qwen Team
+
+
+
+
About
Qwen2.5-Coder 32B Instruct is a language model developed by Alibaba Cloud / Qwen Team. It achieves strong performance with an average score of 64.9% across 15 benchmarks. It excels particularly in HumanEval (92.7%), GSM8k (91.1%), MBPP (90.2%). It supports a 256K token context window for handling large documents. The model is available through 4 API providers. It's licensed for commercial use, making it suitable for enterprise applications. Released in 2024, it represents Alibaba Cloud / Qwen Team's latest advancement in AI technology.
+
+
+
+
Pricing Range
Input (per 1M)$0.09 -$0.89
Output (per 1M)$0.09 -$0.89
Providers4
+
+
+
+
Timeline
AnnouncedSep 19, 2024
ReleasedSep 19, 2024
+
+
+
+
Specifications
Training Tokens5.5T
+
+
+
+
License & Family
License
Apache 2.0
Base ModelQwen2.5 32B Instruct
Performance Overview
Performance metrics and category breakdown
Overall Performance
15 benchmarks
Average Score
64.9%
Best Score
92.7%
High Performers (80%+)
5Performance Metrics
Max Context Window
256.0KAvg Throughput
74.0 tok/sAvg Latency
0ms+
+
+
+
All Benchmark Results for Qwen2.5-Coder 32B Instruct
Complete list of benchmark scores with detailed information
| HumanEval | text | 0.93 | 92.7% | Self-reported | |
| GSM8k | text | 0.91 | 91.1% | Self-reported | |
| MBPP | text | 0.90 | 90.2% | Self-reported | |
| HellaSwag | text | 0.83 | 83.0% | Self-reported | |
| Winogrande | text | 0.81 | 80.8% | Self-reported | |
| MMLU-Redux | text | 0.78 | 77.5% | Self-reported | |
| MMLU | text | 0.75 | 75.1% | Self-reported | |
| ARC-C | text | 0.70 | 70.5% | Self-reported | |
| MATH | text | 0.57 | 57.2% | Self-reported | |
| TruthfulQA | text | 0.54 | 54.2% | Self-reported |
Showing 1 to 10 of 15 benchmarks