Claude 3.5 Haiku
Zero-eval
by Anthropic
+
+
+
+
About
Claude 3.5 Haiku was developed as the next generation of Anthropic's fastest model, offering similar speed to Claude 3 Haiku while surpassing Claude 3 Opus on many intelligence benchmarks. Built with low latency, improved instruction following, and accurate tool use, it serves user-facing products, specialized sub-agent tasks, and data-heavy applications.
+
+
+
+
Pricing Range
Input (per 1M)$0.80 -$1.00
Output (per 1M)$4.00 -$5.00
Providers3
+
+
+
+
Timeline
AnnouncedOct 22, 2024
ReleasedOct 22, 2024
+
+
+
+
License & Family
License
Proprietary
Performance Overview
Performance metrics and category breakdown
Overall Performance
9 benchmarks
Average Score
60.8%
Best Score
88.1%
High Performers (80%+)
3Performance Metrics
Max Context Window
400.0KAvg Throughput
82.0 tok/sAvg Latency
0ms+
+
+
+
All Benchmark Results for Claude 3.5 Haiku
Complete list of benchmark scores with detailed information
| HumanEval | text | 0.88 | 88.1% | Self-reported | |
| MGSM | text | 0.86 | 85.6% | Self-reported | |
| DROP | text | 0.83 | 83.1% | Self-reported | |
| MATH | text | 0.69 | 69.4% | Self-reported | |
| MMLU-Pro | text | 0.65 | 65.0% | Self-reported | |
| TAU-bench Retail | text | 0.51 | 51.0% | Self-reported | |
| GPQA | text | 0.42 | 41.6% | Self-reported | |
| SWE-Bench Verified | text | 0.41 | 40.6% | Self-reported | |
| TAU-bench Airline | text | 0.23 | 22.8% | Self-reported |
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+