Gemini 1.5 Pro
Multimodal
Zero-eval
#1XSTest
#1WMT23
#1FunctionalMATH
+8 more
by Google
+
+
+
+
About
Gemini 1.5 Pro was introduced as Google's advanced multimodal model with an expanded context window, designed to comprehend and reason across long documents, videos, and audio. Built to handle complex, information-rich tasks, it brought breakthrough capabilities in processing extended context while maintaining high-quality reasoning and generation.
+
+
+
+
Pricing Range
Input (per 1M)$2.50 -$2.50
Output (per 1M)$10.00 -$10.00
Providers1
+
+
+
+
Timeline
AnnouncedMay 1, 2024
ReleasedMay 1, 2024
Knowledge CutoffNov 1, 2023
+
+
+
+
Specifications
Capabilities
Multimodal
+
+
+
+
License & Family
License
Proprietary
Performance Overview
Performance metrics and category breakdown
Overall Performance
23 benchmarks
Average Score
72.6%
Best Score
98.8%
High Performers (80%+)
10Performance Metrics
Max Context Window
2.1MAvg Throughput
85.0 tok/sAvg Latency
1ms+
+
+
+
All Benchmark Results for Gemini 1.5 Pro
Complete list of benchmark scores with detailed information
| XSTest | text | 0.99 | 98.8% | Self-reported | |
| HellaSwag | text | 0.93 | 93.3% | Self-reported | |
| GSM8k | text | 0.91 | 90.8% | Self-reported | |
| BIG-Bench Hard | text | 0.89 | 89.2% | Self-reported | |
| MGSM | text | 0.88 | 87.5% | Self-reported | |
| MATH | text | 0.86 | 86.5% | Self-reported | |
| MMLU | text | 0.86 | 85.9% | Self-reported | |
| Natural2Code | text | 0.85 | 85.4% | Self-reported | |
| HumanEval | text | 0.84 | 84.1% | Self-reported | |
| MRCR | text | 0.83 | 82.6% | Self-reported |
Showing 1 to 10 of 23 benchmarks
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+