Natural Questions
text
+
+
+
+
About
Natural Questions is a question answering benchmark featuring real questions that people search for on Google, paired with Wikipedia articles containing the answers. It evaluates models' ability to find and extract answers from real-world documents, testing reading comprehension, information retrieval, and natural language understanding in authentic question-answering scenarios.
+
+
+
+
Evaluation Stats
Total Models7
Organizations2
Verified Results0
Self-Reported7
+
+
+
+
Benchmark Details
Max Score1
Language
en
+
+
+
+
Performance Overview
Score distribution and top performers
Score Distribution
7 models
Top Score
34.5%
Average Score
24.0%
High Performers (80%+)
0Top Organizations
#1Mistral AI
1 model
31.2%
#2Google
6 models
22.8%
+
+
+
+
Leaderboard
7 models ranked by performance on Natural Questions
| License | Links | ||||
|---|---|---|---|---|---|
| Jun 27, 2024 | Gemma | 34.5% | |||
| Jul 18, 2024 | Apache 2.0 | 31.2% | |||
| Jun 27, 2024 | Gemma | 29.2% | |||
| May 20, 2025 | Gemma | 20.9% | |||
| Jun 26, 2025 | Proprietary | 20.9% | |||
| May 20, 2025 | Gemma | 15.5% | |||
| Jun 26, 2025 | Proprietary | 15.5% |
