QMSum
text
+
+
+
+
About
QMSum is a query-based multi-domain meeting summarization benchmark containing 1,808 query-summary pairs over 232 meetings across academic, product, and committee domains. This comprehensive evaluation tests AI models' ability to generate targeted summaries based on specific queries, requiring understanding of meeting dynamics, participant interactions, and selective information extraction from conversational contexts.
+
+
+
+
Evaluation Stats
Total Models2
Organizations1
Verified Results0
Self-Reported2
+
+
+
+
Benchmark Details
Max Score1
Language
en
+
+
+
+
Performance Overview
Score distribution and top performers
Score Distribution
2 models
Top Score
21.3%
Average Score
20.6%
High Performers (80%+)
0Top Organizations
#1Microsoft
2 models
20.6%
+
+
+
+
Leaderboard
2 models ranked by performance on QMSum
| License | Links | ||||
|---|---|---|---|---|---|
| Aug 23, 2024 | MIT | 21.3% | |||
| Aug 23, 2024 | MIT | 19.9% |
