SummScreenFD

text
+
+
+
+
About

SummScreenFD is an abstractive summarization benchmark featuring TV series transcripts paired with human-written episode recaps, challenging models to integrate plot details from dialogues and character interactions. This unique evaluation tests AI models' ability to understand narrative structures, identify key story elements, and generate coherent summaries that capture essential plot developments from screenplay dialogues.

+
+
+
+
Evaluation Stats
Total Models2
Organizations1
Verified Results0
Self-Reported2
+
+
+
+
Benchmark Details
Max Score1
Language
en
+
+
+
+
Performance Overview
Score distribution and top performers

Score Distribution

2 models
Top Score
16.9%
Average Score
16.4%
High Performers (80%+)
0

Top Organizations

#1Microsoft
2 models
16.4%
+
+
+
+
Leaderboard
2 models ranked by performance on SummScreenFD
LicenseLinks
Aug 23, 2024
MIT
16.9%
Aug 23, 2024
MIT
16.0%
+
+
+
+
Resources