TempCompass
multimodal
+
+
+
+
About
TempCompass is a comprehensive benchmark evaluating Video LLMs' temporal perception abilities across diverse temporal aspects and task formats including multi-choice QA, captioning, and yes/no questions. This innovative evaluation uses conflicting videos with identical static content but different temporal aspects, revealing significant limitations in current Video LLMs' understanding of temporal dynamics and video comprehension.
+
+
+
+
Evaluation Stats
Total Models2
Organizations1
Verified Results0
Self-Reported2
+
+
+
+
Benchmark Details
Max Score1
Language
en
+
+
+
+
Performance Overview
Score distribution and top performers
Score Distribution
2 models
Top Score
74.8%
Average Score
73.3%
High Performers (80%+)
0Top Organizations
#1Alibaba Cloud / Qwen Team
2 models
73.3%
+
+
+
+
Leaderboard
2 models ranked by performance on TempCompass
| License | Links | ||||
|---|---|---|---|---|---|
| Jan 26, 2025 | tongyi-qianwen | 74.8% | |||
| Jan 26, 2025 | Apache 2.0 | 71.7% |
