CRUXEval-Output-CoT
text
+
+
+
+
About
CRUXEval-Output-CoT enhances the standard CRUXEval output prediction task by incorporating chain-of-thought reasoning methodology. This benchmark evaluates Large Language Models' ability to predict program outputs while showing their reasoning process step-by-step. CRUXEval-Output-CoT tests both code execution understanding and the capacity to articulate logical reasoning in code analysis scenarios.
+
+
+
+
Evaluation Stats
Total Models1
Organizations1
Verified Results0
Self-Reported1
+
+
+
+
Benchmark Details
Max Score1
Language
en
+
+
+
+
Performance Overview
Score distribution and top performers
Score Distribution
1 models
Top Score
56.0%
Average Score
56.0%
High Performers (80%+)
0Top Organizations
#1Alibaba Cloud / Qwen Team
1 model
56.0%
+
+
+
+
Leaderboard
1 models ranked by performance on CRUXEval-Output-CoT
License | Links | ||||
---|---|---|---|---|---|
Sep 19, 2024 | Apache 2.0 | 56.0% |