Arena Hard

Arena Hard is an automatic evaluation benchmark for instruction-tuned Large Language Models that achieves 98.6% correlation with human preferences from Chatbot Arena. Built from live crowdsourced data, it provides 3x higher model performance separation compared to MT-Bench through pairwise comparisons using strong judge models. The benchmark uses the Bradley-Terry model for scoring and offers cost-effective, fast evaluation with frequent updates from real-world user interactions.