RepoQA

RepoQA is the first benchmark specifically designed for long-context code understanding, featuring 500 code search tasks from 50 repositories across 5 programming languages. Using the Searching Needle Function (SNF) task, this multilingual evaluation tests AI models' ability to retrieve specific functions based on natural-language descriptions within extensive codebases, challenging long-context comprehension capabilities.