AttaQ

AttaQ (Adversarial Question Attack) is a semi-automatically curated dataset featuring adversarial question attack samples designed to evaluate AI systems' robustness against challenging and potentially misleading questions. The benchmark tests models' ability to handle adversarial inputs and maintain reliable performance when faced with questions designed to exploit weaknesses in reasoning or comprehension. AttaQ serves as a critical evaluation tool for assessing AI safety and robustness.