1. Introduction

The Geometry Problem Solving Evaluation Dataset (GeoEval) was constructed by the State Key Laboratory of Multimodal Artificial Intelligence Systems (MAIS), Institute of Automation of Chinese Academy of Sciences (CASIA) and University of Strathclyde. We provide a comprehensive collection that includes a main subset of 2,000 problems, a 750 problems subset focusing on backward reasoning, an augmented subset of 2,000 problems, and a hard subset of 300 problems. This benchmark facilitates a deeper investigation into the performance of LLMs and LMMs in solving geometry math problems.

Download: GeoEval.zip

2. Condition of Use

The application form of the dataset for academic research can be downloaded bellowing:

      Application Form

Reference

If this dataset helps you, please cite the papers below:

   [1] Jia-Xin Zhang, Zhong-Zhi Li, Ming-Liang Zhang, Fei Yin, Cheng-Lin Liu, Yashar Moshfeghi. GeoEval: Benchmark for Evaluating LLMs and Multi-Modal Models on Geometry Problem-Solving, In ACL 2024 Findings.

   [2] Ming-Liang Zhang, Fei Yin and Cheng-Lin Liu. A Multi-Modal Neural Geometric Solver with Textual Clauses Parsed from Diagram, In IJCAI 2023.

   [3] Zhong-Zhi Li, Ming-Liang Zhang, Fei Yin, Cheng-Lin Liu. LANS: A Layout-Aware Neural Solver for Plane Geometry Problem, In ACL 2024 Findings.

Contact

Cheng-Lin Liu (liucl@nlpr.ia.ac.cn), Fei Yin (fyin@nlpr.ia.ac.cn)

State Key Laboratory of Multimodal Artificial Intelligence Systems (MAIS)

Institute of Automation of Chinese Academy of Sciences

95 Zhongguancun East Road, Beijing 100190, P.R. China