AgentProcessBench: Diagnosing Step-Level Process Quality in Tool-Using Agents

FIRST_AUTHOR_LAST, FIRST_AUTHOR_FIRST; SECOND_AUTHOR_LAST, SECOND_AUTHOR_FIRST

AgentProcessBench Diagnosing Step-Level Process Quality in Tool-Using Agents

Shengda Fan^1,*, Xuyan Ye^1,*, Yupeng Huo¹, Zhi-Yuan Chen¹, Yiju Guo¹, Shenzhi Yang¹, Wenkai Yang¹, Shuqi Ye¹, Jingwen Chen³, Haotian Chen², Xin Cong², Yankai Lin^1,†

¹ Renmin University of China, Beijing, China ² Tsinghua University, Beijing, China ³ Beijing Jiaotong University, Beijing, China ^* Indicates Equal Contribution ^† Corresponding Author

Paper Dataset Code arXiv Document

An overview of AgentProcessBench. First, we sample trajectories from four representative agent benchmarks generated by five source models. Subsequently, human experts annotate the data via a specialized platform, achieving an inter-annotator agreement of 89.1%. Finally, we utilize the constructed benchmark to evaluate 20 distinct models across various families and parameter scales using the StepAcc and FirstErrAcc metrics.

Statistics of AgentProcessBench

Overall Performance on AgentProcessBench

Case Study

BibTeX

@misc{fan2026agentprocessbenchdiagnosingsteplevelprocess,
      title={AgentProcessBench: Diagnosing Step-Level Process Quality in Tool-Using Agents}, 
      author={Shengda Fan and Xuyan Ye and Yupeng Huo and Zhi-Yuan Chen and Yiju Guo and Shenzhi Yang and Wenkai Yang and Shuqi Ye and Jingwen Chen and Haotian Chen and Xin Cong and Yankai Lin},
      year={2026},
      eprint={2603.14465},
      archivePrefix={arXiv},
      primaryClass={cs.AI},
      url={https://arxiv.org/abs/2603.14465}
}