ReTraceQA: Evaluating Reasoning Traces of Small Language Models in Commonsense Question Answering

Official repository for the ACL 2026 (main conference) paper "ReTraceQA: Evaluating Reasoning Traces of Small Language Models in Commonsense Question Answering" by Francesco Maria Molfese, Luca Moroni, Ciro Porcaro, Simone Conia, Roberto Navigli. This project provides tools and resources for evaluating the reasoning traces generated by small language models in commonsense question-answering tasks.

---

Table of Contents

• Installation
• Dataset
• Quick Start
• Reproducing Results
  • 1. Generate Reasoning Traces
  • 2. Evaluate with Process Reward Models
  • 3. Evaluate with Critic Models
  • 4. Evaluate with LLM-as-a-Judge
• Main Functionalities
  • Inference
  • Evaluation
  • Benchmarking
• Project Structure
• Citation
• Acknowledgments
• License

---

Installation

1. Clone the repository

git clone https://github.com/framolfese/ReTraceQA.git
cd ReTraceQA

2. Install dependencies

pip install -e .

Requirements:

• Python >= 3.10
• CUDA-capable GPU (recommended for inference and training)
• See requirements.txt for complete dependency list

---

Dataset

The ReTraceQA benchmark is available on Hugging Face:

🤗 sapienzanlp/ReTraceQA

The benchmark includes:

• Reasoning traces from multiple small language models
• Human annotations for step error locations and categorizations
• Benchmark results across CommonsenseQA, OBQA, QASC, and StrategyQA datasets

---

Quick Start

Generate Reasoning Traces

Run inference for a given model on a commonsense question answering dataset as follows:

python src/inference/inference.py \
    --model_name "Qwen/Qwen2.5-1.5B-Instruct" \
    --config_path config/inference/commonsenseqa_test.yaml \
    --device cuda:0

Evaluate with Process Reward Models (PRMs)

Evaluate reasoning traces using different PRMs:

# Math-Shepherd PRM
python src/inference/math_shepherd_prm_inference.py \
    --model_name "peiyi9979/math-shepherd-mistral-7b-prm" \
    --config_path config/inference/prm_benchmark.yaml

# Skywork PRM
python src/inference/skywork_prm_inference.py \
    --model_name "Skywork/Skywork-o1-Open-PRM-Qwen-2.5-1.5B" \
    --config_path config/inference/prm_benchmark.yaml

# Qwen PRM
python src/inference/qwen_prm_inference.py \
    --model_name "Qwen/Qwen2.5-Math-RM-72B" \
    --config_path config/inference/prm_benchmark.yaml

ReTraceQA outputs used in the paper

ReTraceQA items aligned with PRM, along with outputs from LLM-as-a-Judge models:

• DRIVE LINK

---

Reproducing Results

1. Generate Reasoning Traces

Generate traces for all datasets using the provided script:

bash scripts/inference.sh <model_name> <device>

Example:

bash scripts/inference.sh "Qwen/Qwen2.5-1.5B-Instruct" cuda:0

This will run inference on:

• CommonsenseQA (train & test)
• QASC (train & test)
• OpenBookQA (train & test)
• StrategyQA (train & test)

2. Evaluate with Process Reward Models

Evaluate reasoning quality using PRMs:

# Using Math-Shepherd
python src/inference/math_shepherd_prm_inference.py \
    --model_name "peiyi9979/math-shepherd-mistral-7b-prm" \
    --config_path config/inference/prm_benchmark.yaml

# Using GenPRM
python src/inference/gen_prm_inference.py \
    --model_name "GAIR/GenRM-Llama3.1-8B" \
    --config_path config/inference/prm_benchmark.yaml

3. Evaluate with Critic Models

Use large language models as critics to evaluate reasoning:

# Local critic model
python src/inference/critic_model.py \
    --model_name "Qwen/Qwen2.5-72B-Instruct" \
    --config_path config/inference/critic_model.yaml

# Or use the script for SLURM clusters
bash scripts/critic_model.sh

For API-based critics (OpenAI, DeepSeek, Gemini):

# OpenAI
python api/openai/critic_model.py --config_path config/api/openai.yaml

# DeepSeek
python api/deepseek/critic_model.py --config_path config/api/deepseek.yaml

# Gemini
python api/gemini/critic_model.py --config_path config/api/gemini.yaml

4. Evaluate with LLM-as-a-Judge

First, prepare inputs for the judge:

python src/evaluation/prepare_input_for_judge.py \
    --input_path data/reasoning_traces.jsonl \
    --output_path data/judge_input.jsonl

Then run the judge evaluation:

# Local LLM judge
python src/inference/llm_as_a_judge.py \
    --model_name "Qwen/Qwen2.5-72B-Instruct" \
    --config_path config/inference/llm_as_a_judge.yaml

# OpenAI judge
python api/openai/llm_as_a_judge.py --config_path config/api/openai.yaml

Evaluate the judgments:

python src/evaluation/llm_as_a_judge_eval.py \
    --predictions_path evaluation/judge_predictions.jsonl \
    --config_path config/eval/judge_eval.yaml

---

Main Functionalities

Inference

The inference module supports multiple model types and evaluation methods:

• Standard Inference: Generate reasoning traces from LLMs

  python src/inference/inference.py --model_name <model> --config_path <config>

• PRM Inference: Evaluate traces with Process Reward Models

  • Math-Shepherd: src/inference/math_shepherd_prm_inference.py
  • Skywork PRM: src/inference/skywork_prm_inference.py
  • Qwen PRM: src/inference/qwen_prm_inference.py
  • GenPRM: src/inference/gen_prm_inference.py

• Critic Models: Use LLMs to critique reasoning steps

  python src/inference/critic_model.py --model_name <model> --config_path <config>

• LLM-as-a-Judge: Binary judgment of reasoning correctness

  python src/inference/llm_as_a_judge.py --model_name <model> --config_path <config>

Evaluation

Evaluation tools for different metrics and methods:

• xFinder Evaluation: Extract and evaluate final answers

  python src/evaluation/xfinder_eval.py --input_path <path> --config_path <config>

• Majority Voting: Aggregate predictions via majority voting

  bash scripts/majority_voting.sh

• LLM Judge Evaluation: Evaluate judge predictions

  python src/evaluation/llm_as_a_judge_eval.py --predictions_path <path> --config_path <config>

• Human Agreement Analysis: Compute agreement with human annotations

  python src/agreement/human_agreement.py --predictions_path <path> --ground_truth_path <path>

---

Project Structure

ReTraceQA/
├── api/                        # API integrations (OpenAI, DeepSeek, Gemini)
│   ├── openai/
│   ├── deepseek/
│   └── gemini/
├── config/                     # Configuration files
│   ├── inference/              # Inference configs for different datasets
│   ├── eval/                   # Evaluation configs
│   └── api/                    # API configs
├── data/                       # Dataset storage (not included in repo)
├── src/                        # Source code
│   ├── inference/              # Inference scripts (PRM, critic, judge)
│   ├── evaluation/             # Evaluation scripts
│   ├── models/                 # Custom model implementations
│   ├── data/                   # Data loaders
│   ├── benchmark/              # Benchmarking utilities
│   ├── model_utils/            # Model utilities and helpers
│   └── agreement/              # Agreement analysis tools
├── scripts/                    # Bash scripts for running experiments
├── plots/                      # Plotting and visualization scripts
├── requirements.txt            # Python dependencies
└── setup.py                    # Package setup

---

Citation

If you use ReTraceQA in your research, please cite our paper:

@misc{molfese2025retraceqaevaluatingreasoningtraces,
      title={ReTraceQA: Evaluating Reasoning Traces of Small Language Models in Commonsense Question Answering}, 
      author={Francesco Maria Molfese and Luca Moroni and Ciro Porcaro and Simone Conia and Roberto Navigli},
      year={2025},
      eprint={2510.09351},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2510.09351}, 
}

---

Acknowledgments

We gratefully acknowledge the support of Future AI Research (PNRR MUR project PE0000013-FAIR).

---

License

This project is licensed under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License (CC BY-NC-SA 4.0).

See the LICENSE file for details.