MTRAGEval
🎉 Welcome to MTRAGEval! MTRAGEval is a task for Evaluating Multi-Turn RAG Conversations at SemEval 2026. 🎉
Join Our Mailing List!
MTRAGEval Mailing List
Training and Trial Data
The MTRAG Benchmark is released as the trial and training data for MTRAG. You can access the full dataset here
📋 Tasks
- Task A: Retrieval Only
- Task B: Generation with Reference Passages (Reference)
- Task C: Generation with Retrieved Passages (RAG)
Read more about our tasks in our proposal
Evaluation Scripts
Stay Tuned! Coming Soon!
📆 Timeline (Tentative)
- Sample and Training data ready 15 July 2025
- Evaluation start 10 January 2026 Task A and C
- Evaluation end 20 January 2026 Task A and C
- Evaluation start 21 January 2026 Task B
- Evaluation end by 31 January 2026 Task B
- Paper submission due February 2026
- Notification to authors March 2026
- Camera ready due April 2026
- SemEval workshop Summer 2026 (co-located with a major NLP conference)
Task Organizers
Sara Rosenthal ✉️
Yannis Katsis ✉️
Vraj Shah ✉️
Marina Danilevsky ✉️