Reasoning with Machines Lab
@ University of Oxford

Led by Prof. Adam Mahdi, the Reasoning with Machines Lab advances the science of AI evaluation, benchmarking, safety and security. Through rigorous empirical research, we study how LLMs and agentic systems reason, interact with humans, and drive scientific discovery. We work with industry partners deploying AI where reliability matters.

Read our research Engage with OxRML

Papers: 10; open access
Themes: 4; open access
Team: 15; + lead

Evaluation Safety Agentic Human-AI

Engage with OxRML

For individuals

Open

Read, cite, and build on anything we publish.

10 peer-reviewed publications, open access
All 4 research themes
Benchmarks, datasets, prompts, and eval rigs — fully documented
Lab reading list (quarterly digest)

Start reading

For teams

Workshop

On-site sessions for product and ML teams on evaluation, safety, and agent reliability.

Half-day to multi-week formats
Designed for product & ML teams shipping LLMs

Book a workshop

For enterprises

Partner

Applied research collaborations with foundations, governments, and large companies.

Tools co-built with engineering partners
Foundations, governments & global corporates

Discuss a partnership

Research Themes

The questions the lab is built around.

01 / 04
Benchmarks and Evaluation
We develop the science of LLM evaluation, setting the standard for rigorous assessment and identifying hidden risks before they matter.
02 / 04
AI Safety and Security
From bias and toxicity to agentic misalignment, we study the full spectrum of AI risk and develop the technical and governance tools to address it.
03 / 04
Agentic AI for Science
We build agentic systems that automate scientific knowledge synthesis and discovery, with a focus on agents that are reliable, transparent and domain-grounded.
04 / 04
Human–AI Interaction
We run large-scale empirical studies on how people use AI for high stakes decisions, from healthcare and law to policy and beyond.

Research

ICML spotlights, Nature Medicine, NeurIPS Datasets & Benchmarks, ICLR, EMNLP. Click through and you're on arXiv or OpenReview, not a request form.

Search all research

all links resolve direct to the venue

01
01ICML (Spotlight)May 2026
Strategic Navigation or Stochastic Search? How Agents and Humans Reason Over Document Collections
A benchmark that tells real navigation apart from stochastic search when agents work over document collections.
Ł Borchmann, J Van Landeghem, M Turski, S Padarha, RO Kearns, A Mahdi, et al.
ICML (Spotlight)May 2026
Benchmarks and EvaluationAgentic AI
02
02ICMLMay 2026
A Positive Case for Faithfulness: LLM Self-Explanations Help Predict Model Behavior
LLM self-explanations are usually dismissed as unreliable. Measured the right way, they predict model behavior.
H Mayne, JS Kang, D Gould, K Ramchandran, A Mahdi, NY Siegel
ICMLMay 2026
AI Safety and AlignmentBenchmarks and Evaluation
03
03ICLRApril 2026
LingOly-TOO: Disentangling Reasoning from Knowledge with Templatised Orthographic Obfuscation
A benchmark that obfuscates orthography to strip memorised knowledge out of reasoning problems, showing how much "reasoning" was recall.
J Khouja, K Korgul, S Hellsten, L Yang, V Neacsu, H Mayne, RO Kearns, A Bean, A Mahdi
ICLRApril 2026
Benchmarks and Evaluation

Team

15 researchers + 1 PI

Lab Lead · Associate Professor

Prof. Adam Mahdi

Oxford Internet Institute, University of Oxford

Adam leads OxRML. The group studies how language models reason, how people work with them, and how agentic systems behave on real scientific and decision-making tasks.