OxRML · University of Oxfordest. 2023 · Oxford Internet Institute

The science of evaluating AI, and securing it for use where stakes are real.

Reasoning with Machines Lab is an Oxford research lab. We study how language models and agentic systems behave under pressure, and how to deploy them where the stakes are real.

Partner with us Read our research

OxRML-Reason · v1.4Q1 2026 release

GPT-5

71.4

Claude 4.5

68.2

Gemini 2.5 Pro

64.7

Llama 4 405B

52.1

Human baseline

87.3

Multi-step reasoning under adversarial perturbation. 6 domains, 12k tasks, n=3 seeds. CC-BY-4.0.

Benchmarks and Evaluation◇AI Safety and Security◇Agentic AI for Science◇Human–AI Interaction◇Mechanistic interpretability◇Red-teaming◇Capability elicitation◇Benchmark design◇Bias & toxicity◇Domain-grounded agents◇Scientific discovery◇AI governance◇Distributional robustness◇Self-explanation◇Reasoning under pressure◇Benchmarks and Evaluation◇AI Safety and Security◇Agentic AI for Science◇Human–AI Interaction◇Mechanistic interpretability◇Red-teaming◇Capability elicitation◇Benchmark design◇Bias & toxicity◇Domain-grounded agents◇Scientific discovery◇AI governance◇Distributional robustness◇Self-explanation◇Reasoning under pressure◇

01·What we do

Three research lines, asking whether we can trust what these systems do next.

01 / 03

Benchmarks and Evaluation

We work on the science of LLM evaluation: what benchmarks measure, where they mislead, and how to build ones that hold up.

Benchmark design
Statistical evaluation
Capability elicitation
Contamination audits

02 / 03

AI Safety and Security

We work on bias, toxicity, and agentic misalignment, and on the technical and governance tools that address them.

Mechanistic interpretability
Red-teaming
Agentic misalignment
Policy translation

03 / 03

Agentic AI for Science

Agentic systems for scientific work. We focus on keeping them reliable, transparent, and grounded in the domain.

Literature synthesis
Hypothesis generation
Evidence grounding
Domain transfer

02·Selected work

Recent publications

All publications

2026

Strategic Navigation or Stochastic Search? How Agents and Humans Reason Over Document Collections

Ł Borchmann, J Van Landeghem, M Turski et al.

ICML

2026

A Positive Case for Faithfulness: LLM Self-Explanations Help Predict Model Behavior

H Mayne, JS Kang, D Gould et al.

ICML

2025

It's a TRAP! Task-Redirecting Agent Persuasion Benchmark for Web Agents

K Korgul, Y Yang, A Drohomirecki et al.

ICML

2026

Reliability of LLMs as medical assistants for the general public: a randomized preregistered study

AM Bean, RE Payne, G Parsons et al.

Nature Medicine

2025

Measuring what matters: Construct validity in large language model benchmarks

AM Bean, RO Kearns, A Romanou et al.

NeurIPS Datasets and Benchmarks

03·For industry

We help teams ship AI they can defend.

Two ways to work with us: third-party evaluation of your models and agents, or a focused engagement that turns one of our research outputs into a tool you own.

See partnership models Talk to the lab

Evaluation

Pre-deployment audits, custom benchmarks, agentic red-teaming.

Co-build

We work with engineering partners to turn lab work into tools other teams can run.

Sectors

SaaS, public sector, financial services, healthcare.

Engagement

12–24 weeks · NDA-friendly · publishable outcomes negotiable.

04·The lab