I am interested in a basic question: how do we know that a model has actually reasoned rather than produced a plausible answer? I design and audit evaluation tasks in mathematical and scientific domains, with an emphasis on verifiable solutions, failure localization, benchmark validity, and reproducibility.
Research statement
Mathematics is a useful setting for that question because correctness is not a matter of taste. A proof either establishes its conclusion under the stated assumptions or it does not, and the point at which it stops doing so can be identified. That makes mathematical arguments an unusually precise instrument for studying reasoning failures — and an unusually demanding one, because a benchmark built on them is only as good as the verification behind each item.
Most of my current work follows from a practical observation: aggregate accuracy tells you very little about what a system did. Two models with the same score can fail in entirely different ways, and a correct final answer can rest on an argument that does not hold. I therefore build evaluations around localization and justification — which step fails, what was assumed without support, and what would repair it — rather than around a single scalar.
A second thread concerns the populations that evaluation serves. Benchmarks encode assumptions about language, context, and what counts as a typical problem. I work on African-language and multilingual systems partly because they expose those assumptions, and partly because the tooling gap there is concrete and worth closing.
Research questions
01
Mathematical reasoning and proof auditing
Can a system tell an invalid step from one that is merely compressed?
Studying whether reasoning systems can produce, inspect, and repair mathematically valid arguments rather than merely plausible final answers. The interesting cases are not arithmetic slips but arguments that read well and quietly depend on something never established.
—Distinguishing invalid steps from acceptable compressed reasoning
—Locating the earliest point at which an argument stops being supported
—Reconstructing the premise an argument silently assumed
02
Evaluation validity and benchmark design
What has to be true of a task before a score on it means anything?
Designing original, hand-verified tasks and evaluation procedures that expose model limitations while controlling for ambiguity, contamination, and weak answer checking. Difficulty and validity are separate properties, and a hard benchmark can still measure the wrong thing.
—Item construction that resists contamination and shortcut solutions
—Answer checking that does not reward the right answer for the wrong reason
—Reproducible protocols that make cross-system comparison meaningful
03
Scientific and multimodal reasoning
When a task requires reading a figure and reasoning quantitatively, which part actually fails?
Evaluating how models integrate textual, visual, and quantitative evidence in scientific problem-solving settings, including tasks constructed so that neither the image nor the text alone is sufficient.
—Tasks where visual and symbolic evidence are jointly necessary
—Separating perception failures from reasoning failures
—Reference answers that specify what a correct derivation must contain
04
Multilingual and low-resource AI
What does rigorous evaluation look like for languages that mainstream benchmarks barely cover?
Building and evaluating systems for African languages and contexts that are often poorly represented in mainstream AI benchmarks, where the gap is both in model quality and in the measurement apparatus itself.
—Named entity recognition and sequence labelling for African languages
—Per-language error analysis instead of a single averaged score
—Practical tooling for contexts with limited annotated data
Current research
Work in progress2026 –
PremiseGuardDetecting Unsupported Load-Bearing Claims in Mathematical Proofs
Research question
Can language-model-based mathematical critics reliably detect, localize, explain, and repair unsupported load-bearing claims while distinguishing them from ordinary mathematical errors and acceptable omitted details?
PremiseGuard investigates a failure mode in which a proof silently relies on a nontrivial claim that has not been established under the stated assumptions. The project studies controlled proof variants, earliest-error localization, missing-premise reconstruction, dependency analysis, and proof repair.
The failure mode
An unsupported load-bearing claim is a step the argument genuinely depends on, stated as if it were settled, that does not follow from the assumptions available at that point. It differs from an arithmetic error, which is locally checkable, and from a compressed but recoverable step, which a competent reader could fill in. A critic that flags all three alike is not measuring what it claims to measure — which is why the negative cases matter as much as the positive ones.
Methodological elements
Controlled minimal pairs
Clean proofs and corrupted variants that differ in one identifiable respect, so that a detection signal can be attributed to the corruption rather than to surface differences.
Explicit proof contracts
Stating assumptions, the goal, and the results a proof is permitted to invoke, so that 'unsupported' has a definite meaning at each step.
Load-bearing claim annotation
Marking which claims the argument actually depends on, separating them from remarks that could be deleted without affecting the conclusion.
Premise attribution graphs
Representing each step's dependencies on assumptions and prior steps, which makes the notion of 'load-bearing' checkable rather than impressionistic.
Hard negatives
Valid proofs containing compressed or elided reasoning, included so that a critic is penalised for treating brevity as a defect.
Tasks under study
01Detection. Does the proof contain an unsupported load-bearing claim?
02Localization. Which step is the earliest point at which support is missing?
03Explanation. What is assumed there, and why does it not follow?
04Repair. What premise or sub-argument would make the step legitimate, and does the repaired proof hold?
Research methodology
The same procedure applies whether the material is a benchmark item, a proof variant, or a model comparison.
Original task design
Items written from scratch rather than adapted from circulating sources, which keeps contamination a design decision rather than an unknown.
Independent mathematical verification
Every item is solved and checked by hand before use. An unverified item measures nothing reliable.
Reference-solution construction
A complete solution and explicit criteria describing what a correct response must contain, so scoring does not depend on the grader's mood.
Executable or symbolic checking
Where a claim can be checked computationally, it is — as a guard against errors in the reference material itself.
Controlled model comparison
Fixed prompts, fixed conditions, and recorded settings, so differences between systems are attributable to the systems.
Fine-grained failure taxonomy
Failures are categorised by kind and location rather than collapsed into a single accuracy figure.
Reproducible evaluation
Protocols documented in enough detail that another person could rerun the evaluation and obtain comparable numbers.
Contract research work constructing original evaluation material for frontier language and multimodal systems, aimed at multi-step reasoning that standard benchmark items do not reliably require.
—Wrote and independently hand-verified original olympiad-level mathematics problems spanning geometry, number theory, and combinatorics.
—Designed original scientific tasks requiring reasoning over visual and textual material together.
—Delivered each item complete: problem statement typeset in LaTeX, ideal response, and explicit evaluation criteria.
—Assessed how frontier language and multimodal systems handle difficult multi-step reasoning.
Problem design
Independent verification
Ideal-response specification
Ambiguity control
Failure analysis
This material was produced for Turing and remains theirs. Problem statements, ideal responses, evaluation criteria, model outputs, and results are confidential and are not reproduced here.