← Back to Research
Learning to Judge: AI-Supported Calibration Practice for Teaching Expert Reasoning

Learning to Judge: AI-Supported Calibration Practice for Teaching Expert Reasoning

AI in Education Research

Learners are taught expert-reasoning frameworks once but rarely get to practice them with feedback on whether their judgment is actually any good — and people are systematically miscalibrated, unable to tell their sound judgments from their errors. Expert feedback is one of the strongest levers on learning yet doesn't scale to a whole course, and leaning on generative AI passively can make it worse by letting learners offload the thinking. This research asks whether a grounded AI system can be turned into a trainer for judgment rather than a crutch — one that measurably sharpens a learner's own reasoning — and whether that holds across very different fields.

CalibrationMetacognitionFeedbackAI in EducationClinical ReasoningAssessmentHuman-in-the-LoopTransferable Method

The crisis

  • Expert reasoning is taught as a framework once, then rarely practiced with feedback on whether the learner's judgment is actually improving.
  • Learners are systematically miscalibrated — weaker performers over-estimate themselves and cannot distinguish their correct judgments from their errors (the Dunning-Kruger pattern).
  • Expert-quality feedback is among the strongest levers on learning, but it does not scale to every learner in a course.
  • Used passively, generative AI induces metacognitive offloading — learners stop judging for themselves — so a naive AI tutor can erode the very skill it is meant to build.

About this research

This thread takes on a familiar failure in professional education: expert reasoning is taught as a framework once, then rarely practiced with the kind of feedback that tells a learner whether their judgment is improving — and used carelessly, generative AI makes it worse by inviting learners to offload the very thinking they are meant to build. It investigates whether a grounded, cited AI system can instead be used to strengthen a learner's own judgment, and to measure honestly whether that judgment gets better over time — improvement in the learner, not satisfaction with the tool or time saved for the instructor. It is studied in high-stakes reasoning settings such as clinical evidence appraisal and exam answer-writing, chosen because getting the judgment wrong there carries real cost, and it is designed to test whether the same approach transfers across such different fields. It builds on the lab's deployed clinical-evidence work and its expert-validation and evaluation practices, and draws on retrieval-grounded generation, human-in-the-loop design, and rigorous learning-science evaluation. Faculty-advised.