Physics department concludes AI grading experiment, shares mixed results

Last week, Physics professor Lyman Page released the results of his AI grading experiment, conducted in Spring 2026, last week. After physics faculty members graded all 229 final exams for PHY 104: General Physics II by hand, Page used Claude AI to score the tests and compared the two grades to evaluate Claude’s performance.

Page told The Daily Princetonian that the results demonstrate that faculty-trained AI can be a useful tool for improving efficiency and providing more feedback, but it will never completely replace human grading. Given the mixed results, Page said the department remains unsure whether digitized grading tools will be implemented for PHY 104 students this year.

Page said he was first inspired to conduct this experiment after using Gemini to digitize his handwritten notes for new faculty members. 

“It was sort of mind-blowing how well it did,” Page told the ‘Prince.’ The platform was able to transcribe formulas in TeX, which Page described as a “publication-quality” computer language used for writing mathematical equations. He said the accuracy of Gemini’s digitization sparked his curiosity about using it for transcribing exams and, eventually, giving students grades and feedback on test problems.

Gemini Pro was the platform Page initially intended to use for last semester’s experiment, but he ultimately chose to use Claude instead, citing convenience and security reasons. 

Course grades were based solely on faculty grading of all the final exams, as AI scores were determined after letter grades were reported to the registrar. Although all 229 exams were graded by professors, students received the option to opt out of the experiment, and Page said he did not look at any names or connect students to certain grades during the trial. 

Claude’s grading metrics were then set by an adjustable rubric: Page could ask it to set an average for the question, to not take off points for propagated errors, to subtract extra points for incorrect units, or to give nearly full points for conceptually close answers. 

For Page, the most significant benefit of introducing faculty-trained AI was its ability to quickly provide detailed feedback to students. He said that because it takes around 12 hours for one professor to grade an entire class’s exams, it is unrealistic to expect faculty members to offer extensive explanations for every mistake. Claude, on the other hand, can generate paragraphs of critiques in a matter of seconds. 

However, he found that AI still made mistakes. With hand-written student work, Claude can misinterpret handwriting or misplace where students mark their final answers. For those reasons, Page told the ‘Prince’ he would never use AI — to any degree — to grade short quizzes. 

“If you really look at it and you spend an hour grading quizzes, you really learn a lot about the students and just the way they’re thinking, so I can’t imagine replacing that,” Page said.

For the majority of cases, Page said that Claude “did a pretty good job.” He said it was “impressive that you could tell it things like ‘don’t take off for a propagated error.’”

Still, only for major exams or large-scale homework assignments would partial AI use even be considered, Page said. If the physics department were to actually implement this, professors would need to be “holding its hand” through the process, closely monitoring Claude’s output, he added.

Before grading, Claude was given the exam to take for itself, which it was able to do with no issue. However, when grading scanned copies of the exam, Claude had difficulty reading some answers, and its scores differed from faculty’s in around 10 percent of cases. 

“Everyone thinks slightly differently, but you want to learn from [another] person,” Page said. “We’re not just there to have a job; part of it is working with [students] so … I don’t see that changing. That would be a huge mission change and not a good one.” 

Page said he does not foresee a future where Princeton would completely rely on digital tools for scoring assessments. 

“Using AI to provide targeted exam feedback would be very helpful for students like me, especially when I’m frustrated to see why I get points deducted on questions,” Sean Egan ’29, a PHY 104 alum, wrote in a statement to the ‘Prince.’ “However, I am concerned about the effect that faculty use of AI would have on student usage. Most of what we hear from faculty on student AI usage is highly discouraging, and if professors were using AI to augment their core job responsibilities, it would be sending mixed messages to students.”

Tova Jean-Louis ’29, another alum of the course, expressed concern about how difficult it is to separate AI use with environmental issues like energy and water consumption, as well as the construction of data centers. 

“Dr. Lyman’s report admits that it can be difficult to pin down the precise energy use of this project and I cannot help but be concerned about the sustainability of AI grading going forward,” Jean-Louis said.

Physics professor and department chair James D. Olsen found Page’s findings to be “illuminating,” and views AI as a tool to increase human interaction rather than reduce professors’ roles in the learning process. 

“Instructors provide what is uniquely human, the shared experience of learning the same material when we were students  — the struggles, frustrations, and brilliant light-bulb moments not only when the answer is known, but when the principle is understood,” Olsen said. Because of how time-consuming grading by hand is, AI could potentially allow instructors to redirect their hours toward enhanced student interaction instead, he said.

“Conveying the genuine excitement of active learning and the joy of deep understanding to our students is where the humanity lives and breathes in a science course,” Olsen said. “Humans will always be an essential part of that process.”

Claire Ding is a News contributor for the ‘Prince’ from Cherry Hill, N.J.

Please send any corrections to corrections[at]dailyprincetonian.com.