Physics department to run grading experiment using AI

Princeton’s physics department is testing whether artificial intelligence (AI) can grade exams,  comparing faculty-graded exams for PHY 104: General Physics II with those generated by Gemini Pro.

Grades given by AI will not impact official student scores, and students can choose to opt-out of the experiment. The department will be using the enterprise version of Gemini Pro, which “offers enterprise-grade security and data privacy” according to Google Cloud, to ensure that student data is protected. 

In an interview with The Daily Princetonian, course head and Professor of Physics Lyman Page shared details about the experiment, including the decision to use the enterprise version in order to protect the student’s data. 

Google claims that it does not share content inputted into its generative AI software with outside organizations without permission.

“The main goal was to make sure this doesn’t get outside of anything at Princeton, so we’re using the enterprise version of Gemini Pro,” Page said.

Page has been working with Canvas professionals to ensure that “everything is above board and transparent” and that “students aren’t getting a short end of the stick.” 

Regarding what the experiment results could mean for future grading, he explained that it could make the process more efficient. 

“There are about 230 students, and one faculty member grades all of the problems for 230 students over a series of two days,” Page said. “Let’s say it does fantastically, it would probably reduce that time.” 

However, he expressed his belief that, even if the experiment goes well, it will not replace faculty graders. “I don’t imagine it would ever replace the faculty, and I don’t think it would ever not be the faculty,” Page said. “‘It’s an experiment,’ I think is the way to view it, and we’ll just see how it goes.” 

Multiple students in PHY 104 wrote to the ‘Prince’ with mixed reactions about the AI grading. 

“I decided to opt in to the experiment, because all final exams are still going to be graded in person first, so I don’t mind if, afterwards, they use my final to test the accuracy of Gemini,” Sean Egan ’29 wrote. 

“I would definitely have opted out if there were not a qualified human grading my work first,” he added. 

Tova Jean-Louis ’29 agreed that they would not be comfortable with AI fully grading exams in the future, especially noting the importance of originality across the board. 

“Under the Honor Code, we, as students, are expected to produce original work, and, especially in a course like PHY 104, we cannot use any outside resources on exams; accordingly, I would like to think that our work should be treated with the respect it deserves,” they wrote. 

Jean-Louis also noted that “AI can ‘hallucinate’ information and reproduce biases in ways that cannot be easily mitigated.” 

Students also shared the importance of considering a student’s work beyond whether the answer is mathematically correct and concerns whether AI would be able to do this. 

“AI may just detect the right or wrong answer, but professors and TAs try to understand the student’s logic even if there is a calculation error,” Grace Rivas ’29 wrote. “AI cannot interpret student work the same way professors and TAs can, and the needs of the students should be prioritized.”

Page will be working alongside the physics department, Deputy Dean of the College Rebekah Peeples GS ’09, Director of the McGraw Center for Teaching and Learning and Senior Associate Dean of the College Katherine Stanton, and the Office of Information Technology to ensure that the PHY 104 grading process complies with University policies and maintains student confidentiality. 

These administrators and offices were cited in a message sent to PHY 104 students as people whom the course staff were coordinating with.

Peeples, Stanton, and OIT did not respond to multiple requests for comment.

“I can see this being a part of future grading easily … At least for us, where it’s not graduate students doing the grading, it releases faculty time for more help sessions and more one-on-ones,” Page said. “I think if it works, at some point, it will change how grading is done, but it won’t be sudden.”  

Giselle Moreno is a staff News writer who typically covers Town and U. Affairs. She is from Dallas and can be reached at gm2076[at]princeton.edu. 

Please send any corrections to corrections[at]dailyprincetonian.com.