Contact The Editor
Articles By This Author
Roberto Serrano was shocked when he saw the results of the midterm exam in his advanced mathematical economics class at Brown University last semester: The average score was 96, when in the past, it had ranged from the 60s to the 80s. Nearly half of the students this year got a perfect score of 100.
When he and his teaching assistants ran the exam through a large language model, ChatGPT, it gave an odd, convoluted process for solving one problem rather than a straightforward direct proof. So did numerous students.
He had decided, after the deadly classroom shooting at the school in December, to allow take-home exams for the first time. And he had just learned that those can no longer be trusted to measure student learning, even at an Ivy League school.
"I think it's basically impossible," Serrano said, "to come up with an alternative explanation beyond massive cheating to explain the data."
The results in Serrano's class show that higher education is still struggling with a really basic question: As students get better at using artificial intelligence, how can faculty ensure that students are learning?
When ChatGPT was released in 2022, some educators worried that students would use it to cheat, but others said the large language models (LLMs) couldn't write a compelling essay or solve difficult math problems. Now they can.
And new devices are complicating efforts to test students. There are innocent-looking calculators with cameras and generative AI. ("Snap. Solve. Done." is one device's slogan.) Eyeglasses embedded with cameras and AI can provide answers to tests without professors having any idea it's happening.
"The reality is that students are going to be using AI, whether you know about it or not," said Laurent Lessard, an associate professor of mechanical and industrial engineering at Northeastern University. Lessard, who served on an AI assessment task force - something many universities have - said the technology is moving so fast that professors have to adapt quickly or they won't have any understanding of its capabilities and how students are using it.
He said AI could successfully complete a semester-long college course in about two hours.
In many ways, universities are embracing AI, with researchers using it to supercharge their work in some fields, faculty designing tutoring apps to help students grapple with tough problems, and school officials adding courses and degrees related to AI to prepare students for the fast-changing workforce.
But the problem remains: For students who know they could get a better grade if they used an LLM and know they'll be competing for jobs and admissions against classmates who are using it, the incentive to turn to such a quick and easy tool could be powerful, Lessard said; that's why so many faculty members are trying to rethink how they teach, how they test and what they grade.
Some schools are resurrecting or considering, old-school practices such as blue books and oral exams. Stanford and Princeton universities began allowing and, in Princeton's case requiring, proctored exams this year despite their honor codes.
The University of Chicago's law school is piloting a ban on devices in its first-year core classrooms. And some professors have stopped assigning, or grading, homework; emphasized the learning process over the final product; and added more time for in-person discussions.
A majority of teens, 59 percent, think using AI to cheat is a regular occurrence at their school, according to a February report from the Pew Research Center. Experts say there is widespread use of AI among college students.
At Brown, a committee that has been studying AI use for much of this academic year released a report last week that recommended several changes, including examining academic codes to ensure that academic integrity remains a core standard.
The university is looking into what happened in Serrano's Welfare Economics and Social Choice Theory class.
"Brown treats every allegation of academic integrity with the utmost seriousness," said Brian Clark, a spokesman for the university.
Serrano, a professor of economics, knows that learning can be a struggle: At 17, while growing up in Madrid, he began to go blind. He considered not continuing his education. Instead, he learned to use a cane and read Braille. In college, his father would help him reconstruct the notes he had scrawled in class so that Serrano could transcribe them into Braille. After earning his doctorate from Harvard University, he began teaching at Brown.
Many people face far more difficult circumstances, he said, but "it certainly gave me the clear idea that learning and succeeding doesn't come without effort."
Serrano was surprised when 86 students signed up for his course this year, about triple the typical enrollment. Now he wonders if that was because the syllabus made clear the midterm and final would be take-home exams.
After the midterm, Serrano told his class that it appeared there had been widespread cheating, despite students having signed an academic integrity pledge when taking the test.
The response was silence, he said.
He also told them he was giving them a chance to prove him wrong.
After he switched the final exam to an in-person, three-hour test, 27 students dropped the class, he said. Twenty-two of them had gotten a perfect score on the midterm.
The average score on the final was a 48.6.
Given the results, he told the class he was voiding the midterm. And he asked them, if they chose to use AI on an exam: "Why are you here? Why are you attending a university?"
The happy news, Serrano said, was that the course still had a very strong group of students who showed a good command of the challenging material in the final exam. The highest grade on the final was a 95. That student also got a 95 on the midterm.
After he told his story to El PaÃs, a Spanish newspaper, he heard from hundreds of people, at all kinds of schools, most of them confirming his conviction that concern about AI in the classroom is widespread. There has always been cheating in academia, and new technologies have often brought new challenges.
But AI in its various forms came so suddenly - like a tsunami, he said - it caught educators unprepared to not only prevent cheating, but also to assess whether students are actually learning.
Now they're responding, he said. "With enough people thinking, we should all figure out better solutions."
(COMMENT, BELOW)