Cambridge, Massachusetts, US — The most interesting question raised by Harvard’s latest AI-in-education research is not whether a chatbot can answer a physics question. It is whether a machine, when deliberately designed around the principles of good teaching, can change how students learn.
A study led by Harvard lecturers Gregory Kestin and Kelly Miller offers a striking answer. In an experiment involving 194 students in Harvard’s Physical Sciences 2 course, students who learned through a customised AI tutor showed learning gains of about twice those recorded by students receiving the same material through an instructor-guided active-learning classroom. The AI group also reported greater engagement and motivation.
The research, conducted in fall 2023, was later published in Scientific Reports in 2025. The findings have since become part of a wider debate over what generative AI should actually do in education: provide answers, or help students learn how to find them.
Harvard AI Tutor Study: What Happened to the Physics Students?
The experiment involved students taking Physical Sciences 2, an introductory physics course for life sciences majors and one of Harvard’s large physics classes.
Rather than dividing students permanently into an “AI group” and a “classroom group,” researchers used a crossover design.
Students experienced both approaches across two lessons.
One week, a group learned through an instructor-guided active-learning lesson while another group worked with the AI tutor remotely. The following week, the groups switched methods. Researchers used pre-tests and post-tests to measure learning, while also asking students about engagement, enjoyment, motivation and their perceptions of learning.
The result caught the researchers’ attention.
Learning gains with the AI tutor were more than twice those recorded in the active-learning classroom. Harvard’s earlier report also showed a higher mean post-test score for the AI-supported lesson than the active-learning lesson.
That comparison matters because the benchmark was not a conventional lecture in which students simply sat and listened. Harvard was comparing the AI tutor with an already research-informed active-learning model.
This Was Not Simply ChatGPT
That may be the most important detail in the entire study.
Students were not told to open a generic chatbot and ask it to explain physics.
Kestin and Miller built a custom AI tutor using detailed instructions, course material, content-rich prompts and structured feedback. The system was designed to behave more like a knowledgeable tutor than an unrestricted answer-generating chatbot.
The researchers used research-based prompt engineering and scaffolding to structure the learning experience.
That meant the AI could guide students through concepts rather than simply supplying the final answer.
The distinction is crucial. A student who asks a chatbot to solve a difficult problem may finish the homework without understanding the underlying idea. A tutor designed to ask questions, provide feedback and adjust explanations can instead make the student do more of the intellectual work.
Harvard’s own researchers warned that distinction matters.
Why Did the AI Tutor Work So Differently?
Two features stand out: personalised feedback and self-paced learning.
In a classroom, one instructor must respond to students who arrive with different levels of preparation. Some may already understand a concept. Others may need several explanations before it clicks.
An AI tutor can respond individually.
A student who understands the basics can move ahead. Another can stop, ask another question and spend more time on a difficult concept. The system does not need to move the entire class at the same pace.
That flexibility may be particularly valuable during the first encounter with unfamiliar material.
The researchers suggested that AI-supported learning could allow universities to move some basic instruction outside the classroom, freeing face-to-face time for activities that benefit from human interaction.
That could mean more time for advanced problem-solving, projects, discussion and group work rather than using classroom hours primarily to introduce foundational concepts.
Students Were Not Just Learning More — They Were More Engaged
The learning results were only part of the story.
Students using the AI tutor also reported greater engagement and motivation than those participating in the classroom lessons.
That finding is significant because education technology has long faced a basic problem: a tool can deliver information efficiently without necessarily making students want to learn.
The Harvard experiment suggested that personalisation may change that equation.
A student can ask a question immediately instead of waiting for a lecturer to reach that topic. They can revisit a concept without worrying about holding up the rest of the class. And they can work at a pace that matches their current understanding.
For students who are hesitant to speak up in a crowded classroom, that private interaction could also lower one barrier to asking questions.
What the Harvard Study Does — and Does Not — Prove
The headline result is striking, but the study has boundaries.
It involved 194 students at one university, in a particular introductory physics course, and tested specific lessons rather than an entire university curriculum. The researchers measured immediate learning through pre- and post-tests; the experiment therefore should not be treated as proof that AI tutoring will produce the same results across every subject, age group or educational setting.
There is another important qualification.
The AI tutor was carefully engineered by educators with course-specific material and pedagogical guidance. The findings do not establish that opening an ordinary general-purpose chatbot will automatically produce the same learning gains.
The design of the tutor was part of the intervention.
That is why the study is more useful as a lesson in AI-assisted teaching design than as a simple contest between humans and machines.
Harvard’s Message: AI Should Help Students Think, Not Think for Them
The researchers have also drawn a line around what they believe AI tutoring should become.
Harvard’s Kestin cautioned that AI could strengthen learning if used carefully, but could also undermine it if students allow the technology to do the thinking for them. The goal, he argued, should be to develop critical-thinking skills rather than outsource them.
That distinction is becoming increasingly important as generative AI moves deeper into education.
A system that gives students an answer may save time.
A system that helps students understand why the answer works may actually teach.
The Harvard experiment was built around the second idea.
From Physics to Calculus: Where Harvard Took the Experiment Next
The physics study did not remain confined to physics.
Harvard reported plans to adapt the AI-tutor framework for other courses, including multivariable calculus. The university’s Derek Bok Center for Teaching and Learning and Harvard University Information Technology also worked on pilots involving AI chatbots in other large introductory courses.
Harvard’s broader AI teaching resources now describe applications including personalised tutoring, feedback, practice problems and helping students work through course-specific material.
The university has also continued experimenting with AI tutors in other areas. Harvard’s CS50 course, for example, has developed an AI tutor designed to guide students toward solutions rather than simply provide them.
That points toward a larger shift.
The future classroom may not be defined by choosing between a professor and an AI tutor. It may involve deciding which parts of learning are better handled by each.
The Bigger Question for Education
The Harvard AI tutor study does not settle the debate over artificial intelligence in education. It makes the debate more specific.
The issue is no longer simply whether students are using AI.
They already are.
The harder question is what happens when AI is deliberately engineered around educational research, subject expertise and the needs of individual learners.
Harvard’s experiment suggests that a carefully built AI tutor can help students learn more in less time under the conditions tested. But it also points to the role that human teachers continue to play: designing learning goals, creating meaningful problems, challenging assumptions, supervising progress and turning knowledge into discussion and collaboration.
The most consequential lesson may therefore be hiding behind the headline.
The breakthrough was not that Harvard gave students a chatbot. It was that Harvard built the chatbot to teach.
