AI is learning how to tutor -- but will students use it?

Khan Academy's AI-powered tutoring chatbot has been a dud so far, Sal Khan tells Chalkbeat's Matt Barnum. Students don't use it very much.

In 2024, Khan appeared in a “60 Minutes” segment Indiana’s Hobart High School, which was an early adopter of Khanmigo, writes Barnum. The tool relies on students to ask questions, something they're not very good at.   

Kristen Musall's geometry students found Khanmigo frustrating, she told Barnum. “If students don’t engage with the material enough to know what they’re looking for, then an AI like Khanmigo doesn’t necessarily help,” she said. She's stopped using it in class, though a few advanced students use it to learn new topics.

Kristen DiCerbo, Khan Academy's chief learning officer, said, “Students aren't great at asking questions well.” She'd hoped AI would personalize instruction, but that hasn't panned out. “So far I am not seeing the revolution in education,” she said. 

Khan Academy is modifying the AI tool to give students more practice, and hopes it will become more useful.

Researchers are working to improve the effectiveness of AI tutors, reports Hechinger's Jill Barshay. "Some studies have found that chatbot tutors can backfire because students lean on them too heavily, get spoonfed solutions and fail to absorb the material."

But developers hope to do better.

In a study of Taiwanese high school students learning Python programming, all students used the same AI tutor, designed not to give away answers, Barshay writes. Some were given practice problems that progressed from easier to harder, while others problems adjusted for difficulty "based on how the student was performing and interacting with the chatbot."

Students who received personalized problems did significantly better on the final exam that students in the other group, said Angel Chung, a Penn researcher. “Students usually don’t know what they don’t know,” she said. “The student doesn’t have the ability to ask the right questions to get the best tutoring.”

In the pre-AI age, writes Barshay, rigorous studies found that well-designed "intelligent tutoring systems" helped students learn significantly more. However, "their Achilles’ heel was engagement. Many students simply didn’t want to use them."

AI boosters hope chatbots that converse with students could change that.

In the Taiwan study, students in the personalized group spent about three additional minutes more per problem, than the comparison students, researchers observed. Personalizing was more helpful to students who were new to Python and to those from less elite high schools.

All the students were volunteers, and "many were highly motivated, with highly educated parents," she writes. "It’s not clear whether the chatbot would work as well with less motivated students who are behind at school and most in need of extra help."

Ken Koedinger, a Carnegie Mellon professor, "is experimenting with using new AI models to alert remote human tutors who can motivate struggling students who are drifting off, Barshay writes.