AI tools meet classroom reality: How are they used? Are they used?

Is artificial intelligence making students smarter? Rotting their brains?

Instruction Partners' new AI in Action report, based on a year-long effort to observe how students and teachers use AI, analyzes 20 products from chatbots, platforms such as Brisk Teaching, Google Classroom, MagicSchool and Playlab, and targeted instructional tools including Amira Learning, Khanmigo and Quill. They found a mixed bag, reports Greg Toppo.

"General-purpose chatbots like Claude and ChatGPT pose the biggest threat to student thinking," the Nashville nonprofit concluded. The platforms, including Brisk Teaching, Google Classroom, MagicSchool and Playlab, "produced different results depending on how teachers used them." The targeted tools, such as Khan Academy’s Khanmigo tutor and Quill, a writing app, "produced the most consistent and strongest learning experiences."

Some technology that expands teachers' reach, such as giving students detailed feedback or giving teachers"a quick analysis of an entire group’s work," said Emily Freitag, CEO of Instruction Partners and a former assistant education commissioner in Tennessee and Teach for America math teacher.

But what apps are designed to do is not the same as how they're used, she found.

She worries about AI's ability to simplify reading passages for slower readers, writes Toppo. It's a popular feature of most reading tools. Freitag fears releveling can become “a permanent lower setting” rather than a bridge back to grade level.

Designed to mimic a human tutor, Khanmigo provides hints and guiding questions, but doesn't give students the answer, reports Hechinger's Jill Barshay. However, it's hard to get students to use it, according to a study of Tennessee middle schoolers, When they couldn't get the AI tutor to give them the answers, most students gave up.

All the students had been given a remedial-math period to catch up. Those who had access to Khanmigo did slightly better than remedial students who were given other math programs, but no better than those given Khan's non-AI practice sessions, notes Barshay.

The study confirmed other research, wrote CEO Sal Khan in a blog post. Those who use tutoring help improve, but it's very hard to get students to engage.

In the new version of Khanmigo, students who ask for help before trying a problem, get a hint and a pep talk, writes Barshay. "But after a student gets a problem wrong, the AI assistant automatically pops up to offer help and work through the problem together with the student." To get students to use the tutor, "a bot-assisted redo" may "count toward one of the five correct answers in a row that students need before moving on to the next skill." 

It's not just Khanmigo. Educational technology has a 5% problem, writes Rick Hess. Again and again, "only a small share of students use them as recommended."

It doesn't matter how great an AI tutor may be, if students aren't motivated to use it, says Michael Horn, co-host of the Future U podcast, and the author of Blended: Using Disruptive Innovation to Improve Schools. If there's no "human accountability," few students will make the effort.

Horn suggests connecting new learning to background knowledge, and rewarding "the hard grind of mastery with a tangible sense of progress, so students can see the ground they’ve covered." And, he tells Hess, "lock down everything except the AI tutor they’re supposed to be using. Get rid of the distractions."