Computer tutors are learning to read students’ emotions, so they can provide better feedback, reports Annie Murphy Paul on the Hechinger Report.
Analyzing students’ posture in a special chair, how much pressure they exert when they click on a special mouse or the pitch of their voices can reveal “academic emotions” such as “curiosity, delight, flow, engagement, confusion, frustration and boredom.”
Some systems use wireless skin conductance sensors or cameras that analyze facial expressions and track students eyes.
“One computerized tutoring program uses ‘mind-reader software’ to identify 22 facial feature points, 12 facial expressions and six mental states,” writes Paul.
Detecting the learner’s feelings is just the first step.
A computerized tutoring program called Wayang Outpost, developed by researchers at the University of Massachusetts-Amherst, features an onscreen avatar that subtly mirrors the emotions the learner is feeling. When the learner smiles, the avatar smiles too, making the learner feel understood and supported. When the learners express negative feelings, the avatar mirrors their facial expression of, say, frustration, and offers verbal reassurance: “Sometimes I get frustrated when solving these math problems.”
Then — in a shift that researchers have found to be essential — the avatar pivots toward the positive. “On the other hand,” the avatar might add, “more important than getting the problem right is putting in the effort and keeping in mind that we can all do math if we try.”
Researchers try to encourage a “growth mindset,” the belief that ability improves with effort.
If the learner seems bored, for example, (Notre Dame’s Affective) AutoTutor might respond with the comment, “This stuff can be kind of dull sometimes, so I’m gonna try and help you get through it. Let’s go.” If the AutoTutor senses that the learner is confused, it might advise, “Some of this material can be confusing. Just keep going and I am sure you will get it.”
Deep learning requires struggle, say researchers in affective computing. “Students show the lowest levels of enjoyment during learning under the conditions in which they learn the most, and the feeling of confusion turns out to be the best predictor of learning.”
However, repeated failures turn confusion into “frustration, disengagement and boredom (and ultimately, minimal learning).”