Computerized tutor detects mood, emotion

When trying to master a difficult subject, many students may prefer to study with a human tutor rather than a machine. However, computer systems that teach students are rapidly becoming more advanced and may already provide greater learning gains than do human tutors.
Scientists from MIT and Notre Dame have developed a computer software, which they named AutoTutor, to help students learn complex subjects, such as Newtonian physics, computer literacy and critical thinking. This system operates more like a human tutor than any software before and may soon redefine the human-computer interaction.
Humans usually communicate with computers through menus, windows and text. The developers of AutoTutor, however, wanted to create a software that would allow a computer to interact with a student through speech and nonverbal cues, similar to the way humans interact with one another. The AutoTutor software allows a computer to read facial expressions, eye contact, posture and other gestures. This gives the computer information about the mental state and motivation level of the student whom it is tutoring.
Specifically, the AutoTutor technology works by first evaluating the student's current knowledge through his or her responses to questions. The software can find and correct misconceptions, as well as respond to the student's own questions. It also responds to comments and complaints. The interactive tutor provides challenging problems that require collaboration between the student and the computer.
Another novel feature of the AutoTutor is a program known as the Affective AutoTutor, which monitors the student's nonverbal cues to identify his or her emotions. When it spots signs of boredom or frustration, the AutoTutor can alter its teaching strategy to help reengage the student. The researchers gave Affective AutoTutor its name because it can monitor emotions, which are also known as affective states.
See AUTOTUTOR, Page B8

AUTOTUTOR, From B7
AutoTutor is an Intelligent Tutoring System (ITS). These are computer systems that provide instruction to students without the intervention of a human tutor. Some of the features of AutoTutor are that it holds a conversation with a student in natural language, which means that it communicates with the student in a manner very similar to the way a human tutor would communicate. It simulates the teaching and motivational strategies of human tutors, and it can model its students' cognitive states, meaning that it can tailor its interaction to each individual student. It also uses animations, images, and simulations to keep the attention of the student.
Affective AutoTutor is slightly more advanced than AutoTutor. It contains softare that monitors the facial features, body language, and the conversations of the student to identify his or her emotions. When it recognizes frustration or boredom, it will work to change the student's mood. It can provide emotions in response to the student's emotions through the use of its responses and speech intonation. The computer responds to the student through an animated tutor that can speak and has rudimentary facial expressions and gestures.
Other researchers have worked on developing an ITS system that can identify and respond to human emotions, because research shows that a person's emotions are very connected to how they learn. Affective AutoTutor is important because it takes a unique approach; the researchers focused on developing a computer system that could identify a larger number of emotions. They also used unique sensors in the system to identify emotions.
AutoTutor has been tested on over 1,000 students, and the study showed that usage of this program correlated with an improvement of approximately one letter grade. This improvement is beyond what most human tutors could achieve.
The study about the software, "AutoTutor and Affective AutoTutor: Learning by Talking with Cognitively and Emotionally Intelligent Computers that Talk Back," authored by D'Mello, will be published in ACM Transactions on Interactive Intelligent Systems, in a special issue about the most innovative technology of the last decade.