Computers track how humans smile and react

What if computers were able to understand us and what we want? Computer vision systems are becoming increasingly popular in many businesses such as marketing, hospitals and Google.

Face recognition software enables these companies to monitor people’s facial expressions while conducting different activities, and then use the data to modify or enhance their products.

The software, developed at MIT, tracks the movement of 25 spots on the human face.

The expressions are then calculated using the angle between points, for example, around the mouth for a smile, and then compared to a database of expressions.

The creators of this software, Rosalind Picard and Rana el-Kaliouby, are also the co-founders of Affectiva, a company that is beginning to market and sell this software to other businesses.

One such business is Shopper Sciences, a company that studies shopper behavior. Chief executive John Ross believes that Affectiva’s new software will allow the company to study shoppers in a more impartial setting.

“You can see and analyze how people are reacting in real time, not what they are saying later, when they are often trying to be polite,” he said in an interview with The New York Times.

“The technology is more scientific and less costly than having humans look at store surveillance videos.”

Christopher Hamilton, who has worked as a technical director of visual effects on many films such as King Kong and The Matrix, has used facial recognition software in the past to determine the viewer’s reaction to films.

“A director,” Hamilton said in an interview with The New York Times, “could find out, for example, that although audience members liked a movie overall, they did not like two or three scenes.”

“Or he could learn that a particular character did not inspire the intended emotional response.”

Hamilton believes that software like Affectiva is soon to be a large part of the entertainment industry. Hospitals hope to use facial recognition software as well to monitor patients.

In October, Bassett Medical Center in Cooperstown, N.Y. placed cameras in a special care unit which alerted staff when a patient was in danger of falling out of bed, or reminded nurses to wash their hands when coming in and out of the room.

If these methods prove useful and cost effective, Bassett Medical Center hopes to add more features such as facial expression software that will scan patients for signs of pain and distress.

However, there are potential alarming results of this software.

“Machines will definitely be able to observe us and understand us better,” said Hartmut Neven, a computer scientist and vision expert at Google, in an interview with The New York Times. “Where that leads is uncertain.”

Google’s new application, Goggles, allows people to take a picture of something and search the internet for information about it. Google purposefully excluded facial recognition software, however, which would allow people to search for information on other people without their consent.

“Affectiva intends to offer its technology as ‘opt-in only,’ meaning consumers have to be notified and have to agree to be watched online or in stores,” Picard said in an interview with The New York Times.

It has already “turned down companies that wanted to use its software without notifying customers.”

Though Affectiva has taken this moral stance, it cannot guarantee that other manufacturers of similar software will do so as well.

Imagine a world where classrooms are full of cameras that could notify the professor if a student is daydreaming, or where companies could analyze what types of products you might buy through cameras watching you walk down a street.

“With every technology, there is a dark side,” said Hany Farid, a computer scientist at Dartmouth, in an interview with The New York Times. “Sometimes you can predict it, but often you can’t.”