Power of SUNY

BU Researchers Use Human Cues to Improve Computer User-Friendliness

Lijun Yin wants computers to understand inputs from humans that go beyond the traditional keyboard and mouse. Yin’s team has developed ways to provide information to the computer based on where a user is looking as well as through gestures or speech. One of the basic challenges in this area is “computer vision.” That is, how can a simple webcam work more like the human eye?

To some extent, it’s already possible. One of Yin’s graduate students presented a PowerPoint presentation and using only his eyes to highlight content on various slides. When Yin demonstrated this technology for Air Force experts last year, the only hardware he brought was a webcam attached to a laptop computer.

The next step is to enable the computer to recognize a user’s emotional state. Yin works with a well-established set of six basic emotions — anger, disgust, fear, joy, sadness, and surprise — and is experimenting with different ways to allow the computer to distinguish among them.

He’s partnering with Binghamton University psychologist Peter Gerhardstein to explore ways this work could benefit children with autism. Many people with autism have difficulty interpreting others’ emotions; therapists sometimes use photographs of people to teach children how to understand when someone is happy or sad and so forth. Yin could produce not just photographs, but three-dimensional avatars that are able to display a range of emotions. Given the right pictures, Yin could even produce avatars of people from a child’s family for use in this type of therapy.

Yin and Gerhardstein’s previous collaboration led to the creation of a 3D facial expression database, which includes 100 subjects with 2,500 facial expression models. The database is available at no cost to the nonprofit research community and has become a worldwide test bed for those working on related projects in fields such as biomedicine, law enforcement and computer science.