Database is reason to smile genuinely

The smiles of almost five hundred visitors to the NEMO science centre in the Dutch capital have been recorded by computer scientists to create the most extensive database in the world of posed and spontaneous smiles.

The researchers, from the Faculty of Science at the University of Amsterdam (UvA), recorded the genuine and posed smiles of 481 participants aged between eight and 76 years. The visitors were also asked to pose looking angry, happy, sad, surprised and scared. The work formed part of the Science Live project at NEMO, which was made possible by funding from the Netherlands Scientific Research Organisation (NWO) and the Royal Netherlands Academy of Arts and Sciences (KNAW).

Certain characteristics can be analysed from the videos which will allow researchers to develop or continue developing applications to estimate age, recognise emotions and analyse human behaviour. Work on age estimation software in particular is coming on apace, with software that can tell an individual’s age within six years, on average, compared to a human average of within seven years.

ScienceOmega.com interviewed Hamdi Dibeklioglu, a doctoral candidate in Professor Theo Gevers’ group at UvA’s Informatics Institute, to learn more about the computer analysis of facial expressions and why the database is worth smiling about…

What are the differences between a posed and a spontaneous smile?
If you’re watching a posed smile, it is faster in terms of facial dynamics; in other words, the facial muscles move more quickly. A spontaneous smile is slower – not by much – and lasts longer. Eyelid, cheek and lip corner movements are more rapid in posed smiles, and mean eye aperture is smaller during spontaneous smiles in comparison to posed ones.


How good are humans at recognising whether a smile is spontaneous or not?
There are no definite numbers for this, but we know that the circumstances can determine how good humans are at recognising smiles as fake or real. If you take trained people such as actors, who know how to fake and control their muscles, people are not good at saying which smile is which. For people there are no definite numbers, but a computer can analyse the dynamic very carefully because the timeframe is so short. People cannot observe the process very accurately but the difference, for example, the acceleration difference, is recorded in milliseconds so the computer can observe it more closely.


How do different emotions or facial expressions change as an individual ages?
In terms of physiology, young people have faster reactions and therefore their facial expressions are faster. The older you get, the longer the response time. That’s the main difference. The differences are due to the ageing of the muscles. We have a study on this, trying to estimate someone’s age using these dynamic features. When the subject smiles we record the time acceleration, the speed of facial dynamics like lip corner movements, eye aperture and the movements of the cheeks. These help with age estimation because, as I said, the younger you are the faster the movements. During both spontaneous and posed smiles, mean eye aperture is smaller and speed of eyelid, cheek and lip corner movements are faster for young people compared to adults.


What are the potential uses of the database you have created?
Imagine you have accurate age estimation on your PC and you want to block the use of some applications or websites to your kids or young people. The PC could automatically recognise age, and restrict access accordingly. For smiling, fake or posed smiles are used for different reasons; you could be lying, or you can be trying to be social. In daily life people smile a lot, but most of these smiles are posed because we are trying to be social or kind to others. This software can be used to check if people are actually smiling or if they are posing. Processing could be used in interviews or for identification purposes, for instance. There are many potential uses.


What are the next steps for the research being carried out by you and your colleagues?
The next step involves combining the two applications. We want to automatically and directly estimate age information in combination with the spontaneity-determining application. You will then get the age and the smile at the same time, because we know that if one of these is known the accuracy of the other is better. For example, if we know you are ten years old the accuracy with which we can determine whether your smile is posed or real is much better. The idea is to recognise them both simultaneously, and this will greatly improve accuracy. We are already working on this.

We already have emotion recognition systems, but we are looking at more detailed analyses. The system can say if you’re happy, sad or disgusted, for example, but we are aiming for a deeper analysis. It can tell if you look happy, but is it genuine happiness or are you posing?


For more information on the database, visit the UvA-NEMO website