Scientific publications
Read about the research that supports the FaceReader Ecosystem
Over the past 20+ years, our facial coding platform and its embedded technologies have been the subject as well as the preferred instrument for numerous accredited scientific studies. Below we present a comprehensive overview of the literature that has emerged from these studies, highlighting and validating the cutting-edge technology of FaceReader Online.
2016
56 citations
Recognizing Semantic Features in Faces using Deep Learning
A. Gudi
The human face constantly conveys information, both consciously and subconsciously. However, as basic as it is for humans to visually interpret this information, it is quite a big challenge for machines. Conventional semantic facial feature recognition and analysis techniques are already in use and are based on physiological heuristics, but they suffer from lack of robustness and high computation time. This thesis aims to explore ways for machines to learn to interpret semantic information available in faces in an automated manner without requiring manual design of feature detectors, using the approach of Deep Learning. This thesis provides a study of the effects of various factors and hyper-parameters of deep neural networks in the process of determining an optimal network configuration for the task of semantic facial feature recognition. This thesis explores the effectiveness of the system to recognize the various semantic features present in faces. Furthermore, the relation between the effect of high-level concepts on low level features is explored through an analysis of the similarities in low-level descriptors of different semantic features. This thesis also demonstrates a novel idea of using a deep network to generate 3-D Active Appearance Models of faces from real-world 2-D images.
2016
8 citations
Toward physiological indices of emotional state driving future ebook interactivity
van Erp, Hogervorst, van der Werf, Ysbrand
Ebooks of the future may respond to the emotional experience of the reader. physiological measures could capture a reader’s emotional state and use this to enhance the reading experience by adding matching sounds or to change the storyline, thereby creating a hybrid art form between literature and gaming. We describe the theoretical foundation of the emotional and creative brain and review the neurophysiological indices that can be used to drive future ebook interactivity in a real-life situation. As a case study, we report the neurophysiological measurements of a bestselling author during nine days of writing, which can potentially be used later to compare them to those of the readers. In designated calibration blocks, the artist wrote emotional paragraphs for emotional pictures. Analyses showed that we can reliably distinguish writing blocks from resting, but we found no reliable differences related to the emotional content of the writing. The study shows that measurements of EEG, heart rate , skin conductance, facial expression, and subjective ratings can be done over several hours a day and for several days in a row. In follow-up phases, we will measure 300 readers with a similar setup.
2015
8 citations
Who do you want to be? Real-time face swap
T.M. den Uyl, H.E. Tasli, P. Ivan, M. Snijdewind
This demonstration paper presents a face swap application where two people’s faces are automatically exchanged in real-time without any calibration or training. This is performed using the Active Appearance Models technique. A realistic visualization is achieved using an adaptive texture sampling technique. The face swap is performed irrespective of the sex, age or ethnicity of the subject in front of the camera. This application is intended for gaming, shopping, educational or entertainment purposes and will be presented in a real-time setup during the demo session.
2015
4 citations
Real-Time Facial Character Animation
H.E. Tasli, T.M. den Uyl, H. Boujut, T. Zaharia
This demonstration paper presents a real-time facial character animation application where the facial expressions of a person are simultaneously synthesized on a virtual avatar. The proposed method does not require any training or calibration for the person interacting with the system. An Active Appearance Model based technique is used to track more than 500 points on the face to create the animated expression of the virtual avatar. The sex, age or ethnicity of the subject in front of the camera can also be automatically analyzed and hence the visualization of the avatar could be adapted accordingly. This application requires a standard web cam and is intended for gaming, entertainment or video conference purposes and will be presented in a real-time setup during the demo session.
2015
88 citations
Characterizing consumer emotional response to sweeteners using an emotion terminology questionnaire and facial expression analysis
Leitch, Duncan, O’Keefe, Rud, Gallagher
Concerns associated with sugar-sweetened beverages have led to an increased consumer demand for sweetener alternatives that are functionally equivalent to sucrose without the associated health risks. Measuring consumer emotions has the potential to aid the industry in subsequent ingredient decision-making. The purpose of this study was to evaluate the relationship of consumer acceptability and emotional response of sweeteners in tea using a 9-point hedonic scale, an emotion term questionnaire , and a facial expression response . Participants evaluated a water sample , two sucrose-tea samples , and four equi-sweet alternative sweetener-tea samples , divided by category . Sessions were divided by category and emotional response tool in a cross-over design. Facial expression responses were recorded in the first session of both days using FaceReader 5.0 and individual participant videos were analyzed per sample for 5-s post-consumption in the continuous analysis setting using automated facial expression analysis software. Emotional term responses were collected in the second session of each day and count frequencies of each term per sample were tabulated and analyzed. Hedonic acceptability was rated in all sessions on a 9-point scale. Alternative sweeteners were all rated ‘acceptable’ , except for honey in one session. Only one alternative in each category was statistically different in liking from sucrose. Facial analysis showed minimal differences in emotion elicited across sweetener categories. Time series analysis was more robust in showing differences than baseline comparisons. Emotional term selection using a CATA questionnaire showed four unique terms for natural sweeteners and two unique terms for artificial sweeteners. More research exploration related to emotions and food is needed in order to accrue a more accurate picture of consumer product preferences.
2015
226 citations
A multi-componential analysis of emotions during complex learning with an intelligent multi-agent system
Harley, Bouchet, Hussain, Azevedo, Calvo
This study evaluates the synchronization of three emotional measurement methods—automatic facial expression recognition, self-report, and electrodermal activity—and their agreement regarding learners’ emotions. Data were collected from 67 undergraduates at a North American university who learned about a complex science topic while interacting with MetaTutor, a multi-agent computerized learning environment. Videos of learners’ facial expressions, captured with a webcam, were analyzed using automatic facial recognition software . Learners’ physiological arousal was recorded using Affectiva’s Q-Sensor 2.0 electrodermal activity measurement bracelet. Learners self-reported their experience of 19 different emotional states on five different occasions during the learning session, which were used as markers to synchronize data from FaceReader and Q-Sensor. The study found a high agreement between the facial and self-report data , but low levels of agreement between them and the Q-Sensor data, suggesting that a tightly coupled relationship does not always exist between emotional response components.