Scientific publications
Read about the research that supports the FaceReader Ecosystem
Over the past 20+ years, our facial coding platform and its embedded technologies have been the subject as well as the preferred instrument for numerous accredited scientific studies. Below we present a comprehensive overview of the literature that has emerged from these studies, highlighting and validating the cutting-edge technology of FaceReader Online.
2012
536 citations
Emotion-Induced Engagement in Internet Video Advertisements
Teixeira, Wedel, Pieters
This study demonstrates how advertisers can utilize emotion and attention to engage consumers with Internet video advertisements. Through a controlled experiment, the authors evaluated joy and surprise using automated facial expression detection across a sample of ads. They measured attention concentration via eye tracking and assessed viewer retention by monitoring zapping behavior. This approach enabled testing predictions about the interplay between these emotions and individual attention differences during exposure. Findings indicate that both surprise and joy effectively focus attention and retain viewers. Notably, the intensity of surprise has a greater impact on attention concentration than its rate of change , whereas the velocity of joy influences viewer retention more than its level. Additionally, the effect of joy is asymmetric, with increases yielding higher gains than decreases result in losses. Based on these insights, the authors developed representative emotion trajectories to aid in ad design and testing.
2012
16 citations
Behavioural and physiological responses to two food odours
He, Boesveldt, de Graaf, de Wijk
Consumer food choice and intake are largely controlled by unconscious processes, which may be reflected better by implicit physiological and behavioural measures than by the more traditional explicit sensory tests. In this study, 26 human participants were exposed to an orange (pleasant) and a fish (unpleasant) odour presented in three different concentrations perceived as weak, medium and strong intensity, and five replications in a semi-random order via an olfactometer (Burghart OM2). Reactions to these odours were measured implicitly by means of facial expressions (automatically analysed with FaceReader), skin conductance responses and heart rate frequency (automatically analysed by Biolab), and explicitly with pleasantness ratings. Facial expressions reflected the odour’s valence (71% explained variance) and ranged from neutral (orange) to sadness/disgust/anger (fish) but showed additional differentiation with respect to odour intensity (15% explained variance). Skin conductance responses were largest for the unpleasant odour (p < 0.05), but showed no intensity effect. The unpleasant odour resulted in increased heart rate whereas the pleasant odour resulted in reduced heart rate (p < 0.05). The heart rate effects increased with intensity (p < 0.05). Different degrees of exposure were reflected in the implicit behavioural and physiological tests but not in the explicit pleasantness test. In summary, implicit physiological and behavioural responses provide detailed information on specific food odours that may not be provided by other more explicit tests.
2012
127 citations
The effect of emotional feedback on behavioral intention to use computer based assessment
Terzis, Moridis,Economides
This study introduces emotional feedback as a construct in an acceptance model, exploring its effect on behavioral intention to use Computer Based Assessment . A female Embodied Conversational Agent with empathetic encouragement behavior was displayed as emotional feedback. The research investigates the impact of Emotional Feedback on Behavioral Intention to Use a CBA system, Perceived Playfulness, Perceived Usefulness, Perceived Ease of Use, Content, and Facilitating Conditions. A survey questionnaire was completed by 134 students. Results demonstrate that Emotional Feedback has a direct effect on Behavioral Intention to Use a CBA system and on other crucial determinants of Behavioral Intention. The proposed acceptance model for computer-based assessment, extended with the Emotional Feedback variable, explains approximately 52% of the variance of Behavioral Intention.
2012
106 citations
Affective learning: Empathetic agents with emotional facial and tone of voice expressions
Moridis & Economides
Empathetic behavior is considered an effective method for Embodied Conversational Agents to provide feedback to learners’ emotions. This study examines the impact of ECAs’ emotional facial and tone of voice expressions, combined with empathetic verbal behavior, when displayed as feedback to students’ fear, sadness, and happiness during a self-assessment test. Three identical female agents were used:
1. An ECA performing parallel empathy with neutral emotional expressions.
2. An ECA performing parallel empathy displaying emotional expressions relevant to the student’s emotional state.
3. An ECA performing parallel empathy by displaying relevant emotional expressions followed by reactive empathy expressions aimed at altering the student’s emotional state.
Results indicate that an agent performing parallel empathy with emotional expressions relevant to the student’s state may cause the emotion to persist. Moreover, the agent performing both parallel and reactive empathy effectively altered a fearful emotional state to a neutral one.
2012
8 citations
User assisted stereo image segmentation
H.E. Tasli, A.A. Alatan
The wide availability of stereoscopic 3D displays created a considerable market for content producers. This encouraged researchers to focus on methods to alter and process the content for various purposes. This study concentrates on user assisted image segmentation and proposes a method to extend previous techniques on monoscopic image segmentation to stereoscopic footage with minimum effort. User assistance is required to indicate the representative locations of an image as object and background regions. An MRF based energy minimization technique is utilized where user inputs are applied only on one of the stereoscopic pairs. A key contribution of the proposed study is the elimination of dense disparity estimation by introducing a sparse feature matching idea. Segmentation results are evaluated by objective metrics on a ground truth stereo segmentation dataset and it can be concluded that competitive results with minimum user interaction have been obtained even without dense disparity estimation.
2012
79 citations
Speech-based recognition of self-reported and observed emotion in a dimensional space
Truong, van Leeuwen, de Jong
The differences between self-reported and observed emotion have only marginally been investigated in the context of speech-based automatic emotion recognition. We address this issue by comparing self-reported emotion ratings to observed emotion ratings and look at how differences between these two types of ratings affect the development and performance of automatic emotion recognizers developed with these ratings. A dimensional approach to emotion modeling is adopted: the ratings are based on continuous arousal and valence scales. We describe the TNO-Gaming Corpus that contains spontaneous vocal and facial expressions elicited via a multiplayer videogame and that includes emotion annotations obtained via self-report and observation by outside observers. Comparisons show that there are discrepancies between self-reported and observed emotion ratings which are also reflected in the performance of the emotion recognizers developed. Using Support Vector Regression in combination with acoustic and textual features, recognizers of arousal and valence are developed that can predict points in a 2-dimensional arousal-valence space. The results of these recognizers show that the self-reported emotion is much harder to recognize than the observed emotion, and that averaging ratings from multiple observers improves performance.