Scientific publications
Read about the research that supports the FaceReader Ecosystem
Over the past 20+ years, our facial coding platform and its embedded technologies have been the subject as well as the preferred instrument for numerous accredited scientific studies. Below we present a comprehensive overview of the literature that has emerged from these studies, highlighting and validating the cutting-edge technology of FaceReader Online.
2012
127 citations
The effect of emotional feedback on behavioral intention to use computer based assessment
Terzis, Moridis,Economides
This study introduces emotional feedback as a construct in an acceptance model, exploring its effect on behavioral intention to use Computer Based Assessment . A female Embodied Conversational Agent with empathetic encouragement behavior was displayed as emotional feedback. The research investigates the impact of Emotional Feedback on Behavioral Intention to Use a CBA system, Perceived Playfulness, Perceived Usefulness, Perceived Ease of Use, Content, and Facilitating Conditions. A survey questionnaire was completed by 134 students. Results demonstrate that Emotional Feedback has a direct effect on Behavioral Intention to Use a CBA system and on other crucial determinants of Behavioral Intention. The proposed acceptance model for computer-based assessment, extended with the Emotional Feedback variable, explains approximately 52% of the variance of Behavioral Intention.
2012
106 citations
Affective learning: Empathetic agents with emotional facial and tone of voice expressions
Moridis & Economides
Empathetic behavior is considered an effective method for Embodied Conversational Agents to provide feedback to learners’ emotions. This study examines the impact of ECAs’ emotional facial and tone of voice expressions, combined with empathetic verbal behavior, when displayed as feedback to students’ fear, sadness, and happiness during a self-assessment test. Three identical female agents were used:
1. An ECA performing parallel empathy with neutral emotional expressions.
2. An ECA performing parallel empathy displaying emotional expressions relevant to the student’s emotional state.
3. An ECA performing parallel empathy by displaying relevant emotional expressions followed by reactive empathy expressions aimed at altering the student’s emotional state.
Results indicate that an agent performing parallel empathy with emotional expressions relevant to the student’s state may cause the emotion to persist. Moreover, the agent performing both parallel and reactive empathy effectively altered a fearful emotional state to a neutral one.
2012
8 citations
User assisted stereo image segmentation
H.E. Tasli, A.A. Alatan
The wide availability of stereoscopic 3D displays created a considerable market for content producers. This encouraged researchers to focus on methods to alter and process the content for various purposes. This study concentrates on user assisted image segmentation and proposes a method to extend previous techniques on monoscopic image segmentation to stereoscopic footage with minimum effort. User assistance is required to indicate the representative locations of an image as object and background regions. An MRF based energy minimization technique is utilized where user inputs are applied only on one of the stereoscopic pairs. A key contribution of the proposed study is the elimination of dense disparity estimation by introducing a sparse feature matching idea. Segmentation results are evaluated by objective metrics on a ground truth stereo segmentation dataset and it can be concluded that competitive results with minimum user interaction have been obtained even without dense disparity estimation.
2012
79 citations
Speech-based recognition of self-reported and observed emotion in a dimensional space
Truong, van Leeuwen, de Jong
The differences between self-reported and observed emotion have only marginally been investigated in the context of speech-based automatic emotion recognition. We address this issue by comparing self-reported emotion ratings to observed emotion ratings and look at how differences between these two types of ratings affect the development and performance of automatic emotion recognizers developed with these ratings. A dimensional approach to emotion modeling is adopted: the ratings are based on continuous arousal and valence scales. We describe the TNO-Gaming Corpus that contains spontaneous vocal and facial expressions elicited via a multiplayer videogame and that includes emotion annotations obtained via self-report and observation by outside observers. Comparisons show that there are discrepancies between self-reported and observed emotion ratings which are also reflected in the performance of the emotion recognizers developed. Using Support Vector Regression in combination with acoustic and textual features, recognizers of arousal and valence are developed that can predict points in a 2-dimensional arousal-valence space. The results of these recognizers show that the self-reported emotion is much harder to recognize than the observed emotion, and that averaging ratings from multiple observers improves performance.
2012
29 citations
Relating Perceived Web Page Complexity to Emotional Valence and Eye Movement Metrics
Goldberg
Initial impressions of visual complexity significantly impact both consumer and enterprise web page designs. To integrate complexity assessment methods into usability tools, a study was conducted comparing subjective ratings, eye tracking, JPEG-compressed file size, and emotional valence measures. Professional enterprise users performed search tasks and evaluated web page complexity. Multivariate factor analysis and ordinal logistic regressions on subjective ratings revealed that perceptions of page complexity were influenced by self-assessed search difficulty and page density. Lower complexity ratings correlated with increased fixation durations and reduced search areas. Facial analysis indicated that aggregated emotional valence improved with higher clarity ratings. Overall, both pre-attentive eye tracking and emotional valence measures were associated with conscious subjective complexity judgments. Further research is recommended to attribute complexity-inducing features to measurable qualities.
2012
197 citations
Autonomic nervous system responses on and facial expressions to the sight, smell, and taste of liked and disliked foods
de Wijk, Kooijman, Verhoeven, Holthusen, de Graaf
Traditional sensory and consumer tests predict long term consumer acceptance of new products rather poorly, as evidenced by the high failure rates of new market introductions. These tests typical reflect conscious processes whereas consumer acceptance may also be based on unconscious processes, which may be measured by implicit physiological and behavioral measures. This study with 16 children (aged 8–10 years) and 15 young adults (mean age 22 years) explored the use of selected physiological and behavioral measures of the autonomic nervous system (ANS) in the study of foods. Three liked and three disliked foods were selected for each participant and their responses were measured during the first sight of each food and when they received the instruction to either visually inspect, smell or taste the foods. The first sight of disliked foods compared to liked foods resulted in increased skin conductance responses (p = 0.05) and increased facial expressions of sadness, disgust, and angriness (p = 0.05). Skin conductance responses (SCRs) varied significantly with the type of instruction (p < 0.001), with relatively small SCRs when participants were instructed to visually inspect the foods and larger SCRs when they are instructed to taste or smell the foods. When instructed to taste the foods, children showed increased SCRs for disliked foods while young adults showed decreased SCRs (p = 0.02). Heart rate varied with instruction and age group (p = 0.03). Children showed increased heart rate when instructed to visual inspect or taste the foods and reduced heart rate they were instructed to smell them. In contrast, young adults showed reduced heart rate when instructed to visual inspect and increased heart rate with instructed to taste or smell. Finger temperature was higher for liked foods than for disliked foods, irrespective of instruction and age group (p < 0.01). It is concluded that implicit ANS and behavioral responses provide detailed information on food preferences in relation to specific food properties and phases of food sampling that may not be provided by other more explicit tests.