Scientific publications
Read about the research that supports the FaceReader Ecosystem
Over the past 20+ years, our facial coding platform and its embedded technologies have been the subject as well as the preferred instrument for numerous accredited scientific studies. Below we present a comprehensive overview of the literature that has emerged from these studies, highlighting and validating the cutting-edge technology of FaceReader Online.
2020
45 citations
Real-Time Webcam Heart-Rate and Variability Estimation with Clean Ground Truth for Evaluation
A. Gudi, M. Bittner and J. van Gemert
Remote photo-plethysmography enables heart rate estimation using a camera by detecting skin reflectance changes associated with blood volume variations. Beyond HR, heart rate variability —the fine fluctuations between heartbeats—offers insights into physiological and psychological states but requires precise heartbeat timing. This study introduces an efficient, real-time rPPG pipeline with novel filtering and motion suppression techniques that not only estimate HR but also extract pulse waveforms to accurately time heartbeats and measure HRV. The unsupervised method operates in real-time without rPPG-specific training. Additionally, the authors present VicarPPG 2, a new multi-modal video dataset designed to evaluate rPPG algorithms for HR and HRV estimation. The method is validated across various conditions using a comprehensive range of public and self-recorded datasets, demonstrating state-of-the-art results and providing insights into unique aspects of rPPG analysis. Furthermore, CleanerPPG, a collection of human-verified ground truth peak/heartbeat annotations for existing rPPG datasets, is introduced to enhance the accuracy, standardization, and fairness of future rPPG algorithm evaluations.
2020
2 citations
Development of means for assessing the level of student satisfaction with the distance learning process through video conferencing
A. F. Ismagilova, D. S. Dudina, S. A. Aleynikov
This article details the development of tools using FaceReader to assess student satisfaction in distance learning environments by analyzing facial expressions during video conferencing.
2019
13 citations
Measuring emotions during learning: lack of coherence between automated facial emotion recognition and emotional experience
F. Hirt, E. Werlen, I. Moser, P. Bergamin
Measuring emotions non-intrusively via affective computing provides a promising source of information for adaptive learning and intelligent tutoring systems. Using non-intrusive, simultaneous measures of emotions, such as systems, could steadily adapt to students’ emotional states. One drawback, however, is the lack of evidence on how modern relate to traditional self-reports. The aim of this study was to compare prominent area of computing, facial emotion recognition, students’ self-reports of interest, boredom, and valence. We analyzed different types of aggregation recognition estimates compared them after reading text. Analyses revealed no relationship between aggregated software FaceReader estimates, neither epistemic (i.e., boredom, interest), nor valence predicted respective self-report measure. We conclude that assumptions about subjective experience cannot necessarily be transferred to other components, as estimated by computing. We advise to wait for more comprehensive predictive validity before relying on it in educational practice.
2019
36 citations
Efficient Real-Time Camera Based Estimation of Heart Rate and Its Variability
A. Gudi, M. Bittner, R. Lochmans and J. van Gemert
Remote photo-plethysmography utilizes a camera to estimate a person’s heart rate . Beyond HR, heart rate variability offers insights into physiological and psychological conditions by measuring fluctuations between heartbeats. Accurate HRV assessment requires precise heartbeat timing. This paper introduces an efficient real-time rPPG pipeline featuring novel filtering and motion suppression techniques that enhance HR estimation accuracy and extract pulse waveforms for HRV measurement. The method operates in real-time without the need for rPPG-specific training. Validation on a self-recorded dataset under ideal lab conditions and two public datasets with realistic scenarios demonstrates state-of-the-art performance.
2019
87 citations
Facial Expressions of Basic Emotions in Japanese Laypeople
W. Sato, S. Hyniewska, K. Minemoto, S. Yoshikawa
Facial expressions that show emotion play an important role in human social interactions. In previous theoretical studies, researchers have suggested there are universal, prototypical facial expressions specific to basic emotions. However, the results of some empirical studies tested production of emotional based on particular scenarios only partially supported predictions. In addition, all were conducted in Western cultures. We investigated Japanese laypeople (n = 65) to provide further evidence regarding expressions. The participants produced for six emotions (anger, disgust, fear, happiness, sadness, and surprise) scenarios. Under baseline condition, imitated photographs automatically coded using FaceReader in terms of intensities action units. contrast photograph where target shown clearly, scenario condition elicited clearly happy surprised conditions yielded different profiles units associated with tested. These results partial support theory but suggest possibility may need be modified evidence.
2019
179 citations
Assessing the convergent validity between the automated emotion recognition software Noldus FaceReader 7 and Facial Action Coding System Scoring
T. Skiendziel, A. R. Rösch, O. C. Schultheiss
This study validates automated emotion recognition software Noldus FaceReader 7 to a dataset of six basic emotions (Standardized Motivated Facial Expressions of Emotion). Percentages of correctly and falsely classified emotional expressions are reported. The validity of AUs is provided by correlations between the analysis of manual Facial Action Coding System (FACS) scoring for 20 AUs. On average 80% of emotional expressions are classified. The overall validity is moderate with highest indicators for AU 1, 5, 9, 17, 27. These results are compared to performance in previous research, yielding comparable coefficients. Practical implications and limitations of the method are discussed.