Publication Tag: Human-Computer Interaction

In this paper, the authors present initial results from a pilot experiment aimed at interpreting multimodal observations of human experts engaged in solving challenging chess problems. The study investigates how observations of eye-gaze, posture, emotion, and other physiological signals can be utilized to model the cognitive state of subjects. It also explores the integration of multiple sensor modalities to enhance the reliability of detecting human displays of awareness and emotion. Potential applications for such cognitive model-based systems include promoting healthy autonomous aging and developing automated training systems. By observing chess players tackling problems of increasing difficulty and recording their behavior, the researchers aim to estimate participants’ situational awareness and predict their ability to respond effectively to challenging situations. Feature selection was performed to construct a multimodal classifier that relies on the most relevant features from each modality. Initial results indicate that eye-gaze, body posture, and emotion are effective features for capturing such awareness. This experiment also validates the use of the equipment as a general and reproducible tool for studying participants engaged in screen-based interaction and problem-solving.