Abstract
This paper describes a vision-based computational model of mind-reading that infers complex mental states from head and facial expressions in real-time. The generalization ability of the system is evaluated on videos that were posed by lay people in a relatively uncontrolled recording environment for six mental states—agreeing, concentrating, disagreeing, interested, thinking and unsure. The results show that the system’s accuracy is comparable to that of humans on the same corpus.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Baron-Cohen, S.: Mindblindness: An Essay on Autism and Theory of Mind. MIT Press, Cambridge (1995)
Baron-Cohen, S., Golan, O., Wheelwright, S., Hill, J.J.: Mind Reading: The Interactive Guide to Emotions. Jessica Kingsley Publishers, London (2004)
Baron-Cohen, S., Wheelwright, S., Hill, J., Raste, Y., Plumb, I.: The Reading the Mind in the Eyes Test Revised Version: A Study with Normal Adults, and Adults with Asperger Syndrome or High-functioning Autism. Journal of Child Psychology and Psychiatry 42(2), 241–251 (2001)
Ekman, P., Friesen, W.V.: Pictures of Facial Affect. Consulting Psychologists (1976)
Ekman, P., Friesen, W.V.: Facial Action Coding System: A Technique for the Measurement of Facial Movement. Consulting Psychologists (1978)
Fasel, B., Luettin, J.: Automatic Facial Expression Analysis: A Survey. Pattern Recognition 36, 259–275 (2003)
Gu, H., Ji, Q.: Facial Event Classification with Task Oriented Dynamic Bayesian Network. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), vol. 2, pp. 870–875 (2004)
el Kaliouby, R.: Mind-Reading Machines: Automated Inference of Complex Mental States. Phd thesis, University of Cambridge, Computer Laboratory (2005)
el Kaliouby, R., Robinson, P.: Real-Time Inference of Complex Mental States from Facial Expressions and Head Gestures. In: Real-Time Vision for Human Computer Interaction, pp. 181–200. Springer, Heidelberg (2005)
Kanade, T., Cohn, J., Tian, Y.-L.: Comprehensive Database for Facial Expression Analysis. In: Proceedings of International Conference on Automatic Face and Gesture Recognition, pp. 46–53 (2000)
Kapoor, A., Picard, R.W., Ivanov, Y.: Probabilistic Combination of Multiple Modalities to Detect Interest. In: Proceedings of the International Conference on Pattern Recognition (ICPR), vol. 3, pp. 969–972 (2004)
Littlewort, G., Bartlett, M.S., Fasel, I., Susskind, J., Movellan, J.R.: Dynamics of Facial Expression Extracted Automatically from Video. In: Face Processing in Video Workshop at the CVPR 2004 (2004)
Michel, P., el Kaliouby, R.: Real Time Facial Expression Recognition in Video using Support Vector Machines. In: Proceedings of the IEEE International Conference on Multimodal Interfaces (ICMI), pp. 258–264 (2003)
Murphy, K.P.: Dynamic Bayesian Networks: Representation, Inference and Learning. Phd thesis, UC Berkeley, Computer Science Division (2002)
Pantic, M., Rothkrantz, L.J.: Automatic Analysis of Facial Expressions: The State of the Art. IEEE Trans. on Pattern Analysis and Machine Intelligence (PAMI) 22, 1424–1445 (2000)
Pardàs, M., Bonafonte, A., Landabaso, J.L.: Emotion Recognition based on MPEG4 Facial Animation Parameters. In: Proceedings of Internatoinal Conference on Acoustics, Speech and Signal Procssing, vol. 4, pp. 3624–3627 (2002)
Tian, Y.-L., Brown, L., Hampapur, A., Pankanti, S., Senior, A.W., Bolle, R.M.: Real World Real-time Automatic Recognition of Facial Expressions. In: IEEE workshop on Performance Evaluation of Tracking and Surveillance (2003)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2005 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
el Kaliouby, R., Robinson, P. (2005). Generalization of a Vision-Based Computational Model of Mind-Reading. In: Tao, J., Tan, T., Picard, R.W. (eds) Affective Computing and Intelligent Interaction. ACII 2005. Lecture Notes in Computer Science, vol 3784. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11573548_75
Download citation
DOI: https://doi.org/10.1007/11573548_75
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-29621-8
Online ISBN: 978-3-540-32273-3
eBook Packages: Computer ScienceComputer Science (R0)