Predicting affect classification in mental status examination using machine learning face action recognition system: A pilot study in schizophrenia patients

Ran Barzilay, Nadav Israel, Amir Krivoy, Roi Sagy, Shiri Kamhi-Nesher, Oren Loebstein, Lior Wolf, Gal Shoval*

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

5 Scopus citations

Abstract

Classifying patients' affect is a pivotal part of the mental status examination. However, this common practice is often widely inconsistent between raters. Recent advances in the field of Facial Action Recognition (FAR) have enabled the development of tools that can act to identify facial expressions from videos. In this study, we aimed to explore the potential of using machine learning techniques on FAR features extracted from videotaped semi-structured psychiatric interviews of 25 male schizophrenia inpatients (mean age 41.2 years, STD = 11.4). Five senior psychiatrists rated patients' affect based on the videos. Then, a novel computer vision algorithm and a machine learning method were used to predict affect classification based on each psychiatrist affect rating. The algorithm is shown to have a significant predictive power for each of the human raters. We also found that the eyes facial area contributed the most to the psychiatrists' evaluation of the patients' affect. This study serves as a proof-of-concept for the potential of using the machine learning FAR system as a clinician-supporting tool, in an attempt to improve the consistency and reliability of mental status examination.

Original languageEnglish
Article number288
JournalFrontiers in Psychiatry
Volume10
Issue numberMAY
DOIs
StatePublished - 2019

Keywords

  • Affect
  • Clinical psychiatry
  • Face recognition
  • Machine learning
  • Schizophrenia

Fingerprint

Dive into the research topics of 'Predicting affect classification in mental status examination using machine learning face action recognition system: A pilot study in schizophrenia patients'. Together they form a unique fingerprint.

Cite this