Automated Verbal and Non-Verbal Speech Analysis of Interviews of Individuals with Schizophrenia and Depression

Abstract

Schizophrenia and depression are the two most common mental disorders associated with negative symptoms that contribute to poor functioning and quality of life for millions of patients globally. This study is part of a larger research project. The overall aim of the project is to develop an automated objective pipeline that aids clinical diagnosis and provides more insights into symptoms of mental illnesses. In our previous work, We have analyzed non-verbal cues and linguistic cues of schizophrenic patients. In this study, we extend our work to include depressive patients. Using Natural Language Processing techniques, we extract verbal features, both dictionary-based and vector-based, from participants' interviews that were automatically transcribed. We also extracted conversational, phonatory, articulatory and prosodic features from the interviews to understand the conversational and acoustic characteristics of schizophrenia and depression. Combining these features, we applied ensemble learning with leave-one-out cross-validation to classify healthy controls, schizophrenic and depressive patients, achieving to 69%-75% in paired classification, and to predict the subjective Negative Symptoms Assessment 16 scores of schizophrenic and depressive patients, where the effect is especially so for NSA2 of which the prediction result is 90.5%. Our analysis also revealed significant linguistic and non-verbal vocal-based differences that are potentially symptomatic of schizophrenia and depression respectively.