Oral Presentation (max 25mins) The National Suicide Prevention Conference 2024

Real-time measurement and portrayal of empathy through voice analysis (101453)

Mapa Mudiyanselage Ruvini Sanjeewa 1 , Denny Meyer 1 , Ravi Iyer 1 , Pragalathan Apputhurai 1 , Nilmini Wickramasinghe 2
  1. Swinburne University of Technology, Hawthorn, VIC, Australia
  2. La Trobe University, Melbourne, VIC, Australia

Mental health helpline services around the world provide wonderful support to those who are going through a mental health crisis. In Australia, the support offered by such helpline services includes helping callers at immediate risk of suicide. Counsellors are carefully trained in order to ensure that the service provided is appropriate and beneficial to callers. In particular, it is important that counsellors exhibit empathy when talking with such callers. But what is empathy, and how is it best measured?

Empathy is simply the ability to understand and share the feelings of another. Measures of empathy usually rely on human perception, often using standardised empathy scales or empathy frameworks derived using transcribed speech. This study breaks new ground using voice analysis to recognise voice characteristics, such as how the variations of pitch, speech rate and volume are associated with empathy. Voice analysis has previously been used to detect suicide risk and distress; however, this is the first study that uses voice analysis to detect empathy.

Our research uses N=120 helpline calls from callers presenting with varying levels of suicide risk to determine which counsellor vocal characteristics are associated with a portrayal of counsellor empathy. Empathy ratings were collected for segments within each call using a subjective rating by three raters. In addition, the raters assessed each call recording using a modified version of the Mayer-Salovey-Caruso Perceived Emotional Intelligence Scale and The Active-Empathic Listening Scale. The reliability of the rating process was confirmed by comparing the evaluations of the three raters.  Using these data, statistical modelling can be used to develop models that provide empathy ratings for any voice segment for any speaker.

There is a broad range of possible future applications for this research in the mental health care sector. One application could involve the creation of an empathetic conversational agent (chatbot), used to routinely collect information from callers while waiting for a counsellor. A better presentation of empathy in the voice of the chatbot will improve user acceptance of this new technology, providing a timely solution for the long waiting times experienced during peak hours while providing the data needed to allow counsellors to engage with callers more productively. The algorithm used to measure empathy in counsellor voices can also be used to assist mental health helpline services in training counsellors to speak with greater empathy, thereby improving the service provided to callers.