Categorisation of spoken social affects in Japanese: human vs. machine

Jean-Luc Rouas (1) , Takaaki Shochi (2, 1) , Marine Guerry (2) , Albert Rilliard (3)

1 LaBRI - Laboratoire Bordelais de Recherche en Informatique
2 CLLE-ERSS - Cognition, Langues, Langage, Ergonomie
3 LIMSI - Laboratoire d'Informatique pour la Mécanique et les Sciences de l'Ingénieur

In this paper, we investigate the abilities of both human listeners and computers to categorise social affects using only speech. The database used is composed of speech recorded by 19 native Japanese speakers. It is first evaluated perceptually to rank speakers according to their perceived performance. The four best speakers are then selected to be used in a categorisation experiment in nine social affects spread across four broad categories. An automatic classification experiment is then carried out using prosodic features and voice quality related features. The automatic classification system takes advantages of a feature selection algorithm and uses Linear Discriminant Analysis. The results show that the performance obtained by automatic classification using only eight features is comparable to the performance produced by our set of listeners: three out of four broad categories are quite well identified whereas the seduction affect is poorly recognised either by the listeners or the computer.

Mots clés: Social attitudes; Speech perception; Ex- pressive speech; Prosodic analysis

Domaines: Traitement du signal et de l'image [eess.SP]; Linguistique

Jean-Luc Rouas, Takaaki Shochi, Marine Guerry, Albert Rilliard. Categorisation of spoken social affects in Japanese: human vs. machine. 19th International Congress of Phonetic Sciences, Aug 2019, Melbourne, Australia. ⟨hal-02317743⟩