Skip to Main content Skip to Navigation

Managing heterogeneous cues in social contexts : A holistic approach for social interactions analysis

Mahmoud Qodseya 1
1 IRIT-SIG - Systèmes d’Informations Généralisées
IRIT - Institut de recherche en informatique de Toulouse
Abstract : Social interaction refers to any interaction between two or more individuals, in which information sharing is carried out without any mediating technology. This interaction is a significant part of individual socialization and experience gaining throughout one's lifetime. It is interesting for different disciplines (sociology, psychology, medicine, etc.). In the context of testing and observational studies, multiple mechanisms are used to study these interactions such as questionnaires, direct observation and analysis of events by human operators, or a posteriori observation and analysis of recorded events by specialists (psychologists, sociologists, doctors, etc.). However, such mechanisms are expensive in terms of processing time. They require a high level of attention to analyzing several cues simultaneously. They are dependent on the operator (subjectivity of the analysis) and can only target one side of the interaction. In order to face the aforementioned issues, the need to automatize the social interaction analysis process is highlighted. So, it is a question of bridging the gap between human-based and machine-based social interaction analysis processes. Therefore, we propose a holistic approach that integrates multimodal heterogeneous cues and contextual information (complementary "exogenous" data) dynamically and optionally according to their availability or not. Such an approach allows the analysis of multi "signals" in parallel (where humans are able only to focus on one). This analysis can be further enriched from data related to the context of the scene (location, date, type of music, event description, etc.) or related to individuals (name, age, gender, data extracted from their social networks, etc.). The contextual information enriches the modeling of extracted metadata and gives them a more "semantic" dimension. Managing this heterogeneity is an essential step for implementing a holistic approach. The automation of " in vivo " capturing and observation using non-intrusive devices without predefined scenarios introduces various issues that are related to data (i) privacy and security; (ii) heterogeneity; and (iii) volume. Hence, within the holistic approach we propose (1) a privacy-preserving comprehensive data model that grants decoupling between metadata extraction and social interaction analysis methods; (2) geometric non-intrusive eye contact detection method; and (3) French food classification deep model to extract information from the video content.[...]
Document type :
Complete list of metadatas
Contributor : Abes Star :  Contact
Submitted on : Friday, January 22, 2021 - 9:58:07 AM
Last modification on : Saturday, January 23, 2021 - 3:25:41 AM


Version validated by the jury (STAR)


  • HAL Id : tel-03118308, version 1


Mahmoud Qodseya. Managing heterogeneous cues in social contexts : A holistic approach for social interactions analysis. Computers and Society [cs.CY]. Université Paul Sabatier - Toulouse III, 2020. English. ⟨NNT : 2020TOU30099⟩. ⟨tel-03118308⟩



Record views


Files downloads