INSENSION: Personalized smart platform for interaction with digital services for people with profound and multiple learning disabilities

INSENSION addressed a specific need: people with PIMD can only interact with their environment through unconventional, non-symbolic means, which limits their access to digital services and their ability to make decisions about their own environment. The platform provided the technological means to recognize, transparently and adapted to each individual, each person's behavioral signals, detecting patterns associated with specific situations. These patterns were translated into affective intentions, such as approval or disapproval of a given situation, which the system communicated to the support service.

The objectives of the project were:

• Develop an ICT platform capable of recognizing non-symbolic behavioral signals in a way that is adapted to each person.
• Translate the detected behavior patterns into affective intentions that can be communicated to care services.
• Increase the capacity for self-determination and the quality of life of people with PIMD.

The system architecture was structured around four behavior recognition modules: body gestures, facial expressions, vocalization, and physiological parameters. Integrating them required advanced artificial intelligence techniques and state-of-the-art Internet of Things models.

The project brought together an interdisciplinary consortium of specialists in computer science, special education, and care for people with intellectual disabilities, with the direct participation of people with PIMD and their caregivers throughout the research and development process, under an inclusive design paradigm. It was the first study of this kind carried out in real care centers for people with PIMD. Among its notable results, the platform demonstrated the ability to substantially reduce the time a caregiver needs to learn to interpret the communicative signals of a person with PIMD, a process that in the home environment can take months. The consortium also developed two questionnaires for assessing interaction and communication, and collected data on this population in the Global PIMD Atlas.

CTIC's participation focused on developing the artificial intelligence for two of the platform's four main modules: body gesture recognition and facial expression recognition. To do so, it applied machine learning techniques aimed at pattern recognition and deep learning models for detecting facial and body landmarks.