AI to assist the elderly

Caring and expressive artificial intelligence? This concept that seems to come straight from a man-machine romance like the movie “Her”, is in fact at the heart of a Horizon 2020 project called EMPATHIC. The project aims to develop software for a virtual and customizable coach for assisting the elderly. To learn more, we interviewed the project’s Scientific Director for Télécom SudParis and expert in voice recognition, Dijana Petrovska-Delacretaz.

The original version of this article (in French) was published on the Télécom SudParis website.

What is the goal of the Empathic project?

Dijana Petrovska-Delacretaz: The project’s complete title is “Empathic Expressive Advanced Virtual Coach to Improve Independent Healthy Life-Years of the Elderly”. The goal is to develop a virtually advanced, empathetic and expressive coach to assist the elderly in daily life. This interface equipped with artificial intelligence and featuring several options would be adaptable, customizable and available on several types of media: PC, smartphone or tablet. We want to use a wide range of audiovisual information and interpret it to take the users’ situation into account and try to respond and interact with them in the most suitable way possible to offer them assistance. This requires us to combine several types of technology, such as signal processing and biometrics.

How will you design this AI?

DPD: We will combine signal processing and speech, face and shape recognition using “deep networks” (networks of deep artificial neurons). In other words, we will reproduce our brain’s structure using computer and processor calculations. This new paradigm has become possible thanks to the massive increase in storage and calculation capacities. They allow us to do much more in much less time.

We also use 3D modeling technology to develop avatars with more expressive faces–whether it be a human, cat, robot, hamster or even a dragon–that can be adapted to the user’s preferences to facilitate the dialogue between the user and virtual coach.   The most interesting solution from a biometric and artificial intelligence standpoint is to include as many options as possible: from using the voice and facial expressions to recognizing emotions.  All of these aspects will help the virtual coach recognize its user and have an appropriate dialogue with him or her.

Why not create a robot assistant?

The robot NAO is used at Télécom SudParis in work on recognition and gesture interpretation.

DPD: The software can be fully integrated into a robot like Nao, which is already used here on the HadapTIC and EVIDENT platforms. This is certainly a feasible alternative: rather than have a virtual coach on a computer or smartphone, the coach can be there in person. The advantage with a virtual coach is that it is much easier to bring with me on my tablet when I travel.

In addition, from a security standpoint, a robot like Nao is not allowed access everywhere. We really want to develop a system that is very portable and not bulky. The challenge is to combine all of these types of technology and make them work together so we can interact with the artificial intelligence based on scenarios that are best suited to an elderly individual on different types of devices.

Who is participating in the Empathic project?

DPD: The Universidad del Pais Vasco in Bilbao is coordinating the project and is contributing to the dialogue aspect. We are complementing this aspect by providing the technology for vocal biometrics and emotional and face recognition, as well as the avatar’s physical appearance. The industrial partners involved in the project are Osatek, Tunstall, Intelligent Voice and Acapela. Osatek is the largest Spanish company specializing in connected objects for patient monitoring and home care. Tunstall also specializes in this type of material. Intelligence Voice and Acapela, on the other hand, will primarily provide assistance in voice recognition and speech synthesis. In addition, Osatek and Intelligent Voice will be responsible for ensuring the servers are secure and storing the data. The idea is to create a prototype that they will then be able to use.

Finally, the e-Seniors association, the Seconda Universidad of Italy and the Oslo University Hospital will provide the 300 users-testers to test the EMPATHIC interface throughout the project. This provides us with significant statistical validity.

What are the major steps in the project?

DPD: The project was launched in November 2017 and our research will last three years. We are currently preparing for an initial acquisition stage which take place in the spring of 2018. For this first stage, we will make recordings with a “Wizard of Oz” system in the EVIDENT living lab at Télécom SudParis. With this system, we can simulate the virtual agent with a person who is located in another room in order to collect data on the interaction between the user and the agent. This system allows us to create more complex scenarios that our artificial intelligence will then take into account. This first stage will also provide us with additional audiovisual data and allow us to select the best personifications of virtual coaches to offer to users. I believe this is important.

What are the potential opportunities and applications?

DPD: The research project is intended to produce a prototype. The idea behind having industrial partners is for them to quickly obtain and adapt this prototype to suit their needs. The goal of this project is not to produce a specific innovation, but rather to combine and adapt all these different types of technology so that they reach their potential through specific cases—primarily for assisting the elderly.

Another advantage of EMPATHIC is that it reveals many possible applications in other areas: including video games and social networks.  A lot of people interact with avatars in virtual worlds today–like in the video game Second Life, which is where one of our avatars, Bob the Hawaiian, comes from. EMPATHIC could therefore definitely be adapted outside the medical sector.

Do not confuse virtual and reality

The “Uncanny Valley” is a central concept in artificial intelligence and robotics. It theorizes that if a robot or AI possess a human form (whether physical or virtual), it should not resemble humans too closely–unless this resemblance is perfectly achieved from every angle.  “As soon as there is a strong resemblance with humans, the slightest fault is strange and disturbing. In the end, it becomes a barrier in the interactions with the machine,” explains Patrick Horain, a research engineer at Télécom SudParis, specialized in digital imaging.

Dijana Petrovska-Delacretaz has integrated this aspect and is developing the EMPATHIC coaches accordingly: “It is important to keep this in mind. In general, it is a scientific challenge to make a photo-realistic face that actually looks like a loved one without being able to distinguish the real from the fake. But in the context of our project, it is a problem. It could be disturbing or confusing for an elderly person to interact with an avatar that looks like someone real or even a loved one. It is always preferable to propose virtual coaches that are clearly not human.”

Do not confuse virtual and reality

The “Uncanny Valley” is a central concept in artificial intelligence and robotics. It theorizes that if a robot or AI possess a human form (whether physical or virtual), it should not resemble humans too closely–unless this resemblance is perfectly achieved from every angle.  “As soon as there is a strong resemblance with humans, the slightest fault is strange and disturbing. In the end, it becomes a barrier in the interactions with the machine,” explains Patrick Horain, a research engineer at Télécom SudParis, specialized in digital imaging.

Dijana Petrovska-Delacretaz has integrated this aspect and is developing the EMPATHIC coaches accordingly: “It is important to keep this in mind. In general, it is a scientific challenge to make a photo-realistic face that actually looks like a loved one without being able to distinguish the real from the fake. But in the context of our project, it is a problem. It could be disturbing or confusing for an elderly person to interact with an avatar that looks like someone real or even a loved one. It is always preferable to propose virtual coaches that are clearly not human.”