User immersion, between 360° video and virtual reality

I’MTech is dedicating a series of success stories to research partnerships supported by the Télécom & Société Numérique (TSN) Carnot Institute, which the IMT schools are a part of.

To better understand how users interact in immersive environments, designers and researchers are comparing the advantages of 360° video and full-immersion virtual reality. This is also the aim of the TroisCentSoixante inter-Carnot project uniting the Télécom & Société Numérique and the M.I.N.E.S. Carnot Institutes. Strate Research, the research department at Strate School of Design which is a member of the Carnot TSN, is studying this comparison in particular in the case of museography mediation.  

 

When it comes to designing immersive environments, designers have a large selection of tools available to them. Mixed reality, in which the user is plunged into a more or less interactive environment, covers everything from augmented video to fully synthetic 3D images. To determine which is the best option, researchers from members of the TSN Carnot Institute (Strate School of Design) and the M.I.N.E.S Carnot Institute (Mines ParisTech and IMT Mines Alès) have joined forces. They have compared, for different use cases, the differences in user engagement between 360° video and full 3D modeling, i.e. virtual reality.

“At the TSN Carnot Institute we have been working on the case of a museum prototype alongside engineers from Softbank Robotics, who are interested in the project,” explains Ioana Ocnarescu, researcher at Strate. A room containing exhibits such as a Minitel, tools linked to the development of the internet, photos of famous researchers in robotics and robots has been created at Softbank Robotics to create mediation on science and technology. Once the object is in place, a 3D copy is made and a visit route is laid out between the different exhibits. This base scenario is used to film a 360° video guided by a mediator and to create a virtual guide in the form of a robot called Pepper, which travels around the 3D scene with the viewer. In both cases, the user is immersed in the environment using a mixed reality headset.

Freedom or realism: a choice to be made

Besides the graphics, which are naturally different between video and 3D modelling, the two technologies have one fundamental difference: freedom of action in the scenario. “In 360° video the viewer is passive,” explains Ioana Ocnarescu. “They follow the guide and can zoom in on objects, but cannot move around freely as they wish.” Their movement is limited to turning their head and deciding to spend longer on certain objects than others. To allow this, the video is cut in several places allowing a decision tree to be made that leads to specific sequences depending on the user’s choices.

 

3D modeling, on the other hand, grants a large amount of freedom to the viewer. They can move around freely in the scene, choose whether to follow the guide or not, walk around the exhibits and look at them from any angle, which is where 360° video is limited by the position of the camera. “User feedback shows that certain content is better suited to one device or the other,” the Strate researcher reports. For a painting or a photo, for example, there is little use in being able to travel around the object, and the viewer prefers to be in front of the exhibit in it its surroundings with as much realism as possible. “360° video is therefore better adapted for museums with corridors and paintings on the walls,” she points out. On the other hand, 3D modeling is particularly adapted to looking at and examining 3D artefacts such as statues.

These experiments are extremely useful to researchers in design, in particular because they involve real users. “Knowing what people do with the devices available is at the heart of our reflection,” emphasizes Ioana Ocnarescu. Strate has been studying user-machine interaction for over 5 years to develop more effective interfaces. In this project, the people in immersion can give their feedback directly to the Strate team. “It is the most valuable thing in our work. When everything is controlled in a laboratory environment, the information we collect is less meaningful.”

The tests must continue to incorporate a maximum amount of feedback from as many different types of audience as possible. Once finished, the results will be compared with those of other use cases explored by the M.I.N.E.S Carnot Institute. “Mines ParisTech and IMT Mines Alès are comparing the same two devices but in the case of self-driving cars and exploration of the Chauvet cave,” explains the researcher.

 

Carnot TSN, a guarantee of excellence in partnership-based research since 2006

The Télécom & Société numérique (TSN) Carnot Institute has partnered companies in their research to develop digital innovations since 2006. On the strength of over 1,700 researchers and 50 technology platforms, it offers cutting-edge research to resolve complex technological challenges produced by digital, energy and environmental and industrial transformations within the French production fabric. It addresses the following themes: Industry of the Future, networks and smart objects, sustainable cities, mobility, health and security.

The TSN Carnot Institute is composed of Télécom ParisTech, IMT Atlantique, Télécom SudParis, Institut Mines-Télécom Business School, Eurecom, Télécom Physique Strasbourg, Télécom Saint-Étienne, École Polytechnique (Lix and CMAP laboratories), Strate School of Design and Femto Engineering.