Microsoft Holoportation combines AI, mixed reality and real-time translation

Summary

• Microsoft Holoportation translates speeches using artificial intelligence

While companies like Niantic, Magic Leap, Apple or Facebook are exploring the possibilities of mixed reality for gaming, Microsoft is not. Since the first HoloLens, the Redmond firm focuses on business use cases and in particular on collaboration.

Over the past few years, researchers from the American company have developed a technology called "Holoportation". This allows a mixed reality headset user to see in 3D and interact with distant interlocutors as if they were present in the same room.

Different iterations of Holoportation have been presented in the past. During the Inspire / Ready 2018 conference, Microsoft notably presented a demo of the future of work. We had been able to discover the Valorem HoloBeam holographic telepresence technology, one of Microsoft's Mixed Reality Agency partners.

This year, again as part of the Inspire 2019 conference which took place on July 17, Microsoft presented a new version of the Holoportation during the keynote of CEO Satya Nadella.

Azure Corporate Vice President Julia White donned a HoloLens 2 helmet and then generated a life-size hologram of her. This hologram was able to transcribe his oral speech in real time, translated into Japanese while maintaining cadence and intonation.

Microsoft Holoportation translates speeches using artificial intelligence

This impressive demonstration with many promises is based on the combination of several technologies. In addition to mixed reality, several artificial intelligence technologies from Microsoft's cloud services were used: Azure speech-to-text, to transcribe oral speech to audio, and Azure Speech Translation, to translate a speech into a language foreign. These services are based on artificial intelligence technology for natural language processing.

This technology offers many possibilities for the business world. It will be possible to collaborate remotely, with speakers of a foreign language, easily and intuitively. This will greatly simplify cooperation and communication.

However, we can also imagine use cases for the general public, in particular for “social” mixed reality. In 2017, Microsoft bought the AltSpace VR social VR platform and could well integrate its Holoportation technology for more natural exchanges. It would be a way of competing with Facebook and its increasingly realistic avatars ...