Brandenburg Labs GmbH, Consensive GmbH, and Technische Universität Ilmenau conducted the joint research project “MULTIPARTIES – Multi-Party Augmented Reality Telepresence System.” The focus is on the development of a 3D communication system that enables realistic online meetings between several people over distances. The two-and-a-half-year joint project is being funded by the German government as part of “KMU-innovativ: Interaktive Technologien für Gesundheit und Lebensqualität”.
Online meetings still differ a lot from real meetings. A circumstance many people experienced during the COVID-19 pandemic. Important interpersonal aspects of communication, such as body language or the ability to whisper to the other person, are lost. Online meetings also lead to a higher cognitive load than real meetings. MULTIPARTIES aims to overcome these limitations and enable spatially separated people to have more natural conversations online.
Using augmented reality (AR), i.e., the computer-aided enhancement of reality perception and spatial audio technologies, the people participating in a MULTIPARTIES meeting will be able to perceive, communicate and interact with each other as naturally as possible. For this purpose, realistic 3D avatars of people with expressive gestures and facial expressions, as well as spatially plausible audio, are seamlessly integrated into the real environment. This creates the impression of actually meeting the other conversation partners. Prof. Karlheinz Brandenburg, CEO of Brandenburg Labs GmbH, sees the future of online conferencing in so-called “AR telepresence systems”: “The goal of such systems is by no means to replace real interpersonal contact. Rather, we want to facilitate digital learning as well as work processes, promote togetherness, and create more diverse opportunities for social participation.”
The interdisciplinary consortium of two Thuringian SMEs and the Technische Universität Ilmenau brings with it many years of experience in their respective fields. The Brandenburg Labs GmbH, the project coordinator, is dedicated to spatial audio reproduction via headphones, which are integrated into the system. These enable a natural reproduction of sound, allowing participants in the virtual meeting to speak simultaneously, for example, or to “whisper in the ear” of other participants.
The Weimar-based start-up Consensive GmbH brings its expertise in 3D representation and interaction of participants in mixed reality into the project. An important starting point in the project is, for the start-up, the use of “digital twins”. This involves digitally recreating the participants’ real-life environments and recombining them to create a shared space.
The results of the project were presented on August 21, 2025, at the Eigenheim Gallery in Weimar. During the presentation, the guests and representatives of the project partners were able to see for themselves how MULTIPARTIES meetings create the impression of actually meeting with the other participants. For the demonstration, an online meeting was held with groups of participants from Weimar, Ilmenau, Aachen, and Lübeck. The participants met in a virtual space composed of representations of the real-world rooms of all participants. Unlike in conventional online meetings, they were not connected via camera feed but wore VR headsets to experience the space immersively.
In this environment, they were represented as avatars and could move freely, interact, and even shake hands within the virtual space. Through headphones, they received an extremely realistic reproduction of spatial sound in real time, allowing them to form small groups or lean in toward a conversation partner and whisper to them. The system is based on the VR4More platform from Consensive GmbH and the Deep Dive audio technology from Brandenburg Labs.