Looking to the future: Learning on the test bench with HoloLens and Mixed Reality

The introduction of smart glasses in the work environment offers many new opportunities for designing future work processes. VIRTUAL VEHICLE is working with Microsoft HoloLens as part of a feasibility study. We use the Microsoft HoloLens to display non-visible processes and states virtually on an air condition test bench.

The HoloLens, a Mixed Reality (MR) device released by Microsoft in March 2016, can enrich a user’s field of view with additional information. In contrast to virtual reality (VR) glasses, the user continues to see the real environment (transparent see-through device). The Microsoft HoloLens can be controlled by gestures, language and a clicker.

In a feasibility study by VIRTUAL VEHICLE, the Microsoft HoloLens is used to show non-visible processes and conditions (eg flow behavior, temperature and pressure) on an air condition test bench in the visual field. The aim is to create a better understanding of how the test bench works and to make it easier to learn the coherencies on this test bench. Based on the first design data (CAD) of this test stand and the thermodynamic simulations based on it, a HoloLens app was developed in this study, while at the same time the test rig was also set up in reality.

The HoloLens App includes two different operating modes:

Operating mode 1: The test bed as a virtual model within the field of view

In virtual mode, the entire test bench is displayed as a virtual model in the field of view. With this first step, it is possible to study the test bench and the processes on the test bench based on the design data, even at a very early stage of the test bench development, while the real test bench does not yet exist. In this mode, this virtual model is already displayed in its original size. In a learning or training scenario, new employees can already be trained on the future test bench at this early stage. Individual training scenarios with different temperature distribution simulations can be played through. For example, from the perspective of occupational safety, it is important for the user to know where the cold and hot areas, represented by red and blue waves, are located on the test bench.

Operating mode 2: Real testbed with mixed reality information

Once the real test bench has been completed, it can be switched to the second operating mode of the HoloLens app. In this mode, computer-generated information is augmented directly on the real test bench. Thus, real measurement data or information from simulations can be visualized directly in the field of view. Thus, very different, even dangerous situations, can be played through directly on the test bench.

As part of the development of the HoloLens app, a generic workflow was also created, and a tool chain was developed to display design and simulation data as automated as possible in the Microsoft HoloLens.

Based on the CAD and simulation models of the test bench, several different conversions were required to provide a suitable presentation format for the Microsoft HoloLens. On the other hand, a form of presentation had to be developed to dynamically represent pressure and temperature conditions of the test bench, in which e.g. the colors of the tubes can be adapted to the current measurement or simulation data.

Only in the very last step was then developed the HoloLens – App itself. Based on this approach, the tool chain can be efficiently adapted to different representation devices.

VIRTUAL VEHICLE is using the developed tool chain very successfully in other mixed-reality projects.