Immersive Media Lab

AR Interfaces for the Industrial Internet of Things (AIIT)

Duration: 01/01/2020 – 31/12/2021 (Covid-related extension)

In the use case "AR Interfaces for the Industrial Internet of Things” (AIIT), the application of augmented reality technologies (AR) was prototypically implemented in a modern, industrial production environment to support staff during working and maintenance processes. Cyber physical systems – machines that have a digital twin in addition to the real-world equivalent – allow for the virtual real-time representation of their characteristics. The objective is for augmented reality to give people faster insights into production processes through the local visualization of data in places where there are usually no displays.

In the company Neuman Aluminium, the researchers found an industrial partner that granted them access to both a real production environment and the data of an Industrial-Internet-of-Things (IIoT) infrastructure. An important component in the production of Neuman is a melting furnace for the processing of aluminum scrap. In case of an incident, it is crucial that staff members gain access to machine data in a fast and uncomplicated manner.

For the use case AIIT, an analysis process identified an application on a glass-based AR display (Microsoft HoloLens 2) as the most suitable technology because contrary to mobile AR (smartphone, tablet, etc.), it keeps the user's hands free. Great attention had to be paid to intuitive user interaction. Moreover, it was important to make sure that virtual elements would not interfere with the field of vision during interaction with the real environment.

In order to achieve a suitable visualization of the machine and data in terms of location using the HoloLens 2, several subtasks had to be solved in the AIIT use case:

  • Creating a data interface between the HoloLens 2 and the industrial network
  • Preparing a 3D model of the melting furnace based on CAD data for visualization in the HoloLens 2
  • Exactly matching the virtual 3D model to the real melting furnace on site for the visualization
  • Visualizing machine data and highlighting relevant assembly units in AR at the machine
  • Drafting and implementing the application logic for AR applications based on a user story

The AIIT prototype is made up of two parts: a miniature model of the furnace that can be placed in front of the user and a real-size AR overlay to complement the actual furnace. Both modes show similar AR extensions but are suited for different situations. The miniature model has a high potential for the onboarding process of staff members as it offers a quick overview of most major components and how they interact. Users can walk around the model and examine the various components at their own pace. To help them in their exploration, a storytelling module offers detailed descriptions of the various parts that can be viewed either chronologically or randomly.

The other part, a life-size overlay of the real furnace's production data, is displayed on the actual site and has potential for maintenance and repair applications. As the interior of the furnace is not immediately visible, the AR glasses can be used to visualize the data directly at their origin. In both cases, however, the abstract machine data had to be converted into intuitive visual metaphors.

Use case lead

  • Lecturer
  • Department of Media and Digital Technologies
P: +43/676/847 228 250

Team

  • Junior Researcher
    Digital Technologies Research Group
    Institute of Creative\Media/Technologies
  • Department of Media and Digital Technologies
P: +43/2742/313 228 684
  • Junior Researcher
    Media Creation Research Group
    Institute of Creative\Media/Technologies
  • Department of Media and Digital Technologies
  • Research Assistant
    Media Creation Research Group
    Institute of Creative\Media/Technologies
  • Research Assistant Institute of Creative\Media/Technologies
  • Department of Media and Digital Technologies
  • Research Assistant
    Center for Digital Health and Social Innovation
  • Department of Health Sciences