Immersive Media Lab

Intermedia Motion Tracking in AR/VR (IMTA)

Duration: 01/01/2020 – 31/12/2021 (Covid-related extension)

The use case Intermedia Motion Tracking in AR/VR (IMTA) establishes a “3D Mesh Motion Capture Storytelling Matrix” by means of motion capturing and motion tracking. To do this, the rooms of the Immersive Media Lab are adapted and operationalised with regard to the spatial dimension and in terms of media technology to ensure the best possible technical integration. By using motion capture suits and 3D scanners and in collaboration with a professional dancer, the researchers define and digitalise choreographed body movements, gestures, motion sequences and “choreographic patterns”, and analyse them as storytelling motion paths. The relationship between body and space in general as well as body posture, elasticity, speed and rhythm in relation to the whole body in space is subsequently archived as a choreographic substrate in the form of a digital action track and then translated into immersive storytelling scenarios.

The body motion tracks recorded and archived with regard to their performance are then put into context and weighted according to narrative structures. The obtained data is stored and systematically refined in a 3D motion database, an “avatar” archive. The archive will serve as a data storage for motion data and can be understood as a “storytelling matrix”. The track left behind by the body in the form of motion tells a story in space – it writes a narrative code in it. This code can be analysed, evaluated and adapted as a versatile layer for different immersive storytelling guidelines. 

Objectives:

  • Establishment of a “3D Motion Capture Storytelling Matrix”
  • Recording of motion capture and tracking data as “3D Scan Mesh”
  • Prototypical examination of bodies in motion (gestures, routines, characteristics, individuality)
  • Development of a motion capture storytelling database

Methods:

  • Research on motion tracking and capturing as storytelling
  • Adaptation of the Immersive Media Lab towards an immersive AR/VR storytelling space configuration
  • Recording of motion data
  • Prototypical examination of bodies in motion
  • Development of a catalogue of parameters and a motion capture database
  • Evaluation of the results in the form of a public presentation (body performance)

The project takes place in cooperation with dancer and choreographer Andrea Nagl. The team thanks Univ. Prof. Karlheinz Essl for his Open Source Library.

Use case lead

  • Deputy Academic Director Digital Design (MA)
  • Lecturer
  • Department of Media and Digital Technologies
P: +43/676/847 228 620

Team

  • Junior Researcher
    Media Creation Research Group
    Institute of Creative\Media/Technologies
  • Department of Media and Digital Technologies
P: +43/2742/313 228 649
  • Junior Researcher
    Media Creation Research Group
    Institute of Creative\Media/Technologies
  • Department of Media and Digital Technologies
  • Junior Researcher
    Media Creation Research Group
    Institute of Creative\Media/Technologies
  • Department of Media and Digital Technologies