Timeframe: March 2024 - December 2024

In its fourth and final segment of VRinMotion (ExperiMotion 4) the project highlights the integration of the classical stop-motion technique, pixilation, in the immersive medium VR. At the core of this undertaking is a collaboration with invited guest-artist Paul Wenninger, whose experience as a dancer, choreographer, and filmmaker with a focus on pixilation (see Trespass (2012) and Uncanny Valley (2015)) provides the project with valuable, artistic expertise. Dramaturgical processes in which the viewers are simultaneously actors and interact in and with the space give the narrative a contemporary choreographic dimension. Raul Maia, a physical theater artist, who worked on several of Wenninger’s films as an actor, is also joining the project.

Building on the groundwork laid by pixilation pioneers like Norman McLaren and Jan Švankmajer (see Neighbours (1952) and Food (1992)), VRinMotion aims to integrate pixilation into VR. This integration is achieved through a workflow involving a 3D reconstruction methods called 3D Gaussian Splatting. While there are previous projects dealing with VR puppetry and 3D-capturing miniature characters such as (Nitsche & McBride, 2020) and (Sjoerd van Acker, 2024), there are some striking differences in the current approach. Apart from the obvious size differences in the recorded characters, the aim is not to create a manipulable, three-dimensional character model, but rather to capture, generate and assemble individual frames from a performance. The desired result would be to seamlessly transfer the tactile aesthetics of pixilation into the immersive and interactive spaces of VR, expanding the expressive and narrative potential of both mediums.

Beyond the inherent affordances of virtual reality, such as unrestricted movement, numerous less evident opportunities arise. For instance, the ability to mix scales through 3D capturing backgrounds or environments at a miniature level while capturing human actors and objects at actual scale, as is common in pixilation. Aspects like these not only have the chance to generate interesting new visual aesthetics in VR but also open up possibilities for exploring the perceptual dynamics of spatial relationships in immersive storytelling.