Data Analysis with Sound and Image

The Project “SoniVis – Data Analytics using Sonification & Visualization“ at the St. Pölten University of Applied Sciences aims to develop techniques to simplify data analysis by combining visualisation and sound.

Data analysis, such as the analysis of production processes, includes a number of steps that cannot be completely automated. Therefore, the work of human analysts is often required. Approaches to analytics at the interface of automated processing and human interpretation use graphical representation in data visualisation, and audio elements in sonification, i.e. the representation of data with sounds.

“With the increasing volume and complexity of data, both approaches have reached their limits, particularly seeing as the human perceptual capacities serving as the basis for interpreting images and sounds are limited“, says Wolfgang Aigner, Head of the Institute of Creative\Media/Technologies at the St. Pölten UAS.

Novel Combination of Images and Sounds

Although extensive research has been carried out on auditory and visual representation of data, there have been few attempts to combine the two. “Techniques that combine both channels in a systematic and complementary manner, i.e. consider the interrelation between the channels, are missing“, says Aigner.

Previous attempts have often focused on one modality while neglecting the other. An established approach to combining them has yet to be developed. The project SoniVis therefore aims to bridge the gap between sonification and visualisation. In doing so, the researchers hope to develop the basis for a unified design theory for audio-visual analytics.

Design and Experiment

The project’s objective is to outline the range of design options for complementary audio-visual data analysis approaches, examine the practical feasibility in the form of a design study based on the analysis of production processes, and conduct an experiment to compare different approaches.

“We have adopted an interdisciplinary approach that makes our project special: our project team consists of researchers from both fields. Our expected research outcomes include adequate design principles and opportunities for technical implementations. We are working towards developing a design theory for integrating interactive visualisation and sonification for data analysis tasks“, says Aigner.

SoniVis – Data Analytics using Sonification and Visualization

The project SoniVis is funded by the Austrian Science Fund FWF.