The Vision

Performance Sonification is pushing the boundary of motor learning and human performance. We work at the cutting edge of neuroscience, psychophysics, sport engineering, and introspective phenomenology. What makes us unique is how we combine these perspectives, mining substantive insights from each to push our vision forward. That vision sees audition (hearing) as an information channel that can be used to bypass the limits of natural proprioception to improve bodily awareness, putting us in better contact with our bodies. We are prototyping, researching, and testing lightweight wearable sensor systems that convert motion into sound with the aim of improving motion control and sport performance.

What’s Known // What We Do

Sonification is the conversion of data into sound. Think of a Geiger counter clicking to signal radioactive particles. Much research has already been done on the sonification of motion data, demonstrating its effectiveness in sport and rehab. Users receive precision, real-time data about body position through changes in a tone’s pitch and volume. This often allows for improvements in motor control. Why? The brain is deeply plastic and deeply multimodal. If a sense provides usable information, the brain often will learn to use it. Bodily awareness and motor control is not limited to just the stretch receptors in your muscles and joints. The brain naturally uses visual and auditory information as well. Many common examples demonstrate the deep connection in the brain between sound and the body, including the obvious way in which we use music beats to guide dance or the common experience of tactile “tingles” in ASMR in response to sound. We aim to push the boundaries of research by sonifying explosive sport movements, like pitches (baseball), passes (football), squats and cleans (weightlifting), starts (cycling), and high jumps (track and field).

Prototype Wearables

Wearable physical devices that track body motion through 3D space and generate sound.

Design Sonifications

Innovative mathematical representations of motion. Neuroscience-based mappings from motion into sound.

Test and Research

Collaborations with leading university research labs to test and develop effective sonifications for sport performance.

Science

Many applications today treat sonification as a form of expression or art: Sonification applications often use sensed motion to control a melody or piece of music, or give users a rhythm to follow. These apps create something artistic, but abstract away from the fine details of motion and don’t give users an improved stream of information about body position. Our sonifications are a form of precision sensory substitution, or sensory augmentation. Natural proprioception is slow, suppressed, and inaccurate. We aim to bypass natural proprioception and deliver accurate position information to the brain faster, using sound. The aim is to give users a way literally to perceive their movement better than they normally would.

Current Partners

We are partnering with the Multisensory Integration Laboratory at York University to test and develop our sonifications, as well as explore how technique affects sport performance. We aim to use cutting-edge motion tracking to explore how different motion paths affect sport performance, and how sound affects motion paths.