Hacklab 2024
Project name : Raíces
Position : Programmer
Workplace : Société des Arts Technologiques, Montréal, Québec, Canada
Residency : Hacklab 2024 by MusicMotion
Category : Immersive experience, multimedia installation, performance
Timeline : 2 days
Skills : Experience design, prototyping, hardware & software testing, soldering, wiring, microcontrollers
Concept
An interactive audiovisual performance inclusive of an audience with physical impairments. Anyone can participate and jam with our cellist and singer, with the help of a midi controller to play sounds, a tablet to add augmented visuals, a vest to feel the vibrations of the music. Every part of the performance interacts with another, creating a unique experience.
Tools I used
-
Arduino Nano ATmega328 Mini Type C, Arduino Software
-
Adafruit Analog Accelerometer ADXL 326
-
AbletonLive, Max4Live
-
TouchDesigner
-
OSC
-
Cables and soldering iron
How it works
We had a real-time interaction between the cello's arch movements and 3D visuals in Unity, with Microelectronics tracking the position of the arch allow our cellist to interact with 3D visuals in real time. Those 3D visuals are integrated in Montreal's visual landscapes with the help of Google's 3D Tiles API.
A Midi controller is put in the center of the space, allowing anyone to improvise with sounds mixed in Ableton Live. The LED lamp above it is controlled by LivePose, a motion tracking tool provided by la SAT.
A subpack is available to allow one person to feel the music not from the ears but from the vibrations of the real-time audio signal.
While playing with sounds, the audience can also be part of the visuals thanks to a Spark AR filter, added to the VJ's roster in TouchDesigner.
All video and audio signals are sent through OSC on a local server.
