'Crush'

Composer and project leader: Natasha Barrett
Spatial-audio programming: Natasha Barrett
Interactive sensor interfacing and programming: Oyvind Hammer and Svein Berge
PGP and external collaborators: Karen Mair, Steffen Abe, Alexandre Schubnel.
Crush is an interactive sound-art installation exploring the microscopic forces released during the process of crushing rock. The installation focuses on two research projects at PGP in Oslo: 3D numerical simulations of grain fracture and fault gouge evolution during shear (the work of Steffen Abe and Karen Mair), and the study of real acoustic emissions from granite, basalt and sandstone under compression (the work of Alexandre Schubnel).

Crush consists of 3D electroacoustic sound, a loudspeaker array, wireless headphones, a motion tracking system, still images and a real-time 3D video projection. In this installation, the audience can move through a virtual, immersive space, experiencing the dynamics of deformation from "inside" the rock.

This first version of Crush is located in two attached spaces: a dedicated room houses a loudspeaker system and video, while in the foyer the visitor wears a motion tracking system and wireless headphones to interact with the work.

The interactive system
For Crush, Natasha Barrett, Svein Berge and Oyvind Hammer designed a motion tracking system that allows each user to physically navigate through the 3D sound composition. The user wears a head-mounted Nintendo Wii-Remote with the Motion Sensor Plus. This unit contains 3D accelerometers, gyroscopes and an infrared camera. Seven targets with infrared light constellations and a picture surround the interactive space. Motion data are sent to a computer over blue tooth to provide the userÕs position and direction of view. This information is used to modify the spatial sound image. The 3D sound is rendered using head-related transfer functions (HRTFs) in wireless headphones. In the area defined by the targets, two people may interact with the work at any one time.

The loudspeaker array
In a dedicated space, an eight-channel loudspeaker system plays the 3D sound-field encoded with higher-order ambisonics based on one of the two interactive streams. This gives a higher quality and more precise sound picture than that heard over the headphones. If no one is currently interacting with the work, a preset cycle begins.

Video projection located in the same room as the loudspeakers.
A 3D interactive virtual world is created with the open source gaming software Irrlicht. As the user navigates through the real space these movements control the camera view inside the virtual world containing the following elements:
- The targets corresponding to the real interactive space
- A cloud derived from the dynamic rupture propagation (3D co-ordinates and magnitude) of a granite sample from La Peyratte (France) under compression (courtesy of Schubnel).
- 80 images derived from X-ray CT scans of sandstone (from Schubnel), one picture every mm, to reconstitute the full fracture in 3D, where the projected location, size and brightness of each image is controlled by the location, amplitude and frequency of the loudest component of the 3D sound-field. In this way the sound may be regarded as a mediator between visualised science and art.

The process
Work on Crush began with accurate sonification of data from simulations and real acoustic emissions. The results of this process were used to guide the correlation between 3D sound and the patterns and processes found in the geological systems. In the final work, micro-scale processes are enlarged into a dynamic system audible through sound colour, texture, shape and spatial geometry.

Sound, transformation and mapping
The real acoustic emissions were recorded at a frequency of 4 MHz. Three of these recordings from different rock samples were transposed into the audible range and used as sound material in Crush. In addition I made my own recordings of rocks being crushed, scraped and generally destroyed, using ultrasonic transducers capturing up to 90 kHz as well as recordings in the audible range.

Parameters such as sound type, volume, transience, frequency, filter, pitch shift, grain, continuation, resonance and spatial location were mapped in various ways to the source data parameters such as fracture magnitude, spatial displacement and particle source cluster origin.

Science to art
The two data sources are at odds not only in modeling versus real observation but in the way in which the forces were applied. The unfolding temporal patterns revealed clear differences in the underlying processes. These differences proved important in Crush.

Making audible sense of enormous data sets
The simulations involved hundreds of thousands of fractures and necessitated data reduction to make the processes audible. Different magnitudes were mapped to different sound types. An enormously complex initial rupture was followed by an exponentially decreasing density of information uniformly distributed over the 3D space. Likewise, to track the motion of all particles would render the process inaudible. Instead, 30 particles were chosen at random. In contrast, data from the real emissions involved a more moderate number of fractures, a less uniform spatial-temporal distribution where the rupture occurred after a built-up of compression energy, and all information was used in the sound mapping process.

Time and space
Both research projects involved data grouped into a number of time steps. In Crush this temporal axis is malleable Š some layers of sound span 10 minutes, others a fleeting few seconds. During 10-minute expansions, temporal randomisation was applied to data within each time step to give the impression of continuity. Spatial information in the scientific data likewise finds a malleable scale in the artwork, ranging from one to many meters. Changing the temporal scale of the sonified simulations did little to improve how we aurally perceive the process. The experimental data yielded a contrasting result: a shorter timescale was useful in aurally revealing the processes taking place.

In Crush the fracture simulations occupy 10-minute 3D layers where the listener can navigate through the crushing rock structure. The acoustic emissions occupy time spans from 10 seconds to 2 minutes, navigable in a similar way. The tracking of individual particles and their fracturing from larger particle clusters occupy time spans of one to two minutes. The 10-minute layers play constantly until an interacting person looks directly at a target from close range. At this point, a shorter duration process will be triggered while the 10-minute layer is paused. There are seven targets, each with their own sounds. To discover all elements takes approximately 20 minutes.