-nature -data -space -sound
Sciart project on sound design | sonification | scientific data | 3D design | virtual worlds
Which kind of sound do drosophila larvae make? And what about zebrafish embryos? How do honeybees navigate through a space while only seing and hearing smells? The project -nature -data -space -sound is about the experimental use of sound [noise/voice/sound/music] to enrich interactive virtual worlds.
Based on experiments conducted by researchers from the Centre for the Advanced Study of Collective Behaviour, students from the Merz Akademie developed interactive virtual worlds for honeybees, drosophila larvae, and zebrafish.
Exploring the World of Larvae, Bees, and Zebrafish in Virtual Reality
This project was a student-led effort to explore the unique soundscapes of different species of larvae, bees, and zebrafish in virtual reality. Using data provided by scientists, the students created immersive virtual environments that represent the natural habitats of these species. The focus of this project was on the sonification of the species' behaviour and sounds, as well as the environment that they are integrated in.
By immersing users in these virtual environments, the project aims to give them a deeper understanding and appreciation of the complexity and diversity of life on our planet. Additionally, it provides a unique opportunity for people to experience and learn about species they may never have the chance to observe in real life.
Project Results
Bee Maze by Cora Marie Lenz
The Bee Maze is an interactive VR experience where the user is a bee that gets lost and has to find its way home in the dark.
You flew to the city because there are more flowers there towards the end of the year. But unfortunately, you get lost in an apartment. You get stuck in a glass and everything goes black. When you wake up again, you are lying on a flower in a forest far away from home. Now you have to find your way back home.
It is now evening and pitch black. That means you can only orient yourself by smells (sounds) and crawl. You should be coming home soon, but keep an eye out for enemies.
Underwater by Nicoleta Domenti
This VR experience showcases the natural habitat of zebrafish and provides a look at the different stages of its life: embryo development, juvenile stage, and adult stage.
In this VR experience, users will be transported to a virtual underwater environment where they can explore the different stages of the zebrafish's life. The graphics provide a depiction of the zebrafish's natural habitat, allowing users to compare and contrast the different stages of the zebrafish's life. The soundscape of the underwater environment is enhanced with sonification, providing a more engaging and sensory experience.
Larvae_sound by Jonas Langenbrinck & Alonso John
For this project, we combined the insight we got on how experiments are held with larvae and sonificaiton into a simple yet cohesive virtual reality experience.
To accomplish this, we decided to use data from Dr Katrin Vogt on larvae movement and replicate their experiments in virtual reality. We tried to stay as close to the original experiments as possible, while combining them with our idea of larvae sounds in different experiments. All our sounds have been made without using the help of sound libraries.
The course was a great opportunity for students to gain hands-on experience in virtual reality development and sonification, while working with scientists to understand the intricacies of the species behaviours. It was a true collaboration between students, scientists, and the teachers, working together to create a unique and innovative experience, both educational and entertaining. The result provides a valuable learning experience for all involved.
Larvae_sound by Jonas Langenbrinck and Alonso John
Work on the project
The project started with a kick-off visit at the CASCB in Konstanz in October 2022. During the project, the students and researchers exchanged ideas. For the final presentation, the researchers from Konstanz visited the Merz Akademie.
-
Project start, Copyright: E. Böker, CASCB -
Hernan Morales Navarrete shows the lab where he conducts the zebrafish embryo experiments. Copyright: E. Böker, CASCB -
Hernan Morales Navarrete shows the lab where he conducts the zebrafish embryo experiments. Copyright: E. Böker, CASCB -
Katrin Vogt and Akhila Mudunuri perform behavioural experiments with drosophila larvae and track the movement. Copyright: E. Böker, CASCB. -
Giovanni Galizia studies how odors are coded in the insect brain, using several species (mostly honeybees and fruit flies). Copyright: E. Böker, CASCB -
Warm welcome at the Merz Akademie. Copyright: E. Böker, CASCB -
Project presentation. Copyright: E. Böker, CASCB -
Project presentation. Copyright: E. Böker, CASCB -
Collective work on the projects. Copyright: E. Böker, CASCB
Involved People
Course leaders: Mario Doulis, Jörg Frohnmayer and Michael Baur, Merz Akademie
Researchers: Giovanni Galizia, Patrick Müller & Hernan Morales Navarrete, Katrin Vogt & Akhila Mudunuri, CASCB/University of Konstanz
Students: Nicoleta Domenti, Alonso John, Jonas Langenbrinck, Cora Lenz, Merz Akademie
Organization: Elisabeth Böker, CASCB