Jakob Kudsk Steensen’s The Deep Listener combines art, ecology and technology

STIR speaks to New York-based Danish artist Jakob Kudsk Steensen, whose project The Deep Listener was selected from over 350 global applications to the Serpentine Augmented Architecture commission, a new programme developed in collaboration with Google Arts & Culture and Sir David Adjaye OBE. It looked into the future and the production of new realities through architectural experimentation, and a rethinking of spatiality in a non-physical world.

Steensen’s project is an immersive augmented reality (AR) and spatial audio guide to the specific living ecosystems that live within Kensington Gardens and Hyde Park, creating new networks of audio-visual interaction with the environment.

04 min watch

Sukanya Garg (SG): How did you first begin working on projects that lie at the intersection of ecology and technology?

Jakob Kudsk Steensen (JKS): I belong to a generation where interactive technologies such as video adventure games, interactive novels, and artistic virtual worlds became widespread because of advances in commercial and consumer friendly 3D technologies. When I was 10, I build my first computer because the 3D graphics of the game Unreal enthralled me completely. I loved drawing, art, and comics as a kid, and when we got our first computer in our home, I was fascinated. The first video game that really got me interested in technology was Fallout 1. It came out in 1997 when I was 10-years-old. It imagines an alternative post-nuclear timeline where you explore California, 200 years after a nuclear disaster that happened during WWII. You could play the game entirely by talking with virtual characters and visiting different towns, people and landscapes in the video game. The poetics, narratives and virtual landscapes of that game had inspired my sense then. Video games have become a lot more commercial since then, and a bit cliché and stereotypical often. So instead of working in video games or as an animator I became an artist.

At the same time, as I grew up with video games, I also attended a Rudolf Steiner Kindergarten and an outdoor school in Denmark for the first years of my life as a kid. There were no toys, everything took place outside. So while I grew up being fascinated by virtual landscapes and imaginary spaces, I also spent a lot of time climbing, diving, and in the woods. It is a life-long passion for me and part of my identity. My primary interest with technology is not about technology itself. I am interested in how technology can be used to create strongly poetic and sensory virtual landscapes, which we can share, explore and experience. Today I am interested in how technology can be used to enhance our senses by letting us experience things in nature that we otherwise cannot perceive.

My primary interest with technology is not about technology itself. I am interested in how technology can be used to create strongly poetic and sensory virtual landscapes, which we can share, explore and experience. – Jakob Kudsk Steensen

SG: What inclined you to work on the project The Deep Listener? What was the original design or artistic inspiration?

JKS: The Deep Listener is an AR project that uses your phone. AR has become increasingly commonplace but I do not like phones too much, and I did not at first want to use phones to create cell-phones, because they appeal too closely to modalities we are used to sensing the world with. So, over the past couple of years, I started to think about how AR and your phone can change sounds and respond to your bodily movement through space - the pace of your walk, the movement of your hand, and how it can make you explore places otherwise perceived as commonplace.

So, with The Deep Listener, I created a design where people are able to listen to species in Hyde Park in ways they otherwise cannot. By moving physically you speed up or slow down recordings of insects, birds and plants in the park. So to engage with the work you need to listen, look at the landscape, and move physically throughout a fairly large area in the park. It will take you about 40 minutes to one hour to experience the entire work.

SG: The Deep Listener is a poetic manifestation of using technology to preserve and present the beauty of the natural, yet with a sci-fi orientation. Is there a basis for this juxtaposition? What do you hope to bring forth through this exhibit?

JKS: The Deep Listener follows a formula the video game Fallout also followed. It is based on real places and actual natural histories, but it changes them to create new imaginations and ideas about what it means to be human living with technology and nature today. In the past, my works have had certain dystopian aesthetics, but I want to move away from that. I want increasingly to make people feel inspired and curious about different species and places.

To do that, The Deep Listener is based on actual audio recordings of species and photographs of their skin, which is remixed to create playful virtual creatures you can interact with. At the same time, you have to go to specific locations in the park where the species live. In the work there is also text that tell stories about the different species and why they are in the park. The best comment I got was from an assistant curator at Serpentine Galleries, who told me that she had never noticed the presence of blue damselfies in the park before she started working on The Deep Listener. I have heard this from many now. I have heard how the work is playful, and brings new perspectives on existing species in the park, in spite of its ’sci-fi‘ orientation. To me, I simply interpret and portray species as an artist, a bit like a painter or a writer might do it.

I have heard how the work is playful and brings new perspectives on existing species in the park, in spite of its "sci-fi" orientation. To me, I simply interpret and portray species as an artist, a bit like a painter or a writer might do it. – Jakob Kudsk Steensen

SG: What is your creative process like?

JKS: I start by researching natural histories and stories about species within the greater landscape an exhibition is going to be in. In this way my work is quite location specific. I then contact field biologists to learn about their perspectives on a place. Their stories and work informs how I then start working with different species in a place, photographing insects, plants and get audio from them. I then remix the material digitally to create entire worlds that people can experience. 

SG: What kind of technology was involved in the creation of this project?

JKS: The Deep Listener was made with Unreal Engine, a software used to create video games. I started using this as a teenager because it came together with the game Unreal that inspired me early on. I also worked with sound artist Matt McCorkle, the Museum of Natural History in London, and a great team at Serpentine Galleries. Programmers Ivaylo Getov and Troy Deguid worked on the programming, making sure the AR functioned as it should. Artist Rindon Johnson narrated the text I wrote. So a project like The Deep Listener does not only use technology to be created. It involves many people with different technical and artistic interests and skills. 

SG: What kind of response has the experiential project evoked in viewers?

JKS: The response has been amazing. One that surprised everyone was how it felt using the work collectively as a group. When 50 people use the work at the same time outside, in Hyde Park and Kengsington Gardens, their phones become collective speaker systems that exchanges audio of species in the park. This social and performative aspect is inspiring me a lot right now to do more work that way - ’performing‘ collectively in landscapes through AR. The response has been very good, with people finding the work informative, playful and grand. I think there have been made many AR works before, but nothing at a larger scale like this.

SG: Could you talk about the spatial and relational aesthetics of this work and the process of developing them?

JKS: People interact with specific species and locations in the park, so AR is used in The Deep Listener to bring new and imaginative perspectives on places people normally look by. Many species are often overlooked even though they are beautiful and have interesting natural histories. Why are they there? What biological rules do their lives follow? How do they sound? I think The Deep Listener has relational elements between people and the park, and between people using the work, and people walking past someone using it, to an extent that is far greater than I had anticipated. It is very hard to document how powerful it can be.

The installation The Deep Listener was on view at the Serpentine Gallery from July to September 2019.