Bridging the real and virtual: Exploring the responsive installations of arc/sec Lab

A line of artistic inquiry that has long persisted now is the nature of relationship that the physical shares with the digital; it’s push and pull and the interstices wherein the line between the two begins to blur. The arc/sec Lab, based out of the University of Auckland, sits at the forefront of the many creatives that are involved in such explorations, and as Associate Professor for Design and Design Technology, as well as the Founder of the lab, Uwe Rieger tells STIR, “The arc/sec Lab explores cyber-physical design principles with the aim to create new forms of reactive architecture and responsive environments.” The terminology “cyber-physical”, as Rieger uses it, refers to interlinked systems wherein the physical influences the digital, and vice-versa. Rieger continues, “While it is based at the School of Architecture and Planning, the Lab operates in a much wider, trans-disciplinary context. arc/sec works with a variety of artists as well as computer scientists and even medical professionals.” Rieger is joined by Research Associate at the arc/sec Lab and Co-director of arc/sec Solutions, Yinan Liu. She continues, explaining the raison d’etre of arc/sec Solutions, that as the Lab itself is focused on experimental and artistic design research, it is arc/sec Solutions that functions as a link to connect the Lab and its wider university context with industry and community. Here, Liu works towards translating basic research into customised applications.

01 min watch

While a cursory glance at the arc/sec Lab’s phenomenal installations would direct one to believe light is the primary driver here, their works are in fact far more complicated than that. Rieger explains, “We are fascinated by this combination of light and design. Light projections are an effective way to bring dynamic digital information to a physical architectural scale.” Liu continues, “With our projects we are exploring a new form of cyber-physical architecture; hybrid constructs in which the traditional separation between the computer and the physical environment is being dissolved.” The architecturally sublime installations of the arc/sec team are meant to be reactive environments in real-time, wherein data is connected to physical features, and human interaction operates without the use of interfaces. Understandably, the Lab carries out a great deal of precision engineering, and their installations could have been quite intricate and esoteric. However, Rieger and Liu pay a great deal of attention to fabrication, ensuring that their installations may travel long distances and be setup with ease at different locations. To this end, they generally use lightweight materials such as carbon fibre and aluminium to construct parts that have been optimised in form and function after a rigorous R&D process. Ensuring that the systems they create can withstand real-world conditions has always been a point of focus for the arc/sec team.

02 min watch

Discussing their process, Liu tells STIR, “When we work on our hybrid structures, we pay equal attention to the physical and digital components throughout the whole process of the design, fabrication and inhabitation. Digital content and physical constructions undergo a calibration process in order to be merged precisely with each other. We create a feedback loop, consisting of sensors meant to monitor the environment and user behaviour, and this information is sent to a network of computers which process it in real-time. The processed result is then pushed back out to the physical world and mediated through actuators such as projectors, sound systems, ARVR headsets or robots. Creating a closed loop out of these components forms the basis of our cyber-physical design.” Rieger says that their project LightScale II entailed the team looking closely at the navigation of a haptic-digital space. The project was built through a 20m long tubular carbon fibre structure, upon which holographic images were created through a precise series of projections. The structure’s body was created to resemble a mesh. Here, using a highly sophisticated motion tracking system, the object would be enabled to respond in real time to the touch of a visitor, allowing them to interact with a variety of audio-visual narratives.

After subsequent projects, LightTank and LightWing II, the team has recently undertaken the fabrication and installation of its latest project, LightSense. Discussing this, Liu says, “We are a bit held back by the recent lockdown in Auckland, but we are about to finalise this cyber-physical architectural system which truly operates between the material and the networked world. LightSense is an AI-XR installation that augments a 12m long suspended structure with 3D anaglyph projections and spatial sound. The structure is placed in balance by motorised counterweights. The brain of the sensing system is an integrated AI component, which has been trained to learn 60,000 poems.” This allows LightSense to engage, lead and even sustain a coherent oral conversation with its audience members, and to bring it all together, the emotional tenor established during these conversations is used to actuate architectural behaviour in LightSense, which instigates immersive pavilions themed around love, hate, curiosity and joy.

Thinking about the future of arc/sec, Rieger begins by taking stock of the past. He says that we, as a culture, are currently entering the world of spatial computing. This, in a sense, constitutes a radically new paradigm wherein the computer, material and user are interlinked. However, while a wide range of digital spatial technology has been available for some time, little is known of how to design these hybrid worlds. This calls upon transdisciplinary investigations and artistic explorations to find radically new applications and principles. Rieger is excited to be treading this uncharted territory, and concludes, saying “The arc/sec Lab focuses on cyber-physical design principles that allow us to leave behind the reductive understanding that architecture is a fundamentally static craft. Our new project LightSense points towards the creation of a new breed of buildings that are responsive in real-time. We will continue to explore AI not for the optimisation of building performance or the substitution of human action, but to design architecture that is capable of improvisation, and that may accompany us as a creative partner in our lives.”