When environments begin to sense us, who gets to decide what we feel?

Imagine entering a room after a difficult conversation. Your heartbeat is elevated. You are unusually quiet. The room notices. Its lighting softens and the soundscape shifts. Perhaps a digital assistant offers reassurance. The response appears thoughtful: the environment has detected a signal and acted.

But what if you are not distressed? Perhaps you are angry and need space to think. Perhaps your silence is deliberate. The room has detected something real. Somewhere between detection and response, however, it has made a decision about what that signal means. Yet the reason for the change remains unexplained, and you were given no opportunity to choose.

As sensors and AI enter the environments we inhabit, designers are encouraged to create spaces that anticipate our needs. Yet a responsive environment is not necessarily an understanding one. The more consequential question may be not what a space can sense, but who has the authority to interpret what it senses and when that interpretation permits action.

When a signal becomes a story

A physiological measurement is not an emotion. It is an observation that acquires meaning through context. An elevated heart rate might accompany fear, excitement, physical exertion or something else entirely. Stillness might mean concentration, exhaustion, resistance or comfort. A signal may be accurate while the story attached to it is wrong.

In a 2019 scientific review¹, Canadian-American psychologist Lisa Feldman Barrett and her co-authors (Ralph Adolphs, Stacy Marsella, Aleix M. Martinez and Seth D. Pollak) challenged the idea that particular facial movements reliably reveal specific emotions across people and situations. Though their work concerns facial expression rather than directly examining responsive rooms, its broader warning is against treating ambiguous human signals as self-explanatory emotional labels.

A responsive system may appear to follow a simple sequence: observe a change, infer its meaning, select a response. But these are separate judgments. Even a plausible interpretation does not itself authorise intervention. A system can be technically responsive while remaining socially presumptuous.

The quiet authority of space

Architecture has always shaped how we feel. Scale, acoustics, light and material can invite intimacy or exposure, concentration or withdrawal. The difference with an adaptive environment is that the space may begin interpreting its inhabitant in return. Rather than offering conditions to experience, it may silently decide what an experience means and modify those conditions accordingly.

A useful counterpoint is Random International’s Rain Room², first presented in 2012 at the Barbican in London. Its falling water creates an intense experience of the sound and smell of rain while leaving the visitors dry within a continual downpour. The installation’s response is tied to a visitor’s presence and movement, not a claim about their feelings. That distinction matters: adapting to an observable position is a different design claim from inferring distress and changing an atmosphere on someone’s behalf.

Consider a room that dims its lighting when someone grows quiet. The adjustment may be welcome. Or it may interrupt a moment of concentration. The change itself is not necessarily the problem; its unexplained arrival can be. Consent to sensing is not automatically consent to interpretation, and consent to interpretation is not permission for every possible response.

When the environment changes without asking

The distance between what an environment detects and what a person actually experiences became central to my own design research. While developing Therapeutic Realms, an immersive-design investigation for my MA in Game Development Design at Kingston University, London, I began examining what happens when responsive environments change before their occupants understand why. My exploratory research involved 25 participants through interviews and questionnaire responses. Some accounts of unexpected changes in virtual environments described confusion, overwhelm or a loss of control. These were qualitative observations, not clinical outcomes.

In my interview notes, P03 questioned what a system had detected if the environment changed without explanation and preferred being prompted before an adjustment. P11 wanted to feel that he was making a decision, rather than having it made for him. P17 associated unprompted changes with a loss of predictability and control. In a synthesis of the same participant material, the concern was not necessarily the adjustment itself, but its arrival without warning.

These accounts cannot establish what all participants, or future users, would prefer. But they reveal three useful design concerns: transparency, agency and predictability. Together they sharpen the question: how can a space respond without taking ownership of someone else’s experience?

From virtual immersion to familiar space

As part of the exploration, I examined existing VR experiences and biofeedback-related design approaches. This exploratory work raised a design question: repeated, unexplained transitions could disrupt rather than support reflection. I also consulted one therapist and one psychiatrist. In those consultations, sudden changes were discussed as a possible source of confusion that could make it harder to reflect on what triggered a response. These discussions informed the concept; they did not constitute a clinical evaluation.

The resulting move towards mixed reality was a design hypothesis, not evidence that it reduces anxiety or improves clinical outcomes. Keeping familiar physical surroundings visible while introducing a simulated social encounter offered another possibility: rather than replacing the setting when someone becomes uncomfortable, the design could preserve the context in which reflection takes place.

Holding a moment instead of correcting it

Building on my original MA research at Kingston University London, I subsequently developed Therapeutic Realms: The Living Scene, a proposed therapist-guided mixed-reality experience. While the initial investigation explored virtual reality and its potential for supporting therapist-guided reflection, The Living Scene extends this design direction into mixed reality, where familiar physical surroundings remain visible alongside simulated social interactions. This shift emerged from a central design question: could an immersive environment support reflection without removing the person from the context in which that reflection takes place?

Imagine rehearsing a difficult conversation with a colleague. You want to raise an issue but hesitate, unsure whether to speak or stay silent. The proposed experience is intended for therapist-guided sessions in which individuals can revisit and reflect on challenging interpersonal situations, such as workplace disagreements, difficult conversations or moments of social uncertainty, such as workplace disagreements, difficult conversations or moments of social uncertainty. Within a mixed-reality setting, the participant encounters a simulated social interaction while remaining aware of their physical surroundings. Rather than interpreting their emotional state automatically, the experience allows the therapist and participant to pause, examine and explore alternative responses together. The aim is to create space for reflection and choice, rather than prescribe how someone should feel or respond.

Within the mixed-reality experience, Hold the Moment is a proposed interaction designed to preserve the context of a difficult social exchange. The participant remains aware of their physical surroundings while a simulated person and conversation are layered into that space. Suppose a difficult moment arises; if the participant hesitates after something the simulated colleague says, the therapist can pause the interaction at that exact point. The conversation stops progressing, but the participant can still see the colleague and the room around them, preserving the context of what has just happened.

Instead of the environment automatically interpreting the hesitation and changing itself, the pause creates an opportunity for the therapist and participant to reflect on the moment together. Physiological signals, if available, could be visible privately to the therapist through the proposed Director Desk as additional context, rather than as a diagnosis or instruction to intervene. The participant would also retain the ability to pause or withdraw from the experience.

Through Unfold the Scene, the therapist and participant could explore three hypothetical continuations: speaking up, remaining silent or seeking support. Speaking might invite clarification or disagreement. Silence might leave uncertainty unresolved or provide time to reflect. Seeking support might bring in another perspective. These are not predictions, prescribed decisions or claims about what another person will do. They are possibilities made visible for discussion.

The spatial decision matters. The original moment remains available as a reference instead of disappearing beneath a supposedly corrective emotional landscape. The participant can explore alternatives without being told which interpretation is true. These interactions exist as sketches and design proposals, not as a functioning prototype; no therapeutic efficacy is claimed.

Designing the possibility of restraint

Questions raised while developing Therapeutic Realms later led me to formulate Interpretation-Gated Adaptation (IGA) as a broader design framework. Therapeutic Realms raised a recurring design tension: detecting a physiological or behavioural signal does not necessarily mean understanding what that signal means, nor does it automatically justify a response. IGA develops that question beyond the therapeutic context by proposing an Interpretation Gate between sensing and action.

It asks what was observed rather than inferred, which alternative explanations remain plausible, what consequences might follow and whether permission exists. A system could then move towards one of four response modes: ACT, ASK, OFFER or WITHHOLD. This is a design proposition, not a validated technical or clinical system.

Not every adaptive action needs the same procedure. A previously configured, reversible accessibility adjustment is different from an intervention based on an uncertain emotional inference. Risk, prior agreement, reversibility and context should determine whether a system acts automatically, asks a question or does nothing. A prompt is not inherently humane if it interrupts constantly; restraint must also include knowing when not to ask.

The ambition, then, is not to make environments passive. It is to make their authority legible and proportionate. The best response may be an adjustment, an offer, a question—or a deliberate decision to leave the room unchanged.

The freedom to remain unread

There is something seductive about a room that understands us before we speak. It promises care without effort. But human experience is not always immediately legible, even to the person living it. We change our minds. We hold contradictory feelings. Sometimes we do not want to explain ourselves.

As environments acquire the ability to perceive and respond, designers play a critical role in deciding not only what these systems can detect, but how those signals are interpreted and when that interpretation is allowed to trigger a response.

Perhaps the most humane environment is not one that claims to understand us before we speak, but one that recognises the limits of what it knows and leaves us room to decide what our experience means.

References

^1.Barrett, L. F., et al. (2019). Emotional Expressions Reconsidered: Challenges to Inferring Emotion From Human Facial Movements. Psychological Science in the Public Interest, 20(1), 1–68. https://doi.org/10.1177/1529100619832930
^2.Random International. Rain Room (2012). Project documentation: https://www.random-international.com/rain-room ; Barbican Centre: https://www.barbican.org.uk/read-watch-listen/random-international-rain-room