Borrow a familiar surface, displace its meaning

Entry begins as an almost exact recreation of an Instagram reel. A 19 by 19 grid of 361 apartments scrolls vertically and the visitor enters the work using only the gestures they already know. Onboarding cost approaches zero. On top of that familiar interface a social critique unfolds, in which the value of a home is reduced to the two-dimensional plane of location and brand. The interface stays familiar; only the meaning is displaced.

The single act of tapping an ad is the real entry point. After that one tap the camera, microphone, accelerometer, gyroscope, and multi-touch are each unlocked. There is no tutorial page; visitors discover the capabilities of their own device inside the work. Learning about the medium happens within the medium.

A cold-medium principle, the zero-text interface

There are no parameter labels on screen, no dimension numbers, no instructions such as “shake here.” Every input is taught only by affordance and by the work's own advertising copy. The artist refers to McLuhan's notion of a cold medium. It is close to the furthest extent of textual-onboarding minimisation that contemporary UX practice has aimed at.

Ref-based gesture stack, bypassing React state

To work around a known react-spring v10 issue (transforms reset on every render through useIsomorphicLayoutEffect), the work's camera control, comment sheet, and plane gestures bypass React state and write transforms directly to DOM via refs. Pose updates are throttled with requestAnimationFrame; bounding rect lookups are cached on a ref. The result is fourteen simultaneous deformations running at 60fps on a single screen. A practical pattern for any UI that has to combine many gesture-driven transforms in parallel.

Two strategies for combining multi-user input

How inputs combine differs by modality. Voice RMS and shake combine as a vector sum, producing collaborative deformation. When several visitors push the same plane in the same direction at the same moment, the deformation grows. Camera-based head tracking, gyroscope tilt, and finger skating combine as latest-wins. As more visitors join, occupation contention emerges naturally. The choice between these two strategies is itself a critical surface of the work.

A Markov-chain ad navigation, finding paths without menus

Movement between the twenty-seven links is governed not by a menu but by weighted Markov-chain advertisements. Three groups (M↔D↔E) share family-level weights; an unvisited-family hard filter, a recent-family avoidance, and collision suppression all sit on top. After five seconds without an action a hamburger menu appears as a fallback. It is a single concrete answer to the IA problem of avoiding a menu while preventing visitors from getting lost.

What this implies for designers and brand teams

Borrowing a familiar surface (scroll, ads, Instagram reels) and displacing only the meaning makes it possible to keep onboarding cost near zero while still mounting a critique of the medium itself. It is an alternative to the tutorial-first pattern that dominates branded experiences.

A bring-your-own-device model lets N visitors play in ensemble at once without dedicated kiosks or hardware. The pattern transfers directly to pop-up stores, flagship retail, and conference booths.

A single source of truth (a zustand store on the media wall) broadcast to every client at roughly 180ms intervals is the simplest way to enforce consistency across many devices. The architecture transfers to other systems where distributed agreement is required, including collaborative tools and virtual events.