[Eng] Crossing Realities: From Code to Choreography
- momoko91
- 2 days ago
- 7 min read
Text: Zelia ZZ Tan@TechDanceLab
Practice-as-Research: When Code Becomes Choreography
I begin with a simple question: what happens when physical sensation is treated as data? Drawing on practice and theory, I treat coding not as backstage support but as a fully fledged choreographic medium.
Open reading resources such as Expanded Choreographies by Anna Leon and Verena Anker’s dissertation Creating Dance with Bytes and Pixels inspired me to think of emerging technologies not as neutral tools, but as cultural techniques that shape how dance is created, perceived, and embodied. Whether I am working in Unreal Engine, Touchdesigner or Max/MSP, each platform imposes a logic that influences how bodies appear in space and time. My practice is to restructure that logic—to decide what may vary, what must remain visible, and what should resist translation. Even when the visuals vanish and only 0s and 1s remain, choreography persists—unfolding between hardware and flesh, software and imagination.
Building network flows between different programming languages and establishing agency within them remains a challenge. I’ve come to see node-based patches, Python scripts, and sensor inputs as forms of notation—much like a musical score—defining timing, intensity, and relationships. I now treat suspension, latency and glitch as choreographic materials, asking:
How does a half-second lag alter a gesture being received?
What does resistance feel like inside a “coded” body?
Introduction: Why Start with “Crossing Realities”?
In recent years, the borders between choreography and new media have become increasingly porous. As a dance artist and researcher, I find myself drawn to these intersections — between body and machine aesthetic, avatar and immersive narrative, virtual worlds and real-time mocap (motion capture) performance. The subject of “Crossing Realities” speaks to my curiosity to explore not only how dance can integrate technology, but how choreography itself can be reimagined through creative languages like coding, and physical-digital embodiment. It honours the discoveries that have shaped my practice, inspired by Kate Ladenheim, Lisa Jamhoury, and Gilles Jobin’s pioneering work in XR (Extended Reality) dance and interactive performance.
Challenges: What breakthroughs emerge when we translate movement into Code and Engines?
First, it's important to say this question is lived rather than theorized. In rehearsal rooms and research labs, we experience it: spiraling movements that feel natural to the dancer may become flattened, distorted, or rigid when captured by technology. Coding "movement" is not merely recording — it is structuring, filtering, interpreting. "Translation" feels like a clichéd term, yet we use it constantly to navigate this gap between intention and result.
For example, in Jobin’s Studio44 Mocaplab, where I had the opportunity to work, the challenge of maintaining high-quality optical tracking through a game engine, Unity, led to the development of specialized skeletons and mapping processes. Dancers are carefully guided to maximize expressivity within digital translation limits — with evolving choreographic strategies between dance and the hybrid system.
My encounter with Ladenheim’s work, beginning with her paper about her creation Babyface, illuminated how installations and robotics could centre the body. She reframes this through feminist and performative lenses. Babyface challenges ideas around servitude and spectacle, creating tension between human and machine through minimal yet powerful design with code. It was a revelation: performance does not require a grand scale to be profound — it can be deeply rooted in body and concept.
I discovered Jamhoury's work through the SloMoCo 2021 residency. Her research focuses on foundational questions in dance tech, emphasizing subject over spectacle. In conversation, we explored her Unreal-based “imperfect average body,” which reveals how digital systems aestheticize and distort human movement while preserving its resistance. Her work values ecological creativity over technical display, reminding me that every system—Max, TouchDesigner, Unity—carries embedded assumptions that shape how we move and think.
In my practice, I combine node-based programming with physical movement to create an expressive dialogue between body and machine. For example, even a phone gesture—like a tilt or tap—can trigger an AR (Augmented Reality) sculpture in space. These interactions are choreographed through code, where movement continuously negotiates with the system’s logic, more cross realities designed around our daily life.
I started with no coding experience, but curiosity—and necessity—led me to build interactive systems using tools like Isadora and TouchDesigner. What began as simple experiments became essential. Over time, I modified codes in my choreography and articulated the movement language shaped through scores, interactable systems.
Two months ago, at the recent Game Developers Conference, I performed a new interactive XR piece with Cie Gilles Jobin’s team and residency artists, deepening the inquiry:
What happens when dance is embedded inside a game engine?
How does the body translate into the logic of digital space?
Movement Through Installation and Interaction
XR dance has existed for over a decade, and its potential lies not in novelty, but in redefining embodiment.
Posthuman theory reframes dance as more than personal expression. Jamhoury's approach to utilizing methodology to open up social narratives is truly inspiring. The holistic nature of the project, bridging performance art and sculpture, resonates deeply with my practice. The scale of her installation and its interaction with public spaces are particularly intriguing. The minimalist design, allowing viewers to see through the form, offers a unique perspective. As I develop my own AR work, I have found balancing scale and interactivity to captivate audiences is a common challenge.
Listening to Jamhoury’s work, I was struck by how clearly motion and narrative are conveyed through description. The phrase "The Leg Betrayed the Body" poignantly explores themes of departure and return. Discussing body weight, angles, joints, and camera perspectives vividly brings the narrative to life, opening the imagination to broader connections beyond the "individual body".
Last month, witnessing Monumental Death in Ladenheim’s presentation at CalArts (California Institute of Arts) gallery night, I was struck by the animated flux of a soft monument rising and falling with audience participation. It reimagined the monument not as a fixed structure but as a living, collective act — linking memory, culture, and bodily presence. This questioning of how we memorialize, how we "witness," resonated with my own research on memory, disappearance, and transformation.
How Have Lisa Jamhoury, Kate Ladenheim, and Gilles Jobin Influenced My Thinking About XR Dance?
The work of these three people further reinforced my exploration into XR dance. Encountering Ladenheim’s work taught me that design and performance can be intimately, elegantly intertwined — that simple, thoughtful systems can create profound emotional experiences. Her installations and performances revealed how minimal movement, embedded in robust conceptual frameworks, can critique cultural systems and offer new visions of agency.
Jamhoury’s exploration of absence and presence pushed me to rethink what it means to "be" in a digital space — how motion, memory, and feeling persist beyond physical borders.
Her practice opens portals into how the body can be absent yet present — questioning the very notion of perception, presence, and human connection in mediated spaces.
Jobin’s integration of dancers and avatars in immersive XR environments expanded my understanding of ensemble work, of collaboration across human and digital performers, and of how we choreograph worlds. Meanwhile, his pioneering approach in building entire ecosystems of motion capture dance performance challenges how we think about ensemble, collaboration, and audience experience in hybrid realities.
Together, their works ignite my pursuit of crossing realities: building choreographic systems that live between worlds, inviting audiences to imagine new forms of connection, memory, and movement. I see a future where dance is not merely enhanced by technology but fundamentally reshaped — where the virtual becomes a site for both political reflection and poetic embodiment.
Final Thoughts
Coding, like choreography, is rooted in choice and the process of making. Shaped by the creator’s hand, it transforms sensed movement into lived experience. I imagine a “choreographic coding map” — a series of toolkits where people can build, remix, and translate movement through code. The focus lies in how we use these tools to listen, reflect diverse bodies, and tell embodied stories.
======
Zelia ZZ Tan
Zelia ZZ Tan is a choreographer, dancer, and multimedia artist weaving movement, code, and memory into hybrid worlds. Her work traces presence across physical and virtual landscapes, curious about the future of bodies, exploring identity through motion capture, XR, and immersive systems. She directed REfract, a performance between avatars and bodies, commissioned by West Kowloon and Goethe-Institut. In 2022, she co-founded TechDanceLab, an experimental platform for embodied technology. Zelia is currently pursuing an MFA at the California Institute of the Arts, double-majoring in Choreography and Interactive Media for Performance.
![[Eng] OOTFest25 (UN)SEEN: Vision, Dialogue, Experimentation and Paradoxes](https://static.wixstatic.com/media/f99599_c976f913ce8e4317816053a968bacc70~mv2.jpg/v1/fill/w_980,h_551,al_c,q_85,usm_0.66_1.00_0.01,enc_avif,quality_auto/f99599_c976f913ce8e4317816053a968bacc70~mv2.jpg)
![[中]《 街舞劇場試煉場》—— 記 【街舞劇場發展計劃2024-2025】The Box 6.0](https://static.wixstatic.com/media/f99599_316817aceb79487a9eb020524852d637~mv2.jpg/v1/fill/w_980,h_551,al_c,q_85,usm_0.66_1.00_0.01,enc_avif,quality_auto/f99599_316817aceb79487a9eb020524852d637~mv2.jpg)
![[中] 舞影山海—— 與藍嘉穎談舞蹈影像創作](https://static.wixstatic.com/media/f99599_f7c50a29186e41bc8748755970cb2f9d~mv2.jpg/v1/fill/w_980,h_551,al_c,q_85,usm_0.66_1.00_0.01,enc_avif,quality_auto/f99599_f7c50a29186e41bc8748755970cb2f9d~mv2.jpg)
留言