Ryuichi Sakamoto was a composer, producer, and artist born in Tokyo. His film soundtracks received prestigious awards, including an Academy Award, two Golden Globes, a Grammy, and several others. He passed away in March 2023 at the age of 71. In late 2020, in collaboration with Tin Drum, Sakamoto participated in motion capture work for the mixed-reality project KAGAMI. Through wearable headsets, KAGAMI aims to enable audiences to feel as though they are witnessing a live performance in real time within a shared, immersive space.
Unlike other works of technological art, KAGAMI debuted posthumously after Sakamoto's passing, prompting media descriptions that go beyond "immersive experience" to terms such as "face-to-face," "resurrection," "rebirth," and "final solo concert." But can such an experience genuinely be achieved? From these descriptions, it is evident that the work relies on two basic elements: a spatial setting and the presence of both "Sakamoto" and the audience within that space.
This research explores how immersive experiences are created in the mixed-reality performance KAGAMI, drawing on Lefebvre's Three Levels of Social Space (spatial practice, representations of space, representational space) and Merleau-Ponty’s concepts of bodily perception, particularly "made flesh." Merleau-Ponty argues that space is rooted in the body but pays little attention to the body’s social and historical dimensions. In contrast, Lefebvre’s concept of space situates the body within a broader societal context. By engaging these two theorists in dialogue, this research aims to move beyond critical theories of space toward a deeper understanding of how space generates immersive experiences.
KAGAMI encompasses three distinct spaces: the first is the physical space of the theater; the second is the space created by the performance, akin to a traditional piano recital; and the third is the space of the audience's body. The immersive experience emerges when these three spaces converge into a unified whole. Merleau-Ponty describes bodily awareness as being oriented toward specific tasks or potential actions: "My body appears to me as an attitude directed towards a certain existing or possible task. And indeed, its spatiality is not...a spatiality of position, but a spatiality of situation" . Immersion is not solely about the creativity of the artists or the technical expertise of the production team; it also depends on the physical space and the audience's embodied practices. From Merleau-Ponty's perspective, immersive experience is an interplay where we and all things around us mutually capture one another, suspending reflection and thought. This study challenges the notion that immersive experiences must involve zero distance, focus on the sense of distance between the audience's body and the space, as well as the movement between different spaces.
This research offers a new framework for understanding mixed-reality experiences, emphasizing the significance of spatial production and bodily engagement over simplistic ideas of complete sensory encapsulation. It aspires to inspire future innovations in the field of technological art.
References
1.Chen, A., &Chuang, Y. (2024). Ryuichi Sakamoto's one-year memorial: Rebirth of his final live solo concert KAGAMI. Vogue Taiwan. Retrieved from https://www.vogue.com.tw/article/ryuichi-sakamoto-tim-drum
2.Factory International. (2023). In the studio with Todd Eckert Tin Drum [Video]. YouTube. Retrieved from https://www.youtube.com/watch?v=7hRwOP890a0
3.Fuchs, C. (2019). Henri Lefebvre's theory of the production of space and the critical theory of communication. Communication Theory, 29(2), 129–150.
4.Lefebvre, H., & Nicholson-Smith, D. (1991). The production of space. Blackwell Publishing.
5.Ling, M.-X. (2023). Ryuichi Sakamoto's final solo concert coming to Taiwan next year, inviting the audience onto the stage to meet the master face-to-face. Liberty Times. Retrieved from https://art.ltn.com.tw/article/breakingnews/4503407
6.Merleau-Ponty, M. (1973). The prose of the world. Northwestern University Press.
7.Merleau-Ponty, M. (2009). Eye and mind · The prose of the world. In Collected works of Merleau-Ponty (Vol. 8, D. Yang, Trans.). The Commercial Press. (Original work published 1969).
8.Merleau-Ponty, M. (2023). Phenomenology of perception (D. Yang, Y. Zhang, & Q. Guan, Trans.). The Commercial Press. (Original work published 1945).
9.Opentix. (2024). 2024 TIFA Ryuichi Sakamoto: KAGAMI. Retrieved from https://www.opentix.life/event/1714929180109553665
10.Polydorou, D. (2024). Immersive storytelling experiences: A design methodology. Digital Creativity, 35(4), 301–320.
11.Rauschnabel, P. A., Felix, R., Hinsch, C., Shahab, H., & Alt, F. (2022). What is XR? Towards a framework for augmented and virtual reality. Computers in Human Behavior, 133, 107289.
12.Škola, F., Rizvić, S., Cozza, M., Barbieri, L., Bruno, F., Skarlatos, D., & Liarokapis, F. (2020). Virtual reality with 360-video storytelling in cultural heritage: Study of presence, engagement, and immersion. Sensors, 20(20), 5851.
13.Trunfio, M., Jung, T., & Campana, S. (2022). Mixed reality experiences in museums: Exploring the impact of functional elements of the devices on visitor's immersive experiences and post-experience behaviours. Information & Management, 59(8), 103698.
14.Xuci Editorial Team. (2023). AI ushering a new era in music production: Deceased musicians "resurrected," transcending time through technology. P-Articles. Retrieved from https://p-articles.com/heteroglossia/3841.html
15.Zhang, C. (2020). The why, what, and how of immersive experience. IEEE Access, 8, 90878–90888.
