Dissolving and Emphasizing the Physical in the Digital Age

Reality, Appearance, and Virtual Perspectives in Alexander Schubert’s Work

Rainer Nonnenmann

“I go to the window and I am opened.”
Peter Handke, The Innerworld of the Outerworld of the Innerworld (1969)¹

“Late in the afternoon, the teacher turns off the lights. Like everyone around him. Like thousands throughout the city, re-enacting their boring, quiet lives of yesteryear in virtual rooms. Back then, when they still had a function.”
Sybille Berg, GRM—Brainfuck (2019)²

The term “digital revolution” is a misnomer, as the technological developments since the 1940s have taken place in countless independent steps, causing effects on professional, social, and private lives occurring in several spurts rather than as a revolution all at once. For the generation of “digital natives” born around 1980, this term is above all an atavism because the digital upheaval—if there ever was one—is long gone. As early as 1998, the director of the Media Lab at the Massachusetts Institute of Technology (MIT) Nicholas Negroponte proclaimed: “The digital revolution is over.”³ For at least three decades digital means of production, storage, distribution, and presentation have been the natural rule rather than the rare exception in all areas of life. Thus, the term “post-digital” emerged as early as 2000. It describes the fact that everything digital, including the internet, social media, and a plethora of ever-new apps have profoundly changed human self-portrayal, world perception, communication, and interaction and have begun to influence non-digital aspects of analog life as well.⁴ Within a few years, mobile devices have changed the behavior and movement patterns of private and public life, just as the hygiene and distance rules imposed in the course of the Corona pandemic re-choreographed all interpersonal interactions within a few weeks. What was initially experienced as a disruption of the social status quo soon established itself as a new convention and form of interaction. The use of smartphones, screens, touchpads and Bluetooth reconfigures the haptic and audio-visual perception of self and the world. Therefore, Immanuel Kant’s classic essential questions resurface: What can I know? What should I do? What may I hope? What is man?

Today, almost all information can be digitized, edited, and thus manipulated. Digital media not only depict reality, but also create realities, identities, corporealities, and virtualities that could exist in any number of modifiable possibilities without having to correspond to anything that physically exists in the analog world. In many cases, the digital representation of people, actions, and things allows little or no conclusions concerning their real presence. Original and representation, cause and effect are decoupled. The distinction between manipulation and authentication, simulation and identification, transparency and non-transparency is becoming increasingly difficult. The concepts of reality and appearance are merging into simulacra. At the same time, the social acceptance of virtual content in real life is growing.⁵ In large stadiums, tens of thousands no longer follow the competition of real soccer teams on the field, but the duel of gamers playing the computer game FIFA or other popular e-sports and e-games on large video screens. They control game characters, which in turn are modeled on well-known real-life soccer stars, and likewise hold proper tournaments based on the model of national leagues and international championships.

In Japan, the emphatically artificially designed anime figure Hatsune Miku appears as the lead singer of a rock band playing on stage in real life. Instead of a physically present singer, this projected hologram is cheered on by thousands of fans as if it were a real person. The augmented reality game Pokemon Go, which has been played by millions since its release in 2016, places computer figures at predefined locations in public spaces all over the world that can be found, captured, and collected by using smartphones and GPS. Real places turn into the scenery of the computer game. Common computer programs for image processing allow subtle retouching as well as serious transformations such as morphing, face-swapping, and style copies of portrait photos as charcoal drawings, pencil sketches, oil paintings, classical, impressionist, or cubist portraits. In YouTube videos and TV satires, facial expressions and speech movements of one face are transferred to another, making appearances of real politicians with fabricated gestures and statements that seem so realistic that they can only be unmasked as fake by elaborate digital multimedia forensics.

Thanks to tablets and smartphones, there has long been parity in public space and passenger transport between the media representation of absent people and the presence of real people. In Amsterdam, the Nxt Museum opened in 2020 as one of the first exhibition houses for immersive digital media art. The founder of the Tesla company, Elon Musk, had a chip implanted in the brain of a pig in the summer of 2020 with the hopes of using the research to optimize the cerebral memory and communication abilities of humans in the near future. Halo wristbands, which tell the wearer how they are feeling at any given moment, are already commercially available. Social media like YouTube, Facebook, Twitter, WhatsApp, Instagram, and TikTok are increasingly designed to suit certain age groups and usually show content specifically produced for these platforms and peer groups, in which the boundaries between private and public are blurred. Furthermore, bloggers, influencers, fake accounts, automated social bots, and trolls obscure the differences between private opinions and commercial advertising, documentation and fiction, critical reporting and targeted disinformation, factual enlightenment and manipulation that endanger democracy. Rankings and hit lists of intermediary search engines and streaming portals such as Vimeo, Spotify, Soundcloud, or Pinterest often do not document actual preferences, but are bought and are therefore more an expression of tangible economic interests of marketing, prestige, and image cultivation than they are a yardstick for reliable quantities or even quality. Companies, politicians, YouTubers, and artists invest in fake likes, comments, user-, follower- and click-statistics, which are “legitimized” by fictitious people in order to advertise for comments, services or consumer goods.

Hidden manipulation and its influence on the forming of opinions has always existed and is present everywhere. However, what is new are the mechanisms and dimensions made possible by cookies, Google analytics, filter bubbles, and big data, with which digital media both open up and distort the world, or even create new virtual realities. The proportions are reversed. Those searching for something via Google are at the same time found by the search engine. They open an internet page and are opened. They surf the World Wide Web and are pupated into a highly personalized filter bubble tailored to their search- and streaming behavior. Freely adapted from Kant, enlightenment is nowadays becoming increasingly necessary as a way for man to emerge from his self- or third-party-imposed filter bubble.

Disembodiment and Embodiment

In music, multimedia works have become increasingly popular since the 2000s, with more and more pieces exhibiting the use of computers, live sampling, video, and graphic programs. Only later did composers increasingly focus on the consequences of the use of digital technology for production, presentation, and reception. The physicality and movement of musicians played a central role in this. After making and listening to music had been exclusively physical—and often also written down—for centuries, the immediate resonance of musical ideas, notes, playing styles, sounds, and psychophysical reactions was now being replaced by binary coding through the discrete individual values 0 and 1, which elude perception until they are transformed again into sensually experienced continuums, words, images, sounds. R. Murray Schafer already described the decoupling of audible and visible results from the time and place of their causal production in The New Soundscape (1969), using the term “schizophony” which he borrowed from psychopathology.⁶ However, the phenomenon did not only become virulent through digitization, but already through analog recording, storage, and reproduction technologies, to which artists reacted even at that time.

From the 1960s onwards, Dieter Schnebel and Mauricio Kagel no longer used playing gestures primarily to produce sound, but instead designed movements of singers and instrumentalists in a compositionally, gesturally, and mimically independent way. A little later, Nam Jun Paik transformed the bodies and actions of musicians into videos. Since the 1990s, as a matter of course, more and more composers have been working with live video, video feeds and various editing and manipulation possibilities. Examples include Michel van der Aa, Simon Steen-Andersen, Mark Applebaum, Michael Beil, Brigitta Muntendorf, Stefan Prins, Matthew Shlomowitz, Jogoda Szmytka, and Jennifer Walshe. Performers like Stellarc, Laetitia Sonami, and Suguru Goto hybridize their own bodies with sensors, robotics, video glasses, headphones, and artificial ankles. Using full-body interfaces, body movements control light, videos, and electronic sounds as well as various pneumatic percussion and robotic instruments. As early as the 1960s, Marshall McLuhan understood media analogous to the use of tools as “extensions of man.”⁷ Technical devices turn the human body into an “extended” or “amplified body” with expanded possibilities for perception, action and communication. The utopia or dystopia of a further development of the human being into a cyborg is emerging.⁸

Moving bodies in the context of media extensions are also at the center of the work of Alexander Schubert (*1979). He studied computer science, bioinformatics, and cognitive science in Leipzig before completing a degree in Multimedia Composition in Hamburg. With a high level of technical skill, socio-cultural reflectiveness, and aesthetic flair, he addresses aspects of interpersonal interaction, self-perception, perception of others and body perception. Schubert explicitly sees his work as a “tool for making digital effects visible and to sensually experience them in the analog world.”⁹ Like other artists and composers—in whose context he explicitly sees himself—he uses interactive control possibilities such as live electronics and live video to illuminate the function of performers and the relations between action and effect through dysfunctional interference.¹⁰ In performative, music-theatrical, and installation projects, he lets the performers and the sometimes actively involved visitors act in reality and at the same time experience unreal transformations of their own actions and bodies through digital settings. On the one hand, he demonstrates the rampant disembodiment of the human being through flat monitors and, on the other hand, emphasizes the existential significance of proximity, corporeality, and kinesis, both through bodily presence in space and through participation and mutual observing, touching, grabbing and virtual body exchange. Schubert turns the fundamental realization that virtuality and deception are inherent components of every digital mode of working into a “virtual imperative” in order to address precisely this in his pieces.¹¹ Thus, he represents the media-critical approach of making it possible to experience the altered expectations and modes of perception and action brought about by digital technologies. By refraining from evaluating the phenomena he demonstrates, he aims to make the immersiveness of his multimedia productions and the ambivalence of the “post-digital” in all areas of life all the more forceful for the audience.

The following chapters evaluate selected works by Schubert, ranging from the sensor-based interactive performance Weapon of Choice (2009) to the most recent real-life computer game Genesis (2020), with regard to the relationship between moving bodies and their technical transformation. The technologies used for the purpose of increasing virtualization are developing rapidly. Whereas in the beginning it was electronics, sensors, and light in predominantly conventional concert and stage situations, later it expanded to include VR glasses, internet, and computer-game settings in complete buildings with actively involved visitors, including the detachment of bodies and movements from their physical bond to space, time, and person. On the one hand, Schubert follows the principle that Mauricio Kagel had already formulated in the early 1960s: “As soon as a musician enters the stage, there is also the aspect of dramatization/presentation.”¹² On the other hand, Schubert establishes media-related set-ups, the function of which he disrupts in the course of his pieces, thus making their fabricated nature and effectiveness discernible: “A mistake’s characteristics also point to the functioning of the underlying system.”¹³ The following contribution does not so much aim at a fundamental theorization of the artistic engagement with virtual reality as at an empirical description of post-digital perspectives during the 2010s using the example of works by Alexander Schubert, experienced live at premieres and subsequent performances.

Sensor-Based Performance

Schubert developed his first interactive composition Weapon of Choice for violin, sensor, live electronics, and live video (2009) with and for the violinist Barbara Lüneburg.¹⁴ The violin bow is equipped with a motion sensor and serves as a “weapon of choice” for the musician in a “duel” with the technology, like a foil. By tilting the bow from vertical to horizontal and rotating it around its own axis, control voltages are triggered depending on the speed of the movements, which activate predefined sound and graphic programs. Bow and instrument serve only secondarily to generate violin sounds, and primarily as an audio-visual interface, whether in combination with conventional playing techniques or completely independently of them. The instrument and the musician’s embodiment merge with the digital set-up, forming an analog-digital hybrid. Kinesis, which is otherwise treated as secondary in music-making, is moved to the center and expanded theatrically. A video screen behind the performer shows her gestures enlarged and processed abstractly in color, tempo, and time. The musician draws black-and-white or colored structures with the bow as if with an oversized Apple Pencil on a magic board, constantly overwriting the hatchings and erasing them again after a few seconds as if by magic. Guiding the bow through the air, for instance, causes fan-like patterns to unfold and fold back together in slow motion. Likewise, col legno accents are multiplied into repetitions by means of delay, whose pitch and timbre then modify silent bow gestures. The actions are not specified in detail in the score because the performer is also supposed to react spontaneously to the events she produces. Schubert merely provides an interactive framework in which the performer acts with her specific motor activity.

Figure 1: Video still Weapon of Choice with violinist Barbara Lüne­burg (2009). © Alexander Schubert

In Point Ones for extended conductor, small ensemble, and live electronics (2012), the usually silent conductor produces sounds by means of movement sensors on his wrists. Not an instrument, but the human body itself serves as the interface here. The conductor’s task is not so much to coordinate the ensemble, to beat time, to give cues, or to shape tempo and dynamics. Instead, by moving his hands and arms up, down, right, left, hard or soft, he accesses numbered “cues” with pre-produced electronic sounds and live transformations of the amplified ensemble.¹⁵ The score notes arrows for the left and right hand as well as keyword-like events of electronics and instruments for the purpose of coordinating the “E” and “I” systems as exactly as possible. At first, the ensemble acts analogously to the conductor’s gestures with glissandi, repetitions, runs, staggered or synchronous beats. Conducting and ensemble-playing converge so that the audience experiences the usual unity of visible signs and audible sounds. Later, however, there is increased divergence between conducting and playing. The conductor’s gestures coincide with pauses and cause divergent or no reactions in the ensemble, only silence or instrumental or electronic sounds played from offstage. The connection of visible cause and audible effect is disturbed. The functionality of conducting and ensemble is dysfunctionally decoupled and thus becomes the center of attention.

Music example 1: Point Ones for extended conductor, small ensemble, and live electronics (2012), bars 70-78. © Alexander Schubert

In the course of the piece, the discrepancies between audible electronic and instrumental sounds and visible actions of the conductor and musicians increase even further. In places, the “augmented conductor” is no longer supposed to conduct the ensemble, but to act like a soloist with powerful, wild movements. During a cadenza (bars 180 to 184) he completely abandons his traditional role and semantics in order to make his range of motion theatrically independent.¹⁶ His bizarre gestures take on “a visual intrinsic value” in order to “entirely fathom the connection between sound and body language, thus making the interplay of those components directly tangible for the audience.”¹⁷

Figure 2: Enno Poppe as conductor in Schubert’s Point Ones with Ensemble Mosaik 2014, video still of the conductor’s solo. © Evy Schubert

What Schubert says about the conductor’s role as a whole applies in particular to this cadenza: “Think of the conducting for the piece as a choreography.” Although no instrumentalist plays during the solo, high-energy electronics beat down on the audience, since every gesture evokes sound files: buzzing, crackling, snapping, droning, booming. Since every twitch is accompanied one-to-one by sound, the causal chain seems to be reversed. The conductor leaves the impression that he is not controlling the electronics, but that he himself is under high voltage: the maestro as a wriggling marionette.

Digitalization through Light

Schubert’s “interactive sensor pieces”—as he refers to them¹⁸—allow the bodies and movements of musicians to appear as themselves, despite all the enhancements by audio and video electronics. Pieces with strictly timed sound and light impulses go one step further in the medial transformation of living bodies. In Lucky Dip for midi-drum kit, keyboard, electric guitar, light, and projection (2013) as well as in Scanners for string quintet, choreography, and electronics (2013, rev. 2016), and Supramodal Parser for singer, electric guitar, saxophone, percussion, piano, electronics, and light (2016), Schubert dissolves the continuous presence of the actors. Their positions on an otherwise completely dark stage are only made to appear through rapid blinking lighting. The discontinuity of the light resembles the binary 0 and 1 grid of digital codes. The musicians are not simply illuminated, but switched on and off, as it were, like a sample. Binary coded, their bodies and gestures appear flattened, pixelated and fragmented like video clips. They lose their three-dimensionality and turn into liquidities or virtualities that can be switched on and off at will: “The individual on stage is transformed into many small virtual, digital clips in real time. The continuous presence of an expressive performance gives way to a series of isolated, mechanical image sequences.”¹⁹ The performers no longer appear merely “extended,” but hybridized with the technical set-up to such an extent that they appear two-dimensional, depersonalized and disembodied, and it is no longer possible to distinguish whether they control the technology or are merely a manifestation of the multimedia set-up.

In Sensate Focus for electric guitar, bass clarinet, percussion, violin, live electronics, and light animation (2014), four musicians act side by side on a pitch-black stage, each two meters apart. They are microphoned, coordinated by click track and connected via an interface to a computer program that only casts bright cones of light on the musicians and activates electronic feeds or live electronic distortions during short playing gestures or silent turns, tremolos, freezing poses, etcetera. The “Technical Rider” explicitly demands loudspeakers and subwoofers that are suitable “for a techno club and not for subtle amplification of chamber music.”²⁰ The high speeds, volumes, standing drones, hard beats, piercing basses, and clanging highs of Schubert’s music and the physical-gestural access to the instruments, which are electronically reshaped by microphones, amplifiers, and loudspeakers, show a strong affinity for genres such as metal, industrial, electronica, noise, hardcore or free jazz and club culture. The haptic sound production aims at an immersive extension of “mere” listening into a comprehensive, particularly an overwhelmingly powerful physical and vegetative experience for all senses, as is common at rock concerts or rave parties. Schubert’s music thus follows a paradigm shift that can also be observed in contemporary theater and dance: away from the aesthetics of the work, towards physical performativity and eventfulness.²¹

In Sensate Focus, the instrumentalists are illuminated by LED spotlights with stroboscopic flashes for only tenths of a second. Instrumental and electronic extremes correspond to the alternation of light and dark. Short crackling sounds and long reverberations suggest spatial proximity or rather distance, narrow or wide sound spaces. The design levels of light, sound, acoustics, and gestures follow the poles of light-dark, instrumental-electronic, dry-echoed, with or without instrument. However, the simple parameters on their own generate an extremely complex overall audiovisual event in various combinations. The strict parallelism of seeing and hearing creates the impression—like the conductor’s cadenza in Point Ones—that it is not the musicians controlling the electronics, but that they themselves are video projections which can be switched on and off, that seem to buffer during rapid strobe flashes and get caught in loops. For example, as the musicians play silent tremolos, one hears electronic rattling and sees equally rapid flashes of light. Parallel to the synchronously accelerating and slowing sequences of beats, the blinking of the light cones also accelerates and slows down. The performers appear like remote-controlled robots or animated artifacts of a digital automatism. As the speed of the light flashes continues to increase, however, at some point the musicians can no longer keep up. The ensemble beats and further accelerated flashes therefore diverge, resulting in surreal overlaps. Contrary to anatomical possibilities, some movements suddenly seem to run backwards like the spokes of a wagon wheel rotating in a film. Other movements stand still, as they are continuously repeated in accordance with the frequency of the light pulses. As long as the spotlights are slightly offset, the cones of light flash across the stage. At the end, the musicians, caught up in the world of machines, tremble at the playing symbol “⚡” with their hands, arms and whole bodies to the loudly crackling electronics. When the power is turned off, their upper bodies topple over as if dead. At the stage of total exhaustion, their staged collapse also serves as a bow for the final applause.²²

Music example 2: Codec Error, score, bars 54-61. © Alexander Schubert

In Codec Error for double bass, two percussionists and light (2017), Schubert extended his work with light to the entire stage. Remote-controlled spotlights direct colored light at any point with great precision and speed. The three parts each consist of three systems in the score: 1. choreographic gestures performed independently of the instrument in all directions and at varying speeds and dimensions—stopping, turning, freezing, tremolo-like repetitions, etc.; 2. actions with the instrument including conventional playing styles; 3. different positions on stage. The musicians act via click track precisely synchronized with light flashes and audio electronics. All visible and audible events therefore appear as a unity in which the performers seem like holograms due to markedly artificial movements and color-shifting stroboscopic flashes. When the players move very slowly to the flashes of light, they appear as lifeless sculptures shifted in space as if by magic. If they change their position quickly during dark phases, they look like photos or video samples shifted on the computer screen during the next flash of light. Rapidly flickering illumination in red, green, and blue from different parts of the stage finally create effects like those of defective video files whose offset image structure falters and is shifted by a few pixels.²³ The title Codec Error aptly names these staged coding errors.

Figure 3: Codec Error, production photograph. © Alexander Schubert

In the commentary on the work, Schubert describes the intention of his piece as follows: “It examines the body of the performer and its representation in the digital age.”²⁴ The musicians appear as images of themselves as in video clips, staged, manipulated, pixelated, in cuts, excerpts, and slow motion. The media unreality is additionally reinforced by flashes of light that hit the audience head-on, brutally blinding them so that the technical set-up cannot be grasped. It is only in the middle of the piece that the complete setting is revealed. Apparently as a result of a computer crash, the stage lights suddenly turn on and for the first time the audience overlooks the stage space, instruments, tablet computers, microphones, spotlights, and loudspeakers, which then also allows the coordinating click track to be heard in the hall. All at once, the operating system reveals its functional mechanisms. Previously, digital technology simulated the stage as a video screen or hologram. Now it demonstrates itself as a disenchanted technical apparatus that allows the actors to take the place of media representation in their own corporeal presence. The digital framing loses its immersiveness and provides the audience with the opportunity to draw conclusions from the degree of their previous amazement in the concert hall to the degree of digital penetration of the general “Lebenswelt” (lifeworld). The complementary stage situation—virtual illusion versus sober technology—follows a central strategy of Schubert: “I understand my compositional work as a tool to describe and explore the dividing line between the analog and digital worlds […] or also to make visible the invisible of the digital.”²⁵

Virtual Reality

In his most recent works, Schubert expands the digital setting to include video cameras, microphones, virtual reality glasses, and headphones. In addition, the entire space of the performance is captured and the audience is included in a participatory way. Interfaces and collisions between real movements and sensations and simulated images, bodies, and sounds are staged to provoke a constant comparison between virtuality and identity. By both connecting and separating incorporeality and corporeality, distance and proximity, machine and human being, Schubert wants—in the sense of artistic research—to make it possible for the audience to experience the unconscious transfer of digital modes of representation, perception, and action to the analog world:

One opportunity of contemporary multimedia composition can be to depict and address this change of perspective. In this context, a work of art can be thought of as a tool to promote sensitization, visualization, or a change of perspective. Since the crucial area of the post-digital runs precisely at the interface of the analog and digital worlds, this art form can be a suitable tool because of its proximity to both worlds.²⁶

The one-and-a-half-hour performance installation Control (2018)—created for the Ultima Festival Oslo—provides the audience with three phases of thirty minutes each, during which the visitors experience more or less the same thing three times from three different perspectives. A first group of visitors walks with VR glasses, headsets and attached webcams including a microphone through an empty building like through a labyrinth. In the rooms, twenty performers create small situations, pretending to be homeless people, drug addicts, psychopaths, or epileptics. Aspects of control and hierarchy are addressed and interactions are provoked. One moment visitors are asked to sit on a chair, take a seat at a table, start a game with someone, or take a broom in their hand. The next moment, someone lies on the floor as if fainting, or people squeeze through a dark corridor where they can hardly get past each other. Various props are used, including a piano, a computer, screens, mirrors, and a camera, which focus on the relation between the original and the image, live music and technical reproduction. The visitors can only see everything as media modification through their VR glasses, slightly darkened, pixelated and with a limited field of vision that is displaced by a few centimeters in relation to the visual axis of the eyes. Furthermore, it is only through headphones that they hear the atmospheric charges, which can be either pleasant or threatening, as well as pleasant sounds, drones, banging gunshots, and disturbing electronic noises. Events recorded by each visitor using their camera and microphone are simultaneously transmitted to a screen and loudspeaker in a control room, where another visitor from a second audience group observes how the visitor from the first group assigned to him makes his way through the building. Since the visitors in the second group have previously walked through the building, they can now give instructions and advice to the following visitors via their headphones as to where they should go and how they should behave. The real, but primarily media-mediated experience onsite is thus followed in the second group by a spatially separated, purely media-mediated audio-visual image of what is happening.

Due to the head start in experience, the second group has a certain responsibility towards the following first group. At this point, each visitor has to decide how to communicate with their assigned visitor in the first group: in a commanding or collegial way, informing or irritating, caring or inconsiderate. At the same time, the second group also receives text instructions on the screen, for instance telling the assigned visitor from the first group to lie down on the floor. The participants of both groups then have to negotiate with themselves and the other whether they want to follow these orders or if they prefer to maintain certain boundaries and refuse. In a third stage, the visitors of the second group finally arrive in a supervision room where they follow the entire proceedings without having the possibility to interact themselves. A third group now sees on monitors both the inside view of the VR glasses of the visitors of the first group in the building and the instructions and reactions of the visitors of the second group in the control rooms, who in turn now become the observed. Finally, the supervision room also gives the organizer the opportunity to monitor the entire building including all participants in order to intervene if necessary.

In Unity Switch (2019), Schubert positions each performer across from a visitor at a table. They sit opposite each other wearing VR goggles with video cameras and microphones that allow them to see and hear what they are actually seeing and hearing, only transmitted through the media. The half-hour “Interactive Virtual Performative Installation” is moderated by recorded voices that give explanations and call for small actions: putting hands on the table, looking at hands, joining hands, synchronizing with the other person, imitating small actions, striking a tuning fork, listening, clapping, flicking. Then letting the other person take a picture of them with their mobile phone, signing a contract for the video documentation of the ongoing performance, putting the pen back, finally taking off the headphones and glasses, and leaving the room. Mobile spotlights provide for changing lighting situations: frontal on the opposite person, profile from the left and right, or as in the case of interrogation directly into one’s own face with the silhouette of the other person remaining dark. All audio and video channels can be interconnected via a central mixing console—the eponymous Unity Switch. Initially, only the channels of the performers and visitors assigned to each other are switched so that they no longer see the realistic video of the person sitting opposite them on their VR glasses, but instead see themselves from the perspective of the other person. The self-perception and the external perception, the internal and external perspective are exchanged. One’s own actions and interactions with the partner can now only be controlled and synchronized by means of the other person’s external view of oneself. While a participant is interacting with their own mirror image, they sense at the same time that the real person behind it leads a motoric-haptic life of their own that deviates from the movements of their own mirror image. The digitally mediated sense of sight and the physical sense of touch are decoupled. Instead of small, more or less trivial actions, the focus is on one’s own view, movement, and joint communication. In a second step, the previous pairings are finally dissolved and the audio and video outputs of different people are combined. The previously established view of self and others by the partners at the table is now replaced by the perspectives and actions of completely different people from completely different tables. When performers and visitors touch each other in real life, they see completely different people who are either also holding hands or doing something completely different. By virtually reshaping physical reality through convergent and divergent videos, Schubert once again provokes the participants to compare the analog and digital spheres.

Figure 4: Unity Switch. © Carl John Hoffmann

Real-Life Therapy and Game

In A Perfect Circle for audience, two announcers, two assistants and supervisor (2019), Schubert likewise works with pairwise assignments. Instead of performers and visitors, however, he brings two visitors into interaction with each other. In a “Real Room,” twenty people sit in pairs on yoga mats facing each other. As in Unity Switch, they initially see each other through VR glasses until the video image is switched and the people see themselves from the perspective of the other’s video camera, as if in a mirror. An assistant plays sounds over headphones and a female announcer gives instructions to perform small actions in a heavily electronically altered voice. These are discreet, cautious, and careful at the beginning, but then demand intensification. The visitors are to turn their heads to the left and right, stretch out their arms, shake hands, touch each other’s shoulders, caress each other’s cheeks, feel each other’s warm breath, hug each other, tighter and tighter … The participants of the forty-minute “therapeutic group session”²⁷—this is Schubert’s characterization in his commentary on the work—have to decide for themselves and together how far they want to commit to these instructions or whether they prefer to keep their distance. Do they feel kindly guided by them to a togetherness or coerced and brought into line by an anonymous authority? Do they feel suggestively supported by warm wellness sound currents or manipulated by technoid, aseptic electronic sounds?

Figure 5: A Perfect Circle. © Carl John Hoffmann

Simultaneously, another twenty people sit in pairs on yoga mats in a second, so-called “VR Room.” Instead of seeing their physical counterpart, however, they see the image of the video camera of a person from the “Real Room” through their VR glasses. Instead of their own gaze, they see another person through their VR glasses from another person’s point of view. In the process, the people in room 2 follow the same instructions as in room 1, in that they also move, touch, and hug each other. Instead of their own hands and the partner sitting opposite them in real life, however, they see the hands and bodies of two completely different people. The real touches on the spot collide with completely foreign gazes. The palpable body of the person sitting opposite them is overridden by the media-mediated appearance of a completely other body with a different appearance, gender, and age. Reality and unreality intertwine to form a virtual intermediate sphere. The sensorimotor unity of seeing, hearing, feeling, and acting is disturbed. Disembodied medial distance intrudes into the experience of real physical proximity. Vice versa, the media distance also creates a voyeuristic closeness to real distanced bodies that can be observed undisturbed from a distance and even virtually touched with the hands of other people.

Schubert created Genesis (2020) for the fifth Hamburg International Music Festival. Like the story of creation in the first book of Moses, this “real-life computer game” lasts a full seven days from Monday to Sunday, twenty-four hours non-stop. In the historic Bille power plant in the east of Hamburg, four members of the Decoder Ensemble see and hear everything only by means of a video camera and microphone via VR glasses and headphones. The audience connects for one hour each via the internet on their home computers to see and hear what a performer assigned to the visitor sees and hears on site via VR glasses and headphones. Although physically completely separated from the situation, audience members (or players, as they are in this case) can follow the action up close through the eyes and ears of the performer. The play is real and live, but for the audience it only takes place online. Just like in the computer game Minecraft, you see the hands of your “game figure” in the foreground of the room they are in. You give verbal instructions to the player via your home computer microphone. Simultaneously, three other players do the same with their characters, so that one can let them interact with each other. Conversely, each “computer game character” can also give its player elementary feedback or ask questions via short messages: “Yes. No. Please be more specific. Thank you. Do you really want this? Unfortunately, this is not possible!”

Via a menu bar, the player can select five objects from a pool of hundreds, which his character can then pick up at a distribution point and put to use. Depending on when a player logs in, he or she will already find various objects in the room, such as plastic sheets hung up like tents, spotlights, furniture, machines, tools, record players, television sets, household objects, items of clothing, or instruments such as electric guitars, keyboards, and drums, which can be played on, moved around, changed and added to. Instead of merely passively receiving, the player should actively navigate and become creative. Everyone has control over what happens, whether it is exciting or boring, constructive or destructive. As in the “survival mode” of Minecraft, a diagram in the upper right corner of the home monitor provides information about the physical condition of the game character (thirst, hunger, tiredness, body temperature), as each participant is responsible for their game character, who sometimes also needs refreshment or rest. In reality, of course, it is a human being who does not have any number of lives, but only one.

Since the players do not act with an animated computer figure as is customary, but rather give verbal instructions to a living human being, they are not merely spellbound as in an immersive computer game, but are challenged by the one-to-one relationship with their „avatar“ to reflect on their own ability to control and on the other’s to be remote-controlled. Moreover, it provokes them to confront the fact that the duality of a person and a medium is transformed here into the virtuality of a medial person and a humanized medium. In particular, the lack of content in the action, which does not reveal any given theme or narrative, makes the player and his or her actions the focus of attention. Everyone has to think about what they do and do not expect of their player. The player puts himself in the position of his human game character in a way that is not the case with merely animated computer game characters. Schubert’s claim to let the players of Genesis create a new world is admittedly ambitious. For the time being, creating something out of nothing is reserved for the gods—for now! However, the technical interconnection of people and places with other people, places, devices, and media-mediated sensory impressions via the internet across all spatial distances reaches a level of virtual reality in which a new post-digital in-between world is emerging. In any case, we are not far off from what Werner Heisenberg once anticipated in 1953: “Perhaps later the many technical apparatuses will belong just as inevitably to human beings as the snail shell belongs to the snail or the web to the spider.”²⁸