¶ Interaction and Distance
In short, scenic telepresence is a form of live art involving interaction between individuals who are geographically separated. Most often, this interaction is made possible by a technological setup that allows near-instant communication between different locations.
Today, telepresence systems almost always require an internet connection to create a shared virtual event. The purpose of this practice is to connect several geographically distinct spaces by partially reproducing them through various technical means. The main goal of such a setup is to foster the sensation or effect of presence among physically separated individuals, giving the impression that they are gathered in a common space.
Telepresence requires that the senses of the involved individuals are subjected to stimuli—most often audiovisual, but sometimes also haptic—from a remote site, and that they can interact in the same way with it. Thus, the position, movements, actions, and words of individuals can be perceived, transmitted, and duplicated in real time to the desired destination, making interaction effective.
¶ Scenic Telepresence
Julien Brun, a director specializing in telepresence, defines it as:
“Telepresence is presence at a distance, the use of digital tools to facilitate the meeting of living beings separated by geographic distance.” Julien Brun
Scenic telepresence differs from the more widespread form of telepresence—videoconferencing—by focusing more deeply on the effect of presence made possible by the technical setup. While videoconferencing has become common for allowing several distant individuals to meet virtually, usually in a face-to-face configuration, scenic telepresence is interested in broader meeting configurations—especially in the performing arts—and often involves entire bodies or a portion of a space or stage. Another notable aspect distinguishing scenic telepresence from videoconferencing is the higher quality and stability of sensory stimuli transmitted between the involved sites.
See the section About scenic telepresence for more information.
¶ Interdependence
For a telepresence project to be fully interactive, it is recommended to establish interdependence between scenic elements in different locations. The audience’s interest is stimulated when all connected venues have a precise and necessary function in the event; each space connected by telepresence must be meaningful and in constant interaction with the others.
In a dramatic work, for example, an entity in location X needs to interact with an entity in location Y to advance their quest. When there is text in the work, the dynamism of the relationship between locations is especially optimized when sentences are short. Monologue is a possible form, but it is important to consider what happens in other locations where the monologue performer is not physically present. Narratives are thus composed of several parallel realities and can involve different means of expression simultaneously (images, lights, sounds, voices, movements, etc.).
In a more playful situation, such as a telepresence game—where each team is in a different location but shares a common virtual playing surface that would not exist without the telepresence setup—both spaces are also completely interdependent; the game would be impossible without the other location, regardless of whether participants are rivals or collaborate as a team. The audience immediately perceives the relationship to the other location and the live interactivity.
During the Nuit Blanche 2020 at the SAT, participants in the telepresence Tic-Tac-Toe game experienced an inspiring example of interdependence. Two connected spaces, located on two different floors of the building, shared a common virtual surface for the famous game. Participants upstairs played with « X » and those downstairs placed « O ». Both spaces were completely interdependent; the game was impossible without the other location. The audience, by participating, immediately understood the relationship to the other location and the live nature of the activity.
The collaborative puzzle game for children, Pipoes, developed by ABLBLALAB and presented at the three-room event Bibliomix, is another good example. In each room, there was a group of children, separated by three colors. For example, the reds were spread across three cities. The goal was to get their team to guess as many words as possible at a distance by miming or making a sound. In this way, the game offered real interaction and collaboration between the rooms.
In all cases, it is essential to have a rich and complete, yet unique, experience from each point of view throughout the performance. No location should be neglected; there must be balance overall.
Questions to consider:
- What is the purpose of having one or more other locations?
- How does one location need the other in the work or event?
- How does this change the environment?
¶ A Variety of Possible Relationships Between Locations
Seeking balance does not mean that all locations have the same role or are identical reproductions. Different relationships are possible, ranging from complementarity to equality. Here are some typical examples:
- Equality: several distant locations interact together and mutually present all content.
- Complementarity: one location presents most of the content to satellite locations, and individuals in those satellite locations can respond and interact with the main location.
- Mixed: several interacting locations jointly present most of the content to one or more satellite locations, and individuals in those satellite locations can respond and interact with these locations.
In all cases, roles may vary, but interaction between locations remains essential; otherwise, it would be simple broadcasting, indistinguishable from a delayed event.
¶ Role and Engagement of Involved Individuals
In all cases, each location presents one or more individuals who will interact with those in another location. However, their role and the importance of their respective level of engagement can vary greatly. Thus, each location can:
- Host a physically present audience or not;
- Feature artists or performers or not;
- Provide an animation or mediation service or not;
- Host one or more facilitators;
- Offer a space or stage where any present individual is free to interact with the other.
These possibilities are obviously not mutually exclusive and can be combined. But it is essential to consider how interaction will take place between locations, by asking questions such as:
- Who can interact with the other location?
- Is it direct interaction (with capture) or indirect (with mediation, animation, etc.)?
- Is it a telepresence situation that is temporally measured or improvised? And to what degree?
- Is the interaction continuous or occasional over time?
- What role does each individual involved in the interaction have (facilitator, stage management, animation, mediation, assistance, performer, technical, artistic, audience, etc.)? And who has authority over what?
¶ Possible Themes
¶ Artistic
Telepresence technology has inspired many creators in the performing arts. Some have used it to develop a work, others have employed this medium to fulfill an artistic intention. Certain themes go hand in hand with telepresence and highlight an artistic message. By analyzing some telepresence works brought to the stage, we notice a recurrence of certain themes:
- Dialogue at a distance, situations of separation between protagonists;
- Communication challenges;
- Notion of absence-presence (relationship between physical bodies and mediated/virtual bodies);
- The body as a vector of communication;
- Virtual identity and personality;
- Real versus imaginary space;
- Hypermediatization of society;
- Otherness of the body / specter;
- International issues, etc.
The creation CorresponDanse de Guerre addresses the situation of distance, where in one location there is a soldier during World War I and in the other, the family waiting for their return.
¶ Social and Cultural
Telepresence is a powerful tool for bringing together communities and groups. It can be used for cultural mediation or social projects. Here are some examples of social or cultural topics and contexts that have used telepresence:
- Bibliomix: Ideation on digital technology in libraries;
- Interregional numérique: Digital appropriation for territorial development;
- Ulysse et Pénélope: Travel and correspondence;
- Bibliolab: Distance learning;
- Speed-booking: Discovery of literary works through rapid argumentation;
- Radio-Ressource: Research phase for a theatrical documentary on social issues in regions and their resources.
The Radio-Ressource project is a theatrical research project questioning our relationship to territory.
Questions to consider:
- What theme emerges from the telepresence situation?
- How does telepresence highlight or deepen elements present in the artistic material?
- What topics are suitable for telepresence?
¶ Audience Engagement
Some project leaders choose to place spectators as witnesses to what happens. Others involve spectators in the decisions of the actors. These different choices define the spectators’ role during performances (immersion as external voyeur or participatory voyeur) - Gabrielle Godin
In telepresence, given the novelty of the experience, the audience needs time to master the situation and the devices. They must first understand the distance between the locations in the story and be sensitive to both physically present individuals and those whose images come from other connected locations.
The audience must quickly grasp which audiovisual elements come from other locations and that it is a real-time transmission. It is good to regularly remind them of the immediacy of the performance. One frequently used method is to start the event with interaction between audiences. When the audience is involved, it is important to respond to their engagement. This exchange avoids a one-way relationship and immerses the audience in the action: they understand that their presence can influence the course of the event live.
For example, in the show Tshimushuminutsh, the audience is invited during the performance to ask questions to the Indigenous elders on stage.
Questions:
- What is the relationship between audiences?
- Is the audience from the other location visible in the installation, scenography, etc.?
- What are the audio solutions for perceiving the distant audience?
- How to maintain audience interest in what is happening elsewhere?
¶ Technical
¶ Latency
In telepresence, there is always a short delay between when information is sent and when it is received. In most cases, this minimal latency is quite acceptable. But in cases where fine synchronization is important, such as in music or dance, latency can become a significant issue that may influence the presented content.
This short delay is unfortunately unavoidable due to the nature of the transmission chain: audio and video streams are first captured, then digitally encoded by a computer system with its own « clock, » which transmits them over the internet to one or more other locations; conversely, the received streams must be digitally decoded by another computer system (another « clock ») before they can be broadcast.
Although audio signals can be transmitted faster than video streams—because they use less digital data—synchronizing audio with video is usually essential. Therefore, the overall latency time is generally determined by the video stream transmission.
Since the simultaneity of interaction between locations is never perfect in telepresence, its effects must be anticipated, contrary to a non-telepresence situation.
If telepresence exchanges are generally in the form of dialogues—thus masking latency between each intervention—the issue becomes particularly apparent when musicians try to play together in telepresence. In the case of moderately rhythmic music at a moderate tempo, for example, a latency as short as 30 ms can be perceptible and destabilize performers and the audience. There are, however, some proven strategies to overcome or adapt to this latency:
- Alternation: the simplest strategy is to avoid musicians playing simultaneously, but rather successively, taking advantage of pauses and silences between each musical phrase to mask the effect of latency. A representative musical game of this strategy is call-and-response, which imitates a dialogue between individuals.
- Canon: similar to alternation, canon allows some simultaneity of instrumental play in telepresence because it consists of imitating musical phrases in delay. The time difference between the original phrase and its imitation can be synchronized with the telepresence system’s latency to remain musically coherent in each location.
- Guide and accompaniment: this simple strategy can be combined with others, but essentially consists of designating a musician as a musical guide, like a conductor or a percussion metronome, for example. This guide does not listen to the others and plays their part as a solo, while musicians in other locations hear them and play following the guide’s rhythm. Unfortunately, the result will not be simultaneous for the guide. That’s why it may be recommended to have a silent guide—literally a conductor who provides the tempo from a location separate from all the musicians.
- Lento/Largo: this strategy refers to a slow or broad musical tempo (between 70 and 40 beats per minute), more favorable for masking latency through « silences » and « soft » or gentle attacks, for example.
In music, the notion of attack characterizes the beginning of the dynamic profile of a produced sound. For example, a « soft » attack is heard as a reinforcement of resonance—or attenuation of the onset—of a sound; a « gentle » attack has a rather null dynamic profile, often gradual and without apparent attack, etc.
- Rubato: like music at a slow tempo, this strategy aims to mask latency or apparent asynchrony of sound events within a variable or improvised tempo.
In music, the term rubato refers to great rhythmic freedom, either by advancing or delaying certain notes of a melody compared to a fixed tempo, according to the performer’s inspiration.
- Virtual instrumentation: using virtual rather than acoustic instrumentation eliminates the analog-to-digital conversion step in each location, but can be relatively complicated to set up and remains less natural than acoustic instrumentation. Moreover, this strategy is difficult to adopt for singing. The principle of this approach is not to transmit audio signals directly to other locations, but only data from a control surface using the MIDI or OSC protocol. These data are much faster to transmit because they are less voluminous than an audio signal and do not require analog-to-digital conversion upstream. Once received in the other location, these data can be used to control musical parameters (pitch, duration, timbre, intensity, etc.) of a virtual instrument whose generated signal would then be broadcast by the local sound system.
In the creation of the project Ailleurs Ensemble, an artist or duo in Montreal develops a blended musical creation with an artist in Saint-Camille or Rimouski. Each duo develops their methodology for working with latency. The duo of Estelle Charron and Robin Servant explores, within the same composition, moments of alternation and then guide and accompaniment.
Questions:
- How is latency an issue or an asset for the project?
- How to work with this constraint?
- How to adjust the pace of communication to provide new rhythmic cues to performers and the audience?
¶ Superimposing Distant Spaces by Keying
Keying is a visual effects technique commonly used in film, television, and photography, allowing objects filmed separately—in another location—to be integrated into the same image.
Thanks to this technique, it is possible to combine (superimpose) two distant locations within a single virtual telepresence space—a single video stream.
The keying technique is summarized as follows:
- A subject is filmed in a location against a blue or green background;
- Using real-time visual processing software, the blue or green background is transformed into a black mask—a technique called chroma key—to obtain the subject’s silhouette transparent against a black background;
- The black background can then be replaced by another video capture, from another location, for example.
The project Masqu’ensemble invites children from Rimouski and Saint-Camille to create a masked work bringing both groups together on the same virtual surface.
The choice of blue or green background is explained by the near absence of these colors in human skin. Although blue is closer to the complementary color of flesh, offering optimal chrominance contrast, green is most often used because it is the brightest primary color in video. Indeed, it allows better luminance contrast with the subject to be keyed—especially due to YUV encoding, where green is directly related to luminance (Y ≃ Red + Green + Blue), while blue is also related to chrominance (U ≃ Blue – Y; V ≃ Red – Y).
Questions:
- Is it desirable to broadcast telepresence spaces separately or to superimpose them within a single virtual broadcast space?
- How will the audience perceive interactions between different subjects?
- How will subjects perceive themselves during their interactions?
- How is superimposition relevant to interaction between locations?
