¶ Designing the telepresence space
To design a telepresence project, it is essential to clearly define the intention and implement it while considering the overall experience; the design of the space must follow a coherent artistic vision for all viewpoints — both on-site and remote. This overall vision must include all connected locations and consider both the spatial characteristics of the physical spaces used and their respective representations. The stage pre-production work that follows can become quite complex and often requires the involvement of the technical managers of each location. Indeed, to properly assess the scenic possibilities offered by each space, the seating plan is not enough; interaction with a physically present person provides a better sense of the real constraints of a space. In summary, here is a list of recommendations for properly preparing the staging of all involved spaces:
- Have one person responsible for the overall view in each venue. For each point of view to be coherent, a balance must be found between them in the staging. It is a constant compromise between the viewpoints of each audience;
- Create a diagram illustrating the installation of each venue to better evaluate their scenographic coherence and share it with the other people involved in the project;
- Establish direct communication with the managers of each of the involved venues to obtain an accurate understanding of the real constraints and possibilities of the space.
Reflections:
- Where are the cameras placed?
- From which viewpoint do we want to see a given performer in the other venue?
- What is seen in the camera’s eye that we want to see in the venues?
- Where are the projection surfaces located?
- Where in the room should what is filmed in the other venue be projected?
- Is the local and/or distant audience captured and projected in the venues?
- What distance should the camera be to get the most interesting framing for the proposed experience? Are the cameras fixed or can they change position during the performance?
- What differences in the architecture of each venue affect what is sent to the other spaces?
¶ Balancing the spaces
The type of space, its configuration, and its size all influence the telepresence experience between locations. It is important to determine whether it is desirable to have a similar or different experience for each venue involved. In general, the more a symmetrical layout is used between locations — such as a face-to-face arrangement — the more it tends to reproduce conditions similar to a natural communication between parties. In this case, a mirror layout is ideal when emphasis is placed on meeting and exchanging between people. Conversely, artists may also choose to create completely different spaces, either to create a unique experience in each location or to give a more ambiguous sense of the nature of the distant place.
In theater, unequal stages are often used to express a dramatic difference between the spaces involved, while equal stages tend to give the impression of an equivalent or balanced dramatic relationship.
In the performance Tshimushuminutsh, both stages are equal; the experience is similar and the two participants are dramaturgically equivalent.
In cinema, when the optical axis of the camera is not horizontal, the shots are either high-angle or low-angle:
- High-angle: when the view is taken from a point of observation higher than the subject;
- Low-angle: when the viewpoint is lower than the main subject.
Depending on the situation, these shots are used to induce a dramatic effect on the object or subject captured, such as a feeling of oppression and fragility for a high-angle shot or, conversely, a feeling of scale and power for a low-angle shot.
In stage telepresence, these strategies — drawn from both theater and cinema — can be combined.
Examples:
- A group of students from École Nationale de théâtre created a theatrical experience where a group of performers in a small room interact with a large theater hall. The audience only had access to fragments of the other venue, creating an effect of fragmentation aligned with the dramaturgy used.
Human-scale visual projection and eye contact help reinforce the feeling of presence by reproducing the natural conditions of in-person conversation. In contrast to this sensation, working with different projection sizes creates different relationships between the characters. For example, a close-up of a performer often creates a sense of intimacy with the audience and a different relationship with the smaller physical body on stage. Camera movement and shot changes more typical of cinema add a more omniscient narrative through the image.
Reflections:
- Do we want a similar or different experience in each venue?
- Is it desirable to simulate the conditions of natural communication between people in different venues?
- Do we want to create a dramatic effect or illustrate an asymmetrical relationship between the spaces?
- Is it better to play with the balance between stages, the camera framing, or a combination of both?
¶ Reproducing a natural communication
In telepresence, it is generally desirable to reproduce an interaction that feels natural between people physically present and those who are remote. Human-scale visual projection and eye contact, for example, are strategies that all promote a feeling of presence through an effect of realism.
The effect of realism (or mimesis) is based on the anthropomorphic cognitive process and describes any strategy or aspect of an interaction anchored in situations or cognitive parameters encountered in the real world. The effect of realism — i.e., the subjective recognition of a certain level of plausibility — tends to provide a context conducive to adopting an inclusive mimetic interpretative behavior that is broadly intelligible to most people.
That is why the most commonly used telepresence systems tend to adopt, as much as possible, the usual codes of interpersonal communication in order to minimize the immersion gap induced by mediation through telecommunication technology. They notably optimize empathy, proprioception, and reproduce several elements of natural in-person communication. However, since the architectures of connected venues are never identical, it is often preferable, or even necessary, to play with the differences in the setup.
Examples:
-
In 1993, Paul Sermon’s installation Telematic Dreaming gives the sensation that participants from distant locations are sitting on the same bed;
-
In the piece Bluff, there are several moments where communication feels natural and the protagonists appear to be side by side.
The observed elements include:
- For visual contact between participants, the camera is positioned at the eye level of the projected distant person.
- To allow expression of non-verbal language, the framing is wide and the camera is placed directly in front of the subject.
- The projection of distant people is at 1:1 scale, and large screens are placed close to the floor so that the characters can be seen from head to toe.
- The projected person must be well positioned to align the gaze.
¶ Equipment and configurations
Depending on the objectives to be achieved, different equipment can be used, or even required, for telepresence projects: cameras, projectors, projection surfaces, microphones, audio monitors, adapters, cables, lighting, computer systems, software, motion sensors, actuators, connected objects, and more.
¶ Video
In telepresence, video capture and projection are important aspects. The images of the venues and the subjects must be transmitted and then visible in the other locations.
¶ Cameras
The type of camera, its positioning, the framing used, and its settings will influence image quality.
Reflections:
- What lighting level will be used during the project? If the camera needs to be used indoors in a low-light context, it is advisable to choose a light-sensitive camera.
- Will the camera be on stage? What form factor should it have? A compact format is often preferable to limit its visual footprint.
- What angle of view is required on the filmed subjects? Usually, standard lenses allow proportions to remain at a human scale. Wide-angle lenses allow capturing a larger area, such as when filming an entire audience.
To learn more about the technical aspects of video display, see the section Using the right type of camera in the Telepresence technical best practices.
¶ Positioning and framing
The placement of cameras — whether they are located on stage or in the room, operated fixed or mobile — is a crucial decision.
¶ Fixed
If the cameras and projection surfaces remain fixed in the same place throughout the performance, you must ensure the space is configured so that the cameras do not interfere with the audience’s visibility or the projection surfaces.
This type of installation has the advantage of offering viewers a consistent, clear, and understandable reference. The lack of camera mobility also supports the impression of a virtual “window” onto the distant space and is less likely to break the co-presence effect that visual stability establishes between locations. Indeed, any camera movement reminds the audience of its existence and that the mediated experience is not intentionally neutral.
¶ Mobile
Using mobile cameras, on the other hand, requires accepting that the technical apparatus may sometimes be visible and may even play a role in the overall choreography. Whether handled by performers or an operator, shoulder-mounted or remote (robotic cameras), mobile cameras allow for expanded shot and framing possibilities. For example, a wide shot provides a general view of the space, while a close-up creates a sense of intimacy with the performer.
In the performance Bluff, at certain moments, the camera is positioned facing the audience so that the performer appears at 1:1 scale in the foreground with the audience behind. This creates a large 360-degree assembly where audiences can see each other.
¶ Human-scale visual representation
To create a space where communication between venues feels as natural as possible, a few basic recommendations apply:
- Position the camera frontally, with no angle, therefore no high-angle or low-angle effects;
- To promote eye contact, the height of the camera should match the subject’s eye level, or otherwise the waist height for a full-body shot;
- Adjust the camera distance and zoom to obtain an image of the subject at a human scale (1:1);
- Conduct a projection test to compare the size of the projected subject with that of the in-person subject, side by side. It is recommended to adjust the projection so that the projected subject is 5–10% larger than the in-person subject. This helps amplify the presence of the distant subject, whose flattening by projection tends to reduce it;
- To achieve a corresponding scale between locations, it is preferable to have similar projectors, projection surfaces, and cameras — ideally with the same settings (position relative to the subject, zoom, focus, white balance, color level, etc.). Otherwise, you must manually adjust the cameras’ settings in each venue to obtain an apparent equivalence in size;
- Angle of view: does the camera capture all essential elements?
- When a camera captures subjects head-on, it should be parallel to them;
- During multi-camera events, it is preferable to avoid overlapping shots; it is not desirable for a subject to appear duplicated in more than one shot at the same time, as this could harm the sense of presence.
¶ Video processing
There are several video processing software tools for developing an optimal image composition for telepresence, including OBS Studio (integrated into the SCENIC station), Modul8, MadMapper, Millumin, and Resolume. With SCENIC, video processing can be done in real time on the video stream and then sent to the connected venues. To combine multiple cameras into a single video input, a video switcher is required.
¶ Projection surface
Using different types of projection screens allows you to specify the artistic intent. This makes it possible to develop scenographies where projection surfaces have different levels of opacity, different shapes, and different textures.
Here are some examples of scenographic effects possible depending on the type of projection surface:
- Opaque, rigid panels: for a clearly visible projection that captures light well.
- Tulle, mosquito netting, glass, plastic film: to play with transparency or layering of subjects.
- Spandex or other stretch fabrics: to touch and manipulate the projection.
- The Pepper’s ghost technique: to create a holographic-style presence.
- Rear-projection screens: to place elements just in front of the screen without blocking the image or casting shadows.
- LED screens: to optimize image visibility without distortion from stage lighting, ideal when you want to film a projection.
- Using performers or subjects on stage as a projection screen: to create a different effect on the relationship between the in-person and distant performers.
In the performance Onirisme, artist Isabelle Clermont uses spandex as a scenic element to create a tactile relationship with the video projection.
The LED screen was used by director Françoise Dancause in her performance CorresponDanse de guerre to film characters from the other venue.
Screens can be positioned in different ways in the space, and their size greatly influences the experienced effect. Often, venues are equipped with a large screen at the back of the stage. Although its size allows for a large surface, creativity is required for performers to interact with a projection located behind them. Using a screen within the playing space allows for an easier relationship between the in-person and distant subjects. For example, a human-sized screen positioned vertically on stage reinforces the illusion of the real presence of protagonists located in another venue, and people on stage then interact more naturally with the projection. In the early explorations of the Bluff project, simple banner supports were used.
To learn more about the technical aspects of projection surfaces, see the section Using the right type of video display in the Telepresence technical best practices.
¶ Video projectors
Depending on the surfaces available, the type of projection may vary. Unless using video mapping software, you must ensure the projector beam is aimed directly at the screen, whether it is a front or rear projection. You must also take into account that some venues do not offer the same flexibility in terms of projector placement and distance to the projection surface.
- The larger the surface and the brighter the space, the more powerful the projector must be for the image to remain sufficiently sharp;
- Wide-angle projectors with a short focal length should be considered when the space is more confined, as they can be placed closer to the surface without significantly reducing the projection dimensions.
To learn more about the technical aspects of video projection, see the section Using the right type of video display in the Telepresence technical best practices.
¶ Lighting
Integrating one or more video projections into the stage requires a careful balance between stage lighting, necessary for the cameras to capture a good-quality image, and the brightness of projections on screens. Properly calibrating the light on the performers in relation to the projection’s power is a recurring challenge that should not be overlooked.
To learn more about the technical aspects of video projection lighting, see the section Using the appropriate type of lighting in the Telepresence technical best practices.
¶ Audio
¶ Microphones
In telepresence, sound must be captured, processed, and compressed before being transmitted to the other venues. Ideally, one microphone (hypercardioid, unidirectional, or dynamic) is planned per participant, and omnidirectional microphones are avoided to prevent feedback. Therefore, it is difficult to capture the audience as a whole. In some projects where the audience participates, using a mobile microphone can be useful. For example, a Catchbox-style microphone can be thrown to participants who want to speak.
In the project Masq’ensemble, young people used a Catchbox microphone to easily pass it between participants.
¶ Mixing
Although the inevitable transmission delay (latency) between locations limits the feedback recursion that causes howling, it introduces an echo effect. Like feedback, the louder a sound is played through speakers and picked up by microphones, the more the system amplifies the echo.
To counter echo, maintain a sufficient dynamic gap between the desired sound (higher level) and unwanted sounds (lower level) to optimize the effectiveness of the gate applied to each mixer channel receiving a microphone signal. This is achieved by positioning the main speakers farther away and directing them away from the microphones. Conversely, place the microphones as close as possible to the source to keep the input gain low.
To learn more about technical aspects of audio capture and reception, see the Audio section of the Telepresence technical best practices.
¶ Spatialization
Distributing sound sources across multiple speakers generally provides a better listening experience for the audience. It promotes immersion and preserves coherence between image and sound. For example, if the sound source transmitting the voice of a person in another venue is positioned where their image is projected, it helps the audience better locate the origin of the voice and facilitates decoding through lip reading.
To learn more about technical aspects of sound diffusion in a venue, see the section Spatializing sound sources in the Telepresence technical best practices.