What is Transmission?
Transmission is a proposed research project at Bornemouth University that focuses on telematic performance and real time sonification and visualization of brain waves. A remote user changes and reacts to someone else’s brain activity, eploring a feedback loop of action and interaction. This project is a collaboration between artists and scientists. There is not much research on the topic of sound, visuals and brain activity, so it is hoped that the art piece generates scientific data.
The piece will be a 3D holographic immersive projection, in which users visually and sonically explore their brain in real time to develop “a new language that engages their senses on multiple layers”. This allows the player to learn about the reactivity of their own nervous system on both a conscious and subconscious level.
Interactive art has a history of shifting perception though engaging kinetic activities, exploring chain reactions of cause and effect and Transmission explores this. Transmission explores the sense of presence – of “being there, being elsewhere” – though remote interaction. Users never get to see each other, but effect each other and their own minds.
The project was inpired by the “therapeutic practice of nurofeedback applications”. The collective plans to use the piece to improve nurofeedback in therapeutic applications.
Aims and Questions of the Project
Artistically, the aim of Transmission is to create a self reflective experience of communication though interactive engagement. In the immerse space, the internal experience of dialogue will be externalized though visual and sonic representation of stimuli. The juxtaposition of kinetic, physical movement and invisible, psychological impulses allows the audience to reflect on their perception of the internal and external, body and mind.
The questions asked in this project are: “Is the mind located in the brain?”, “How do we react to external presence – especially if this presence is only mediated, no visual and not representational?”, “What effect does presence have on our neurological activity?”
Within their academic research, the collective sought a classification system to explain telepresence, with various academics breaking up the phenomenon based on cues of individual variables. The most concise of these details how it can be narrowed down to three factors: immersion (Barfeild and Weghorst, 1993, Prothero and Hoffman, 2007, cited by this paper), involvement (Hoffmann et al, 1998, cited by this paper) and realism (Barfirld and Hendrix, 1995, cited by this paper)
Telepresence is now a feature of everyday life and has changed dramatically from its origins. Models for measuring data on telepresence must focus on the user’s individual experience and perception and be egocentric and subjective.
Digital environments and visualizations can be used to draw the player into a virtual space and create a meditative state. “Virtuality is conceptualized as a sub-component of interactivity: for us in digital environments, 3D display technologies are a meditating presence as much as networked environments, mobile phones and other handheld devices.”
Immersion is a perceptive and psychological phenomenon. It can be seen as a physical reality or as a subjective condition. To be immersed, there must be a degree of absorption into an environment, with the exclusion of the real world or the media reality being the dominant actuality. The degree of immersion can be measured by the amount of senses that are affected by the media environment. Immersion is not a condition by itself, but a factor of presence.
Interactivity can be described as a perceptive reality. How real this is is determined by the engagement and personal involvement of the user. The “Transmission” project looks at interactivity as a factor in the user’s perception of a remote presence, being facilitated though visual and audio technological cues.
The setup of the peice uses two dedicated spaces: a motion capture studio for real time transmission and a holographic studio for display and visualization. The motion data controls a particle steam displayed on the holographic interface. Sound and motion data are generated in real time, allowing it to be modified. The holographic setup creates the illusion of 3D using optical cues. These interfaces allow for cinematic, immersive experiences, which aids Transmission’s “social engagement with the technological environment.” Brain wave data is soinfied and played though surround sound speakers in conjunction with the visualization, aiding this engagement. The feedback received at this location is the relayed to the performer in the motion capture studio. The data is relayed though a motion analysis server which passes animation data to Maya and converts audio to Open Sound Control signals before also passing this to Maya.
Participants are given tools to understand their own neurological activity though neruo-feedback methods. The player’s brainwaves, when connected to an Emotiv EEG headset can change colours, particle emissions and particle forces, as well as procedural sound generation. The core of the visual experience will be an avatar on the holographic display that works as an extension of the performance space.
“Transmission” is both a subjective (immersion) and social (interactivity) experience. Telepresence can be examined in the space between these two phenomenon. The team can gather both qualitative and quantitative data to examine this. They explored how different visual and audio cues affect the feeling of telepresence. The artistic interpretation of this brain wave information can be used for introspection, therapy or medical applications.
“Transmission” will be performed in art and science centric museums. The team look to having a permanent installation of the piece, with professional dances being invited to act and interact in the motion capture space.
Gingrich, O., Emets, E. and Renaud, A. 2014.Transmission – Sonifying, Visualising and Analysing Neural Activity through Telepresence. In: Electronic Visualisation and the Arts (EVA 2014), London, July 8-10 2014. London: BCS. [online]. Available from:http://www.bcs.org/upload/pdf/ewic_ev14_s22paper1.pdf [Accessed 12 January 2014]