Neurocinematics: How Brain-Computer Interaction Is Changing Cinema

Research is beginning to analyze viewers' brain activity in response to film sequences and this may lead to deeper audience engagement by empowering brain-to-computer driven changes in plot and outcome. JL

Polina Zioga reports in Singularity Hub:

Brain-computer interaction in cinema can enhance audience engagement (and) empower collectively control (of) a film through combined brain activity together with study of the effect of films on the spectators’ brain activity (neurocinematics). The most recent innovations are wireless brain-computer interfaces, available as low-cost headsets. Already used in computer games and the arts, more recently they have been applied in filmmaking. Universal and 20th-Century Fox have released interactive versions of their films where the spectator can control the plot with the use of a BCI headset.
Over the past few years, we have seen the extraordinary development of neural prosthetic technologies that can replace or enhance functions of our central nervous system. For example, devices like brain-computer interfaces (BCIs) allow the direct communication of the brain with a computer. The most common technique applied in these devices is electroencephalography (EEG), a recording of the electrical activity along the scalp.
These technologies are mainly used in health, but our new research shows how they are changing the future of cinema too.
This is no coincidence. Artists have been among the pioneers in the use of these technologies, developing creative applications since their first emergence in the 1960s. Early examples include Music For Solo Performer (1965) by Alvin Lucier, which is considered the first performance using EEG technology. Interactive artworks, like the Brainwave Drawings (1972) by Nina Sobell and installations like Alpha Garden (1973) by Jaqueline Humbert also illustrate how the art world paved the way.

Interactive Filmmaking

During the same period, the first interactive film was presented. The comedy Kinoautomat (1967), which was created in Czechoslovakia, allowed the audience to vote on what should happen next by pressing buttons. Since then, acclaimed filmmakers like Peter Greenaway have been advocating the new possibilities of interactive technologies in cinema.
More recently, the film industry is showing interest in emerging technologies like virtual reality (VR). A milestone in this direction was the special award presented by the Academy of Motion Picture Arts and Sciences Board in 2017 to Carne y Arena, directed by Alejandro Gonzalez Iñárritu. Carne y Arena is a VR installation which was said to be opening “new doors of cinematic perception.” This follows on from the work of an increasing number of festivals (like Berlinale and the Venice Film Festival), filmmakers, and researchers who are investigating the potential of using new interactive technologies in cinema.
Among the most recent innovations are new wireless brain-computer interfaces, which are now available in the market as low-cost headsets. They are already used in computer games and the arts, but more recently they have been applied in interactive filmmaking as well. For example, Hollywood studios like Universal and 20th-Century Fox have released interactive versions of their films where the spectator can control key moments of the plot with the use of a BCI headset.

Multi-Brain Interaction

Our research sheds new light on how our brains can control a film or live cinema event, not just for home entertainment but also in a cinema theater. And it shows how this can be used to bring to the audience a new, engaging, and collective experience.
More specifically, we developed a new system that allows multiple brains to interact. This system was used in a new live cinema event that we created for this purpose and presented at the CCA: Centre for Contemporary Arts in Glasgow.

Illustrative example: The passive multi-brain EEG-based BCI system. Zioga et al. 2016 (Images of human heads originally designed by Freepik).

For the first time, the technology enabled one performer and two members of the audience to interact simultaneously. Using their passive brain-activity separately or jointly, they were able to control aspects of the live projected film. Their transition from relaxed to more alert cognitive states was visualized as a shift from colder to warmer tints. This created the constantly changing color of the live projections and set the overall atmosphere of the narrative.
The event was also a neuroscience experiment in a real-life environment with the presence of a public audience. This allowed us to obtain data from the participants, which have provided us with important results. They showed they were able to understand which parts of the event were controlled by their brain activity and how. These particular scenes made a special impression on them. And at the same time, their attention and emotional engagement were also increased, while having the feeling of being “connected.”

‘Enheduanna―A Manifesto of Falling’ Live Brain-Computer Cinema Performance at CCA: Centre for Contemporary Arts Glasgow. Polina Zioga/Catherine M. Weir.

Cinema’s New Horizon

Our experiment provides new tools and methods for creating interactive films with the use of the brain activity of the spectators. It proves how the use of brain-computer interaction in cinema can enhance the audience’s perception and engagement. And even more, it opens a new horizon of possibilities. Audiences in the future will be empowered to immerse themselves and collectively control a film through their combined brain activity.
Together with studies looking at the effect of films on the spectators’ brain activity (neurocinematics), these new possibilities will push further the motion picture arts and sciences. They will also advance our understanding of how we collectively engage, collaborate, and compete in emotive environments and situations.