Neurocinematics as passive-BCI based Applicaition : The EEG study on neural responses of human during watching movie

Abstract

Traditionally, brain-computer interface is mostly composed of researches for the rehabilitation of paralyzed patients with the objective of controlling bio-signal through external devices. However, recently the concept of the research has widened to understand the user’s processing of cognitive emotional information for non-medical purposes and has classified this as manual brain-computer interface. Among these, neurocinematics is a manual brain-computer interface’s applied research field which tries to understand the changes in the cognitive or emotional state of the viewer while watching a movie. There are two main reasons why this field of study is receiving particular attention recently. First, movies do not only have audio-visual stimulus, but are composed of different factors such as culture and environment, and it can help in studying the human’s social cognitive process. Another reason is that the original survey or post-interview method of audience reviewing about movies has a limitation - audience should be aware of their conditions; so the credibility is low. On the other hand, neurocinematics studies observe it through bio-signal and believes that a more objective verification is possible. However, in existing researches, they mostly used the method of validating findings by comparing the results of the neurocinematics research through bio-signal with the original result from the survey. Also, a lot of researches were done to know if most subjects made the same reaction while watching the same movie, but they obtained the bio-signal through individual viewing. This research has verified the objectivity of engagement index extraction through the introduction of psychophysical methods to overcome the limitations of existing studies. While the subjects were watching a movie in one room, their brainwaves were measured. Changes in the brainwave synchronization levels between the subjects were also checked. Moreover, we proved the changes in the level of brainwave synchronization in two conditions: when people are watching individually and when they are watching in a group. In the first experiment, we used a psychophysical method called Secondary Task Reaction Time (STRT), which is known for representing concentration to evaluate Neural Engagement Index (NEI). STRT is used to check the reaction speed for a tactile stimulus given additionally while the subject is doing the main task. It is known that the reaction rate gets slower when the subject is more engaged to the main task. In this experiment, we measured the STRT and NEI while the subjects are watching 8 movie trailers that are not yet out in the cinema. After watching each trailer, the subjects completed the survey. As a result, there was a significant correlation between STRT and NEI, but in the survey, there was no meaningful correlation. In the second experiment, while four subjects were simultaneously watching the Chaser (2008, Silk road), their brainwaves were measured and analyzed 5 per frequency band different inter subject correlation (ISC). Moreover, using sliding show method, we analyzed the correlation between time: Delta2~4Hz), Theta (4~8Hz), Alpha (8~13Hz), low Beta (13~18Hz), high Beta(18~23Hz). Also, to verify the derived correlation’s significance, the result of checking the correlation of the data that had been time shifted on each window and its 95% range through non-parametric permutation analysis, the researchers observed time slots that had specific significance on each band correlation. As a result, in the span of the movie’s whole running time, researchers observed that there were parts where a significant correlation increased between the subjects’ band power. Especially there were a lot of meaningful correlations found in the movie when it reaches the emotional climax, and they were important scenes in the movie when it comes to development of plot, these scenes the same with the scenes picked by the majority of the audience. In the third experiment, based on the result of the second experiment, the researchers checked if the audience’s reaction changes depending on the viewing conditions when watching the same movie content by applying a brainwave-based response model. The viewing conditions were divided into a group of people who watched the movie together, and the viewing group who watched the movie in separate rooms. We proceeded with the experiment after recruiting 8 subjects per group. The data of two groups watching together and one group watching individually was built. We analyzed the collective responses to the applied brainwave frequency inter subject correlation coefficient of each group: Delta (2~4Hz), Theta (4~8Hz), Alpha (8~13Hz), low Beta (13~18Hz), high Beta (18~23Hz). During the whole period of watching the movie, as the result of analyzing the rate of ISC increased significantly, the rate of ISC increase of the group that watched the movie together had a higher ISC increase rate than the group that watched the movie individually, in all frequency bands