Comparisons of Auditory, Audiovisual, and Visual Modalities in Feature Domain for Auditory Brain-Computer Interfaces

Abstract

The development of non-visual P300-based brain-computer interfaces (BCIs) is needed for patients with unreliable gaze control or healthy users with visual distractors. As an alternative means, auditory BCIs have been developed, but reportedly showed relatively low performance. To elucidate the performance gap, this study investigated the feature domain between the auditory and visual BCIs, along with the audiovisual BCI as the combination of the two. Not only the online test, but also a cross-modality assessment was conducted to compare the performance of three modalities, revealing that the classification performance became significantly low when the feature of the auditory BCI was included. When comparing the features that showed significant differences between the target and nontarget stimuli of each subject in each modality, significant individual differences in selected features were more pronounced in the auditory BCI than others, meaning that the common features across subjects were scarce for the auditory BCI. Moreover, the biggest decrease was shown in the auditory modality when comparing the performance of online test and Leave-One-Subject-Out (LOSO) cross validation, which was conducted with selected features. Our results suggest potential sources of the performance gap between auditory and visual BCIs in the context of feature domain.