Mobile brain imaging in butoh dancers: from rehearsals to public performance

Abstract

Background

Dissecting the neurobiology of dance would shed light on a complex, yet ubiquitous, form of human communication. In this experiment, we sought to study, via mobile electroencephalography (EEG), the brain activity of five experienced dancers while dancing butoh, a postmodern dance that originated in Japan.

Results

We report the experimental design, methods, and practical execution of a highly interdisciplinary project that required the collaboration of dancers, engineers, neuroscientists, musicians, and multimedia artists, among others. We explain in detail how we technically validated all our EEG procedures (e.g., via impedance value monitoring) and minimized potential artifacts in our recordings (e.g., via electrooculography and inertial measurement units). We also describe the engineering details and hardware that enabled us to achieve synchronization between signals recorded at different sampling frequencies, along with a signal preprocessing and denoising pipeline that we used for data re-sampling and power line noise removal. As our experiment culminated in a live performance, where we generated a real-time visualization of the dancers’ interbrain synchrony on a screen via an artistic brain-computer interface, we outline all the methodology (e.g., filtering, time-windows, equation) we used for online bispectrum estimations. Additionally, we provide access to all the raw EEG data and codes we used in our recordings. We, lastly, discuss how we envision that the data could be used to address several hypotheses, such as that of interbrain synchrony or the motor theory of vocal learning.

Conclusions

Being, to our knowledge, the first study to report synchronous and simultaneous recording from five dancers, we expect that our findings will inform future art-science collaborations, as well as dance-movement therapies.

Background

Background information on the brain architecture of dance

In the past two decades, there has been a mounting interest in identifying the neural underpinnings of artistic expression, and of dance, in particular. The first endeavors towards this direction have focused on studying the brain responses during dance observation, namely while dancers, or non-dancers, perceive videos of dance movements of themselves or others. Brain perception signals have been studied for a variety of dance genres, including but not limited to jazz [1], ballet [2, 3], and tango [4], using either electroencephalography (EEG) [1, 3, 4] or functional magnetic resonance imaging (fMRI) [2]. Overall, their findings underscore the power of both techniques to capture distinct patterns that distinguish dance perception, in an array of settings, such as dance perception by expert dancers vs. non-dancers.

Identifying the neural basis of dance performance, involving the actual production of dance movements, has proven challenging, considering the limitations of neuroimaging techniques that render natural movement in space impractical. Still, researchers have come up with creative ideas to address this question. For instance, Brown et al. [5]. used an inclined surface in front of the leg room of a positron emission tomography (PET) scanner, where amateur dancers performed small-scale, cyclic leg tango steps while in a supine position. The same group used fMRI to study bimanual partnered movements, with the experimenter sitting next to the reclined subject holding hands, and alternating between “leading” and “following” joint movements, similar to those used in tango or salsa [6]. In turn, mobile EEG techniques, complemented with motion sensing, have enabled researchers to study brain activity while subjects are dancing freely in the space with the EEG caps on. For example, mobile EEG studies on Laban movement analysis (LMA) dancing [7] demonstrated the feasibility of classifying specific movements and LABAN effort qualities from distinct EEG signals, and proposed a framework for eliminating motion artifacts from dance analysis. EEG has also proven effective in picking up not only sex-specific effects during contemplation of jazz dancing but also sex-independent effects during physically dancing jazz [8].

It is in this context that we decided to study the brain activity of five experienced dancers while dancing butoh via mobile EEG, recorded simultaneously and synchronously (a process known as hyperscanning). To our knowledge, this study marked the first time that both this type of dance (butoh) and this number of dancers (five) were studied. Moreover, this art-science collaboration allowed us to monitor the creative process through EEG recordings during rehearsals culminating in a theater performance in front of an audience. In this paper, we aim to explain the background, design, neuroengineering methods, and technical validation of our EEG, EOG (electrooculography), and IMU (inertial measurement units) procedures. We report in detail the methodology (e.g., bispectrum estimation, filtering, time-windows, equation) that enabled us to generate a real-time visualization of the dancers’ interbrain synchrony on a screen, while we also propose and demonstrate a signal preprocessing and denoising pipeline that can be used for future offline analyses. Importantly, we openly share all the raw data and code resulting from this experiment. Lastly, we discuss specific objectives and hypotheses that our data can address, and how we envision that this interdisciplinary work can inform future art-science collaborations and therapeutic practices, using dance as a therapeutic modality for improving wellness and motor deficits.

Background information on butoh

Butoh is a Japanese avant-garde dance originated by Tatsumi Hijikata and Kazuo Ohno at the height of the counterculture movement in Japan in 1959 [9, 10]. Although it defies a precise definition, butoh has been described as a type of dance that allows the exteriorization of bodily reactions, otherwise suppressed in social settings, such as spasms, involuntary jerks, tremor, facial or bodily distortions, falling, stamping, and rolling on the floor [10]. Unlike in other dance styles (e.g., ballet), butoh dancers do not pursue high jumping or fast spinning, they rather focus on their breath and subtle body reactions [10]. Butoh has also been seen as a “meditational dance”, due to being a contemplative movement practice that includes deep relaxation and meditative calmness [9]. In this, it is similar to other meditative practices, such as Tai Chi or yoga, although, according to Kasai [9], meditation does not picture the essence of butoh as a whole, since the calmness can be interrupted by explosive movements.

These descriptions considered, there are several characteristics that make butoh fall out of the narrow and Western definition of dance [9]. One example that distinguishes butoh from other dances is that butoh dancers must be able to execute movements with a modified use of vision; visual stimuli are often shut down, thus the dancer learns to enhance other senses and focus their receptivity to sound stimuli. Another example is that movements in butoh do not tend to be entrained to a periodic metered rhythm, as is the case for most Western dances. Still, the dancers respond to sounds and musical cues similarly to how ballet, contemporary dance, tango, or hip-hop dancers follow the beat. In the choreography performed for this experiment, half of the soundscape was organized rhythmically, and oftentimes, the dancers were counting, much like in other dance forms. Another section included nature sounds, where the dancers followed sound cues (e.g., pouring rain, bleating of a goat, owl hoot) to inform their movements. There were even parts where the dancers were using the intensity/loudness of the sound as a cue to guide their movements. Thus, while the music and choreography may not be organized in a traditional rhythmic manner, the dancers entrain their movements to other aspects of the sound.

Universal aspects of dance in butoh that are similar to Western dance styles include synchronized movement and coordination among dancers, turn-taking for sequential movement execution, synchronization of movements across various body parts, real-time self-correction within a choreography, and the integration of spatial and temporal parameters (e.g., proprioception). When it comes to learning butoh or other choreographies, they all require basic mechanisms of motor, auditory, and sequence learning, as well as short- and long-term memory to learn a sequence of movements in sync with musical or sound cues. Our choreography and dataset provide avenues for investigating universal aspects of dance, including the analysis of synchronized and distinct movements among the five dancers, facilitating the examination of interbrain synchrony during both similar and disparate motor executions while controlling for acoustic stimuli. Another universal element of dance conducive to study with our dataset is, in a turn-taking context, the exploration of connectivity patterns that evolve from premotor preparation to movement execution. In summary, butoh was considered ideal for this experiment since it offers a great balance: allowing us to study specific characteristics that make butoh unique, while also tapping into common processes employed in a variety of dances.

Methods

Participants

Five healthy female adults with no history of neurological disorder, or movement difficulties participated in this study. The experimental protocol and informed consent (reviewed and signed by each participant) were approved by the Institutional Review Board (IRB) at the University of Houston. All experiments were performed in accordance with the 45 Code of Federal Regulations (CFR) part 46 (“The Common Rule”), specifically addressing the protection of human study subjects as promulgated by the U.S. Department of Health and Human Services (DHHS).

Throughout the experiment, the pregnancy of one of the butoh dancers gave us the opportunity to study brain activity during butoh dancing in a pregnant woman. Concerning the safety of dancing during pregnancy, there is published evidence that dancing can actually be beneficial in pregnancy, as long as it does not include lifting other dancers, or high-impact activities such as jumping and back flips [11, 12]. For the purpose of the experiment, the choreography was specifically altered to suit the movement abilities and safety of the pregnant woman, and the performance was expressly allowed by a doctor. Regarding the use of EEG during pregnancy, there are already published reports on the safety of this technology in pregnancy [13].

Instrumentation & Data collection

To record the dancers’ brain activity via EEG, while controlling simultaneously for eye movements via electrooculography (EOG), both at a sampling frequency of 1000 Hz, we used two different systems: one 128-channel EEG system (BrainAmpDC with Acticap active electrodes; Brain Products GmbH, Munich, Germany) split into four 32-channel systems with a Wi-Fi transmitter, and one 32-channel system (Nautilus, Gtec medical engineering GmbH, Austria) with Bluetooth communications. We distributed the electrodes following the international 10–20 system (Fig. 1a) with slight modifications that allowed for some channels to be used for EOG. Specifically, channels TPO9 and TP10 were removed from the cap and placed on the right and left temples, respectively, to record horizontal eye movement, whereas channels PO9 and PO10 were placed above and below the right eye, respectively, to record vertical eye movements (Fig. 1a, b). The remaining 28 channels were arranged according to the 10–20 system (Fp1, Fp2, F7, F3, F4, F8, FC5, FC1, FC2, FC6, C3, Cz, C4, CP5, CP1, CP2, CP6, P7, P3, Pz, P4, P8, PO9, O1, Oz, O2, PO10), with ground and reference electrodes placed on the earlobes.

Fig. 1

EEG, EOG, IMU locations and impedance values. a, EEG-channel montage according to a modified 10–20 system, with ground and reference electrodes placed on the earlobes, channels TPO9 and TP10 on the right and left temples, channels PO9 and PO10 above and below the right eye, and IMU Opal sensors on the forehead. b, Close-up image of active EEG electrodes, and location of EOG and IMUs. c, Impedance values (kΩ) of the 32-channel EEG for each of five subjects (A-E) at the beginning and the end of the experiment. Impedance values > 60 kΩ are considered low quality, and < 25 kΩ, of high quality. Subject E was the only participant who used different EEG equipment, with Bluetooth transmission vs. Wi-Fi.

Head motion signals were simultaneously recorded using a distributed system of inertial measurement units (IMU; Opals, APDM Wearable Technologies Inc, Portland, OR) to track the dancers’ head motions. The IMU Opal sensors were placed on the forehead of each dancer, acquiring data at a sampling rate of 128 Hz. The data was stored on an onboard micro-SD card and simultaneously streamed to a PC for visualization and data monitoring using the Biometrics Analysis Software (Biometrics Ltd, Newport, UK). The recording sites were cleaned with an alcohol solution and allowed to dry. The sensors were then fixed to the skin using a double-sided adhesive tape (designed specifically for Opal sensors, so as not to obstruct electrodes). The signals were calibrated and checked for quality while the subjects stood in a neutral posture with their hands by their side, or while moving their head/neck.

Artifact minimization procedures

The EEG setup was rigorously prepared to minimize potential artifacts that typically contaminate the raw EEG measurements [14, 15]. A frequent source of noise in the signal comes from electrode artifacts that can occur when there is a disruption in the contact of the electrode with the scalp. In our experiment, for example, this could be driven by the dancer’s sweat while dancing, something further magnified by possible changes in ambient temperature. In the case of the butoh choreography we tested, electromyographic (EMG) artifacts would be expected during movements that include head contact with the floor (e.g., while rolling on the floor). All these artifacts would lead to changes in electrical impedance, and, hence, would affect the quality of the recording. Dancing itself might also generate EMG artifacts originating from the head and neck musculature recruited during head/face and neck movements, which can increase the noise and add artifacts, particularly in frequencies above 12–20 Hz, including beta and gamma waves [16]. Lastly, although eye movements and eye blinks also typically contaminate raw EEG recordings, there are several methods already published with working protocols to remove ocular artifacts, both offline and online [17], so this artifact did not pose a novel challenge.

To minimize the types of artifacts we mentioned, we followed a set of standard procedures: (a) we measured each participant’s head circumference to allow for selection of an appropriately sized EEG, which would guarantee the right fit of the EEG cap; (b) we asked the participants to refrain from using products in their hair that may increase the impedance at the scalp/electrode interface (e.g., conditioner, hair gel, etc.); (c) before donning the cap, the skin on the face around the eyes, the temples, and the earlobes were gently cleaned with alcohol wipes to remove any dirt and skin oils; (d) we applied viscous hypoallergenic conductive electrolyte gel between the electrode tips and the scalp to further secure the electrodes in place and reduce electrode impedance; (e) we aligned the cap on the head such that the FP1 and FP2 were 10% of the distance from the nasion to the union along the midsaggital plane, and electrode Cz was at the vertex of the head; (f) we used a stretchable netting (e.g., medical grade tubular elastic net dressing) to secure the location of all electrodes on the scalp, as well as the electrode cables that otherwise may pull down electrodes during head movements.

Alongside these commonly employed methods, we implemented a tailored set of measures to address the requirements of our study, which included: (a) incorporating the use of a hybrid gel-based pin electrode to enhance electrical conduction in Subject E, the only participant with a Bluetooth-based device, as previous studies [18] have indicated that dry electrodes may lead to suboptimal impedance values; (b) recording the experiment in a climate-controlled venue; (c) carefully choreographing a piece that was composed to a great extent of slow movements, which helped to minimize motion artifacts; (d) recording head movements using a distributed system of inertial measurement units (IMUs) to track the dancers’ head motions, so as to later be able to remove motion artifacts from our EEG measurements (Fig. 1a, b); (e) designing (i) shock absorbing caps for electrode protection during head contact with the floor and (ii) travel pillows for neck protection, which we emptied and customized with zippers, to place in the EEG WiFi or Bluetooth transmitters, instead of attaching the transmitters at the back of the head, where the device would be damaged during head contact with the floor (Fig. 2a, b); and f) recording EOG signals to be able to later remove ocular artifacts and eye movements off-line (Fig. 1a, b). All these protection constructs were tested to protect the equipment and were also individually adjusted for the dancers’ comfort. A period of acclimatization was necessary to allow for the dancers to become familiar with the equipment, and its proposed setup, so as to minimize interference with their dance (Fig. 2a, b).

Fig. 2

Customized head and neck protectors used in the experiment. a, Shown are scientists carefully placing the electrode caps, processing units, and Wi-Fi transmitters into a neck pillow we customized with zippers (left and right images), as well as three dancers (right image) with custom caps on, which we used as shock absorbing caps to protect the equipment. b, Shown are the dancers while dancing in standing (left image) and lying (right image) positions, with their equipment, and head and neck protectors on. (Dancers’ names: Azumi Oe, Kelsey Strauch, Margherita Tisato, Sindy Butz, and Vangeline Gand)

Technical validation of artifact minimization procedures

To assess potential changes or drifts in impedance values, we recorded impedances twice, at the beginning and the end of the EEG recording, using the Brain Vision Recorder software, which allows for visualization of the topographic position of each electrode with a color-coded display of its impedance value (Fig. 1c). According to this software, impedance values > 60 kΩ are considered low quality, and < 25 kΩ, of high quality. To ensure that our data quality would not be meaningfully reduced by high electrode impedance, before starting the experiment, we strived to maintain all impedance values < 60 kΩ for all participants.

Experimental protocol

The experiments were conducted over a series of 4 days, with each day differing in the data collected. The first recording (Day 1: 12/9/22) consisted of a rehearsal only with subject C, where we were able to test the fit of the equipment and several of the technical procedures we followed. On Day 2 (2/6/23) we recorded both the choreography and all control tasks from all subjects (A-E). On Day 3 (2/7/23), we recorded a subset of control tasks with all subjects (A, B, C, D, and E). Days 4 (2/8/23), 5 (2/9/23) and 6 (2/10/2023) involved calibration, rehearsal and control tasks with all subjects. Day 6 additionally included the recording of a final performance in front of an audience. On Day 4, data recording for Subject D was hindered by a technical issue in data transmission. All the experiments were conducted at the University of Houston Student Center South Theatre.

Before starting the control tasks, 1 min of EEG and EOG were recorded to establish a baseline period of brain and muscle activity. Afterwards, the control tasks included a resting state at both the beginning and the end (1 min); walking (2 min); vocalizations (10 min) that included (a) reading a list of words (“snake, wind, mist, hitch, brief, vent, throat, click, jeeps, mouse”), (b) reading a list of nonwords (“brant, pipso, brab, blave, filt, golk, raint, tane, praine, shaty”), (c) reading a list of “Jabberwocky” words/sentences (“The blay florped the plenty mogg”; “The Gou twuped the vag all lus rall”; “The heafest dropding deak is rhaph phemes away”), (d) producing sentences (“My name is Vangeline. I live in New York. Butoh is great, isn’t it?”), and (e) producing volitionally vocalizations that are typically produced spontaneously (laughter, sneezing, and yawning); seated meditation (5 min); butoh movements done mechanically, without “dancing” them, without being into the butoh mood, and with no music (7 min); and simple movements, such as raising right/left arms and legs, doing cyclical wrist/ankle movements, tongue protrusions, lip movements (e.g. lip rounding), nostril movements, finger/toe movements, and opening and closing the jaw (completed in sequences of 10 movements for each task; 10 min).

The 60-minute butoh choreography the dancers performed was choreographed by Vangeline Gand under a series of specified parameters. These parameters included variations in speed (e.g., no movement, motor preparation, very slow movements not detectable to the eye, slow movements detectable to the eye, pedestrian/walking speed), type of muscle contraction, butoh technique used, choreography type, imagery, gravity (e.g., working against gravity involved muscle tension, while the opposite, muscle relaxation), and emotion.

Interbrain synchrony

The quantitative measurement of interbrain synchrony that was used for the BCI visualization was achieved by calculating the bispectrum between dancer-dyads of EEG data obtained while performing butoh. Specifically, bispectrum combinations were generated (1 Hz) between the following dyads: subjects C-A, C-B, and C-D. Data from Subject E were excluded because of the possible differences in quality due to using a g.tec device.

Bispectrum was estimated across the EEG recordings using 4-second windows with 75% (one-second) overlap. The bispectrum at each time window was estimated using Eq. 1:

$$ B(fi,fj)Pca=log(\mid X(fi\left)X\right(fj\left)X*\right(fi+fj)\mid )$$

(1)

where subscripts Pca are for the data used from Participants “C” and “A”, in this example, and fi the frequency vector for the signal of Pc, and fj the frequency vector for the signal of Pa. X(fi) and X(fj) represent the Fourier transform of window l at frequencies fi and fj respectively. The term X∗(fi + fj) represents the complex conjugate of the Fourier transform of the sum of both frequencies fi and fj [19]. The quantity B for bispectrum is obtained by taking the logarithm of the absolute value of the product of the Fourier transforms and their complex conjugates at frequencies fi, fj and fi + fj. Using this method, bispectrum was estimated for all fi = fj, in 50 frequency bins between 1 and 50 Hz.

The 32 raw channels for each of the 4 subjects (A, B, C, and D) were filtered into 8 channels using a Laplacian spatial filter. Then bispectrum between Subject C and each of the other Subjects for gamma frequency was calculated using the formula (Eq. 1), yielding an 8 by 8 matrix comparing each of the 8 channels between two subjects for the three dyads/combinations we tested (subjects C-A, C-B, and C-D).

Artistic brain-computer interface

The visualization was designed by TouchDesigner (TD, Derivative, Toronto, CA), a visual programming environment aimed at real-time 3D rendering, combined with high-resolution real-time compositing (https://derivative.ca/). MaxMsp, a visual programming language for music and multimedia developed and maintained by San Francisco-based software company Cycling ‘74 was used in this project for data filterings and mathematical operations such as normalizing, scaling, averaging, and calculating min. and max. of input data. Additionally, MaxMsp was used for optimizing the computation and construction of the user interface for the change of the visual scenes.

To establish a mechanism to translate the EEG data to artistic visualization (BCI), the science team and multimedia artist established a communication system between MATLAB (The Mathworks Inc., Natick, MA) and TD. Two data packets were transferred from MATLAB to TD using the networking protocol TCP/IP (https://docs.derivative.ca/index.php?title=TCP/IP_DAT) via a direct Ethernet connection. The first packet of data is the raw EEG data of the dancers with a frequency of 100 Hz. The second packet is the interbrain synchrony with a frequency of 1 Hz. The received data transfers to MaxMsp via Open Sound Control (OSC), a protocol for network communication among computers, sound synthesizers, and other multimedia devices (https://www.cnmat.berkeley.edu/opensoundcontrol). MaxMsp filters/calculates the data and sends them back to TD via OSC.

Signal preprocessing

EEG signals were pre-processed utilizing MATLAB R2023a (MathWorks, MA), and functions from the open-access toolbox EEGLAB. The EEG and EOG raw data were filtered with the Adaptive Noise Canceling (ANC) H ∞ filter. For this step the parameters γ = 1.15, q = 1e-10, and Po = 0.5 were utilized. Following ANC H ∞ 28 (EOG-denoised) EEG signals were obtained. For each EEG session data, frequency peaks from (1–15 Hz) of gravity-compensated acceleration signals were identified and removed from the 28 EEG channels via motion artifact removal. Motion artifact denoised EEG data was further cleaned using line noise removal (60 Hz), with Cleanline, Zapline, EEGLAB plug-ins, or Notch filter (60 Hz). In the Artifact Subspace Reconstruction (ASR) algorithm (EEGLAB), the parameter κ = 15 was applied.

Results

In this experiment, we used mobile EEG recordings to study the brain activity of five experienced dancers while dancing butoh simultaneously. For this purpose, we recruited five female participants (age: 42.6 ± 5.21 years) with no history of neurological disorder. By the time the experiment took place, the participants had on average 15 ± 3.16 years of experience practicing butoh dance.

Impedance values

Our comprehensive artifact minimization strategies, encompassing both conventional techniques and innovative measures (Methods; Figs. 1a, b and 2a), including the strategic placement of data transmitters in a customized neck pillow, were designed to mitigate potential impedance changes inherent in experiments involving movement. While our impedance measurements for Subjects A-D before and after the experiment indicated generally high-quality impedances (< 25 kΩ), Subject E exhibited predominantly elevated values (> 60 kΩ), particularly in recordings preceding the experiment initiation (Fig. 1c). This discrepancy aligns with the fact that Subject E was the sole participant employing distinct EEG equipment, with Bluetooth transmission vs. Wi-Fi (Fig. 1c). Despite our use of a hybrid gel-based pin electrode (Methods), which was designed to address the common issue of poor impedance values coming from dry electrodes in most Bluetooth-based g.tech devices [18], we still observed lower impedance quality compared to the BrainAmpDC devices.

Data synchronization between different EEG measurement modalities and IMU equipment

In our experiment, achieving synchronization at a millisecond range between different measurement modalities and equipment was crucial for ensuring synchronization across different data streams. To address this issue, we implemented hardware synchronization via a custom cable for the wired transmission of Transistor-Transistor Logic (TTL) signals between devices. In this TTL circuit, transistors served as electronic switches, controlling current flow based on the input signals. In detail, as shown in Fig. 3, Brain Products’s ActiCAP headsets for Subjects A-D were wired individually to a MOVE EEG transmitter that wirelessly sent their brain activity data to its corresponding receiver via Wi-Fi (Brain Product’s MOVE wireless system). The MOVE receivers then sent the data, via a fiber optic cable, to a USB 2 adapter capable of translating the information to a USB cable. The latter was then readable to the EEG personal computer hosting the Brain Products’s BrainVision Recorder software. The g.tec Nautilus headset of Subject E was also wired similarly to its respective g.tec transmitter but sent data to the g.tec computer via Bluetooth instead. In all Subjects, IMU sensors -attached to their foreheads- sent data to a receiver. This data being directly transmitted, via USB cable, to the BrainVision and the g.tec EEG personal computers.

Fig. 3

Diagram of the physical configuration of the EEG system in our experimental setup, including hardware elements, connections, and the direction of signal flow. Each box with a name represents an individual device: EEG Wi-Fi-based devices are in blue, EEG Bluetooth-based devices are in green, IMU devices in red, and computer-related devices and other peripherals are in gray. Solid lines connecting boxes represent cable connections between devices. Arrows indicate the direction of signal flow

Synchronizing the Brain Products and g.tec EEG systems, as well as IMU sensors, required the input of a manual trigger box that had three Transistor-Transistor Logic output connectors (Fig. 3): the first one was connected to the Access Point antenna to mark IMU data, the second to the BrainAmpsDC through the USB 2 Adapter to mark WiFi EEG data, and the third to the Base Station to mark the Bluetooth EEG data. The latter was made possible via a modified dual-pin cable connection. Critical points in the performance and control tasks were marked simultaneously via the trigger in all three systems. These stored timestamps can be used to align the signals offline for subsequent analysis. Figure 4 depicts an example of a time-synchronized subset of the recorded raw EOG and EEG signals obtained during 1 s of sitting, speaking, walking, and dancing, with both signals sampled at 1000 Hz. To align signals sampled at different rates, such as those from the IMUs (128 Hz), in the “Signal preprocessing and denoising pipeline” section below, we explain how this can be achieved by upsampling the IMU data to match the EEG data by interpolating the timestamps at 1000 Hz (Fig. 5).

Fig. 4

Time-synchronized subset of EOG and EEG during sitting, speaking, walking and dancing conditions. The timeseries EOGh (horizontal) and EOGv (vertical) are computed as bipolar signals for the horizontal and vertical EOG channels, respectively

Signal preprocessing and denoising pipeline

Artifact identification, minimization, and noise removal are critical steps in data preprocessing, laying the groundwork for subsequent functional analysis or neural decoding. For comprehensive offline analyses using our data, we propose a signal pre-processing pipeline (specific steps outlined in Fig. 5a; see Methods for more details), which ensures the removal of physiological and non-physiological artifacts, including power line noise, as well as achieves a “true” average reference of the signals. To illustrate this pipeline, we offer a demo using a fraction of our data (Fig. 5b). As a first step, since EEG and EOG signals were collected at a sampling rate of 1000 Hz, and the IMU signals at 128 Hz, IMU data was first gravity-compensated and re-sampled to 1000 Hz in order to synchronize it, as well as use it as source noise signal to identify potential motion artifacts in the EEG data in further filtering steps. Following this approach, three-dimensional, gravity-compensated acceleration signals were obtained.

Fig. 5

Proposed signal preprocessing and denoising pipeline. a, A methodology flowchart showing the basic steps of our proposed pipeline. b, Average Power Spectral Density of Subject’s A raw data, data after H-infinity (green arrow) and after the application of ASR (pink arrow) on the interval [0:00–20:00]. Dotted circles highlight the main differences between the graphs, e.g., in peak frequencies

The EEG and EOG raw data were then filtered (H ∞ filter [17]) to allow for the removal of eye-related artifacts such as eye blinks, eye motion drifts, and recording biases. These signals were further filtered using an adaptive, non-linear motion artifact removal algorithm in order to preserve the neural content of the EEG signal while increasing the signal-to-noise ratio by removing motion artifacts [20]. The motion artifact-denoised EEG data underwent additional cleaning through line noise removal. Following these steps, a robust re-referencing using the PREP pipeline and a band pass filter were applied to further refine the EEG data. These obtained signals were further conditioned using the Artifact Subspace Reconstruction (ASR) algorithm from EEGLAB, automatically removing transient or large amplitude artifacts that contaminated the EEG data. The next step consisted of Independent Component Analysis (ICA), which was utilized to retain brain-related and exclude artifact-related ICs (e.g., residual eye, muscle, electrode popping). In the example comparison we showcase (Fig. 5b), we estimated the Average Power Spectral Density of Subject’s A raw data across all 28 EEG channels (1000 Hz) over the first 20 min of the choreography with 95% intervals and band-pass filtering between 0.01 Hz and 50 Hz, highlighting the differences identified after applying H ∞ and ASR. Importantly, we emphasize the differences in the lower frequencies identified, where after H ∞ application, peaks emerge at 10 Hz, 25 Hz, 30 Hz, and near 50 Hz, while after ASR, all peaks fall below 20 Hz.

Real-time interbrain synchronization and EEG-based brain-computer interface visualization

Leveraging mobile EEG and brain-computer interface (BCI) techniques, we artistically visualized the interbrain synchrony of the dancers “in action and in context”, during a live dance performance in front of an audience. The artistic design focused on exploring new ways of projecting real-time interactive animated visualizations of EEG data that are both accessible to a diverse audience and informative to for those with a scientific background.

This required that the dancers’ interbrain synchrony be measured in real-time, while dancing butoh, which implies a continuous computation of a synchrony metric across multiple combinations of electrodes of different subjects. The computational load of this process increases with the number of electrodes and participants [20]. Therefore, to make this online calculation as efficient as possible, we used a Laplacian spatial filter, which filtered the 32 raw channels/Subject into 8 channels (Methods). This filter acts as a spatial high-pass filter applied to the data, which attenuates low-spatial-frequency signals that are broadly distributed across the scalp while preserving more localized higher-spatial-frequency signals. Using these filtered 8 channels/Subject, we calculated the bispectrum between dyads across the gamma frequency band (between 30 and 50 Hz), yielding blocks of 8 by 8 matrixes that were instrumental in generating the BCI visualization.

The BCI visualization of the resulting data drew inspiration from the structure of the music composed for the purpose of the study, the choreography, and the concept of interbrain synchrony. Since butoh is often made up of slow movements, and in contrast, the music for the project is active and repetitive, the visualization aspired to find a middle ground between the physical and sonic rhythms. Overall, the visualization had no identifiable regular pulse but flowed freely in terms of texture, movement, color, and spatialization. It was structured in three sections: (a) abstract scenic monochromatic images, (b) five 3D brains and brain connections portraying interbrain synchrony, and (c) five abstract circles producing a colorful texture (Fig. 6).

In detail, at the beginning of the music and for the first 20 min, the visualization was meant to depict the prevailing sounds of nature with abstract scenic monochromatic images with additive textural visual noise. This noise’s amplitude was the result of mapping normalized raw EEG data of the dance leader (Fig. 6a). The second visualization featured five 3D brains and their connections, aligning with the dancers’ positions on stage to help the audience associate the brain visuals with each dancer. The brain synchrony value was mapped in real-time with the particle flow level between brains, forming a line between them, so that the higher the value of synchrony, the higher the opacity and thickness of the line (Fig. 6b). The third visualization showcased five abstract circles representing the interbrain synchrony between the accompanying dancers and the dance leader. Their interbrain synchrony was mapped to the position of the four circles relative to the central circle, so that the higher the value of interbrain synchrony, the more concentric the circles appeared (Fig. 6c). This visualization was crucial in communicating the essence of this collaboration, right at the intersection of art and science.

Fig. 6

Real-time interbrain synchrony visualization via brain-computer interface. a, Abstract scenic monochromatic images, with additive noise, whose amplitude was the result of the leading dancer’s normalized raw EEG data. b, Five 3D brains and their connections reflecting real-time brain synchrony. c, Five abstract circles showing the interbrain synchrony between the accompanying dancers and the dance leader. In all cases, top images show computer examples from the type of visualization described, and bottom images show real instances of how these visualizations unfolded during a live performance. (Dancers’ names: Azumi Oe, Kelsey Strauch, Margherita Tisato, Sindy Butz, and Vangeline Gand)

Discussion

In this experiment, we used mobile EEG recordings to study the neural dynamics of dance with exquisite temporal resolution (millisecond range) and in ecological settings (i.e., a theater) that are not possible to test with other techniques, such as with fMRI, where the subject is constrained to lay down within the confines of the bore of a scanner in a neuroimaging facility. Still, even with mobile EEG recording, there are important technical challenges that we had to overcome, as such we provided here a comprehensive account of the experimental design, methods, technical validation, and practical execution of a highly interdisciplinary project, intending to offer valuable insights for future endeavors. We detailed the best practices and steps taken to achieve high-quality impedance values, along with observations and suggestions regarding the most effective devices. We also shared our practices in recording different types of artifacts (via EOG and IMU) to maximize the signal-to-noise ratio. Additionally, we explained how we synchronized different EEG devices, via TTL triggers, and how we ran interbrain synchrony analysis in real time that we then used for the BCI visualization. For future reference, we put forward a signal preprocessing and denoising pipeline that could be used to analyze our data offline, by showcasing how a fraction of our data looks before and after going through the pipeline. Lastly, we shared our raw data and code, both for the live recordings and the live visualization of brain activity on a screen, via an artistic brain-computer interface (BCI) while the dancers were dancing. We believe that this data holds potential for addressing different hypotheses, several of which are outlined in the following sections.

Hypotheses to test

Interbrain synchrony

Dance has been posited to have evolved as a form of interpersonal coordination and social communication, which is based on both imitation (matching of movement) and synchrony (matching of time) skills [21]. Different kinds of dances rely on different aspects of interpersonal coordination, including touch, eye gaze, sensory-motor interactions, facial expressions, or even synchronization with other physiological parameters, such as breathing, heartbeat, and sympathetic tone [22]. Thus, EEG recording from different dancers, dancing the same choreography simultaneously, is expected to unravel interbrain neural synchrony in the dance aspects that require interpersonal coordination.

Previous experiments [23] where we examined EEG signals of two dancers while dancing a ballet duet showed high interbrain synchrony in the gamma band of visual brain regions (Broadman area 18) of the dancers. Interestingly, the leading dancer exhibited interbrain synchrony between her visual (BA18) cortex and her partner’s cognitive (BA31) and premotor/supplementary motor areas (BA6). The butoh choreography that we investigated was focused on auditory cues from the music, with dancers transitioning from one move to the next in response to specific auditory cues, such as a sound that resembles an owl hooting announcing a specific move. Throughout most of the choreography, the dancers performed with their eyes closed or half-closed with a soft focus [24], something that gives us the unique opportunity to study interbrain synchrony in a dance form where vision is not expected to be the basis of coordination, reported as the most common form of interbrain synchrony [25]. Lastly, since we recorded both the rehearsals and the final performance, it will be possible for us to assay changes in interbrain synchrony as a function of learning, practice, or the scenic context (e.g., presence or absence of an audience).

Beyond the online estimation methods of interbrain synchrony that we reported in this study, we further recommend specific practices for offline analyses that can address the above questions. For example, in offline analyses, we suggest using more complex and computationally expensive filters, such as robust referencing, Artifact Subspace Reconstruction (ASR), and Independent Component Analysis (ICA), to remove unwanted artifacts from different sources. We also suggest that component space, instead of channel (sensor) space, be used for bispectrum estimations, which allows us to ask questions by zooming in on specific Brodmann areas, as we have previously shown [23]. Another interesting practice would be to use differing time windows and overlaps to yield more fluid changes in interbrain synchrony, such as 2-second windows with 50% overlap and 4-second windows with 75% overlap.

Finally, based on our observation of stronger impedance values with the WiFi-BrainAmpDC devices with gel electrodes compared to the Bluetooth-g.tech device with dry electrodes, we recommend prioritizing BrainAmpDC or a similar device for future experiments akin to the one presented here. Although we utilized a hybrid gel-based pin electrode with the Bluetooth device to address this issue, we still encountered lower impedance quality compared to the BrainAmpDC device. We acknowledge the ongoing debate about the reliability of Bluetooth-based devices, with some studies suggesting interpretable signals and a good signal-to-noise ratio in the frequency domain [18]. While there was an improvement in impedance by the end of the experiment, we assume that the challenge of maintaining low impedance during the initial headset setup may have influenced the signal quality and potentially affected the results. Thus, based on the impedance values we reported in this study, we recommend using EEG devices with gel electrodes.

Motor hypothesis of vocal learning

There is a hypothesis [26] that links the evolution of the neural circuit that is responsible for rhythmic body muscle movement (e.g., head, arm, and leg muscles) to the evolution of the neural circuit that is responsible for the movement of the muscles of the vocal organ during vocal communication (e.g., laryngeal muscles in humans). This hypothesis is built on findings [26] showing that in vocal learning birds, all their cerebral nuclei that are devoted to song learning are adjacent to discrete brain areas active during limb and body movements. Essentially, the hypothesis states that our ability to move in time with an auditory beat (or, dance, in humans) originated from the neural circuitry for complex vocal learning (or, speech learning, in humans).

This prediction became even more pertinent after the finding that only species that communicate with complex vocalizations (i.e., humans and parrots) are able to dance (i.e., to entrain their body movements to a beat) [27], pointing to a common neural substrate in both abilities. Although this hypothesis has not been directly tested in humans, meaning that no one has compared in the same subjects the neural pathways underlying speech (i.e., laryngeal movements) and dance movements (e.g., rhythmic arm movements), cross-studies’ comparison points to an overlap between several of the regions controlling body movements in the primary motor cortex with the regions that control laryngeal movements in the primary motor cortex [28,29,30]. Further, dance has been found to increase network connectivity between the basal ganglia and premotor cortices [31], both of which are co-activated during speech [32, 33].

To explore possible parallels in speech and dance, in our control tasks, we instructed our dancers to produce speech and speech-like vocalizations (e.g., Jabberwocky words), as well as other non-speech vocalizations (e.g., sneeze, laughter, yawn), with the aim to compare their EEG patterns during laryngeal movements vs. movements of other body parts. For a nuanced comparison of these two types of movements, we suggest examining both short (e.g., 1 s) and long (e.g., 4 s) time windows, recognizing that each serves different purposes; longer windows would capture all relevant activity, but might include more artifacts, compared to shorter windows. To mitigate artifacts from activities that further engage laryngeal muscles, such as heavy breathing, we suggest selecting dance movements that do not require a lot of physical strength. One way to control for the laryngeal movements of breathing would be to subtract the brain activity during resting state, when the subjects were seated and breathing, from both the speaking and dancing activities.

For a targeted analysis of neural patterns during speech and dance, we propose specific EEG analyses that are feasible with our shared dataset. Firstly, one example would be to run a topographical analysis to explore potential similarities in EEG signal topographical patterns during speech and dance movements. Secondly, with a functional connectivity analysis, commonalities in functional connectivity patterns during speech and dance could be investigated using a seed analysis on electrodes associated with primary motor cortex activation (e.g., “C3” [34]). Lastly, frequency bands displaying similar modulations during both speech and dance, as well as their interactions, could be identified with a power spectral analysis and cross-frequency coupling analysis. These specific EEG analyses aim to provide a comprehensive understanding of the neural dynamics underlying both speech and dance, shedding light on potential similarities in topographical patterns, functional connectivity, and spectral modulations.

Butoh vs. other meditative practices

As aforementioned, butoh embraces in its practice contemplation and meditation [9], suggesting it may tap into similar processes as those involved in, for example, seated meditation, such as attention mechanisms that guide concentration. Other embodied meditative practices, such as yoga and tai-chi, also share similarities with butoh, in that they all include varying degrees of movement while maintaining focused attention, with some incorporating auditory attention to external stimuli. As such, our butoh EEG recordings offer a comparandum with other meditative practices, where there are already published data on their associated brain activity.

For example, Banquet [35] used spectral analysis of EEG data during transcendental meditation in the early 1970s, a method described as a mental repetition of a special sound or mantra, and showed that meditative states could be distinguished from other states of consciousness based on sequential changes in the alpha, theta, and beta waves in relation to their topographical alterations across the scalp. More recently, EEG recording during meditation in Buddhist practitioners revealed self-induced and sustained high-amplitude gamma-band oscillations [36]. In a different study, meditation training gave rise to increased theta activity in the frontal midline electrodes, which was sustained even during the resting state following meditation training [37]. Xue et al. [38] in a similar experiment on short-term meditation training, found increased theta (and some alpha) activity in the anterior cingulate cortex and adjacent prefrontal cortex, which correlated with improved performance on tasks of attention, working memory, creativity, and problem-solving.

All these studies provide a fertile ground for comparison with the hypothesized meditative aspects of butoh. To make the comparison between butoh and meditation readily possible in our experiment, we included a seated meditation task in our control tasks, allowing us to compare whether the brain activity during seated meditation bears any resemblance to the contemplative practices employed during specific segments of the butoh choreography. Regardless of whether the signatures will be the same or different, given the documented health benefits associated with meditation-like [39], we anticipate that our experiment will shed light on the patterns of brain activity underlying these practices.

Butoh in pregnancy

Throughout the experiment, the pregnancy of one of the butoh dancers gave us the opportunity to study brain activity during butoh dancing in a pregnant woman. To our knowledge, this is the first time to run mobile EEG with a pregnant woman dancing butoh, or dancing, in general [11, 12], offering a potential avenue to uncover butoh’s untapped role as a beneficial practice in pregnancy. In principle, the pregnancy itself could be considered a limitation, since studying dance in pregnant women was not one of the initial goals of our study, and pregnancy might introduce several confounding factors, such as alterations in bodily properties. Nevertheless, we prefer to view this as an opportunity to both record the brain activity of a pregnant woman while dancing and to extend our methodological approach to populations underrepresented in the research realm.

Currently, our understanding of how brain activity patterns differ between pregnant and non-pregnant women, particularly in tasks involving auditory-to-motor integration like dance, remains limited. Interestingly, Plamberger et al. [13] conducted a study using a visuospatial attention task, involving pregnant and non-pregnant participants. In the task, an auditory cue directed the attention of the participants either to the left or to the right visual hemifield, where, following a variable time interval, they had to discriminate between a “p” or “q” sound on the cued hemifield. Both non-pregnant and pregnant women showed a decrease in the alpha amplitude in the fronto-parietal network, which correlated positively with accurate discrimination, with no significant differences in the cases of pregnancy vs. non-pregnancy. Since our butoh choreography is based on correctly perceiving auditory cues in the music, it is tempting to hypothesize that an alpha band desynchronization, leading to the expected cue and right after the cue is perceived, could underlie accurate choreography performance. Considering the alterations in bodily properties during pregnancy, such as increased body mass and slower motion, an additional hypothesis could be that pregnant women may allocate more time to motor preparation after perceiving an auditory cue. In general, there is a need for more detailed investigations involving pregnant subjects to inform specific hypothesis-driven analyses of our data.

Live test for interdisciplinarity

As a collaboration studying butoh in the brain, both the art -butoh- and the science -EEG recording- were equally important towards success [40, 41]. This live test for interdisciplinarity allowed exploration into understanding the unique opportunities and challenges for such a collaboration, including needs for dancers as athletes and subjects, technical requirements for protecting equipment without inhibiting movement, and for synchronizing brain waves across all five dancers, implications for providing education and working with students, and determination of visual projection based on BCI.

The condition of sound was a critical component, not only for live visualization via BCI but also because auditory cues were often the only signals upon which to coordinate motor movements. Thus, any discrepancies or failures in sound quality would greatly impede the dancers’ performance, and potentially relatedly, EEG recordings based on the ability to enter into anticipated parts of the choreography and synchronize with each other. As a result, this would percolate down to the real-time interbrain synchrony calculated and to the BCI-visual projections.

This collaboration further highlighted that bringing various fields together requires clear communication to understand the various needs and expectations of each discipline [42, 43]. For example, for dancers, who are highly skilled athletes [44], a controlled environment, considering aspects of stage size, noise level, temperature, and other factors that may affect the dancers’ performance, must be considered to minimize stress, and maximize their ability to perform. For scientists, it is also critical to factor in human fatigue in EEG recordings; simply recording data, if the question at hand is as specific as that we are asking about butoh dancing, will not suffice, and errors in sound production, unnecessary delays lengthening the time of preparation for study, and any other factors that may impede the dancers’ ability to perform butoh may lead to poorer data outcomes.

Given that the dancers are the subjects of interest, this collaboration also showed that creative solutions may lie in another’s lived experience. One example was that the solution for how to best don the EEG caps, which are sensitive both for capturing brain waves and as a piece of equipment, was found by one of the dancers. Another example comes from a dancer who reported that the control tasks are better to be recorded before any butoh dancing takes place, since the butoh (meditative-like) state may linger after the performance and confound results from control conditions. These situations are great examples of the benefits of this multidisciplinary collaboration, in which the perspectives from experts in different fields came into play and merged into a highly unique project.

Ultimately, the project navigated seemingly competing interests in data needs, considering reporting requirements to meet funder expectations and secure future funding: for dancers, a high-quality video; for scientists, robust data collection; for students, time and attention for hands-on learning. The reader is referred to a recently edited book on Mobile Brain–Body Imaging and the Neuroscience of Art, Innovation and Creativity [45] that addresses the challenges and transdisciplinary opportunities for transformational and innovative research and performance at the nexus of art and science enabled by emergent technologies.

Dance -and butoh- as movement therapy

Dance-movement therapy utilizes creative movement [46] as a healing tool rooted in the inseparable connection between the body and the mind. Rooted in concepts of embodiment and attunement, this approach has an impact on several aspects of human behavior–psychologically, physically, and socially [47]. The therapeutic effect of dance extends across the lifespan, influencing motor development in various populations, including children in general [48], those with Down syndrome [49], cerebral palsy [50], and developmental cerebellar anomalies [51]. In the elderly dance contributes to successful aging, for markers including fitness, functional balance, mobility control [49,50,51], and cognition [52]. Overall, dance-movement therapy and dance leads to psychological health outcomes including decreasing depression and anxiety, increasing quality of life, and expanding interpersonal and cognitive skills [53].

Among patients with Parkinson’s Disease, music, and dance proved to be simple, non-invasive treatment options that promote balance, gait, and cognition [54,55,56,57], decrease psychological symptoms, and improve quality of life [58, 59]. For other conditions, such as schizophrenia and psychotic disorders, many studies tend to have small samples, no randomization, and no adequate control [60]. Yet, there is some support that body-centered interventions do alleviate stress, depression, and anxiety as well as facilitate pain reduction in physical and psychological pathologies via a bidirectional pathway between the brain and body [61]. As one creative therapy, dance has been shown to be effective for severe mental illnesses such as trauma-related disorders, major depression, and bipolar disorder [62, 63].

Because butoh dance is a psychosomatic exploration method [9], this study holds implications for further understanding the healing effects of dance, particularly how dance-movement can be prescribed as a form of therapy by selecting the emotional or physical level of involvement, or dose response, based on the patient’s condition. For example, there may be a relationship between some mental aspects of schizophrenia and butoh performance in terms of the state of consciousness and body-mind vulnerability [64].

Conclusions

The art-science collaboration that we reported here was a unique, complex, multidisciplinary experiment that required the coordination, management, and execution of a diverse team, including dancers, engineers, neuroscientists, musicians, multimedia artists, logistic personnel, facility management crew, and students. In addition, securing funding for the travel expenses and artists’ fees was critical to the success of the project. Last but not least, trust and respect for each other were essential to conduct the project in an accelerated timeline. The resulting data, best practices, approach, code, and audiovisuals present a unique opportunity for the scientific and artistic communities to harness the data, knowledge, and lessons learned from this project, to answer novel questions, deploy new algorithms or computational methods, and create new art-science works.