Multimodal data acquisition in human studies usually involves recording data from sources other than the biosignal acquisition hardware devices (e.g. video camera). Given that the biosignal hardware and the camera are independent recording sources, a common problem when replaying the recording session is the synchronization of both.
Getting Started: Required products
The present article shows how to record and synchronize videos and biosignals data in OpenSignals using the following setup:
- 1x 8-channel biosignalsplux hub included in the following biosignalpslux kits:
- 1x biosignalsplux LED accessory
- 1x external video recording solution
- OpenSignals (r)evolution software
- OpenSignals Video Synchronisation add-on
- Any biosignalsplux sensors of your choice
Important Note
This approach requires the use of a Digital I/O Port (Accessory Port) for the use of the biosignalsplux LED, which is only available in 8-channel biosignalsplux devices. This approach is not supported on 4-channel biosignalsplux hubs.
Step 1: Setting up your hardware
Connect all your sensors to the biosignalsplux hub and configure your system in the OpenSignals device manager accordingly.
Connect the LED to the digital port of the biosignalsplux hub (digital port is highlighted in red in the following image):
When setting up the camera for the video recording, make sure that the LED is visible within the camera’s view.
Step 2: Record data in OpenSignals & set visual markers
Start the acquisitions & set visual markers using the LED Start the video acquisition and proceed with initializing the signal acquisition in OpenSignals. It is recommended to set a synchronization marker before acquiring any signals of your actual experimental acquisition to ensure that you can synchronize both the video and biosignals data at an early stage.
Let the subject being monitored stay at rest and turn the LED on and back off again (in case a permanent light is not desired) by clicking on the circle to the right of DIGITAL OUTPUT (digital output controller) in OpenSignals’ information bar.
This toggling of the LED’s ON/OFF state can be seen in the video and will be used as the synchronization marker after the signal acquisition. Finish your signal acquisition and proceed with the next step.
Step 3: Loading the acquired signal & video data in OpenSignals
Enter the visualization mode in OpenSignals and load your acquisition file. The acquired sensor data and an additional I/O channel will be visualized.
The acquired data can be split up into three phases as seen in the screenshot below:
- Standby: No significant signal acquisition or LED control; can be ignored
- Synchronization Marker: LED control and signal that will be used for the synchronization
- Signal Acquisition: The acquired signals & intervals that are relevant for your actual application
Open the Video Synchronization Add-On using the add-on icon (highlighted in the screenshot below in blue) and click the folder icon in the add-on (highlighted in red) to load your video.
Step 4: Synchronizing signal & video data
The I/O channel shows the status of your LED throughout the signal acquisition (ON = signal = 1, OFF = signal = 0), which is the information that will be used for the synchronization of the video and sensor data. In this specific example, the change of the LED state from ON to OFF will be used as a synchronization point.
First, go through your video using the video controls in the Video Synchronization add-on and stop at the moment at which the LED is turned on.
Second, drag the vertical blue marker of the Video Synchronization Add-On which can be found at the left of the visualization window to the position where the signal of the I/O channel switches from 0 to 1. Lock the marker by clicking on the lock icon next to it.
If done correctly, these steps should be visible as in the screenshot below (first highlighted in red, second highlighted in blue):
Step 5:Replay your acquisition
Step 5: Replay your acquisition
After synchronizing the video acquisitions with the signal acquisitions, click on the play button in the Video Synchronization Add-On and a marker will be moving through the acquired signals as the video is being replayed.
Alternatives
Other than starting the acquisition with the LED is turned off and having to manually turn it on and of first to create a visual marker, it is also possible to start the acquisition with the LED being turned on automatically.
In this case, the visualization marker will consist of the LED change from ON to OFF. For this, turn on the digital output of your biosignalsplux in the OpenSignals device manager (circle must turn blue):
