NEUROFEEDBACK VIA BRAIN-COMPUTER INTERFACE

on By

Here are some of our insights after a few days of exploring the neurofeedback technology called Brain-Computer Interface or BCI.

Brain-Computer Interface (BCI) refers to the communication between the brain and a computer used to interpret the real-time activity of the brain through signal processing and machine learning.

Such communication offers great possibilities for medical research and game development.

Neurofeedback is a part of neuro-science, which uses BCI technology to make human brain and body signal data available to applications, exposing data channels for:

  • EEG - Electroencephalogram, measures the electrical activity in the brain.
  • EOG - Electrooculography, measures the electrical potential generated by eye movement.
  • EDA - Electrodermal, measures the action of sympathetic nerve traffic on eccrine sweat glands.
  • PPG - Photoplethysmogram, measures the blood volume changes in the microvascular bed of tissue.
  • ACCEL - Accelerometer, measures the speed of movement.
  • GYRO - Gyrometer, measures the angle of movement.
  • Analog and other data.

Table of Contents

In this article we'll go through the following topics:

  • The Context.
  • So what happened?
  • All we had to do was.
  • Highlights.
  • Introduction to the channel concept.
  • Commenting on the 6 code samples.
    • Get data from device.
    • Serializing data.
    • Downsampling data.
    • Transforming data.
    • Signal filtering data.
    • Denoising data.
  • Wrap up

THE CONTEXT

The context of the following insights is:

Make a simple proof of concept on how to get access to brain signal- and neurofeedback data,
via a Brain-Computer Interface (BCI) enabled technology,
focusing on open-source API's and platforms.

Insights on headmount devices, Unity3D integration and advanced usage of neurofeedback data concepts, are not included in this article, but we'll get back to these subjects at another time.

So what happened?

We went through the Modern Brain-Computer Interface lectures, by Christian Kothe from Swartz Center for Computational Neuroscience (SCCN).

Bottom line was, check out the open-source project OpenBCI and a supported API called BrainFlow, which supports several hardware devices (boards), synthetic emulation (no hardware) and programming interfaces for Python, Java, C# / Unity Game Dev. integration, R and C++ on Windows, Linux and MacOS.

And BrainFlow’s boards can stream data to different destinations like files, sockets and so on.

Great! We were ready for hours of research, hoping and craving for a quick dopamine rush like,

“Yes we did it, our first code experiment works!!
A tiny victory and the future feels brighter than before!!”

You know the feeling, right?

All we had to do was...

Go through the following task:

  • Choose an API interface / language ( We chose Java, it was quick and simple).
  • Browse the brainflow code on the OpenBCI GitHub. [Link]
  • Download the Java jar and source. [Link]
    • Download Eclipse Java Developer, if you don't have a Java IDE. [Link]
  • Explore the code samples (Java). [Link]
  • Investigate and play around to get acquainted with the API.

HIGHLIGHTS

Besides integration to BCI hardware devices, BrainFlow offers a synthetic device that emulates brain signal data streams, which is great when doing experimental development, because these modern headmount devices can cost hundreds or thousands US dollars.

Lets go through the 6 "simple" code samples:

  • Getting data from a device.
  • Serializing data.
  • Downsampling data.
  • Transforming data.
  • Filtering data.
  • Denoising data.

INTRODUCTION TO THE CHANNEL CONCEPT

For this simple experiment we used the synthetic (no hardware) emulated board (board_id=-1).

int board_id = BoardIds.SYNTHETIC_BOARD.get_code();

A board ( or device) exposes a number of  channels, dedicated to some sort of signaling, may it be EEG, EOG, EDA, PPG, ACCEL, GYRO and other signal data or control data, like sequence numbers and timestamps.

Depending on the board you're integrating to, you'll have a number of signal channels available. And you need to know which channels are connected to what and how you're supposed to process and interpret the data.

The brainflow_boards.json file, contains a configuration for the synthetic board and others.

"boards": {
		"-1" : {
			"name": "Synthetic",
			"sampling_rate": 256,
			"package_num_channel" : 0,
			"timestamp_channel" : 12,
			"num_rows" : 13,
			"eeg_channels" : [1, 2, 3, 4, 5, 6, 7, 8],
			"emg_channels" : [1, 2, 3, 4, 5, 6, 7, 8],
			"ecg_channels" : [1, 2, 3, 4, 5, 6, 7, 8],
			"eog_channels" : [1, 2, 3, 4, 5, 6, 7, 8],
			"eda_channels" : [1, 2, 3, 4, 5, 6, 7, 8],
			"accel_channels" : [9, 10, 11]
		},
...

The synthetic board has 13 channels:

  • Channel 0: Package sequence number.
  • Channel 1-8: EEG, EMG, ECG, EOG, and EDA data signals.
  • Channel 9-11: ACCEL data signals.
  • Channel 12: Timestamp.

You can include your own BCI boards and devices, by adding a configuration to the brainflow_boards.json file.

Getting data from a device

In this sample:

  • The call waits a few seconds, for the buffer to load and gets 30 samples from all channels.

The BrainFlow Java sample: BrainFlowGetData.java

Lets ignore all the initial session code stuff and go directly to streaming data.

double[][] data = board_shim.get_current_board_data(30);

What you get is a 2-dimentional list (array). A list of channels, each holding its own list of decimal values.

The number of samples ( values in each channel) in a reply depends on your query.

You can get a peek at a chunk of the available data, by adding the number of samples to the method call, like above, asking for a sample size of 30 values (in each channel). But this peek call doesn't flush it from the buffer.

Calling the same method without arguments, will implicitly flush the buffer after getting the available data.

SERIALIZING data

In this sample:

  • All channels are serialized out to a file called test.csv and then read back in.

The BrainFlow Java sample: Serialization.java

A simple sample on serializing the data. Not much to say.

DOWNSAMPLING data

In this sample:

  • All EEG channels are downsampled via the AggOperations.EACH operation to half size.

The BrainFlow Java sample: Downsampling.java

In signal processing, downsampling is the process of reducing the sampling rate of a signal. This is usually done to reduce the data rate or the size of the data.

An example of use is to reduce data to half, removing each second element.

Via the DataFilter.perform_downsampling(double[] data, int period, int operation) method, you can define a period and the selected operation via the AggOperations enum values EACH, MEAN and MEDIAN.
double[] result = DataFilter.perform_downsampling( 

	data[index],	// The indexed data channel.
	2,			// Period.
	AggOperations.EACH.get_code()	// Operation.

);

TRANSFORMING data

In this sample:

  • All EEG channels are transformed via the db4 wavelet and inverted back again.
  • All EEG channels are transformed via FFT.

The BrainFlow Java sample: Transforms.java

You use what is call wavelets to transforms data, the wavelet used in the sample transforms data via the db4 - Daubechies 4 wavelet, and a decomposition level of 3.

Pair<double[], int[]> data = DataFilter.perform_wavelet_transform(

	data[index],	// The indexed data channel.
	"db4",		// Wavelet.
	3			// Decomposition level.
);

Wavelets

Wavelets are intentionally crafted to have specific properties that make them useful for signal processing. Using a "reverse, shift, multiply and integrate" technique called convolution, wavelets can be combined with known portions of a damaged signal to extract information from the unknown portions. And it can be inversed too.

Here is a list of the supported wavelet implementations you can use to transform data:

  • db1..db15
  • haar
  • sym2..sym10
  • coif1..coif5
  • bior1.1, bior1.3, bior1.5, bior2.2, bior2.4, bior2.6, bior2.8, bior3.1, bior3.3, bior3.5, bior3.7, bior3.9, bior4.4, bior5.5, bior6.8

Fast Fourier Transform (FFT) and Inverse FFT (IFFT)

Fast Fourier transforms (FFT) are mathematical formulas that relates a signal sampled in time or space to the same signal sampled in frequency. In signal processing, the Fourier transform can reveal important characteristics of a signal, namely, its frequency components.

The perform_fft method is called with a data array and a start- and end position for the data samples to transform.

Complex[] fft_data = DataFilter.perform_fft(

	data[index],	// The indexed data channel.
	0,			// Start position.
	64			// End position.
);

SIGNAL FILTERING data

In this sample:

  • The first 4 EEG channels are singled out and filtered differently.

The BrainFlow Java sample: SignalFiltering.java

A signal filter is a device or process that removes some unwanted components or features from a signal. Filtering is a class of signal processing, the defining feature of filters being the complete or partial suppression of some aspect of the signal.

Filtering types

There are 3 filter types available:

You can read more about the filter types in this Quora article - Which filter type would you use - Butterworth, Chebyshev or Bessel?

 

Filtering methods

There are four main filtering methods available:

  • Low-pass and high-pass filtering, which complement each other.
  • Band-pass and band-stop filtering, which also complement each other.

All the methods works directly on the data channels, after a method call the values in your channel data have changed.

 

Low-Pass Filtering (LPF)

Low-pass filtering (LPF) is a filter that passes signals with a frequency lower than a selected cutoff frequency and attenuates signals with frequencies higher than the cutoff frequency. The exact frequency response of the filter depends on the filter design. The filter is sometimes called a high-cut filter, or treble-cut filter in audio applications. A low-pass filter is the complement of a high-pass filter.

int sampling_rate = BoardShim.get_sampling_rate(board_id);

DataFilter.perform_lowpass(

	data[index],		// The indexed data channel.
	sampling_rate,	// Sampling rate.
	20.0,			// Cutoff.
	4,				// Order.
	FilterTypes.BESSEL.get_code(),	// Filter type.
	0.0				// Ripple.

);

 

High-Pass Filtering (HPF)

High-pass filtering (HPF) is the complement to Low-pass filtering (LTF).

int sampling_rate = BoardShim.get_sampling_rate(board_id);

DataFilter.perform_highpass(

	data[index],		// The indexed data channel.
	sampling_rate,	// Sampling rate.
	5.0,				// Cutoff.
	4,				// Order.
	FilterTypes.BUTTERWORTH.get_code(),	// Filter type.
	0.0				// Ripple.

);

 

Band-Pass Filtering (BPF)

Band-pass filtering (BPF) is a filter that passes frequencies within a certain range and rejects (attenuates) frequencies outside that range.

int sampling_rate = BoardShim.get_sampling_rate(board_id);

DataFilter.perform_bandpass(

	data[index],		// The indexed data channel.
	sampling_rate,	// Sampling rate.
	15.0,				// Center frequency.
	5.0,				// Band width.
	4,				// Order.
	FilterTypes.CHEBYSHEV_TYPE_1.get_code(),	// Filter type.
	1.0				// Ripple.

);

 

Band-Stop Filtering (BSF)

Band-stop filtering (BSF) is a filter that passes most frequencies unaltered, but attenuates those in a specific range to very low levels. It is the opposite of the band-pass filter.

int sampling_rate = BoardShim.get_sampling_rate(board_id);

DataFilter.perform_bandstop(

	data[index],		// The indexed data channel.
	sampling_rate,	// Sampling rate.
	50.0,			// Center frequency.
	1.0,				// Band width.
	4,				// Order.
	FilterTypes.CHEBYSHEV_TYPE_1.get_code(),	// Filter type.
	1.0				// Ripple.

); 

DENOISING data

In this sample:

  • First  EEG channel is filtered via the AggOperation enum MEAN.
  • Second  EEG channel is filtered via the AggOperation enum MEDIAN.
  • All other  EEG channels are denoised via a db4 wavelet.

The BrainFlow Java sample: Denoising.java

Denoising EEG data is about reducing the data noise.

Denoising via MEAN

DataFilter.perform_rolling_filter(

	data[index],	// The indexed data channel.
	3,			// Period.
	AggOperations.MEAN.get_code()	// Operation.

);

 

Denoising via MEDIAN

DataFilter.perform_rolling_filter(

	data[index],	// The indexed data channel.
	3,			// Period.
	AggOperations.MEDIAN.get_code()	// Operation.

);

 

Denoising via a Wavelet

DataFilter.perform_wavelet_denoising(

	data[index],	// The indexed data channel.
	"db4",		// Wavelet.
	3			// Decomposition level.

);

 

Here is an article on wavlet denoising:

Wrap UP

BCI is a very interesting technology and everyone is going to get acquainted with it in the future of human computer interaction.

Our great grand kids will laugh at us, when we tell them that we used a mouse and a board full of keys to communicate with a computer. Their computer will be able to feel what they are thinking. Totally mindblowing and chaotically frightening.

We're amazed how quickly we had something up and running within a few minutes and the programming interface it pretty simple to understand and use. And it runs anywhere. No errors or troubleshooting to decrease motivation. Everything was straightforward and fun to play with.

If you live near Copenhagen, Denmark, and you would like to meet and explore the BCI technology or ideas, feel free to submit a comment and we'll get back to you.

Have good fun with it.

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.