Home Publications Videos Software Contact  
  Brain Signal Decoding
Real-time decoding of different levels of force
Parkinson's disease (PD) patients implanted with deep brain stimulation (DBS) electrodes in the Subthalamic Nucleus (STN) brain area attempts to extend the leg with 3 different force levels (0%, 33%, 66% of maximum force) while sitting. The decoder takes local field potentials (LFP) signals from the STN and predicts the intended force level in real time while the patient is executing the action.

Offline decoding of walking and freezing of gait
PD patients implanted with DBS electrodes in STN area of the brain walks with two different lengths of stride. Occasionally, the patient exhibits freezing of gait (FoG). The decoder takes as input the LFP signals from STN to classify different states of walking and FoG.

Brain signal dynamics while walking
PD patients implanted with DBS electrodes attempts in the STN brain area walks on the ground with different strides and their corresponding brain dynamics are reflected on LFP signals.

Non-invasive real-time decoding of human brain signals
A paraplegic subject attempts to move his own paralyzed legs. His intention is recognised and feedback for the corresponding leg is shown in real time.

Brain-controlled lower-limb exoskeleton
5 able-bodied subjects perform a 3-way control (front, turn left & right) of a powered lower-limb exoskeleton using motor imagery.

Representation Learning
A task representation is learned by observing human demonstrators performing structured actions, where the rule that generates actions is not known to the robot. Task representation is learned by abstracting observed actions as well as finding compounded actions. This allows the robot to not merely imitate actions that humans perform, but also predict the next likely action and filter out inconsistent actions. More importantly, the robot can interpret unforeseen variations of actions that were generated from the same task representation.

Hierarchical task representation learning
One such example is the Towers of Hanoi puzzle, where the robot learns the general solution (task representation) by observing a human demonstrator solving using only 2 and 3 disks and successfully interpret and imitate when the human is performing using 4 disks (or more).

Predictive action execution using learned representation of actions

Musical primitives composed and produced by Jonas Golland

Fun stuff during my student years
Micromouse - Quantum (handmade custom circuit design)

Line Tracer - Goombeng (handmade custom circuit design)

Lightning in Rhythm
The swinging rhythm of a human body is captured at 10 Hz and reproduced in the form of thunder and lightning with electric discharges at 50,000 V. Created for a demonstration at Vision Language Research course in Fine Arts department.