Home Publications Videos Software Contact  
Learning Reusable Task Structures from Demonstrations and
Detecting Human's Intention using Probabilistic Grammars

Project Overview
title image The learning of reusable action components of a task from human demonstrations is useful for various robotic applications, such as imitation learning. In this project, we present a novel activity grammar learning method capable of learning reusable task components from a reasonably small number of samples under noisy conditions.  Our linguistic approach aims to extract the hierarchical structures of actions which can be recursively applied to help recognize unforeseen, more complicated tasks that share the same task components.


The robot learns a task representation in the form of stochastic context-free grammars from human demonstrations using our learning method. The learned grammars act as a belief system, which enables a robot to correct contextually inconsistent actions that include both detection errors and human errors. Note that these errors also exist in training dataset and our method can effectively extract the important local constraints.

The Towers of Hanoi and Dance

Video Demonstration

  • Kyuhwa Lee, Yanyu Su, Tae-Kyun Kim and Yiannis Demiris, "A Syntactic Approach to Robot Imitation Learning using Probabilistic Activity Grammars", Robotics and Autonomous Systems Journal (RAS), Elsevier, Volume 61, Issue 12, pp.1323-1334, 2013.
  • Kyuhwa Lee, Tae-Kyun Kim and Yiannis Demiris, "Learning Reusable Task Representations using Hierarchical Activity Grammars with Uncertainties", IEEE International Conference on Robotics and Automation (ICRA), pp.1994-1999, St. Paul, USA, 2012.