I spent the last year or so developing an approach for combining planning of actions with planning sequences of actions based on their effects. Things came together with a batch of experiments performing a simple magnetic structure assembly task.

The robot chooses which pieces to grab and which grasps to use based on a library of demonstrations provided by the user. This allows ordinary end-users to teach robots how to perform extremely complex tasks – and it allows the robots to adapt them intelligently to solve new challenges.