One possible application of our new models is to allow the robot to plan in domains where planning is currently very difficult.

Take surgical robotics as an example. Surgical robots have a problem: simulating what will happen is often very difficult. Our approach allows us to see the next couple actions an expert might perform, and their effects on the world.

Check out the YouTube video above. You’ll notice some jitter: this is a result of uncertainty. We used the JIGSAWS dataset, which is unfortunately fairly small.