CoSTAR continues to improve as a system. One of our recent tests was to run a lot of repeated evaluations of our vision system in a structure assembly task. In the end, our trials went very well: 10/10 node-link-node structures were completed successfully. Here’s a video:

The robot is actually very good at repeatedly performing this task, despite variations in object pose. It is actually differentiating between the objects on either side of the table in this video: it knows that it should not pick up objects from its left.

Of course, it’s not all perfect. Here’s an example where the computer vision component was slightly wrong about where an object was, but the robot still handled things just fine:

Part of the art of using a system like this is figuring out what the robot can and cannot do. We teach grasp and mate positions for all our different pieces. Before creating the task plan you saw above, we created this one:

Clearly moving straight towards the top of the red link was a bad idea! Instead we changed the robot’s configuration so it reorients its gripper. In the future (using techniques such as my planning projects, discussed here on other occasions) we would like to give the robot a little more freedom to perform these tasks and make these decisions for itself.