CoSTAR Instructor

The Instructor BT Editor

As of today, CoSTAR’s Instructor UI is available on GitHub. You can also find it with the rest of our project.

Getting started with Git

It has come to my attention that a lot of our new and incoming students have not really used Git before. Git’s an incredibly useful tool for software engineering; it’s particularly important for working on group projects.

There are a couple reasons for this:

  • It tracks user contributions and file history, making it easier to figure out when bugs were introduced
  • It allows us to easily maintain different versions of a project with features that are currently in development
  • It makes it easy to combine different subsets of our code.

Let’s get started. On Ubuntu you can install git like so:

sudo apt-get install git

On Windows, the process is barely more complicated. Download Git for windows and add it to your system path. Quick windows command line refresher: to bring up the command prompt, press windows+R, then type “cmd” and press enter. Everything else will be the same after this.

Now, there are a couple million guides that are a Google search away if you want to learn how to use Git with a new repository (see, for example, this one or this one or all these. In this example, I’m going to talk about how to specifically start contributing to one of my projects as an example for how to get things running.

What follows is a brief, high-level guide to how to get started using Git. This is not meant to be comprehensive, and those well-acquainted with git may not love how I use some of these terms. The point here is to get the high-level concepts across for people who have not yet dug into all the details.

Set Up Your Account

You’ll need to create an account on GitHub or whatever other site you are using. For GitHub, create an SSH key as per the instructions here. For GitLab (which we use internally at JHU), the instructions are similar.

Clone The Project

We use git clone to download the project and its history. To clone CoSTAR, for example:

git clone

Now, CoSTAR is a ROS project, which means you clone it into your Catkin workspace. For something else, like this java game, you can clone it wherever seems appropriate. For MATLAB code like this, you would normally clone it into your MATLAB workspace, for example.


Git lets us track code in different branches. For example, if I cd costar_stack, then I can run git branch:

chris@blunderbuss:~/catkin_ws/src/costar_stack$ git branch
* master

This means I am currently on branch master.

When you are starting out, you probably want to create your own branch for whatever feature you will be working on. For example, if I am adding support for the Barrett WAM arm to CoSTAR, I might do this:

git checkout -b barrett-wam

This creates a new branch called barret-wam and moves me into that branch.

In the future, we can switch between branches with git checkout:

git checkout master
git checkout barrett-wam

Making Changes and Pushing

Let’s say I add a new file: In this case, I use the git add command:

git add

This tells Git it should add this file to the set of files it’s currently tracking.

Now I want to commit this change. This actually adds it to my history.

git commit -a

A message will pop up to tell you to enter a message. Try to keep these messages short and sweet, and keep your commits atomic. What that means is that I only change essentially one thing at a time.

We’re not done! Now I’ve added this file to my local history, but I still need to share with others. To do that, I push:

git push origin barret-wam

Substitute barret-wam with whatever your branch is called.

Stay Up To Date

Use the git fetch command to keep your local information up to date. Use git pull to download changes others have made.

Hopefully this helps you get started.

Building a Behavior Tree with CoSTAR

CoSTAR is a framework we have been developing for authoring robot task plans. It is still very much a work in progress, but we are excited about how well it is coming a long.

A crucial part of letting end-users interact and program robots is giving them a framework that is powerful enough to solve their own problems, but adaptable enough that solutions won’t fall apart at the slightest deviation. We achieve all this through our Behavior Tree-based user interface, which you can see in the video below.

One of the key parts of this interface is our SmartMove operation. SmartMoves are special operations that take a list of predicate constraints on what constitutes a valid goal. In layman’s terms, you can describe the qualities you want an object to have before you pick it up: something like “get me the next piece from the right side of the table,” as show here.

CoSTAR Structure Assembly Videos

CoSTAR continues to improve as a system. One of our recent tests was to run a lot of repeated evaluations of our vision system in a structure assembly task. In the end, our trials went very well: 10/10 node-link-node structures were completed successfully. Here’s a video:

The robot is actually very good at repeatedly performing this task, despite variations in object pose. It is actually differentiating between the objects on either side of the table in this video: it knows that it should not pick up objects from its left.

Of course, it’s not all perfect. Here’s an example where the computer vision component was slightly wrong about where an object was, but the robot still handled things just fine:

Part of the art of using a system like this is figuring out what the robot can and cannot do. We teach grasp and mate positions for all our different pieces. Before creating the task plan you saw above, we created this one:

Clearly moving straight towards the top of the red link was a bad idea! Instead we changed the robot’s configuration so it reorients its gripper. In the future (using techniques such as my planning projects, discussed here on other occasions) we would like to give the robot a little more freedom to perform these tasks and make these decisions for itself.

KUKA Innovation Award

KUKA produced this great video showing all of the different teams that contributed to the recent KUKA innovation award. Kel Guerin and I make a brief appearance to describe the CoSTAR System.