Integrated ROS capabilities for planning, predicate inference, gripper control, and perception for use with the KUKA LBR IIWA and Universal Robots.

View the Project on GitHub cpaxton/costar_stack


Collaborative System for Task Automation and Recognition

Build Status

CoSTAR is an end-user interface for authoring robot task plans developed at Johns Hopkins University. It includes integrated perception and planning capabilities, plus a Behavior Tree based user interface.

CoSTAR Expert User Demonstration

Our goal is to build a system which facilitates end-user instruction of robots to solve a variety of different problems. CoSTAR allows users to program robots to perform complex tasks such as sorting, assembly, and more. Tasks are represented as Behavior Trees. For videos of our system in action, you can check out the CoSTAR YouTube Channel.

To take full advantage of CoSTAR, you will need an RGB-D camera and supported hardware:

This is a project by members of the JHU Laboratory for Computational Sensing and Robotics, namely Chris Paxton, Kel Guerin, Andrew Hundt, and Felix Jonathan. If you find this code useful, please cite:

  title={Co{STAR}: Instructing Collaborative Robots with Behavior Trees and Vision},
  author={Paxton, Chris and Hundt, Andrew and Jonathan, Felix and Guerin, Kelleher and Hager, Gregory D},
  journal={Robotics and Automation (ICRA), 2017 IEEE International Conference on},
  note={Available as arXiv preprint arXiv:1611.06145},

Interested in contributing? Check out the development guidelines, which are a work in progress.


Check out installation instructions.

We are working on experimental install scripts:


Run the IIWA test script:

rosrun costar_bringup

It will start gazebo and move the arm to a new position. If this test passes, CoSTAR is set up right.

There is a more detailed startup guide.

CoSTAR Packages

For more information on how to collect data for the “block stacking” task, check out the block stacking data collection notes


More minor utilities:

Sister repositories

These are repositories that have been integrated with costar_stack, though not necessarily required depending on your setup. Also see the .travis.yml in this repository for additional repositories that have been used with costar_stack.


CoSTAR is maintained by Chris Paxton (

Other core contributors include: