In this task you will program the Calliope robot to provision a small
Cobot (based on the Willow Garage Turtlebot) with snacks that it will
in turn deliver to humans. The problem is open ended and will be
explored in stages. The Cobot lives in Manuela Veloso's lab and looks
We have created a Cobot mockup for testing of your solution. We
have also created a simple Mirage world for testing:
In solving this task you will make use of multiple perception and
manipulation mechanisms that we have explored in previous labs. This
is an open ended question, so be creative!
To complete the task you must solve all of these problems:
- Locate and pick up snacks. We will provide you with
several types of sample snacks. In Mirage they are just simple red or
green cubes, but for the real robot we are using real snacks:
pretzels, crackers, and gum. You must visually identify snacks and
then grasp them with the gripper. For initial experimentation you can
mark snacks with small AprilTags or strips of colored tape, but
eventually we will want to use unmodified snacks.
While you can pick up snacks from the floor, it might be advantageous
to create a "snack holder" that holds a snack in an orientation
favorable for grasping. This will probably be most useful for the
bags of pretzels. You are welcome to design and fabricate snack
holders if you wish.
- Navigate to the Cobot. All the snacks are located in the
middle alcove of the playpen; the cobot is located in an adjacent
alcove. Once you have retrieved a snack, you must get close enough to
the cobot to allow you to complete your mission. You cannot assume a
fixed position and orientation of the cobot, only a general location.
So you will need visual cues to identify the cobot's pose. The cobot
itself is white and hence hard to see; you should consider decorating
it with AprilTags or colored tape to make it more visible.
- Transfer the snacks. Snacks must be deposited in the
Cobot's basket, which is 500 mm above the floor. This is at the very
limits of the arm's reach, but should be achievable. Tossing snacks
might be a successful strategy.
- Accept snack requests from the cobot. The cobot will
accept a request from a human for a particular type of snack, travel
to the Calliope, and then tell the Calliope what it needs. This
communication must take place via the network. One simple strategy
would be to have the cobot connect to the Calliope on port 10001 and
issue !msg commands to send text messages requesting snacks.
This complex task has not been attempted with Calliope before, so we
don't know what sorts of problems will arise as you try to solve it.
We will work with you to propose solutions and refine the task
description in order to end up with a working system.
Please organize yourselves into teams of 1-3 students. Each team will
solve the entire task. This is not a competition; teams are permitted
to help each other. But they are also encouraged to explore
alternative strategies so that we can all learn more about what works
and what doesn't. Grading will be based on not just how well your
solution works, but also how well you explored that strategy and can
explain why it works and why it fails.
At the conclusion of the project you will hand in a video that
demonstrates the above behaviors, your code, and a brief writeup with
screenshots that show how the robot identifies the cobot and the
snacks. Please submit these items by Monday, April 9.
Dave Touretzky and