Calypso Module 1: Perception
Learning Goal: This module will show you how to use Calypso to
investigate Cozmo's perception. Cozmo has a camera in his head; he
uses computer vision to recognize objects and faces, and he keeps a
map of the world in his "mind". You can see this map on your computer
screen. Cozmo can also sense when you pick up or tap on one of his
If you don't have a robot and are running Calypso in simulator mode,
skip ahead to the "Editor Mode and Exection
- Wake up Cozmo by putting him on his charger. If he's already on
it, remove him and put him back on it again. You should see his
backpack lights pulse on and off, indicating that he has power and is
- Run the Cozmo app on the Fire tablet, connect to the robot, and
put the app into SDK mode. (To put the app into SDK mode, press the
Settings button in the top right corner of the app display, then
scroll right until you get to the "SDK Mode" button.)
- If Cozmo has driven himself off the charger, put him back on the
charger. Once in SDK mode he will stay put until you tell him to
- Connect the Fire tablet to your computer using the USB cable that came
with the tablet.
- Plug your Xbox 360 game controller into another USB port on your
computer. On Windows you may have to wait a few seconds while the
system locates and installs the appropriate device driver.
- Run Calypso by clicking on the Calypso icon on your desktop.
- Check your Fire for a pop-up message asking whether you want to
allow USB debugging. If you see this pop-up, check the box that says
"Always allow from this computer" and then press "OK" to proceed. You
should only have to do this once per machine.
- If Calypso starts successfully, you should see a Chrome browser tab
that looks like this:
- If Calypso pops up a window telling you that it could not connect
to the robot, see the
Troubleshooting section of the
Calypso Help Center for help.
- You should see the words "Novice Mode" in green in the bottom
right corner of the screen. If you don't see them, it's a good idea
to return to Novice Mode before proceeding further:
- Mouse click on the Home icon in the top right corner of the screen.
- Select "Settings..."
- Click on "Normal Mode" to change to Novice Mode.
The Camera Viewer
- Cozmo has a camera in his head, and built-in computer vision that
allows him to see the world. The Camera Viewer lets you see what
- When Calypso starts up, the Camera Viewer should be visible. If
it's not, click on the camera icon on the right side of the screen to
unhide the Camera Viewer.
- You can change the size of the Camera Viewer, or hide it again,
by moving your mouse over the Camera Viewer window so that the title
bar appears, and then clicking on one of the buttons that appear in
the title bar.
- You can move the Camera Viewer to a different spot on the screen by
moving your mouse over the Camera Viewer window and then clicking and
dragging on any blank spot in the title bar.
- If you hide the Camera Viewer by clicking on the close button
(the "X") in the title bar, and then un-hide it again by clicking on
the camera icon, the Camera Viewer remembers the size and location you
What Can Cozmo See? Recognizing Light Cubes
Cozmo comes with three light cubes that have distinct markers on their
faces. Cube #1 is called the "paperclip" because its markers look
like paperclips. Cube #2 is called the "anglepoise lamp"; you can see
why here. Some people call cube #3 the "deli slicer"
because it sort of looks like this. Others think it looks like a child in a car
seat, or like the letters "bT". We'll stick with "deli slicer".
Here is a diagram showing all the cube markers:
To see Cozmo recognize a light cube:
- Set the Camera Viewer display to the largest size.
- Put a cube in front of Cozmo, about six inches away. You may
need to tilt his head up or down so he can see it properly. (Just
press on his head with your fingers and you can move it.)
- When Cozmo recognizes the cube, you will see a yellow box around
it in the camera viewer.
- Slide the cube to the left or right until it starts to go out of
the camera image. How much of the cube does Cozmo need to see in
order to recognize it?
- Put the cube back in front of Cozmo so he can see it. Put
another cube next to the first one, so he can see both of them. Now
you should see two boxes in the Camera Viewer, one for each cube.
- Move the second cube so that it is slightly behind the first
cube. When one object hides another from view, this is called
"occlusion". Try sliding the second cube around and watching the
result in the Camera Viewer. How much of the second cube has to be
visible for Cozmo to still be able to see it?
- Cozmo is rather near-sighted. If you move a cube further away
from him, at some point he will stop detecting it even though
you can still see it in the Camera Viewer. (If necessary,
adjust his head angle so that the cube remains visible in the camera
image.) If you move the cube closer, Cozmo will be able to see it
again. Using a ruler or tape measure, determine how far away the cube
can be before Cozmo loses sight of it.
Cozmo can recognize human faces. Turn him so that he's facing you and
tilt his head up so his camera can view your face. You should see a green
box drawn around your face in the Camera Viewer.
Editor Mode and Execution Mode
The Calypso software you're using has two modes: editor mode and
execution mode. Editor mode, which you're in right now, lets you write
and edit programs. Execution mode runs the programs. Calypso always
starts out in editor mode. Both modes offer the Camera Viewer.
You can switch between editor mode and execution mode by pressing the
Back button on the game controller (see the colored square in the
diagram above), or by pressing the backspace key (Delete key on a Mac)
on the keyboard. Try it.
Notice that along the left edge of the Calypso window there is "button
help" or "keyboard help" to remind you of the available options. The
icon for the Back button is a leftward-facing triangle.
In execution mode the left half of the screen displays the program
that is running, but you cannot edit the program. You can only
observe it. Since we haven't entered a program yet, there is only one
empty rule, and not much happens on the left half of the screen when
you enter execution mode.
As in editor mode, execution mode allows you to control the size and
placement of the Camera Viewer. You can hide the viewer, or move it
to a more convenient location for execution, and this will not affect
the viewer settings in editor mode. They're kept separate.
The right half of the execution mode screen shows the world map, which
we'll now investigate.
Xbox Game Controller
Cozmo's World Map
Cozmo maintains a map of the world so he can remember where things are
when he's not looking directly at them. The world map is shown on the
right half of the screen in execution mode. Remember that you can
enter execution mode by pressing the Back button on the controller or
the backspace key on the keyboard.
Cozmo is shown on the map as a triangle. His map is constantly
updated. Put a cube in front of Cozmo and tilt his head so he can
see it. (Gently move the head up or down with your fingers.) When
Cozmo sees the cube, a square will appear in the world map display.
Turn the cube and you'll see the square rotate.
Put your hand between Cozmo and the cube so he can no longer see it,
and notice the change of appearance of the square in the world map: it
switches from a thick line to a thin one. If Cozmo can't see a cube,
he assumes it is still whever he last saw it.
Slide the cube so it's at a slightly different location, then take
your hand away so Cozmo can see it again. Notice that the cube
"jumps" in the world map when Cozmo sees it at its new location.
Demo Program: AnnounceCubes
The file AnnounceCubes.calypso
contains a demonstration program that has Cozmo say the name of each
cube he sees. If you hide a cube from him and then show it to him
again, he'll say the name again.
To load the program, simply click on the link above while Calypso is
Sending Cube Taps
A cube can feel when you tap on it. Let's try this out:
- If you're running the AnnounceCubes program, press the Back
button (or the backspace key) to return to editor mode.
- Press the Start button (or click on the Home icon in the top
right corner of the screen) to bring up the Home menu, and select
New. This erases the current program.
- Press the Back button again to return to execution mode.
- Put a cube where Cozmo can see it.
- Tap on the cube with your finger, and you'll see the square in
the world map display pulse.
- You can also simulate a tap by clicking on the square in the
worldmap display with your mouse. What happens when you do this?
Review and Discovery
Do the Calypso Module 1 Review and
Discovery activity to reinforce what you've learned.
In the next module you'll write your
first program for Cozmo, and learn how to drive him around using the
game controller or keyboard. This is called teleoperation.
Back to Calypso Curriculum overview.
Copyright © 2017 Visionary Machines LLC.