15-494/694 Cognitive Robotics

Monday / Wednesday 4:00 ‐ 4:50 in DH 1211 (Doherty Hall)

Friday 3:30 ‐ 4:50 in NSH 3206 (Robotics Education Lab)

Spring 2023

Units: 12.0, Section: A


Instructor: Professor David S. Touretzky (just "Dave" is fine)
  • email: dst@cs.cmu.edu; office phone 412-268-7561
  • Office location: Gates-Hillman Center, room 9013
  • Office hours: by appointment

Teaching Assistant: Lauren Kung
  • email: laurenku@andrew.cmu.edu
  • Office hours: Thursdays 11:00 - 12:00 and also 5:30 - 6:30 in the REL

Course Description

This course explores the implementation of intelligent behavior in mobile robots, focusing on the Cozmo robot by Anki. It consistes of a series of Monday/Wednesday lectures, a parallel series of Friday hands-on labs and problem sets, and a capstone project taking up the last three weeks.

The prerequisite for the course is intermediate-level programming skills and facility with Python. Prior experience in robotics or artificial intelligence is helpful but not required.

Learning Objectives

After taking this course, you will be able to:
  1. Program intelligent behaviors on the Cozmo robot using Python.
  2. Employ computer vision techniques using OpenCV to recognize markers and objects.
  3. Design robot environments that facilitate visual landmark-based localization and navigation.
  4. Use machine learning tools to train convolutional neural networks for robotics applications.
  5. Use speech recognition to provide voice control of a robot.
  6. Assess the strengths and limitations of three Cozmo programming frameworks: the Cozmo Python SDK, Code Lab, and Calypso.

Learning Resources

  • There is no textbook for the course.

  • All software required for this course has been installed on the workstations in the REL (Robotics Education Lab), to which you will have 24/7 access. In addition, you can install the Cozmo Python SDK and cozmo-tools package on your personal laptops if you wish.

  • The Cozmopedia web site contains useful information about both Cozmo and the cozmo-tools package that is the main software framework used in the course.

  • Online documentation for OpenCV 3 is available here.


There are no exams in this class. The final course grade will be calculated using the following categories:

Lab participation10 points
Homework problems60 points
Final Project30 points
Total  100 points

  • Lab participation means showing up for each lab (attendance will be taken) and following the steps in the writeup.

  • Most labs are too long to be completed in the time allotted. The remaining steps and problems constitute homeworks that you will have a week to complete.

  • For the final project, which is 30% of your grade, you can develop a complex robot behavior using the tools you learned in this course, or you can develop a new tool that extends the capabilities of the cozmo-tools software framework we're using. Final projects may either be done solo or in two-person teams.

The following letter grades will be assigned based on calculations coming from the course assessment section.
Grade   Percentage Interval
A 90% - 100%
B 80% - 89%
C 70% - 79%
D 65 - 69%
R (F) below 65%

Grading Policies

  • Late-work policy: Assignments are due at 11:59 pm on the Friday following the lab. They can be submitted up to two days late at a cost of 1 point per day. Assignments more than 2 days late will not be accepted.

  • Make-up work policy: Students can make up work if they miss a deadline due to illness.

  • Re-grade policy: If you believe your assignment was graded incorrectly, please contact the TA who graded it. We will be happy to take another look.

  • Attendance policy: Lab attendance is worth 10% of your grade. A sign-in sheet will be circulated at the beginning of each class. You will be allowed 2 unexcused absences without penalty. (Only the attendance penalty is waived; you must still turn in the assignment if you want to get credit for it.) Additional absences incur a 10 point penalty. Excused absences include illness, conference attendance in connection with research, or participation in certain university-sponsored activities such as a team sporting event. Job interviews and other personal activities do not qualify as excused absences.

Course Policies

  • Academic Integrity and Collaboration: The work you submit in this course must be your own, with the exception of the final project, which can be done in a team of two. You are welcome to help or receive help from your fellow students on general matters such as how to fix a Python error, but you may not share your code with other students, collaborate on writing Python code, or in any other way submit or take credit for work that is not purely your own.

  • Class Communication: We will use Piazza as our primary means of online communication. Please ask questions via Piazza rather than emailing the instructor or TA directly, so that your fellow students can benefit from the discussion. Sometimes a classmate may be able to answer your question more quickly than the instructor or TA.

  • Purchase of Materials: While we will provide you with some materials, you may need to purchase additional materials to complete a final project. Materials such as cardboard or posterboard can be purchased at the CMU Art Store in the Cohon University Center.

  • Accomodations for Students with Disabilities: If you have a disability and have an accommodations letter from the Disability Resources office, I encourage you to discuss your accommodations and needs with me as early in the semester as possible. I will work with you to ensure that accommodations are provided as appropriate. If you suspect that you may have a disability and would benefit from accommodations but are not yet registered with the Office of Disability Resources, I encourage you to contact them at access@andrew.cmu.edu.

  • Statement of Support for Students' Health and Well-Being: Take care of yourself. Do your best to maintain a healthy lifestyle this semester by eating well, exercising, avoiding drugs and alcohol, getting enough sleep, and taking some time to relax. This will help you achieve your goals and cope with stress.

    All of us benefit from support during times of struggle. There are many helpful resources available on campus and an important part of the college experience is learning how to ask for help. Asking for support sooner rather than later is almost always helpful.

    If you or anyone you know experiences any academic stress, difficult life events, or feelings of anxiety or depression, we strongly encourage you to seek support. Counseling and Psychological Services (CaPS) is here to help: call 412-268-2922 and visit their website at http://www.cmu.edu/counseling/. Consider reaching out to a friend, faculty or family member you trust for help getting connected to the support that can help.

Course Schedule

Please see the course schedule page for a list of lectures, assignment issue dates, assignment due dates, and office hours sessions.

How to Succeed in This Course

Most of your grade will be based on the programming assignments you turn in. To do well on these assignments, start on them early, and don't be shy about asking for help. The professor and TA are both happy to help you find tricky bugs or map out a successful strategy for solving a problem. You can post general questions to Piazza; please make them public if possible. If you need to post snippets of your code, use a private post, or send it as an email attachment.

Back to 15-494/694 course home page