16-785: Integrated Intelligence in Robotics: Vision, Language, and Planning
|Instructor:|| Jean Oh (firstname.lastname@example.org)
(Please prefix the subject line with [16-785].)
|Location:||Zoom meeting link|
|Dates/Times:||Tuesday & Thursday, 4:00 - 5:20 PM (Eastern Time)|
|Office hours:||Zoom meeting link, Wed 5:00 - 7:00 PM|
NOTE: This course will be fully virtual. Since the course is discussion intensive synchronous attendance is required.
Course DescriptionThis course covers the topics on building cognitive intelligence for robotic systems. Cognitive capabilities constitute high-level, humanlike intelligence that exhibits reasoning or problem-solving skills. Such capabilities as semantic perception, language understanding, and task planning can be built on top of low-level robot autonomy that enables autonomous control of physical platforms. The topics generally bridge across multiple technical areas, for example, vision-language intersection and language-action/plan grounding.
This course is composed of 50% lectures and 50% seminar classes. Since this is a project-oriented course, we will put a special emphasis on learning research skills, e.g., problem formulation, literature review, ideation, evaluation planning, results analysis, and hypothesis verification.
Prerequisites: There are no explicit prerequisites for this class, but a general background knowledge in AI and machine learning is assumed.
Course GoalsIn this course, we will strive to answer the following research questions and beyond towards the goal of developing cognitive capabilities on robots.
There will be no designated textbooks for this course but recommended ones:
Reading homework (20%)
There will be 9 reading homeworks (+1 recommended reading). For each seminar class, there will be a list of reading material, e.g., either a book chapter or 1-3 technical papers. For each chapter or paper, write 1-paragraph report on how and why you would modify the idea presented in the paper. Include other observations/thoughts/questions for class discussion.
Although it is strongly recommended that the students do all of the reading homework, each student will get one free-pass, i.e., no homework for one week.
Paper presentations (20%)
Each student must take a lead role in a literature review class at least once. A student is expected to prepare a 20-minute presentation of 1-2 technical papers and lead the discussion.
Class Participation (20%)
Students are strongly encouraged for active participation in class discussion. Examples of active participation in a paper discussion include raising/answering insightful or clarification questions, or sharing additional literature review on related work.
Class Project (40%)
A team can have at most 3 students. In all reports, include the names and the Andrew email addresses of the project members. For each report, up to 2 extra pages in addition to the specified page limit are allowed to include references or graphics only.
Weekly project report (5%)
Each team is required to submit a written report on the project progress (1 report per team). The expected length of a report is a half page either in a paragraph or a list of bullets. Extra pages are allowed for figures and references.
Project proposal (5%, 5 pages limit)
A proposal must include:
Midterm report (10%, 5 pages limit)
A midterm report must include:
Final presentation (10%, 20 minutes)
Final report (10%, 10 pages limit)
A final report must include:
We formally follow the guidelines in the CMU's academic integrity policy
Reasonable Person Principle (RPP)
We informally follow Reasonable Person Principle (RPP), a base culture of CMU's School of Computer Science, where everyone gives/gets the benefit of doubt for trying to be reasonable. The four rules of RPP are the following:
Extensions and Late Assignments
Each student will have up to 5 days of grace that can be used for any homework in whatever way without a penalty (Note that there will NOT be any extension for final project presentation and report). For example, you can use all of the 5 days for the first homework assignment, or split into 2 and 3 days to use for the first and the second assignments, respectively. After the 5 grace days have been used up, there will be no additional extensions; 50% will be deducted 1 day after a due date, and no points will be given after 2 days.