** This page is frequently updated. **
16-785: Integrated Intelligence in Robotics: Vision, Language, and Planning
|Instructor:|| Jean Oh (email@example.com)
(It is important that you prefix the subject line with [16-785] when emailing the instructor.)
|TA:||Allan Wang (allanwan at andrew)|
|Location:||Newell Simon Hall (NSH) 3002|
|Dates/Times:||Monday & Wednesday, 09:00 - 10:20 AM|
Allan: NSH 4506, Mon/Wed 10:30 - 11:30 AM |
Jean: NSH 4521, Wed 3:00 - 5:00 PM
Course DescriptionThis course covers the topics on building cognitive intelligence for robotic systems. Cognitive capabilities constitute high-level, humanlike intelligence that exhibits reasoning or problem solving skills. Such capabilities as semantic perception, language understanding, and task planning can be built on top of low-level robot autonomy that enables autonomous control of physical platforms. The topics generally bridge across multiple technical areas, for example, vision-language intersection and language-action/plan grounding. This course is composed of 50% lectures and 50% seminar classes. There are no explicit prerequisites for this class, but a general background knowledge in AI and machine learning is assumed.
Course GoalsIn this course, we will strive to answer the following research questions and beyond towards the goal of developing cognitive capabilities on robots.
Please check this class website periodically for possible updates as the schedule may be subject to change.
* Guest lecture schedules are tentative and can change to different dates.
There will be no designated textbooks for this course but recommended ones:
Reading homework (30%)
There will be 10 reading homework. For each seminar class, there will be a list of reading material, e.g., either a book chapter or 1-3 technical papers. For each chapter or paper, write up to one-paragraph report on how and why you would modify the idea presented in the paper. Include other observations/thoughts/questions for class discussion.
Although it is strongly recommended that the students do all of the reading homework, each student will get one free-pass, i.e., no homework for one week. If the student uses the free-pass, the student must attend the class that discusses the material.
Paper presentations (15%)
Each student must take a lead role in a literature review class at least once. A student is expected to prepare a 20-minute presentation of 1-2 technical papers and lead the discussion.
Review homework (10%)
There will be 5 homework assignments. These assignments will help the students review the material discussed during the class.
Class Participation (5%)
Students are encouraged for active participation in paper discussion. Examples of active participation include raising/answering insightful or clarification questions, or sharing additional literature review on related work.
Class Project (40%)
A team can have at most 3 students. In all reports, include the names and the Andrew email addresses of the project members. For each report, up to 2 extra pages in addition to the specified page limit are allowed to include references or graphics only.
Sample projects (from Independent Study)
Weekly project report (5%)
Each team is required to submit a written report on the project progress (1 report per team). The expected length of a report is a half page either in a paragraph or a list of bullets. Extra pages are allowed for figures and references.
Project proposal (5%, 5 pages limit)
A proposal must include:
Midterm report (10%, 5 pages limit)
A midterm report must include:
Final presentation (10%, 20 minutes)
Final report (10%, 10 pages limit)
A final report must include:
We formally follow the guidelines in the CMU's academic integrity policy (http://www.cmu.edu/policies/student-and-student-life/academic-integrity.html).
Reasonable Person Principle (RPP)
We informally follow Reasonable Person Principle (RPP), a base culture of CMU's School of Computer Science, where everyone gives/gets the benefit of doubt for trying to be reasonable. The four rules of RPP are the following:
Extensions and Late Assignments
Each student will have up to 5 days of grace that can be used for any homework in whatever way without a penalty (Note that there will NOT be any extension for final project presentation and report). For example, you can use all of the 5 days for the first homework assignment, or split into 2 and 3 days to use for the first and the second assignments, respectively. After the 5 grace days have been used up, there will be no additional extensions; 50% will be deducted 1 day after a due date, and no points will be given after 2 days.