CS59200-RM: Robot Manipulation

Full Syllabus

Course Description

There have been tremendous recent advances in the capabilities of robots to achieve complex tasks, and the field sits at a precipice of broader impact; robots are moving beyond a priori engineered factory settings to un/semi-structured domains, such as moving packages around a warehouse, providing automated delivery in restaurants and hospitals, advancing human telepresence in distant locales such as underwater and in space, and more. Despite this progress (and the abundance of cool demo videos), it’s still incredibly difficult to make a robot do any one thing, let alone have a generalizable system that can handle a significant breadth of tasks in various environments. So, what’s so hard about making a robot interact with the world?

This course will answer that question and provide an in-depth understanding of the state-of-the-art in robot manipulation by surveying important landmark papers in the field as well as current recently published works. In particular, this course will have an algorithmic and computational focus, providing an understanding of the fundamental techniques necessary for manipulation. We will also cover modern advances in how statistical machine learning (particularly approaches known as deep learning, generative AI, or foundation models) are applied and used by model- and optimization-based methods to the handle the uncertainties of the real world.

Course Information

Course Number CS59200-RM
Meeting Time TTH 1:30-2:45pm
Course Room LWSN 1106

Resources

There is no required textbook for this course. Lecture notes, research papers, and online resources will be provided throughout the course. For the interested reader, there are recommended textbooks that cover much of the basic background necessary:

A nice high-level overview of controlling manipulators is available in this blog post.

Reading Schedule

TBD

Week 1: Introduction to Manipulation

Week 2: Sampling-based Motion Planning

Week 3: Trajectory Optimization

Week 4: Other Motion Planning Paradigms

Week 5: Task Planning

Week 6: Task and Motion Planning

Week 7: Abstractions and Representations

Week 8: Multi-Modal Manipulation and Locomotion

Week 9: Grasping

Week 10: Benchmarking and Evaluation

Week 11: Sensing and Manipulation

Week 12: Mobile Manipulation and Humanoids

Week 13: Real-time Performance and Hardware Acceleration

Week 14: Behavior Cloning and Imitation Learning

Week 15: Miscellaneous Topics

Week 16: Project Presentations