Using Function Optimization To Find Policies: Walking

This assignment explores using function optimization to do policy optimization.

Many examples of doing this are on Jack Wang's web page. A local example is Seungmoon Song. More interesting examples are on Michiel van de Panne's web page. If you want to use some other walking model, or do a quadruped, go ahead. OpenAIGym has models you can use: 2D Box, 2D Box harder, 2D Mujoco, and 3D Mujoco.

We are distributing a simple simulation in C that works on both Windows and Linux/Mac. Look for a readme.txt file in each directory.

Graphics is provided in a separate animation program, and uses the graphics package from ODE. You can use the mrdplot/matlab plotting software provided, or some other plotting software like gnuplot.

Quick start for Linux:

Quick start for Windows:

Let us know what has to be done to get this code working on a MAC.

Part 1: Implement an optimization procedure to optimize simulation parameters to minimize the cost of a 20 second simulation. The parameters you can modify are listed in walk05/readme.txt

This part is similar to: Optimizing Walking Controllers. Optimizes parameters for Simbicon-like controller.

Part 2: Optimize for a robust gait that can handle large random white noise torso perturbations. What is the largest torso perturbation variance you can handle? What happens to performance on the unperturbed case?

This part is similar to: Optimizing Walking Controllers for Uncertain Inputs and Environments. Optimizes robustness for Simbicon-like controller.

What optimization software should I use? There is a version of CMA-ES for C. The Gnu Scientific Library provides some. A test program for the GSL Amoeba (Nelder-Mead) optimizer is available here. It includes instructions on how to install GSL on Windows. You can do this in Matlab and call the C-code-based simulation from Matlab. There are many other sources of optimization routines available free on the web.

We use SDFAST to generate the dynamics for the robot. If you need to, you can get a manual from Hopefully, you should not need it. SDFAST generates C code which is in the directory sdfast/. sdfast/ specifies the default kinematic and dynamic parameters of the robot. sdfast/xxx.h defines some useful constants for C

If you have problems using the graphics (which are only used by animate.c) try installing ODE.

What to turn in?

You can work in groups or alone. Generate a web page describing what you did (one per group). Include links to your source and any compiled code in either .zip, .tar, or .tar.gz format. Be sure to list the names of all the members of your group. Mail the URL of your web page to and [You complete the address, we are trying to avoid spam.] The writeup is more important than the code. What did you do? Why did it work? What didn't work and why?


Here are some tips to get started under Windows (2015 info):

Here are some tips to get started on a MAC (OS X 10.10.2) running Xcode (2015 info):

Here are some tips to get started on a MAC (OS X) running VirtualBox Ubuntu (2015 info)