Using Function Optimization To Find Policies: Walking
This assignment explores using function optimization to do
Many examples of doing this are on Jack Wang's web page.
A local example is Seungmoon Song.
More interesting examples are on Michiel van de Panne's web page.
If you want to use some other walking model, or do a quadruped, go ahead.
OpenAIGym has models you can use:
2D Box harder,
2D Mujoco, and
We are distributing a simple simulation in C that works on both
Windows and Linux/Mac.
Look for a readme.txt file in each directory.
Graphics is provided in a separate animation program, and uses the
graphics package from ODE.
You can use the mrdplot/matlab plotting software provided,
or some other plotting software like gnuplot.
Quick start for Linux:
Download and uncompress walk05.zip
- you will need to switch the drawstuff library to 32 bit if you are using
The output d0xxxx is a data file that can be animated or plotted.
To animate a data file run
We use mrdplot running in Matlab to plot data.
This will be demo-ed in class.
Quick start for Windows:
Let us know what has to be done to get this code working on a MAC.
Part 1: Implement an optimization procedure to optimize simulation
parameters to minimize the cost of a 20 second simulation.
The parameters you can modify are listed in walk05/readme.txt
This part is similar to:
Optimizing Walking Controllers.
Optimizes parameters for Simbicon-like controller.
Part 2: Optimize for a robust gait that can handle large random white noise
torso perturbations. What is the largest torso perturbation variance
you can handle? What happens to performance on the unperturbed case?
This part is similar to:
Optimizing Walking Controllers for Uncertain Inputs and Environments.
Optimizes robustness for Simbicon-like controller.
What optimization software should I use?
There is a version of CMA-ES for C.
The Gnu Scientific
Library provides some.
A test program for the GSL Amoeba (Nelder-Mead) optimizer
is available here. It includes instructions
on how to install GSL on Windows.
You can do this in Matlab and call the C-code-based simulation from Matlab.
There are many other
sources of optimization routines available free on the web.
We use SDFAST to generate the dynamics for the robot. If you need to,
get a manual from www.sdfast.com.
Hopefully, you should not need it.
SDFAST generates C code which is in the directory sdfast/.
sdfast/xxx.sd specifies the default kinematic and dynamic parameters
of the robot.
sdfast/xxx.h defines some useful constants for C
If you have problems using the graphics (which are only used by animate.c)
try installing ODE.
What to turn in?
You can work in groups or alone.
Generate a web page describing what you did (one per group).
Include links to your source and any compiled code in either .zip, .tar, or
Be sure to list the names of all the members of your group.
Mail the URL of your web page to firstname.lastname@example.org and email@example.com.
[You complete the address, we are trying to avoid spam.]
The writeup is more important than the code. What did you do? Why
did it work? What didn't work and why?
Here are some tips to get started under Windows (2015 info):
0. Install Visual Studio 2010 (or VS 2013).
1. Download and unzip walk05.zip.
2. Within folder walk05/, there should be 2 sub-folders
walk05/useful/ and walk05/walk/. Create a third sub-folder
3. Download and unzip walk05-windows.zip into walk05/lib/. Now
you should have folders walk05/lib/DebugDLL/,
4. If necessary, enter folder walk05/walk/ and delete executables animate.exe
5. Open project simulate.vcproj and follow the steps to convert
to a VS2010 project.
6. Open project animate.vcproj and follow the steps to convert
to a VS2010 project. Open file animate.c. Change line 266 to
give the correct path to textures on your machine. For example:
7. Build and run simulate.vcproj. You should see it print out
some messages in a command window as it simulates.
8. Build and run animate.vcproj. You should see an animation
showing the robot walking.
The header file drawstuff.h is in walk05/useful/drawstuff-windows/.
The provided drawstuff libraries have been tested and work fine on a
64-bit machine if you use MSVC.
There should be a copy of drawstuff.dll in the walk05/walk/ folder.
As long as drawstuff.dll is in the same folder as animate.exe it
should work. If you are running 32-bit Windows, please copy *.dll
from walk05/lib/ReleaseDLL/ to walk05/walk/.
9. Now you can open up controller.c in simulate.vcproj and
apply optimization techniques to complete the assignment. A
cost function is provided. See readme.txt for more
details. learn.c provides a simple model for running multiple simulations
and improving the performance.
Here are some tips to get started on a MAC (OS X 10.10.2) running Xcode (2015 info):
It seems Xcode under OS X 10.10.2 may symlink gcc to
clang, and clang builds C code by default in GNU C11 mode
(same as GCC version 5 and newer).�
The sdfast code is not C99 compatible.
So try adding the following compile flag and rebuilding the
sdfast stuff: -std=gnu89
Here are some tips to get started on a MAC (OS X)
running VirtualBox Ubuntu (2015 info)