Using Function Optimization To Find Policies: Ball Run
This assignment explores using function optimization to do
The problem used is "ball run" also known as "marble run" or "bubble ball".
Please install the free bubble ball app on your phone and start
Get or make a simulation the will support balls falling in air, bouncing,
and rolling/sliding down a surface.
I have written a simulation using Open Dynamics Engine.
An alternative simulator is Box2D.
Make an optimizer that learns to move the ball as far to the right as possible
with one obstacle bar 0.1m long. Implement several optimization approaches,
including a gradient-based approach, CMA-ES, and another non-derivative-based
approach. DON'T write the code yourself! Use a library, such as
Matlab or the Gnu Scientific Library, I include an old version of the
Numerical Recipes library (2nd Edition) in the ODE simulation above.
You can get
CMA-ES code from Hanson's CMA-ES web site.
Google "public domain optimization libraries" for more.
A vision system sees obstacles and a goal at obstacles.txt in pixel coordinates (so positive Y is down, you should fix that).
The ball is dropped at (431, 181) in pixel coordinates.
The obstacles are really 0.25m long and 0.037m high, so you also need
to convert the pixel values to meters. Use optimization to find a
simulated setup that is "similar" that gets the ball in the goal.
A video (slow motion) of the actual ball on this run.
For the system in part 4, an observed trial has the following ball
trajectory: trajectory.txt, sampled at
30 frames per second in pixel coordinates.
Use optimization to adjust the parameters of the simulation so that
the simulated trajectory matches the observed trajectory,
with the obstacles back in the observed positions.
Parameters to change might include gravity, air resistance,
something to do with rolling vs. sliding (friction, moment of inertia
of the ball, some parameters you make up, ...), and in the ODE
simulation the bounce parameters in dynamics.cpp:
else if ( c_type == OBSTACLE )
contact.surface.mode = dContactBounce | dContactSoftCFM;
// friction parameter
contact.surface.mu = 1.0;
// bounce is the amount of "bouncyness".
contact.surface.bounce = 0.3; // used to be 0.1
// bounce_vel is the minimum incoming velocity to cause a bounce
contact.surface.bounce_vel = 0.01;
// constraint force mixing parameter
contact.surface.soft_cfm = 0.01; // 0.001 in bounce
Construct an interesting simulated ball run with four obstacles,
and send Chris the obstacle locations (in metric coordinates).
We will run it in reality, and send you back a video and ball trajectory
that actually happened. You then modify the obstacle locations, and
we repeat, until the desired behavior happens in reality.
We will construct a challenge for you similar to what you have to
do in the bubble ball app.
What to turn in?
You can work in groups or alone.
Generate a web page describing what you did (one per group).
Include links to your source and any compiled code in either .zip, .tar, or
Be sure to list the names of all the members of your group.
Mail the URL of your web page to email@example.com.
The writeup is more important than the code. What did you do? Why
did it work? What didn't work and why?