Blocks World Activity 1: Getting Vision Data


Goal: Demonstrate and capture blocks world build sequences. This data will be used to develop our vision algorithms, and will later be used to instruct the robot to copy the demonstrated sequences, and reason about them. This is known as "learning from demonstration".


The idea is to use your phone to photograph each step of building a structure. You can choose whatever structure you like: an arch, a bridge, a house, a platform, ... If you want inspiration, here is a collection of photos of structures from instruction books. You can also Google Image "Keva Contraptions".


I suggest building the structure on top of a piece of black and white paper or cardboard with a distinctive pattern on it, so 1) we can register the photographs to each other, and 2) you can easily rotate the structure to photograph it from different angles.

Step 1: Put down the piece of paper or cardboard and photograph it.


Step 2: Put down the first block of your structure, and photograph it from several angles.


Step 3: Put down the next block of your structure, and photograph it from several angles.


Step i: Put down the next block of your structure, and photograph the structure from several angles.


Step N: Put all the photos in a directory and make it available to Chris A. (web, dropbox, ...). Preserve the full resolution (size, quality) of the photos.


Try photoshop, snapchat, gimp, or random vision programs you find on the web to see techniques can "see" the blocks best. Here is the OpenCV code I wrote.