15-494/694 Cognitive Robotics Lab 10:
Finishing Up Cube Detection

I. Software Update, SDK Update, and Initial Setup

At the beginning of every lab you should update your copy of the cozmo-tools package. Do this:

$ cd ~/cozmo-tools
$ git pull

II. Experiments with Cube Detection

1. Make a lab10 directory.

2. Copy the files from the class lab10 directory to your lab10 directory.

3. Examine the code in generate.py, which creates the training data we'll be using for our cube detector.

4. Read over the cube detector code in test4.py and run the demo by typing python3 -i test4.py and then call train() several times until all patterns are recognized correctly.

5. Do show_pattern(54) to show an example pattern and test the cube detector; look at the output on the console to verify that a cube was detected. Do show_pattern(2900) to test a pattern with no cube.

6. Run python again, and this time train the model with a learning rate that is too low by typing train(lr=0.001). Why does the model get exactly 2700 patterns correct?

7. Run python again, and this time train the model with a learning rate that is too high by typing train(lr=0.9). What do you observe?

III. Completing the Cube Detector

1. The code in generate.py uses only the 9 background images in the "snaps" directory. Replace this with the larger set of background images that you collected.

2. The code in generate.py only shifts cubes down and to the right. Generalize this code so it can shift cubes in any direction.

3. Another issue with generate.py is that it doesn't shift cubes far enough that they fall partially out of the camera frame. Correct this problem.

4. Rerun test4.py on your expanded training set and demonstrate that the model can detect cubes even when they are partially out of frame.

5. Create a separate test set consisting only of images with cubes that are partially out of the camera frame, plus images with no cubes. Run your trained model on the test set.

Hand In

Create a handin.zip file containing the following:

1. The source code for your modified generate.py, test4.py, and the code that created the test set.

2. A brief report describing the training of your model (how many epochs, what was the final percent correct on the training set), and the performance of your model on the test set.