My goal here is to make a cool app using a camera and a projector.
Basically the notion of
projector based graphics, with a camera and a projector facing the same
image plane, can lead to a number of applications that I find really
interesting. The example that got me started on was this video of people
interacting with a projected box:
The final product I wanted was an interactive photo manipulation
The first feature of this program will be an iphone or microsoft surface
where you move a pile of pictures around with your hands:
From there there are other manipulation tools, like air brush, blur, smudge, etc seem like a natural fit for this format.
For both of these the upper right is the program window, which will eventually be full screen and on a projector. The lower right is what the camera sees, and the lower left is the computed output.
Camera calibration detecting checkerboard pattern (note that the
camera is paused to get a better screenshot):
Background subtraction for finger detection:
Next I did a color calibration, since this is what black looks like:
Here is background subtraction on the final app without the color
correction on top, and with color correction on bottom.
Now the blob based finger detection, basically a threshold of the bottom
of the image:
Then I just calculate the tip of the finger, and use it. To do this I use
a really dumb algorithm, wich just looks for the highest line that has
finger points in it. This is a safe assumption though since the projected
screen is above my head. A more intelligent algorithm that calculated the
tip of the blob intelligently could be used. Here you can see
three images that you can move back and forth, change the order of the
Next I added the blur image feature. Here are two examples, the
first i am blurring the top of the pole, and the second i am blurring
the bush in the center. In each the 4 panels are (starting in the
upper left, going clockwise) blurred expected image, unblured camera
photo, finger detection (filtered threshold of the difference of
first two), and program output screen:
Next a quick reset of the image and a second example:
Now in all of these examples I've shown I wasnt really using a projector. For ease of testing, I used either my laptop or my desktop monitors, but this works depending on the lighting in the area. However I have tested it (and presented it) on a projector, and it works just as well there.