The goal of this assignment was to rectify images and then stitch them into panoramas. When pictures are taken not straight on, there is a distortion which can be fixed with a perspective transform. Once they have been warped correctly, many images can be overlapped and a big panorama can be created.
First the Harris interest point detector is run which results in many many points as follows:
Next Adaptive Non-Maximal Suppression is run which cuts down the number of interest points to:
Then, 40x40 pixel patches around the image are extracted and downsampled so that a better match can be achieved. Next the matching goes through all the images and calculates the SSD on how different they are. At the end, the best match and the second best match are saved. At the end of the whole matching procedure, the average is taken and a threshold value is chosen. All matches that are worse than the threshold are thrown away and only the "good" matches are kept:
Not all the matches are great. Infact, for some panoramas, there are a lot of terrible matches but RANSAC takes care of that. RANSAC runs through and randomly chooses some pairs and sees how many agree with it. That is run a few times and eventually a fairly large consensus is found. From the large consensus, a new transform is computed and used to warp the image and create the panorama automatically:
This panorama lines up well apart from the paper on the table at the bottom. I did not have access to a tripod and suspect that despite trying hard, moved a little which was not as obvious for the far objects but was obvious for the table since it was much closer.