Model Recommendation

People

• Pyry Matikainen
• Rahul Sukthankar
• Martial Hebert

Description

Model recommendation is the application of recommender systems such as those employed for consumer items (e.g., Netflix, Amazon.com) to the problem of selecting a classifier or classifiers from a large library for a particular vision or robotics task. In a typical recommender system, a database of historical user ratings of items is used to predict a new user's ratings of items based on their ratings of a small subset of those items, and thereby return (or recommend) to that user items that they are predicted to rate highly. Analogously, in model recommendation the objective is to predict the accuracies (or other measure of classifier quality) of classifiers on a new task, based on the measured accuracies of a subset of those classifiers on the task.

Simplified Illustrative Example

Consider the problem of trying to detect instances of people walking when viewed from a particular (unknown) camera viewpoint. Suppose that you have been provided with a a library of classifiers produced by training SVMs to detect walking, where each SVM is trained only on samples from a narrow viewpoint, defined by its elevation and mean distance to subject(left). Then, if you could somehow evaluate how good each classifier is on a particular task, the accuracies of all 1600 classifiers on a that task can be visualized as a heat map (right).

If a reasonable amount of training data is available for the new camera viewpoint, then selecting the best classifier from the library is as simple as evaluating each classifier in the library against the training data, and then selecting the classifier with the highest accuracy on the task. However, suppose that very little labeled training data is available for the new task. In the limit, suppose that just one labeled example of walking is available from the new camera (top), in which case the accuracy of every classifier can be evaluated against that example, but the result is a highly quantized map of classifier accuracies (lower left), since every classifier either gets the example right (100% accuracy) or gets the example wrong (0% accuracy). In this situation, there is no way to choose between any of the classifiers that predicted the one example correctly, since they all seem to be perfect. However, model recommendation is able to take this highly quantized map of accuracies and return predicted accuracies which are better estimates of the true accuracies of the classifiers in the library (lower right).

Publications

Classifier Ensemble Recommendation
Pyry Matikainen, Rahul Sukthankar, and Martial Hebert
Workshop on Web-scale Vision and Social Media, ECCV 2012, August, 2012.

Model Recommendation for Action Recognition
Model Recommendation for Action Recognition
Pyry Matikainen, Rahul Sukthankar, and Martial Hebert
Proceedings of the Conference on Computer Vision and Pattern Recognition (CVPR), June, 2012.

Acknowledgements

This work was partially funded by the Army Research Laboratory under Cooperative Agreement #W911NF-10-2-0061.

Copyright notice

The documents contained in these directories are included by the contributing authors as a means to ensure timely dissemination of scholarly and technical work on a non-commercial basis. Copyright and all rights therein are maintained by the authors or by other copyright holders, notwithstanding that they have offered their works here electronically. It is understood that all persons copying this information will adhere to the terms and constraints invoked by each author's copyright.