Machine Learning Thesis Proposal

Thesis Proposals
Machine Learning Department
Shape-constrained Estimation in High Dimensions
Friday, January 31, 2014 - 2:30pm
Traffic21 Classroom 6501 
Gates&Hillman Centers
Abstract:
Shape-constrained estimation techniques such as convex regression or log-concave density estimation offer attractive alternatives to traditional nonparametric methods. Shape-constrained estimation often has an easy-to-optimize likelihood, no tuning parameter, and an adaptivity property where the sample complexity adapts to the complexity of the underlying functions. In this dissertation proposal, our thesis is that some shape-constraints have an additional advantage in that they are naturally suited to high-dimensional problems, where the number of variables is large relative to the number of samples. 
 
In the first part of this proposal, we show that convex functions have an additive faithfulness property, where the additive approximation is guaranteed to capture all relevant variables even if the true function is not additive. We design computationally efficient sparse convex additive models and prove that it achieves variable selection consistency with good sample complexity. The overall work provides a practical tuning-free semi-parametric generalization of the Lasso. 
 
We then propose three directions of development. First, we propose to loosen the convexity assumption by learning convex-plus-concave functions, which is a vastly more general function class than convex functions. Second, we consider variable selection on general smooth functions by first decomposing the function into a convex part and a concave part and then exploiting additive faithfulness. Finally, we  study graph structure learning on a shape-constrained multivariate distribution. 
 
Thesis Committee:
John Lafferty (Chair)
Aarti Singh
Larry Wasserman
Ming Yuan (University of Wisconsin)

Copy of Proposal Document

Keywords:
For More Information, Please Contact:

diane [atsymbol] cs.cmu.edu