### Regression

Build a function that maps $X$ to a real value $Y$.
• Linear regression
• Nonlinear/Nonparametric regression (Tree regression, Kernel, etc.)

#### Linear Reression

• For linear regression minimizing least-square loss is equivalent to maximizing likelihood, with the assumption that the error term is zero-mean gaussian distributed.
• Closed-form solution:
• $\theta=(X^TX)^{-1}X^Ty$
• But really this is not good. Inversion is expensive and exact solution is very sensitive to outliners. not good. we need some regularization.
• Lasso:
• Or

#### Tree Regression

• Each terminal node is a real value

#### Kernel Regression

• Locally weighted average using kernel as the weighting function