###
Regression

Build a function that maps $X$ to a real value $Y$.
- Linear regression
- Nonlinear/Nonparametric regression (Tree regression, Kernel, etc.)

#### Linear Reression

- For linear regression minimizing least-square loss is equivalent to maximizing likelihood, with the assumption that the error term is zero-mean gaussian distributed.
- Closed-form solution:
- $\theta=(X^TX)^{-1}X^Ty$
- But really this is not good. Inversion is expensive and exact solution is very sensitive to outliners. not good. we need some regularization.

- Lasso:
- Or

#### Tree Regression

- Each terminal node is a real value

#### Kernel Regression

- Locally weighted average using kernel as the weighting function