Part V: Face detection

Here you will repeat problem 2 of HW2, using ICA and NMF bases to represent images.

PART V.A: Face detection with ICA representations

• Training: From the collection of training face images, learn the set of ICA bases (instead of PCA bases). To do so, first reduce the dimension of the images to $N$ (which we will specify below) using PCA, and subsequently learn the ICA bases from the $N$-dimensional representations. Use the JADE algorithm (code here) for ICA.
• Decomposition:Project every training and test instance onto the $N$ ICA bases to get an $N$-dimensional representation in terms of the independent components.
• Classification Using the ICA representations of the images (instead of their eigen-face weights) repeat problem 2 of assignment 2. Do this for $N$ (no of bases) = 10,15 and 20.

PART V.A: Face detection with NMF representations

• Training: From the collection of training faces, learn 30 NMF bases. To do so, represent the set of training faces as a matrix $D$. Then use the multiplicative NMF update rules described in class to factorize $D$: \[ B = B\otimes \frac{\frac{D}{BW}W^\top}{1.W^\top} \\ W = W\otimes \frac{B^\top\frac{D}{BW}W^\top}{B^\top.1} \] Here $B$ is the matrix of bases, and $W$ is the matrix of weights. If $D$ is $K\times M$, and we learn $N$ bases, $B$ will be $K \times N$ and $W$ will be $N\times M$. The symbol $\otimes$ represents schur (component-wise) multiplication, the divisions are all also component-wise, and $1$ is an $K\times M$ matrix of ones.
• Decomposition: Once bases are learned, decompose all training and test data into a non-negative linear combination of the bases. To do so, for each vector $V$, you must represent $V=BW_V$. $W_V$ can be estimated by inserting $V$ instead of $D$ into the update rule for the weights given above. Only iterate the update rules for weights, keeping $V$ and the learned bases $B$ fixed.
• Classification Represent every image in terms of its NMF weights $W_V$ (instead of their eigen-face weights), and repeat problem 2 of assignment 2. Do this for $N$ (no of bases) = 10,15 and 20.

Due date

The assignment is due on 6 Nov 2014. The solutions must be emailed to Bhiksha and Zhiding. Please send the solutions as a zipfile called yourandrewid_mlsp_hw2b.zip. The zipfile should have the same format as in assignment2.