## DistLearnKitA Matlab Toolkit for Distance Metric Learning |

Welcome! This is a Matlab toolkit for distance metric learning, including the implementation of a number of published machine learning algorithms in this area. The first version of this toolkit has been available since Oct. 28, 2007.

- Supervised Distance Metric Learning
can be divided into two categories: the global distance metric
learning, and the local distance metric learning.
The first one learns the distance metric in a global sense, i.e., to satisfy all the pairwise
constraints simultaneously by keeping all of the data points in each class close together
while ensuring that data points from different classes are separated. The second approach
is to learn a distance metric in a local setting, i.e., rather than satisfying all of the pair-wise
constraints simultaneously, only to satisfy "local" pairwise constraints. This is particularly
useful for information retrieval and the KNN classifiers since both methods are influenced
most by the data instances that are close to the test/query examples.
Methods Locality Linearity Learning Strategies Code Download Publication Probablistic Global Distance Metric Learning (PGDM) global linear constrained convex programming by Eric P. Xing [pdf] Relevant Components Analysis (RCA) global linear capture global structure; use equivalence constraints by Aharon Bar-Hillel and Tomer Hertz, [pdf] Discriminative Component Analysis (DCA) global linear improve RCA by exploring negative constraints by Steven C.H. Hoi [pdf] Local Fisher Discriminant Analysis (LFDA) local linear extend LDA by assigning greater weights to closer connecting examples [by Masashi Sugiyama] [pdf] Neighborhood Component Analysis (NCA) local linear extend the nearest neighbor classifier toward metric learing [by Charless C. Fowlkes] [pdf] Large Margin NN Classifier (LMNN) local linear extend NCA through a maximum margin framework [by Kilian Q. Weinberger] [pdf] Localized Distance Metric Learning (LDM) local linear optimize local compactness and local separability in a probabilistic framework [by Liu Yang] [pdf] DistBoost global linear learn distance functions by training binary classifiers with margins in a boosting framework by Tomer Hertz and Aharon Bar-Hillel notes on calling its kernel version [pdf] Kernel DistBoost [pdf] Active Distance Metric Learning (BAYES+VAR) global linear select example pairs with the greatest uncertainty, posterior estimation with a full Bayesian treatment [by Liu Yang] [pdf] - Unsupervised Distance Metric Learning
(manifold learning) can be categorized along the following two dimensions:
first, the learnt embedding is linear or nonlinear; and second, the structure to be preserved
is global or local. All the linear manifold learning methods except Multidimensional Scaling (MDS),
learn an explicit linear projective mapping and can be interpreted as the problem of distance metric learning;
and nonlinear manifold learning also has its essentially connections to
distance metric learning.

See*The Connection Between Manifold Learning and Distance Metric Learning*^{new!}(written in Oct., 2007)Methods Locality Linearity Learning Strategies Code Download Publication Principal Component Analysis(PCA) global structure preserved linear best preserve the variance of the data [by Deng Cai] Multidimensional Scaling(MDS) global structure preserved linear best preserve inter-point distance in low-rank [ included in Matlab Toolbox for Dimensionality Reduction] ISOMAP global structure preserved nonlinear preserve the geodesic distances [by J. B. Tenenbaum, V. de Silva and J. C. Langford] [pdf] Laplacian Eigenamp (LE) local structure preserved nonlinear preserve local neighbor [by Mikhail Belkin] [pdf] Locality Preserving Projections (LPP) local structure preserved linear linear approximation to LE [LPP by Deng Cai] [Kernel LPP by Deng Cai] [pdf] Locally Linear Embedding (LLE) local structure preserved nonlinear nonlinear preserve local neighbor [by Sam T. Roweis and Lawrence K. Saul] Hessian LLE can be found at [MANI fold Learning Matlab Demo, by Todd Wittman] [pdf] Neighborhood Preserving Embedding (NPE) lobal structure preserved linear linear approximation to LLE [by Deng Cai] [pdf]