Machine learning ranking
Goal:
To learn a function $F(D,q)$ that is able to rank the input $D$ (items) usually with regard to a query $q$. It can take on a few forms:
- listwise ranking: directly output a ranked list
- pairwise ranking: given a pair, output their relative order
- pointwise ranking: output a score for each individual item
Methodology:
- pointwise: treated as a regression problem. just use your favorite learner+loss. note you can also directly optimize on the metrics such as DCG(http://www.yichang-cs.com/yahoo/cikm09_smoothdcg.pdf) as long as it's smooth or you can find an approxiate that is smooth.
Evaluation:
After getting a new ranking function, how to verify if its better than existing ones?
- DCG
- Chi-Square test: https://en.wikipedia.org/wiki/Wilcoxon_signed-rank_test
- Wilcoxon test: Non-parametric paired test. https://en.wikipedia.org/wiki/Wilcoxon_signed-rank_test
Reading