This directory contains the feature files and lexicon used in the
following paper:

  Behrang Mohit, Nathan Schneider, Rishav Bhowmick, Kemal Oflazer, and Noah A. Smith (2012),
  Recall-Oriented Learning of Named Entities in Arabic Wikipedia. Proceedings of EACL.

and can be downloaded at 

  http://www.ark.cs.cmu.edu/AQMAR/

This dataset is licensed under a Creative Commons Attribution-ShareAlike 3.0 Unported License (see LICENSE).

The lexicon directory holds two lexicons that we heuristically
constructed from the list of Arabic Wikipedia titles: (1) A lexicon of
titles which are expected to be named entities; and (2) a lexicon of
titles which are expected not to be named entities.

The dev and test directory hold the complete features that were
extracted using different resources including the MADA toolkit and our
heuristically constructed lexicons of Wikipedia article titles (for
NEs and non-NEs). Here is the list of features. The list includes
four features (26-29) which were turned off during the development process.

0. Token (T1,T2,...Tn-1, Tn)
1. T1
2. Tn
3. T1,T2
4. Tn-1,Tn
5. T1,T2,T3
6. Tn-2,Tn-1,Tn
7. T2,T3
8. Tn-2,Tn-1
9. T2,T3,T4
10. Tn-3,Tn-2,Tn-1
11. T3,T4
12. Tn-4,Tn-3
13. token length
14. is the word's gloss translation capitalized?
15. POS
16. case
17. aspect
18. number
19. person
20. gender
21. state
22. normalized spelling (romanized)
23. is there a mada analysis
24. is the word Arabic UTF-8 (vs. Latin)
25. is base form same as the normalized form
26. Wikipedia: is this a hyperlink or regular text (noisy; turned off)
27. Wikipedia dict: noisy (noisy; turned off)
28. is the token utf-8 or ascii (noisy; turned off)
29. does the term have any vowels (noisy; turned off)
30. is the token in Wiki NE dict
31. is curr+next token in Wiki NE dict
32. is curr+prev token in Wiki NE dict
33. is the token in Wiki non-NE dict
34. is curr+next token in Wiki non-NE dict
35. is curr+prev token in Wiki non-NE dict
36. class

