NEWS 2016
Img


NEWS 2016: The Sixth Named Entities Workshop

Shared Task Official Results

These are the offcial results for NEWS 2016 Shared Task on Transliteration of Named Entities. These results are also available at CodaLab's Standard Submission and Non-standard Submission competition pages.

English to Chinese Transliteration Task

Task Team Type ACC F-Score MRR MAP
EnCh Team-1 standard 0.3165 0.6643 0.4130 0.3086
EnCh Team-4 standard 0.2808 0.6434 0.3007 0.2812
EnCh Team-2 standard 0.2659 0.6227 0.3185 0.2549
EnCh Team-3 standard 0.1062 0.5160 0.1062 0.1014
EnCh Team-0 baseline* 0.1935 0.5851 0.1935 0.1830

Chinese to English Back-transliteration Task

Task Team Type ACC F-Score MRR MAP
ChEn Team-1 standard 0.2139 0.7292 0.3026 0.2119
ChEn Team-4 standard 0.1619 0.7253 0.1816 0.1620
ChEn Team-3 standard 0.1099 0.6687 0.1099 0.1066
ChEn Team-2 standard 0.0834 0.6564 0.1425 0.0830
ChEn Team-0 baseline* 0.0981 0.6459 0.0981 0.0953

English to Thai Transliteration Task

Task Team Type ACC F-Score MRR MAP
EnTh Team-1 standard 0.1869 0.7784 0.2789 0.1869
EnTh Team-5 standard 0.1553 0.7537 0.1561 0.1553
EnTh Team-2 standard 0.1456 0.7514 0.2181 0.1456
EnTh Team-0 baseline* 0.0680 0.7070 0.0680 0.0680

Thai to Ennglish Back-Transliteration Task

Task Team Type ACC F-Score MRR MAP
ThEn Team-1 standard 0.1958 0.7881 0.2914 0.1958
ThEn Team-2 standard 0.1286 0.7624 0.1966 0.1286
ThEn Team-0 baseline* 0.0914 0.7397 0.0914 0.0914

English to Persian Transliteration Task

Task Team Type ACC F-Score MRR MAP
EnPe Team-1 standard 0.6958 0.9466 0.7952 0.6799
EnPe Team-2 standard 0.5816 0.9267 0.7116 0.5673
EnPe Team-3 standard 0.5029 0.9104 0.5029 0.4678
EnPe Team-3 non-standard 0.4136 0.8846 0.4136 0.3856
EnPe Team-0 baseline* 0.4818 0.9060 0.4818 0.4482

English to Hindi Transliteration Task

Task Team Type ACC F-Score MRR MAP
EnHi Team-1 standard 0.7150 0.9371 0.7814 0.7091
EnHi Team-2 standard 0.3480 0.8349 0.4745 0.3434
EnHi Team-3 standard 0.1090 0.7292 0.1090 0.1061
EnHi Team-0 baseline* 0.2700 0.7992 0.2700 0.2624

English to Tamil Transliteration Task

Task Team Type ACC F-Score MRR MAP
EnTa Team-1 standard 0.6290 0.9216 0.7171 0.6280
EnTa Team-2 standard 0.3240 0.8369 0.4461 0.3235
EnTa Team-0 baseline* 0.2580 0.8117 0.2580 0.2573

English to Kannada Transliteration Task

Task Team Type ACC F-Score MRR MAP
EnKa Team-1 standard 0.5830 0.9089 0.6815 0.5819
EnKa Team-2 standard 0.2860 0.8224 0.4019 0.2856
EnKa Team-0 baseline* 0.1960 0.7833 0.1960 0.1955

English to Bangla Transliteration Task

Task Team Type ACC F-Score MRR MAP
EnBa Team-1 standard 0.4980 0.8955 0.6266 0.4978
EnBa Team-2 standard 0.3460 0.8600 0.4737 0.3438
EnBa Team-3 standard 0.1990 0.8063 0.1990 0.1969
EnBa Team-0 baseline* 0.2870 0.8360 0.2870 0.2837

English to Hebrew Transliteration Task

Task Team Type ACC F-Score MRR MAP
EnHe Team-1 standard 0.1891 0.8031 0.2679 0.1877
EnHe Team-2 standard 0.1591 0.7976 0.2377 0.1582
EnHe Team-3 standard 0.1482 0.7817 0.1482 0.1466
EnHe Team-3 non-standard 0.1755 0.7806 0.1755 0.1739
EnHe Team-0 baseline* 0.1091 0.7715 0.1091 0.1077

Englisg to Korean (Hangul) Transliteration Task

Task Team Type ACC F-Score MRR MAP
EnKo Team-1 standard 0.3524 0.7067 0.4509 0.3526
EnKo Team-0 baseline* 0.2181 0.6177 0.2181 0.2176

English to Japanese (Katakana) Transliteration Task

Task Team Type ACC F-Score MRR MAP
EnJa Team-1 standard 0.4647 0.8386 0.5916 0.4637
EnJa Team-0 baseline* 0.2091 0.7048 0.2091 0.2060

English to Japanese (Kanji) Back-transliteration Task

Task Team Type ACC F-Score MRR MAP
JnJk Team-1 standard 0.3269 0.5886 0.3883 0.2465
JnJk Team-0 baseline* 0.4612 0.6518 0.4612 0.2967

Arabic to English Transliteration Task

Task Team Type ACC F-Score MRR MAP
ArEn Team-0 baseline* 0.4810 0.9127 0.4810 0.1476

* Baseline results are based on a MT implementation at the character level using MOSES. All baseline results have been generously provided by UPC, Barcelona.