Shared Task Official Results
These are the offcial results for NEWS 2016 Shared Task on Transliteration of Named Entities.
These results are also available at CodaLab's
Standard Submission and
Non-standard Submission competition pages.
English to Chinese Transliteration Task
| Task |
Team |
Type |
ACC |
F-Score |
MRR |
MAP |
| EnCh |
Team-1 |
standard |
0.3165 |
0.6643 |
0.4130 |
0.3086 |
| EnCh |
Team-4 |
standard |
0.2808 |
0.6434 |
0.3007 |
0.2812 |
| EnCh |
Team-2 |
standard |
0.2659 |
0.6227 |
0.3185 |
0.2549 |
| EnCh |
Team-3 |
standard |
0.1062 |
0.5160 |
0.1062 |
0.1014 |
| EnCh |
Team-0 |
baseline* |
0.1935 |
0.5851 |
0.1935 |
0.1830 |
Chinese to English Back-transliteration Task
| Task |
Team |
Type |
ACC |
F-Score |
MRR |
MAP |
| ChEn |
Team-1 |
standard |
0.2139 |
0.7292 |
0.3026 |
0.2119 |
| ChEn |
Team-4 |
standard |
0.1619 |
0.7253 |
0.1816 |
0.1620 |
| ChEn |
Team-3 |
standard |
0.1099 |
0.6687 |
0.1099 |
0.1066 |
| ChEn |
Team-2 |
standard |
0.0834 |
0.6564 |
0.1425 |
0.0830 |
| ChEn |
Team-0 |
baseline* |
0.0981 |
0.6459 |
0.0981 |
0.0953 |
English to Thai Transliteration Task
| Task |
Team |
Type |
ACC |
F-Score |
MRR |
MAP |
| EnTh |
Team-1 |
standard |
0.1869 |
0.7784 |
0.2789 |
0.1869 |
| EnTh |
Team-5 |
standard |
0.1553 |
0.7537 |
0.1561 |
0.1553 |
| EnTh |
Team-2 |
standard |
0.1456 |
0.7514 |
0.2181 |
0.1456 |
| EnTh |
Team-0 |
baseline* |
0.0680 |
0.7070 |
0.0680 |
0.0680 |
Thai to Ennglish Back-Transliteration Task
| Task |
Team |
Type |
ACC |
F-Score |
MRR |
MAP |
| ThEn |
Team-1 |
standard |
0.1958 |
0.7881 |
0.2914 |
0.1958 |
| ThEn |
Team-2 |
standard |
0.1286 |
0.7624 |
0.1966 |
0.1286 |
| ThEn |
Team-0 |
baseline* |
0.0914 |
0.7397 |
0.0914 |
0.0914 |
English to Persian Transliteration Task
| Task |
Team |
Type |
ACC |
F-Score |
MRR |
MAP |
| EnPe |
Team-1 |
standard |
0.6958 |
0.9466 |
0.7952 |
0.6799 |
| EnPe |
Team-2 |
standard |
0.5816 |
0.9267 |
0.7116 |
0.5673 |
| EnPe |
Team-3 |
standard |
0.5029 |
0.9104 |
0.5029 |
0.4678 |
| EnPe |
Team-3 |
non-standard |
0.4136 |
0.8846 |
0.4136 |
0.3856 |
| EnPe |
Team-0 |
baseline* |
0.4818 |
0.9060 |
0.4818 |
0.4482 |
English to Hindi Transliteration Task
| Task |
Team |
Type |
ACC |
F-Score |
MRR |
MAP |
| EnHi |
Team-1 |
standard |
0.7150 |
0.9371 |
0.7814 |
0.7091 |
| EnHi |
Team-2 |
standard |
0.3480 |
0.8349 |
0.4745 |
0.3434 |
| EnHi |
Team-3 |
standard |
0.1090 |
0.7292 |
0.1090 |
0.1061 |
| EnHi |
Team-0 |
baseline* |
0.2700 |
0.7992 |
0.2700 |
0.2624 |
English to Tamil Transliteration Task
| Task |
Team |
Type |
ACC |
F-Score |
MRR |
MAP |
| EnTa |
Team-1 |
standard |
0.6290 |
0.9216 |
0.7171 |
0.6280 |
| EnTa |
Team-2 |
standard |
0.3240 |
0.8369 |
0.4461 |
0.3235 |
| EnTa |
Team-0 |
baseline* |
0.2580 |
0.8117 |
0.2580 |
0.2573 |
English to Kannada Transliteration Task
| Task |
Team |
Type |
ACC |
F-Score |
MRR |
MAP |
| EnKa |
Team-1 |
standard |
0.5830 |
0.9089 |
0.6815 |
0.5819 |
| EnKa |
Team-2 |
standard |
0.2860 |
0.8224 |
0.4019 |
0.2856 |
| EnKa |
Team-0 |
baseline* |
0.1960 |
0.7833 |
0.1960 |
0.1955 |
English to Bangla Transliteration Task
| Task |
Team |
Type |
ACC |
F-Score |
MRR |
MAP |
| EnBa |
Team-1 |
standard |
0.4980 |
0.8955 |
0.6266 |
0.4978 |
| EnBa |
Team-2 |
standard |
0.3460 |
0.8600 |
0.4737 |
0.3438 |
| EnBa |
Team-3 |
standard |
0.1990 |
0.8063 |
0.1990 |
0.1969 |
| EnBa |
Team-0 |
baseline* |
0.2870 |
0.8360 |
0.2870 |
0.2837 |
English to Hebrew Transliteration Task
| Task |
Team |
Type |
ACC |
F-Score |
MRR |
MAP |
| EnHe |
Team-1 |
standard |
0.1891 |
0.8031 |
0.2679 |
0.1877 |
| EnHe |
Team-2 |
standard |
0.1591 |
0.7976 |
0.2377 |
0.1582 |
| EnHe |
Team-3 |
standard |
0.1482 |
0.7817 |
0.1482 |
0.1466 |
| EnHe |
Team-3 |
non-standard |
0.1755 |
0.7806 |
0.1755 |
0.1739 |
| EnHe |
Team-0 |
baseline* |
0.1091 |
0.7715 |
0.1091 |
0.1077 |
Englisg to Korean (Hangul) Transliteration Task
| Task |
Team |
Type |
ACC |
F-Score |
MRR |
MAP |
| EnKo |
Team-1 |
standard |
0.3524 |
0.7067 |
0.4509 |
0.3526 |
| EnKo |
Team-0 |
baseline* |
0.2181 |
0.6177 |
0.2181 |
0.2176 |
English to Japanese (Katakana) Transliteration Task
| Task |
Team |
Type |
ACC |
F-Score |
MRR |
MAP |
| EnJa |
Team-1 |
standard |
0.4647 |
0.8386 |
0.5916 |
0.4637 |
| EnJa |
Team-0 |
baseline* |
0.2091 |
0.7048 |
0.2091 |
0.2060 |
English to Japanese (Kanji) Back-transliteration Task
| Task |
Team |
Type |
ACC |
F-Score |
MRR |
MAP |
| JnJk |
Team-1 |
standard |
0.3269 |
0.5886 |
0.3883 |
0.2465 |
| JnJk |
Team-0 |
baseline* |
0.4612 |
0.6518 |
0.4612 |
0.2967 |
Arabic to English Transliteration Task
| Task |
Team |
Type |
ACC |
F-Score |
MRR |
MAP |
| ArEn |
Team-0 |
baseline* |
0.4810 |
0.9127 |
0.4810 |
0.1476 |
* Baseline results are based on a MT implementation at the character level using MOSES. All baseline results
have been generously provided by UPC, Barcelona.
