Dialog System Technology Challenges


June 6, 2018 Decided potential venues for DSTC7 at NIPS, AAAI or IWSDS. Workshop proposals to NIPS and AAAI will be submitted soon.

June 1, 2018 Development phase has been started (Registration Form)

April 1, 2018 DSCT 7 website launched

Background and Motivation

Starting as an initiative to provide a common testbed for the task of Dialog State Tracking, the first Dialog State Tracking Challenge (DSTC) was organized in 2013, followed by Dialog State Tracking Challenges 2 & 3 in 2014. More recently, Dialog State Tracking Challenge 4 and Dialog State Tracking Challenge 5 and Dialog System Technology Challenge 6 have been completed in 2015, 2016, and 2017 respectively. Since DSTC4, the challenge has evolved in several ways. First, from human-computer interactions, the challenges started to investigate human-human interactions. Then, the event started to offer pilot tasks on Spoken Language Understanding, Speech Act Prediction, Natural Language Generation and End-to-end System Evaluation which increased the reach of the challenge into the research community of dialog systems and AI.

Given the remarkable success of the first five editions, and understanding both, the complexity of the dialog phenomenon and the interest of the research community in a wider variety of dialog related problems, the DSTC rebranded itself as "Dialog System Technology Challenges" for its sixth edition.

On the other hand, although DSTC4 and DSTC5 started using human annotations, since DSTC6 the goal has shifted towards End-to-End systems with the aim of minimizing the human annotation effort while exploring more complex tasks.

In this seventh edition, the call for task proposals has resulted into three different tracks, (1) Sentence Selection: using the Ubuntu corpus and Flex Data: Student – Advisor dialogues, (2) Sentence Generation and (3) Audio Visual Scene-aware dialog. The objective of these tracks is to invite interested organizations conduct dialog related challenges in specific areas of research and under the umbrella of the DSTC.

DSTC 7 Organizations

The 7th Dialog System Technology Challenge (DSTC7) is supported by Microsoft Research, Mitsubishi Electric Research Laboratories, Adobe Research, Naver Labs Europe, Heriot-Watt University, Nara Institute of Science and Technology, Institute for Infocomm Research, and COLIPS.