Second Workshop on Chatbots and Conversational Agent Technologies

Shared Task Description

As part of WOCHAT program activities, the Shared Task on Data Collection and Annotation will continue. In this task, participants are required to generate human-machine and human-human dialogues, as well as to produce turn-level annotations on them. Human-machine dialogues can be generated by using different online and offline chat engines made available for such purpose. Annotations are generated following the provided set of guidelines. The collected dataset will be made publicly available to the research community for further research and experimentation in future editions of the workshop.

Metrics and Resources for Chat-oriented Dialogue Evaluation

The Shared Task in WOCHAT is part of a larger scope initiative aiming at both collecting chat-oriented dialogue data that can be made available for research purposes and developing a framework for the automatic evaluation of chat-oriented dialogue. This effort comprises three interdependent tasks:

Similar to RE-WOCHAT, the current Shared Task in WOCHAT focuses on Tasks 1 and 2 only. Task 3 will be addressed in future editions of the workshop after enough annonated data has been generated to make feasible the use of machine learning approaches.

Ways of Participation and Registration

There are four different ways of participating in the shared task:

You can register for participating in one or more of the roles described above by using this form.

Report Format and Submissions

A short report will be required to accompany all participating role activities in the Shared Task. Report submissions to WOCHAT should be in Springer LNCS format (up to 4 pages). Paper submissions must be done in electronic format through the following conference submission system. All submitted reports will be presented at a Shared Task Poster Session the day of the workshop.

Important Dates