Research Output
A Game-Based Setup for Data Collection and Task-Based Evaluation of Uncertain Information Presentation
  Decision-making is often dependent on uncertain data, e.g. data associated with confidence scores, such as probabilities. A concrete example of such data is weather data. We will demo a game-based setup for exploring the effectiveness of different approaches (graphics vs NLG) to communicating uncertainty in rainfall and temperature predictions (www.macs.hw.ac.uk/ InteractionLab/weathergame/ ). The game incorporates a natural language extension of the MetOffice Weather game1. The extended version of the game can beused in three ways: (1) to compare the effectiveness of different information presentations of uncertain data; (2) to collect data for the development of effective data-driven approaches; and (3) to serve as a task-based evaluation setup for Natural Language Generation (NLG).

  • Date:

    10 September 2015

  • Publication Status:

    Published

  • Publisher

    Association for Computational Linguistics (ACL)

  • DOI:

    10.18653/v1/w15-4720

  • Library of Congress:

    QA75 Electronic computers. Computer science

  • Dewey Decimal Classification:

    006.3 Artificial intelligence

  • Funders:

    Engineering and Physical Sciences Research Council

Citation

Gkatzia, D., Cercas Curry, A., Rieser, V., & Lemon, O. (2015). A Game-Based Setup for Data Collection and Task-Based Evaluation of Uncertain Information Presentation. In Proceedings of the 15th European Workshop on Natural Language Generation, 112-113. https://doi.org/10.18653/v1/w15-4720

Authors

Keywords

Natural Language Generation, NLG, data-driven, task-based evaluation.

Monthly Views:

Available Documents