Evaluating human-machine conversation for appropriateness.

Webb, Nick, Benyon, David, Hansen, Preben and Mival, Oli (2010) Evaluating human-machine conversation for appropriateness. In: Proceedings of the Seventh conference on International Language Resources and Evaluation (LREC'10). European Language Resources Association (ELRA), Malta. ISBN 2-9517408-6-7

[img] PDF
Restricted to Registered users only
Available under License Creative Commons Attribution Non-commercial.

Download (665kB) | Request a copy


    Evaluation of complex, collaborative dialogue systems is a difficult task.Traditionally, developers have relied upon subjective feedback from the user,and parametrisation over observable metrics. However, both models place somereliance on the notion of a task; that is, the system is helping to userachieve some clearly defined goal, such as book a flight or complete a bankingtransaction. It is not clear that such metrics are as useful when dealing witha system that has a more complex task, or even no definable task at all, beyondmaintain and performing a collaborative dialogue. Working within the EU fundedCOMPANIONS program, we investigate the use of appropriateness as a measure ofconversation quality, the hypothesis being that good companions need to be goodconversational partners . We report initial work in the direction of annotatingdialogue for indicators of good conversation, including the annotation andcomparison of the output of two generations of the same dialogue system

    Item Type: Book Section
    ISBN: 2-9517408-6-7
    Uncontrolled Keywords: Dialogue; Evaluation methodologies; Usability; user satisfaction;
    University Divisions/Research Centres: Faculty of Engineering, Computing and Creative Industries > School of Computing
    Dewey Decimal Subjects: 000 Computer science, information & general works > 000 Computer science, knowledge & systems > 004 Data processing & computer science
    Library of Congress Subjects: Q Science > QA Mathematics > QA76 Computer software
    Item ID: 3767
    Depositing User: Computing Research
    Date Deposited: 24 Jun 2010 15:50
    Last Modified: 12 Jan 2011 04:55

    Actions (login required)

    View Item

    Document Downloads

    More statistics for this item...

    Edinburgh Napier University is a registered Scottish charity. Registration number SC018373