INSPIRING FUTURES

How Was Your Day? evaluating a conversational companion.

Benyon, David, Gambäck, Björn, Hansen, Preben, Mival, Oli and Webb, Nick (2012) How Was Your Day? evaluating a conversational companion. IEEE Transactions on Affective Computing. (In Press)

Full text not available from this repository. (Request a copy)

Abstract/Description

The “How Was Your Day” (HWYD) Companion is an embodied conversational agent that can discuss work-related issues, entering free-form dialogues that lack any clearly defined tasks and goals. The development of this type of Companion technology requires new models of evaluation. Here, we describe a paradigm and methodology for evaluating the main aspects of such functionality in conjunction with overall system behaviour, with respect to three parameters: functional ability (i.e., does it do the ‘right’ thing), content (i.e., does it respond appropriately to the semantic context), and emotional behaviour (i.e., given the emotional input from the user, does it respond in an emotionally appropriate way). We demonstrate the functionality of our evaluation paradigm as a method for both grading current system performance, and targeting areas for particular performance review. We show correlation between, for example, ASR performance and overall system performance (as is expected in systems of this type) but beyond this, we show where individual utterances or responses, which are indicated as positive or negative, show an immediate response from the user, and demonstrate how our combination evaluation approach highlights issues (both positive and negative) in the Companion system’s interaction behaviour

Item Type: Article
Electronic ISSN: 1949-3045
Additional Information: Submitted for Review “© 2012 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.”
Uncontrolled Keywords: Affectice computing; companions; evaluation; spoken interfaces;
University Divisions/Research Centres: Edinburgh Napier University, Institute for Informatics and Digital Innovation
Dewey Decimal Subjects: 000 Computer science, information & general works > 000 Computer science, knowledge & systems > 006 Special Computer Methods > 006.3 Artificial intelligence
Library of Congress Subjects: Q Science > QA Mathematics > QA75 Electronic computers. Computer science
Item ID: 5309
Depositing User: Computing Research
Date Deposited: 31 May 2012 15:59
Last Modified: 07 Aug 2013 15:30
URI: http://researchrepository.napier.ac.uk/id/eprint/5309

Actions (login required)

View Item

Edinburgh Napier University is a registered Scottish charity. Registration number SC018373