Estimating Spoken Dialog System Quality With User Models
Download Estimating Spoken Dialog System Quality With User Models PDF/ePub or read online books in Mobi eBooks. Click Download or Read Online button to get Estimating Spoken Dialog System Quality With User Models book now. This website allows unlimited access to, at the time of writing, more than 1.5 million titles, including hundreds of thousands of titles in various foreign languages.
Estimating Spoken Dialog System Quality with User Models
Author: Klaus-Peter Engelbrecht
language: en
Publisher: Springer Science & Business Media
Release Date: 2012-08-04
Spoken dialog systems have the potential to offer highly intuitive user interfaces, as they allow systems to be controlled using natural language. However, the complexity inherent in natural language dialogs means that careful testing of the system must be carried out from the very beginning of the design process. This book examines how user models can be used to support such early evaluations in two ways: by running simulations of dialogs, and by estimating the quality judgments of users. First, a design environment supporting the creation of dialog flows, the simulation of dialogs, and the analysis of the simulated data is proposed. How the quality of user simulations may be quantified with respect to their suitability for both formative and summative evaluation is then discussed. The remainder of the book is dedicated to the problem of predicting quality judgments of users based on interaction data. New modeling approaches are presented, which process the dialogs as sequences, and which allow knowledge about the judgment behavior of users to be incorporated into predictions. All proposed methods are validated with example evaluation studies.
Simulation-Based Usability Evaluation of Spoken and Multimodal Dialogue Systems
This book describes an extension of the user behaviour simulation (UBS) of an existing tool for automatic usability evaluation (AUE). This extension is based upon a user study with a smart home system. It uses technical-sociological methods for the execution of the study and the analysis of the collected data. A comparison of the resulting UBS with former UBSs, as well as the empirical data, shows that the new simulation approach outperforms the former simulation. The improvement affects the prediction of dialogue metrics that are related to dialogue efficiency and dialogue effectiveness. Furthermore, the book describes a parameter-based data model, as well as a related framework. Both are used to uniformly describe multimodal human-computer interactions and to provide such descriptions for usability evaluations. Finally, the book proposes a new two-stage method for the evaluation of UBSs. The method is based on the computation of a distance measures between two dialogue corpora and the pair-wise comparison of distances among several dialogue corpora.
Towards Adaptive Spoken Dialog Systems
Author: Alexander Schmitt
language: en
Publisher: Springer Science & Business Media
Release Date: 2012-09-19
In Monitoring Adaptive Spoken Dialog Systems, authors Alexander Schmitt and Wolfgang Minker investigate statistical approaches that allow for recognition of negative dialog patterns in Spoken Dialog Systems (SDS). The presented stochastic methods allow a flexible, portable and accurate use. Beginning with the foundations of machine learning and pattern recognition, this monograph examines how frequently users show negative emotions in spoken dialog systems and develop novel approaches to speech-based emotion recognition using hybrid approach to model emotions. The authors make use of statistical methods based on acoustic, linguistic and contextual features to examine the relationship between the interaction flow and the occurrence of emotions using non-acted recordings several thousand real users from commercial and non-commercial SDS. Additionally, the authors present novel statistical methods that spot problems within a dialog based on interaction patterns. The approaches enable future SDS to offer more natural and robust interactions. This work provides insights, lessons and inspiration for future research and development, not only for spoken dialog systems, but for data-driven approaches to human-machine interaction in general.