Experiences of usability evaluation of the IMAGINE speech-based interaction system

IMAGINE is a recently completed multi-partner, multi-national European 5th Framework project aimed at developing systems for interaction with e-business applications through a multilingual natural language interface from mobile and other devices. The vision embodied in IMAGINE is universal access to electronic services by anyone (including disabled users) at any time. Since this long-term vision is not achievable with current capabilities in speech and natural language technologies, we have concentrated initially on a monolingual (English) application, namely a speech-enabled interface to an Online Shop. This paper is primarily concerned with usability evaluation of a prototype system for this application. The principal finding is that considerable challenges and barriers remain against the successful deployment of speech systems in real applications. The problems are not just technical, but systemic (to do with the nature of speech) and are exacerbated by lack of shared background and difficulties of communication between different professionals involved in the design, commissioning and maintenance of speech systems. Formerly Royal National Institute of the Blind.