Chapter 7

Multilingual and Multimodal Dialogs

Source code:

Available in GitHub

Exercises:

Localisation x 2

The chapter describes how to localize strings to be used for TTS using the resources directories. But, how about the other way round? How can we localize ASR? We have shown how to specify the locale for open ASR, but if our app used handcrafted grammars, we would need to have a grammar for each supported language.

Use the directories approach to save the urls of different speech grammars for different languages, so that once a language is selected by the user or automatically obtained from their device settings, the ASR is started using a particular handcrafted grammar.

Multimodality meets semantics

Improve the MultimodalFormFillLib to support the use of semantic tags in grammars, so that in the SendMessage app, for each contact in the contact list the recognition obtains their phone number and not their name, so that they can be phoned directly.

Suggested reading:

Lopez-Cózar Delgado, R. & Araki, M. (2005) Spoken, Multilingual and Multimodal Dialogue Systems: Development and Assessment. Wiley.

Phillips, M., Nguyen, J., and Mischke, A. (2010) Why Tap When You Can Talk? Designing Multimodal Interfaces for Mobile Devices that Are Effective, Adaptive and Satisfying to the User. In A. Neustein (ed.), Advances in Speech Recognition: Mobile Environments, Call Centers and Clinics, Springer Science+Business Media, LLC 2010