Architecture

Speech Recognition

ASR is supported by CMU Sphinx and Amazon Lex. All recognition is offline by default.

Speech-to-Text

Whether Lex manages to resolve and fulfill an intent or not, it will still return the recognised utterance in text (unless it did not hear anything at all). LexASR and CMUSphinxASR provide a method waitForUtterance() which blocks until the speech to text service returns a string.

NLP - Text to Action

If Lex does manage to resolve and fulfill (to the point where it delegates to client-side fulfillment) an intent by invoking a Lamba function then LexRecognizer notifies a NlpResultListener that the request has been fulfilled or failed etc.

NlpProvider defines a method processUtterance() which takes a string utterance and context. LexNlp implements NlpProvider and notifies the NlpResultListener.

Text-to-Speech

TTS is supported by MaryTTS and Amazon Polly. Speech synthesis is offline by default.