Providing a Custom Speech Recognition UI

The listening, confirmation, and disambiguation screens provided by the SpeechRecognizerUI may not suit your particular purposes and can be obtrusive under some circumstances. Fortunately, you can replace this GUI with your own by working directly with the SpeechRecognizer object rather than the SpeechRecognizerUI.

In this section you see how to assemble an interface that provides feedback to the user during speech recognition. The sample code for this section is located in the Speech/CustomUI directory of the WPUnleashed.Examples project in the downloadable sample code.

We begin by looking at how to retrieve a SpeechRecognizer from a SpeechRecognizerUI via its Recognizer property, as shown in Listing 23.10.

The sample RecognitionUIViewModel’s GetSpeechRecognizer method creates a new SpeechRecognizerUI object and then subscribes to its various events (see Listing 23.10). The AudioCaptureStateChanged event provides notification when the recognizer begins listening for speech, at which point it is placed in a Capturing state. The AudioCaptureStateChanged event is also raised when the speech recognizer has finished capturing and is processing, and is placed in an Inactive state.

The SpeechRecognizer’s AudioProblemOccurred event is raised when there is an audio problem that could affect recognition accuracy.

The SpeechRecognizer in this example uses the default dictation grammar.

LISTING 23.10. RecognitionUIViewModel.GetSpeechRecognizer Class

Click here to view code image

SpeechRecognizerUI GetSpeechRecognizer()
{
    if (speechRecognizerUI == null)
    {
        speechRecognizerUI = new SpeechRecognizerUI();

        SpeechRecognizer speechRecognizer = speechRecognizerUI.Recognizer;

        speechRecognizer.AudioCaptureStateChanged
            += delegate(SpeechRecognizer sender,
                        SpeechRecognizerAudioCaptureStateChangedEventArgs args)
            {
                if (args.State == SpeechRecognizerAudioCaptureState.Capturing)
                {
                    Status = "Capturing";
                }
                else if (args.State == SpeechRecognizerAudioCaptureState.Inactive)
                {
                    Status = "Inactive";
                }
                else
                {
                    Status = string.Empty;
                }
            };

        speechRecognizer.AudioProblemOccurred
            += delegate(SpeechRecognizer sender,
                        SpeechAudioProblemOccurredEventArgs args)
            {
                Status = "Audio Problem Occurred.";
            };
    }

    return speechRecognizerUI;
}

The viewmodel’s Prompt method retrieves the SpeechRecognizerUI and uses its SpeechRecognizer to perform the speech recognition (see Listing 23.11). The SpeechRecognizer.RecognizeAsync is analogous to the SpeechRecognizerUI.RecognizerWithUIAsyc method, but does not present the user with the built-in displays; all interaction must therefore be handled by custom code.

LISTING 23.11. RecognitionUIViewModel.Prompt Method

Click here to view code image

async void Prompt()
{
    SpeechRecognizerUI recognizer = GetSpeechRecognizer();

    SpeechRecognitionResult result;
    recognitionInProgress = true;

    try
    {
        result = await recognizer.Recognizer.RecognizeAsync();
    }
    catch (Exception ex)
    {
        Debug.WriteLine("Unable to recognize text. " + ex);
        return;
    }
    finally
    {
        recognitionInProgress = false;
        Status = string.Empty;
    }

    RecognizedText = recognizedText + " " + result.Text;
}

A recognitionInProgress Boolean field prevents the PromptCommand from being executed during speech recognition, which could otherwise lead to a race condition. The AppBarIconButton automatically toggles its enabled state, depending on the executable state of the PromptCommand.

The PromptCommand is initialized in the viewmodel’s constructor, as shown:

Click here to view code image

public RecognitionUIViewModel() : base("Custom UI")
{
    promptCommand
        = new DelegateCommand(obj => Prompt(), o => !recognitionInProgress);
    PropertyChanged += delegate { UpdateCommands(); };
}

Replacing the GUI provided by the SpeechRecognizerUI can enable your app to streamline the recognition process, allowing you to customize the speech recognition experience to suit the purposes of your app. This also means abandoning the familiarity of the built-in speech recognition screens, and therefore should be done only if it makes sense for your app.