Think about it – this is not a problem faced by touch-tone IVRs (formally known as DTMFs) or by web apps that rely on button clicks and navigation with a mouse. These applications are never ambiguous. A touch-tone ‘2’ cannot be mistaken for a ‘1’ or a ‘3’. User error aside, a mouse-click on a button to ‘proceed to checkout at an online shopping site cannot be misinterpreted as a search for another item. And if it’s the user who makes an error, there is always the ‘back’ button.

Speech is different. Human beings sometimes mishear what is said to them and despite all the improvements made every day to the accuracy of automated speech recognition, software can mishear, too. Sometimes.

What’s a poor application designer to do?

The Artificial Intelligence (AI) tools we have to work with

First, remember that the results of speech processing are usually accompanied by a confidence score which tells the app how certain the recognition engine is that it made a good match of the words spoken. If the score is high enough, the application can consider the results valid, and go ahead and process the user’s choice. If the confidence is too low, the app must come up with a way to try again — and figure out a way to do that without giving the user heartburn.

But there are at least two more possibilities. First, the results of recognition may fall into a grey zone: neither high confidence nor low confidence. Instead, the recognition engine may come back with a hypothesis that needs confirmation. That must be handled delicately, as well.

Finally, the user may not have replied at all, or done so in a manner that the app simply failed to hear.

So, putting aside a nice and clear high-confidence result, there are three scenarios to deal with: medium confidence, low confidence, and no results. For each one, the app finds itself in a recovery scenario and that means prompting the user again – which no one enjoys. But each one of these scenarios should be handled in subtly different ways. On top of devising a re-try prompt to suit the occasion, the application designer has also to consider how often to re-prompt before the user gets fed up.

Complicated, yes, but here are some simple things to keep in mind to guide the process.

Develop clear strategies from the start

First, avoid re-prompting as much as possible by making your initial prompt as clear as possible. For instance, if you are asking for a user’s telephone number, specify the number of digits the app expects to hear. Even though North America began standardizing 10-digit phone numbers over 20 years ago, it is still worthwhile to specify that. Consequently, a prompt like:

“Please say or enter your 10-digit telephone number” is preferable to  “Please enter your phone number.”

It’s a subtle difference, but it clarifies things and people like and need clarity. Do I include my area code? Yes. Do I include my extension? No. A simple thing to do, but helpful. Another example for users in the US: the US Postal Service’s zip codes come in 5-digit and 9-digit versions. If an app is to collect a zip code, tell the user right away which one to use, as in “Please tell me your 5-digit zip code.”

How often should you ask the user to re-prompt?

As a rule of thumb, it is better to re-prompt than to give up, so at least one attempt is typical. On the other hand, people get fed up if the app fails them repeatedly, so while a second re-prompt may be a good idea, more than that usually leads to frustration. Simply put, seldom less than one re-prompt, and very seldom more than two.

What to say during a re-prompt? Hopefully, the initial prompt has stated as clearly as possible what information the application is trying to get from the user. If they failed to provide it, they may not have understood, so the first re-prompt can add some instructions. For example, an initial prompt from an insurance company for a policy number might say “Please tell me your 9-digit policy number.” If that elicits only a confused grunt from a user that leads to a very low-confidence result, a reasonable first re-prompt with instructions might be “Sorry, I did not understand that. Please tell me your 9-digit policy number. It can be found on the upper right-hand corner of your contract.”

If that doesn’t work, add an example in the second re-try: “Sorry, but I still did not understand that. Please tell me your 9-digit policy number, like this: one-two-three, four-five-six, seven-eight-nine. Once again, it can be found on the upper right-hand corner of your contract.”

Notice that not only should re-try prompts add increasing levels of instruction and example, but they also should briefly apologize in a way that frames the fault onto the app, and not the user.

Other options might be to suggest to users that they move to a quieter area, away from background noise. Even though the user may already be in a quiet area, this telegraphs to the user that clarity is important, and it encourages them to speak clearly. Sometimes, it can also get them to turn down their television or ask people around them to be quiet.

Here’s an example on ways to handle a low-confidence scenario

What happens when a user does not respond at all? The strategy of increasing instructions with each retry still holds, but it is important to distinguish not hearing a response from failing to understand one. When a user replies with something the app does not understand, at least they did respond. When the app hears nothing and times out, it should acknowledge the difference. Instead of apologizing with “I didn’t understand”, the prompt should start with “I did not hear you,” or something similar. This tells the user that the app is actually listening and that some kind of input is needed.

Finally, confirmations. When a user’s response is recognized with a so-so level of confidence, the app can respond to the user with “I think you said…” followed by what it understood, and it can close the retry prompt with an explicit yes/no question like “Is that correct?” If the hypothesis was right, the user is encouraged to proceed in the knowledge that the app is not only smart enough to understand him or her, but also smart enough not to drive off in the wrong direction when it is not certain. When the hypothesis is wrong, again the user may be relieved that the app did not charge off into left field and is given a chance to correct the error by trying again.

The human being on the other end of the line

In a world of high-tech engineering, re-prompting strategies are guided not so much by complex math and fancy algorithms, but instead by an understanding of human nature. People will cooperate with a machine asking them questions if they feel understood, if they feel that the interactions are leading somewhere useful and that the app is handling them respectfully.

Failing that, they will likely press ‘0’ determined to get to an agent because no one likes to be toyed with, especially not by a machine. Worse still, they will hang up and call your competitor!

And finally, note that these are guidelines, not hard-and-fast rules, so designers need to think deeply about how to build a persona that reflects the enterprise (More on personas in the coming months) and apply these strategies with care and consistency, but also with the discretion to bend the rules when necessary. Exceptions will always come up, but applied thoughtfully, re-try prompting can substantially increase the user’s success with speech apps, and that means a better customer experience, less time spent with agents – and customers who value and stay loyal to your brand.