I just finished reading Michelle Goodall Faulkner’s article in CRM Daily, “When Getting Human isn’t Enough," in which she pretty succinctly lays out what she terms the “comprehensive approach to the design and deployment of customer-facing applications in the contact center.” She brings up the downside of self-service applications that lead to caller frustration such as misrouted calls, hang-ups, and other factors such as those tediously long menus that we all suffer through when companies fail to apply best practices to IVR design. Which, I might add, after 20+ years of IVR deployments, there is just no excuse for, but I digress.
The gist of her article however is on testing, and here she lays out the three main components of testing a customer service application – usability testing, automated functional testing (AFT) and load testing. She also references the 10 “standards” for application and design testing detailed in gethuman standard v1.0 on Paul English’s Get Human web site.
I agree with Michelle. Testing is everything and it should be done before deployment, not after you use your customers as guinea pigs.
If you deploy with inadequate testing not only do make your customers madder than a hornet’s nest, they might not stay customers. Inadequate testing also means taking the system down to rework the application, potentially breaking something in the process, and then feeding it to those guinea pigs again, and so on. Bad idea. For example, let’s pretend you are calling into your insurance company, where you have home and life insurance, and you hit an IVR. Things are just fine until, after getting a question answered about your life insurance, you get it in your head to ask a question about your home. Somehow, you chose an option that wasn’t what you intended, so you say “help”, and the system plays a help message. But that didn’t do it, so you say help again, and the system plays another help message. If it’s a remotely good application the second help message will be different than the first, but no matter, it didn’t help, so you ask for help again. You never get out. You are stuck in a dreaded infinite loop! You are now madder than a really pissed off hornet.
Here is the part that Michelle doesn’t touch upon. Whereas testing is everything, I’ll posit that the issue of creating truly 100% usable applications, particularly in IVR, isn’t as easy as it seems, even with our current tools. We have years of best practices on usability and design, training classes, seminars, and books written by experts. We have AFT tools and load testing equipment and applications, but we also still have hang-ups, infinite loops, places where customers have to re-enter information, and lots of zero outs by frustrated callers. Why is that?
It’s because IVR scripts are complex! Even the simplest application has dozens of states with maybe two to five options at any point. Many have potentially thousands of paths that a call can take, making testing each avenue extremely hard if not impossible. Just thinking about trying to write a test plan for our imaginary insurance application makes my head hurt.
Furthermore, automated calls for testing cost time and money, and take the production system down, just at the point where people are eager to deploy. Testing can be a real time sink. Can you imagine the time and money it would take to test every avenue of a complex insurance application that has thousands of potential paths?
So what do most companies do? They test for the 20%. Yes, the Pareto Principal applies to IVR systems too. If 80% of the callers use the application in a similar way, why spend time testing the other 20%?
There is a problem with this. It means that 20% of the time the experience for the caller sucks! They hang up. They don’t call back. Or when they do they are angry. Comedians make fun of these applications. It means that unfortunately the industry has accepted mediocrity with a price.
I’d like to hear what companies are doing to address the 20% What is the impact of inadequate testing on the customer experience? What methods do we have for adequately testing voice applications? Where do you draw the line on testing? Is 90-10 good enough? Are we willing to give 20% of our callers a bad time? Any thoughts?