Getting the experience right is one of the most difficult things when building chatbots and voice assistants. Your backend may be very powerful, but if the experience is not engaging and compelling, users will just leave. The amazing tech behind it will be useless if the users are actually confused by the interaction. With Botsociety, is now very easy to perform user testing on your design before actually building the chatbot.
Do not feel like reading? Check out the video tutorial! Otherwise, keep reading
1. Design your chatbot or voice experience
The first step is to design the experience you would like your users to have. For the purpose of this tutorial, I’ve come up with a classic Acme Chatbot. It is an e-commerce chatbot that is capable of selling equipment to Wile the Coyote. You get the idea 🙂
I’ve started by designing the happy path and added the other scenarios (If you want to learn how to design a conversational experience, check out this blog post first). This is what my design looks like:
While designing the experience, I took my time to craft not only the main flow but also the alternative paths — for example, when a user selects “Add to my wishlist” instead of “Buy”:
Now that I designed multiple scenarios, I’m ready to start the testing session
2. Create a user testing session
In order to create a testing session, simply click on the blue button on the top right corner that says “Test:
I can now set up a minimum tests goal. Optionally, I can also select an end date for this session
You can now choose if you want to involve your own set of testers, or you would like to use the Botsociety testers. I’ll go on and select the second one:
Botsociety testers are high-quality, confidential users that will try out your design and give you a more precise idea about how your future users will interact with your chatbot.
Now I confirm the purchase and just lay back for around 20 minutes. Results will show up automatically, and I will receive an email when that’s done. I can check partial results by clicking on the “Test” button.
Once the tests are completed, I will be able to check how people have used my design. Turns out, 75% of the testers selected “I need 10 [gadgets]” as the first option
After that, 38% of the testers chose the “Purchase” option under “Tennis balls”
Only 25% of the people chose “100 [gadgets]” as the first option
After such choice, all the users decided to continue with 10 (the Acme chatbot will not support 100 gadgets when launched). Nobody decided to get notified, which is surprising. I guess the feature is not perceived as useful as I thought. Maybe it’s not that clear.
I can also check out the results in a flowchart format:
Very interesting results. I now have lots of ideas about how I can improve my design.
3. Iterate you design and test the new one
Ok so now I will revise my design — the notification feature needs to be explained better; we should propose a different choice as first (10 or 100 gadgets) as people do not seem to care a lot.
So by clicking on the “Back” arrow, I will be back to design mode. I can now change the design and run another user test. I will see something like this:
I will be able to check the results of the second test with the new design. I will also be able to check my old user testing session (which will show the old design) and make a comparison. Hopefully, the users will respond better to the features I care.
Once the design is finished, I will be ready to develop it!
Hope you find this tutorial useful. Please let me know if you have any questions in the comments. Try out Botsociety here
Also published on Medium.