Benefitting by Seizing Opportunities – The “Usability TestEssen”
Since the last quarter of 2016, we have constantly devoted valuable time to research and development related to Conversational Interfaces. Conversational Interfaces are about natural user interaction, about context and situation awareness, and, of course, about natural language as an important factor in the interaction between humans and computational devices. However, in itself, the Conversational Interface paradigm is not particularly new. In any event, the fact that platforms, such as Amazon Alexa and Google Assistant are now reaching great numbers of people, it is evident that the game is changing.
by Steffen Blümm
At the moment, our focus in this field is on conceiving and developing a set of approaches and methods which lead to effective workflows for designing and implementing Conversational Interfaces. Therefore, UX design methods and usability testing are central within our selection of tools. We have conceived our own prototyping tool and prototyping tool-chain to allow us to test usage scenarios quickly.
Speed-Testing and Pizza
The Usability TestEssen (https://usability-testessen.org/nuernberg/) itself, is a community event where people come together to test software products, eat pizza, and discuss the testers’ experiences. Thus, one could consider it a special case of Discount Usability Testing. It is a great opportunity for us to test our scenarios for Conversational Interfaces. Of course, those usability tests are not at all about laboratory conditions, or about a hand-picked set of testers. However, at those events, people who enjoy using and testing software come together. One part is eager to perform tests, the other, to get insights and collect findings. The testers have no financial interest, instead, they are spending their precious leisure time experiencing and testing new concepts, designs, and services.
The procedure is relatively straight forward – usability speed dating 😉 Each product is tested by six people for 12 minutes each – i.e. 12 minutes for consent, tests, and questionnaire. Hence, it should be clear at this point, that one will not test a whole array of aspects. Instead, it is best to concentrate on a particular one.
Needless to say, it is no substitute for usability tests in the lab. Nevertheless, that casual format also offers a different set of opportunities.
Seize the Opportunities
Just because this kind of usability testing event is far more affordable than controlled laboratory tests, this does not mean we should dispense with serious preparation to get the most out of those tests. We consider them a great opportunity to leave standard paths and to experiment. As stated previously, Conversational Interfaces are about usability within the context of the user and about the situations the user encounters in life. Hence, we tried test setups which placed the user not directly in front of the device, but rather, on the couch. Thereby, the user spoke to a device placed somewhere in the room and had to retrieve a mobile phone for specific aspects of the scenario.
At the most recent event, we brought an A/B-test for a multi-character voice assistant with us. How does the user feel when having to choose a character? What is the feeling when a character assigned to the user? It was informative and interesting to observe our testers.
For more about this, see https://medium.com/@cui.lab.