Qualitative user research enables researchers to gain a deep understanding of users’ feelings, motivations, and experiences. This is why companies nowadays not only put their trust in quantitative user research data but also look for qualitative data analysis to support their findings.

The “fold in the rug” experiment / Flattening the “fold in the rug”

A really tangible idea of usability testing and a good user experience is the paradigm of the “fold in the rug”: Let’s imagine a team of researchers learning about a “fold in the rug” that only a small number of passers-by stumble upon — and the team eliminates the obstacle after this discovery. In consequence, the whole experience is improved, not only for the minority that stumbled before but as well for the other people that could “cross the room without tripping over the fold in the rug. This improvement of user experience can be felt consciously as well as unconsciously. 

Hence, a major upgrade was introduced for all passers-by — thanks to a small insight from a few research data pieces. Providing evidence of how qualitative user testing is key for successful improvement, is the following use case which we conducted for Burda.

Status Quo: The User Experience of einfachbacken.de 

Together with Userlytics and with the use of our platform consider.ly, our team conducted a usability test for Burda’s website einfachbacken.de. The website is a curated collection of simple baking recipes, suitable for beginners and those who are already more experienced in the kitchen. Currently, einfachbacken.de receives about 10 million visits per month and takes on the role of an impactful Burda brand that will be developed further in the future.

The website’s design is bright and friendly. A light color scheme combined with appealing food photography invites users to look for recipes and find baking inspiration. The main menu displays a clear overview of what is provided: There are different categories for the recipes, a baking “news” section, and information about baking accessories. At the top of each recipe page, you will find a photo of the final result, e.g., a chocolate cake, followed by instructions on how to recreate the baked goods. At the bottom of the page, users can comment and rate the recipes using stars on a scale from 1 to 5.

Since Burda didn’t have much experience with qualitative user testing, this kind of study was quite new and unexplored ground. So far, the company had been concentrating heavily on quantitative research. Hence, Burda did not have a specific, designated data storage for UX-related insights. Since Burda wanted to be more prepared for future studies they chose to collaborate with us — using consider.ly to store their research findings.

Working with a strong companion: Userlytics

Our first step was reviewing einfachbacken.de to gain a broad understanding of how the page is structured.

Next, we searched for a companion in our journey to help Burda: A colleague willing to collaborate with us through recruiting participants for our usability study and its conduction subsequently. Luckily, we found Userlytics and its platform — this way, we were able to offer Burda full-service support: 

While Userlytics was responsible for both a fast and targeted adequate participant recruiting and the use of their qualitative user experience testing platform, Usertimes focused on the evaluation and documentation of the results — supported by the Userlytics platform as well as our tool consider.ly.

With their proprietary global participant panel, Userlytics makes it possible to find the right demographic. Thanks to its user panel of more than 750,000 users from all over the world, its advanced demographic filters, and customizable screeners, Userlytics was a perfect match. Because of its advanced UX testing platform and feature set, its integration into consider.ly’s work environment was simple. 

The Userlytics platform includes a series of very useful features to perform usability tests — such as branching logic, advanced metrics (for example NPS or SUS) to quickly locate key insights and specific capabilities for improvement of information architecture like Card Sorting or Tree Testing. Userlytics also allows you to create video clip compilations organized around key themes, as well as machine-based transcription in multiple languages. The Userlytics platform is also highly scalable, allowing the addition of more participants to a study via a simple click. 

After the testing sessions, we imported the resulting videos to consider.ly, tagged important passages and created a holistic picture out of them. This process enabled us to analyze the usability testing results. 

All in all, it only took one week for our team at consider.ly to finish screening, crafting the study and its documentation.

How usability testing results deliver answers

Based on our findings, Burda derived important insights on succeeding features of einfachbacken.de. Small changes were made immediately.

The product team at Burda was especially indecisive about certain “customer rating features” on einfachbacken.de. The qualitative user testing helped the team to better understand why customers want this kind of feature and what they expect in terms of look and feel or usefulness. 

An Example: “Real” 5-Star Recipes

On the existing site, every customer could rate the recipes 1 to 5 stars. The site then automatically aggregated the ratings and showed them along with the recipe. 

The problem: All ratings were rounded up during the aggregation. So, every recipe above 4.4 stars was displayed as a 5-star rating. This led to disbelief among the testing participants and shed a negative light on the whole site.

The research results helped Burda learn more about how they are perceived by their customers and give the product management a better understanding of their target audience. 

Next steps: Ongoing collaboration

In practice, continuing to work with results sometimes forms an obstacle for companies.

With consider.ly we address this challenge early on: We offer a convenient option of visualizing results. In our tool, the raw data is linked to the results. This way you can answer specific enquiries quickly without hesitation. Having results linked to their proof also supports a better understanding of the conclusions made. Additionally, this way it’s easy to involve a whole team in the process.

Thanks to our usability study, Burda has laid the foundation for the following work. The collected data is further exploitable and can support future evaluations. To do so, it is crucial to keep the gained knowledge actionable. A good way to do so is to save raw data, notes, and the procured insights in a repository, just like our tool consider.ly offers.

Burda is now in a position where every new study contributes to its ever-growing research repository. Thus, helping the organization better understand its users and share this knowledge inside their organization.

We say “Thank you, Burda” and a huge thank you as well to our partner Userlytics for the great outcome of our cooperation.