CHALLENGE: WHAT ARE AREAS OF OPPORTUNITY TO IMPROVE THE USER EXPERIENCE OF THE PRODUCT?
The company wanted to conduct a competitive analysis to direct competitor and other CX products. The intentions were to analyze each product, identify areas of success, and uncover development opportunity for the free trial product.
Client: Customer Experience Management Software Company
Team: design researcher (myself)
My Deliverables:
Select product analysis tool and plan test
Facilitate test
Synthesize insights and themes
Findings and recommendations report
Presentation of findings
Research Approach
The company created a free trial product whose success relies on ease of use. We conducted a competitive analysis to compare the ease of use against similar products that offer a free trial in the market. In order to identify areas of success and uncover development opportunity for the the free trial and main products, our goals were to synthesize results of three previous usability tests conducted with end users and a new competitive analysis exercise. It was important to benchmark the product and understand usability issues based on knowledge of past user experiences and behaviors.
Research Plan
Based on my experience with the product and the company, I knew the product’s major barriers to use were primarily usability concerns including, but not limited to, the nomenclature/language used, hierarchy of features, and feature clarity. In preparing for the study, I guided the discussion with the design manager to identify the ultimate use of the results and set appropriate goals for this analysis. We identified that a list of features (one of the initial goals) would only be a part of the study and that the most beneficial results would come from a heuristic or similar evaluation. While we discussed evaluating the products with users, we had a limited time and budget for the tests and decided to keep the project in house. Ultimately I recommended conducting a PURE assessment and involving other design team members to take a multidisciplinary approach to the feature/flow analysis. In the past, I observed a product bias within the design team and wanted to leverage this opportunity to bring them into the analysis. I knew we would be able to leverage their expert knowledge to discuss the successes and pain points from a design perspective and compare it to my experience working closely with users and understanding their needs.
Once we settled on the method, the manager and I set about choosing the products that we would evaluate. We ended up selecting products that were both direct competitors and others that were in a similar market. By looking at a range of products, we hoped to identify themes over the successful products for improving the usability of the designs.
Running the Study
Before beginning the testing, the evaluators had preconceived assumptions around the ease of use and better free-trial product options. In testing these assumptions through a common UX lense, we were surprised to find that the order and more successful products were not what we expected and we were able to analyze the features from a holistic point of view and have a fresh perspective. evaluators the best feeling of success were strongly correlated to the products that used guidance or suggestions to help move users through the process. For example, one product achieved this with their linear process, indicators where to edit/change content, and task completion notifications.
Throughout the tests, our team noticed two recurring sentiments: 1) feeling of success and 2) guaranteeing the credibility of the survey and results. For example, users have raised concerns during past usability tests that they don’t feel confident they are building the best practice survey nor are they sure when they have completed every step necessary before moving on to the next task. Products that were able to give evaluators the best feeling of success were strongly correlated to the products that used guidance or suggestions to help move users through the process. For example, one product achieved this with their linear process, indicators where to edit/change content, and task completion notifications.
Insights
ALERTS AND GUIDANCE GIVE USERS A SENSE OF SUCCESS. The products with simple guidance, good instruction copy, and alerts throughout the free-trial received lower scores and gave users a sense of success and confidence. When the promise of the product is for a specific outcome, a certified NPS score, users need to feel confident that they are meeting best practice standards and are using the program the way it was intended to be used. However, not all tutorials or guidance is helpful or conducive to success.
LIVE PREVIEWS, IN-LINE CHANGES, AND WORKING ON ONE SCREEN STREAMLINE THE EDITING PROCESS, AND GIVE USERS A GOOD SENSE OF FINAL LOOK/FEEL AND TASK COMPLETENESS. Throughout the PURE test and past usability tests, evaluators and users have repeatedly responded well to live previews and inline changes. Several products used live-previews during the initial phases making the process more streamlined and giving evaluators a quick sense of confidence that their survey has the desired look and feel. In the current product, users are asked to open and close several windows in Preview and Style to see the final look and feel. When users close either feature and are opening and closing additional windows, they end up feeling confused and overwhelmed and still aren’t 100% sure their survey looks the way they want it to.
FLOW OF THE COMPETITOR PRODUCTS DIFFERS WIDELY AND SOME WERE MORE SUCCESSFUL THAN OTHERS. Some of the competitor products followed a linear approach to their organization by walking users through individual tasks or steps and showing progress bars.
Final Results
The result of the evaluation was a lengthy report highlighting major observations and insights of the analysis as well as considerations for the product moving forward. In addition to writing the report and documenting the process, I also presented the results and opportunities to the product team (designers, project managers, engineers, and CTO).
Based on these reports and conversations, the product team has incorporated some of these insights into the new designs for the next launch and are using this to help shape strategy for their full-feature product. In the case of the program building phase, they have simplified some of the language and begun making tweaks in the hierarchy. As we begin to test these concepts, we hypothesize that these solutions should start to remove some of the confusion current users experience and will increase the speed and accuracy of this stage.