Overwhelmed by Insights
Feedback is great — but what if you have too much?
As I described in this post, I recently started a UX design internship and conducted a usability test for the product, a mobile app. There had been no formal user testing done until this point, so I really wanted to put some solid research practices in place.
The testing went fairly smoothly, aside from some technical hiccups and difficulties with remote mobile testing. We ended up with 8 participants, and they each performed 6 key tasks with the mobile app.
Because I wanted to gather as much feedback as possible in this round, we ended up with a lot of insights and potential changes. Honestly, it was a little overwhelming. In the academic settings where we conducted usability tests, the products and tasks themselves were small and contained enough that most of the insights were easily implemented. That was not the case here.
Because we tested most major features of the app (except for a few which are still in development), and asked for open ended feedback — our participants supplied a plethora of thoughts and insights.
We had to visualize and organize our data in some way beyond a spreadsheet. For this case, I chose a to plot suggested changes on a matrix, ranking each change on 2 axes: effort and impact.
The level of effort referred to both design and development effort, and impact was determined by how often a particular issue was brought up by test participants, and their general feeling towards that issue.
The effort vs impact matrix definitely helped organize our thoughts, but it was still fairly overwhelming to look at as a whole. From there, we decided to focus on the tasks that were rated the lowest in terms of “ease of completion” (we had asked participants how easy it was to complete each task, on a scale of 1 to 5, 5 being very easy). Essentially, we are assuming that redesigning elements of the “worst” tasks will lead to the most improvements in usability. This will also help keep us focused — we’re not thinking about different parts of the app all at once, only on specific parts one at a time.
For a small UX team of 2 with only 24 hours in a day, this segmented approach seems to be the most sensible plan of action. If anyone has any better ideas, I’d love to hear them! Drop them in the comments if you wish.
