Hi, I’m Michael Reiss, one of the UX Designers in the strategy team here at Catch. I'm hugely passionate about research and the value it brings to UX and digital strategy work, and I particularly champion the customer through my work. In the below I talk about the very real possibility for bias to interfere with research and share my tips for remaining objective.
If you've opened this post you may already be familiar with audience research. And not only are you familiar, but you're convinced of its benefits. The likes of Amazon’s Jeff Bezos, Glossier’s Emily Weiss, and AirBnb’s Brian Chesky and Joe Bebbia have attributed their success to having an understanding of their users. Instead of continuing its praises, I think it’s more interesting to consider how even those of us who conduct audience research for a living are not as objective as we think.
According to the below study, we're actually pretty terrible at being objective, despite thinking we’re better at it than those around us.
Out of a sample of 600+ people, 85% believed they were less biased than the average person. This study led to social psychologist Emily Pronin coining the term "bias blind spot," which explains how not only does everyone have cognitive bias, but everyone also believes they are less biased than others.
Out of a sample of 600+ people, 85% believed they were less biased than the average person.
(Source: Princeton Publications)
And while it’s great that just reminding ourselves of this bias is a huge step in avoiding it’s potential influence, there are a few measures we can take to prevent falling into the bias trap.
The numbers don't lie?
When we begin the research process, it's common to begin by deciding who to interview and what to ask them. For example, we've decided to collect user input for a sports drink company. Great, right? While perhaps a bit exaggerated, the below may sound familiar, especially when time and budget is tight.
We know the product so well, we have lots of site data and market research, so we know what our audience thinks of us. Not only are we the users (because we happen to also drink sports drinks), but we can imagine what our different audiences think of us and our competition.
Unfortunately, we're likely too engulfed in the day to day, to get accurate data from speaking just within our immediate teams. And if we do decide to conduct user interviews, bias can influence who we choose to interview (eg failing to consider all relevant groups of people) and the types of questions asked (eg leading questions).
While we've all heard that "the numbers don't lie," suggesting quantitative data is always objective, what about the steps we take to find this data, or how it is collected? That initial email request for specific datasets, or the Google search and selection of the study that most affirmed your hypothesis? That darn bias sneaks in yet again!
So how do we avoid UX bias?
Looking as far and wide as possible: of course make sure to source for any and all existing information on audience. But don’t stop there. Ensuring that we’re engaging with people from multiple departments, at different levels, and asking who they believe their audiences to be (and who they aren't, and why? …) The process of collecting multiple perspectives and asking them to explain their reasoning will provide us with great insight.
Try to do the above activity in person, if possible during a workshop. Invite people from different departments with different levels of seniority. Separate people from their managers or those they work with on a daily basis to encourage open and honest input. Use ice-breakers to help people loosen up and feel more comfortable.
When you determine who your audience is, print out their persona and make sure everyone on the team can see it as a reminder of who you're working for!
Collect a list (or slack your team members) of the most surprising information you've learned during your audience research. The habit of light heartedly calling out your assumptions and reminding yourself of surprises leads to a more open-minded approach to setting up audience research.
Ensure the whole team stays aware of the real possibility of having to pivot the project based on future research findings. We all need to be aware and educated of this possibility: providing updates on the research process as it progresses definitely helps. Don’t be afraid to expose all team members to rough & ready “work in progress” brainstorms to give exposure to the work and provide space for far and wide input. It may take some initial effort, but you'll save yourself from future stress by having the flexibility to provide the best insights and recommendations.
The good news among all of this bias is that our users will quickly prove or disprove assumptions by telling us what they need, want, and expect! We just have to be comfortable with having assumptions disproven and allow time to adjust direction of research.
Bias can be hard to shake, but we're adept at avoiding it’s traps and giving accurate insight: if you’re thinking about audience research or wider UX strategy, we're here to help.
Related news and work
The new impact of UX metrics on SEO
Google has announced that new metrics called “Core Web Vitals,” will soon impact your site’s SEO ranking
We're proud to announce that we've made it into the top 100 of The Drum’s annual Independent Agencies Census.
We've earned our 76th spot through winning some fantastic clients (some we can talk about, some we can't), growing our UX and strategy offering while creating exciting platforms for some of the UK's best known brands.
Jonathan Smith, Founder & Managing Director at Catch, said; "This is our 3rd time in recent years in a UK Top 100 list, and we couldn't be happier at this recognition for a great team effort. Being in The Drum’s annual census makes it all the more special as we're listed against other independent, owner-managed, agencies which we respect and admire"
The annual Independent Agencies Census is the most comprehensive study of its kind on UK marcomms firms. The Drum crunch two years’ worth of financial figures from agencies across the country to compare their performance on a range of financial metrics including turnover, turnover per head and gross profit.
Cameron Clarke, Deputy Editor at The Drum, said; “The Census is the definitive barometer of agencies' financial performance and it was pleasing to see an increase in agencies participating in our rigorous research process this year".
Head on over to The Drum to see the full list.
Related news and work
Our new Strategy Director
We're excited to announce we've appointed Polly Lygoe as Strategy Director. Joining our Senior Management Team, Polly will head up the Strategy & Creative department
Developing our teams
We're committed to the development of our teams, and recently undertook a leadership and management training session