Andrea Knight Dolan on AI in research

An 18-year Google veteran,
Andrea Knight Dolan
is a leading expert in applying AI to UX research. She offers consulting, teaching, and mentorship services to help organizations harness the power of AI for better user experiences.
Here's what I learnt from her discussion with
Christine Perfetti
last week:
1/ AI enables scale - and raises expectations
"With AI moderators, it's not 10 interviews anymore. It's 30. It's not 30 interviews total. It's 30 per country. As the magnitude of data increases, it is where the opportunity and the necessity to rely on AI to process the data comes in."
2/ There are a range of products available (general and specialised), each with it's use case for Research. In a recent project, she used:
ChatGPT-4o for:
- Domain research and literature reviews
- Brainstorming and drafting interview scripts
- Translating responses (e.g., Spanish to English)
- Summarizing respondent answers
- Cross-checking responses across interviews
Voicepanel
for AI-moderated interviews in Spanish (a language she doesn't speak💡)
AddMaple
(Quant analysis tool) for Open-ended coding and thematic tagging. "AddMaple did in 3 minutes what would usually take me 45-50 mins of manual work. And those 3 minutes were reviewing it's work"
CoLoop
for segment analysis
Reveal
for hypothesis testing with PII stripping
3/ Synthetic (AI-generated) user data may be convincing, but they do NOT represent real user behaviour. Use it carefully.
AI has never truly lived - it hasn't slept, exercised, or eaten, and it can only regurgitate existing discussions, and not truly respond to new stimuli. Using an AI-generated user is like modeling an expert user who knows exactly what to do and don't make any mistakes.
4/ If you are a skeptic, a safe and effective way to start is to use AI for Desk Research
Use ChatGPT as your resource librarian, ask it to quickly find relevant resources and summarize key points. If you find any good reports, share it with GPT to chat with them
5/ Micromanage the tools until you trust them
Treat AI tools as research assistants and give them specific, targeted tasks instead of broad assignments. Maintain control and oversight, ensuring the quality and relevance of insights while leveraging AI's efficiency for tasks like data processing and analysis.
"You don't ask it to boil the ocean. You ask, 'This is what the stakeholder needs to know. Do we have evidence of this here?'"
6/ The most important skill for UX Researchers? Interviewing.
AI tools help scale up the extent and speed up the process, yet if you don't ask the right questions, you don't collect the right information. Even with an AI moderator, you have to program the interview, and you have to know if it's doing a good job. The classic principle still holds: Garbage in, garbage out.