1. What did AI catch that you missed?
There were some details I missed in the transcript, which had resulted in me generalizing a bit. I had evaluated Participant A’s answer as “needing to chase people down,” but AI managed to catch the detail that the reason she needed to chase people down was “outdated documentation,” since she works at a legacy company. Overall, AI was decent at pulling out specific quotes and helping me catch small oversights.
2. What did it flatten into something generic?
It highlighted patterns, but oftentimes, only 2 out of 3 of my participants mentioned anything about the pattern. Therefore, it sometimes overstated the pattern itself unless I checked closely.
3. Did it assert anything that isn’t in the transcripts?
Fortunately, it did not assert anything that wasn’t in the transcripts. I just had to be careful to keep track of who said what, so that I wasn’t overstating the pattern.
4. What changed between Milestone 1 and Milestone 2?
I found Teresa Torres’ advice and video examples around asking story-based answers to be super helpful. Funnily enough, I found that in my first interview, I had approached it more like a coffee chat than a user interview. Once I reframed the interview to picking out stories, as opposed to summarized generalizations, I found that the stories themselves became more specific. Rather than questioning a participant about generalizations, I shifted my focus to asking about specific moments, like “When was the last time…” or “Can you tell me about a recent time…” I feel like I did the best job of asking open-ended questions in my final interview, which was also where I had managed to pull out the most insights.
