Needlefish – Usability Testing: Designing a Solution

Story Map

We started by coming up with goals and motivations for our users which are stepping out of your comfort zone, forming connections, not feeling lonely, having company, “no fomo”, and reducing social anxiety. From here, we figured out tasks for our users that might help achieve these goals, such as initiating conversations, spending time in communal areas, going to social events, and saying yes to plans more often. We came up with a general daily plan for Stanford students living in dorms which includes getting ready in the communal bathrooms in the morning, going to class, eating lunch in the dorm, going to study or more class, eating dinner, and working later that night or exercising. From here we diverged this general story line into a sunny vs. rainy day scenario. 

Sunny: Essentially meeting lots of new people and making connections, plans, and friends whenever possible while going to lots of dorm events.

Rainy: Goes to dorm events alone, feels awkward/unwelcome, lacking community in their dorm, this user stops going to social events, they see their old friends hanging out with other people. They also have a very busy schedule, their friends are busy too, and they go days without seeing people they know.

These needs led to some actionable things these users could do to increase social interaction. 

• Find activity based events
• Come up with conversation topics beforehand
• Hang out with people while getting work done / find a study buddy
• Find someone to eat with
• Find friends to go to events with and ask them to bring their friends too
• Make more plans with people in general

 

Some features we thought about to make these more accessible through an app could be:

• In app journaling with prompts to help you reflect more easily about your social interactions that day
• An accountability tracker that helps you check off what you’ve done each day and what you still have to do
• An activity idea prompts list to help you figure out what to do with friends in the area: food, hike, bowling, etc.
• Conversation prompts to help get you out of a scenario where you don’t have anything to talk about with someone you just met (or a friend)
• Letting friends know where you’re going so they can join you

MVP Features 

1. Activity idea prompts 
2. Conversation prompts 
3. See events near you
4. Journaling 
5. Accountability Tracker

System Paths

To draw out the system paths for two personas, we imagined how each persona would be best helped to solve their problems. Each persona opens up the APP for different reasons and goals of usage, illustrating the point that the APP could help both log emotional status about social life and get nudges about making a change to the user’s social life. Comparing the two system path graphs below, we can see that the first system path focuses on goal-setting and progress-tracking while the second system path focuses on providing prompts. It emphasizes that the first persona needs more help with longer-term goal setting while the second persona mainly needs the APP to reduce the friction of talking to people and getting to know them. These paths helped us delineate the main functionalities of the APP and how they are connected as one single flow. We noticed in the system maps, that users exit and enter the APP frequently because these first few interactions tend to be short and ideally we want to collect interaction data as soon as possible.

 

Bubble Map

Once we figured out our System Maps and the list of features, we decided that our app would have 5 main tabs (Profile, Journal, Prompts, Activity Ideas, and Goal Progress). We wanted the user to be able to take some actions within each of these tabs:

1. Profile
   1. Personal Interests – a questionnaire to help us get to know the user better so our AI can start providing them with custom ideas (optional)
   2. Settings – a place to change account details, profile picture etc.
2. Journal – This would be a place where the user can journal about their feelings and their interactions, we give them some journal prompts to help guide them in case they feel stuck or are new to journaling. They can also choose the free response option if they believe that is more suitable for them. We will also use these responses (if the user gives us permission because privacy) to get to know the user better so our AI can start providing them with custom ideas
3. Prompts – This is a tab that the user would open if they are looking for conversation ideas. We provide general conversation ideas that would be updated daily to be made relevant to events/occasions (eg. Super bowl, Valentines day etc.) as well as personalized prompts generated by our AI especially for each user.
4. Activity Ideas 
   1. Nearby activities – If the user shares their location, they can see events happening near them 
   2. Recommendations – These include ideas for activities you can do with someone. These are again general and personalized depending on what the user prefers.
5. Goal Progress – This is a way for us to help keep our users on track 
   1. Set new goals – The user can set new goals related to making/maintaining friendships here. They can divide their goal into subgoals to make it more manageable 
   2. Check off sub goals – They can check off their sub goals once they achieve them 
   3. Check progress – A way for them to check their progress towards their goal

 

While doing this activity, we were able to really put ourselves in the shoes of the user and think critically about how we can help them achieve their overall goal of making/maintaining friendships through our app. It allowed us to break down bigger features that we had come up with and figure out what would go into each of them and make solid progress towards our final project.

 

Assumption Map

After creating our assumption map, we identified five key groups of assumptions:

 

1. Intrinsic Motivation for Relationship Building
   1. Users are intrinsically motivated to make and maintain relationships. They want more social interaction.
2. Communication Challenges and Assistance:
   1. Users face challenges in remembering conversations and generating conversation topics. This highlights the need for features that assist users in overcoming these challenges, such as conversation prompts and tools for logging and reflecting on interactions.
3. User Engagement and Motivation Factors:
   1. The effectiveness of notifications in prompting users to open the app and the motivation provided by progress indicators are key factors influencing user engagement. Designing a user-friendly and motivating experience, possibly incorporating gamification elements, will be important for long-term app usage.
4. User Privacy
   1. We assume that users will be willing to provide personal information, such as their hobbies/interests and location, for us to create personalized experiences for them.
5. Real-life Interactions and Event Participation:
   1. Users express a willingness to attend events, and we assume that users have time in their schedule for conversations + real-life interactions. 

 

These key groups are reflected in the following highlighted points: 

• Wants to journal/reflect on social interaction
• Will be motivated by a progress indicator
• Can’t come up with activities to do with friends/topics for conversation

These are key assumptions we have made that are all situated within the ‘important’ + ‘no evidence’ quadrant of the chart.

 

Assumption Tests

• Assumption 1 – Are people motivated by a progress indicator?

1. 1. Show people 2 different paper prototypes – one has a progress bar and one does not. Let people choose if they would continue to complete tasks or not in either cases and then follow up with questions about which one was more effective. 

Having a control group would allow us to see how having vs not having a progress bar affects how people respond and act

• Assumption 2 – Can’t come up with topics for conversation

1. 1. Ask people to think back to their last interaction with a friend/someone new and ask them what they spoke about. Did they think of ideas beforehand? Or just let the conversation flow? Why did the conversation end? Was it because they were headed out, just finished talking, or because they ran out of things to say even though they wanted to keep going?

This will help us figure out and understand how people approach conversations and social interactions and what goes on in their mind during this time.This will give us deeper insight into whether or not our assumption they people find it tough to come up with conversation topics is correct or not.

• Assumption 3 – People find it helpful to journal / reflect on social interaction

1. 1. Ask people to journal/reflect (write or type) on their social interactions today or yesterday for at least 5 minutes. After they’re done, ask them if
      1. It was helpful?
      2. It led to any new insights or perspectives?
      3. They would do it again? 
      4. They could see this becoming a habit?

Based on these answers, we can likely determine whether or not journaling is an effective way for people to become more comfortable in social interactions. We can also ask them how likely they would be to continue daily journaling / social reflections and follow up with a longer study if needed.

Intervention Study Findings

Given 9 participants, we are able to collect 44 Google form responses over the 5-day study while one participant dropped in the middle due to COVID-19 infection. Thirty-nine out of 44 responses completed the challenge to start up a conversation with a friend or a stranger. Among responses that successfully used the prompt and completed the challenge, 82% of responses stated that the prompt helped their conversations. Among all initiated conversations, 30.8% of them led to a future meeting plan.

There are many insights from our intervention study:

1. 89.7% of conversations happened between people who already know each other. The common reason is that participants still find it difficult to talk with strangers and do not have many opportunities to meet new people.
2. Provided prompts are effective and easy to use. 82.1% of responses agree that having these prompts ready is helpful. The common comments are like the following examples — “It was easy — someone already told me what to think of so I just had to say it”, “It helped because I think sometimes it’s hard to start a conversation”, and “It was nice to have somewhere to start!”
3. People prefer questions that fit the context. The keyword ‘natural’ is mentioned in multiple text responses. Several examples are “it was natural to continue the convo by asking how their Valentine’s day was and what they did”, “asking about their classes seem to be the most natural choice” and “Prompts 1–3 felt very natural and so I used them.”
4. The initiated conversation usually led to a nice ending. 64.1% of the conversations felt great for the participants and 30.8% of the time still ended better than expected despite the awkwardness at the beginning.
5. We have also acquired some useful suggestions from participants about what kind of prompts that are not included in the study but would be helpful. 
   1. have some of the prompts also be applicable to talking with people from off-campus
   2. one thing that may be helpful is to have them be a bit personalized to the person as well.
   3. I found it easier to just strike up a convo about what we were doing at the moment than the given prompts

Given the insights above, we plan to make these changes to our design:

1. User profile tracking
   1. Track the comfortable topics for the user
   2. Track the people the user has been talking with
   3. Track user’s motivation and goals in terms of making new friends
2. A mixture of Experts (Personalised Prompts & General Prompts)
   1. Specify categorized contexts to generate more context-aware prompts
   2. General prompts are specially designed for first-time or second-time conversations
   3. Utilize the user profile information to allow prompts to have a follow-up functionality
3. Notification Timing 
   1. Morning around 10 am seems to be an active time to set goals
   2. Evening around 10 pm has also been proven to be an effective time to nudge people to log their activities