Assumption Tests Report

Note: We tweaked our experiment designs slightly after submitting the Intervention Study document, since we realized that our previous weeklong study plans would limit the number of people that we would be able to recruit for the tests. Hence, we’ve summarized each experiment design in the first part of each section.

The response form that was used to collect data for the first two studies can be found here: https://docs.google.com/forms/d/e/1FAIpQLSe8uxlES9ou8ej8xIcc_Rm_DoMeQm–et-U2LIfaAAbs-xw4A/viewform.

Test Name: Mood & Response Tracker

In this test, we attempted to figure out the effect that responding to a text had on someone’s mood. We did this by asking participants to rate their moods before and after responding to a text from someone. We recruited participants from a diverse set of dorm lounges, which helped us make sure that the participants were from different friend groups. This was important because certain friend groups have certain standards for responsiveness, so we needed diversity.

Test Name: Mood & Response Tracker
Assigned to: Sumedha Kethini
Step 1: Hypothesis We believe that… Users will experience boosted happiness if they respond to texts that they’ve put off. If this is true, then it gives users motive to use our app.
Step 2: Test To verify that, we will… Ask users to rate their mood on a scale of 1 to 10. Then, ask them to respond to texts that they have been putting off. Finally, ask them again to rate their mood.
Step 3: Metric And measure…  Their rated moods before and after responding to the text.
Step 4: Criteria We are right if… All three of the participants experienced heightened moods after responding to friends. This would indicate that responding to texts improves moods.

Learning Card

Insight Name: Mood & Response Correlation  
Person Responsible: Sumedha Kethini
Step 1: Hypothesis We believed that… Users will experience boosted happiness if they respond to texts that they’ve put off. If this is true, then it gives users motive to use our app.
Step 2: Observation We observed that… 6 out of 7 respondents reported an improved mood by at least one point after responding to a text. The 7th participant had no change in mood.
Step 3: Learnings & Insights From that we learned that… Users do seem to experience boosted happiness if they respond to texts that they’ve put off. 
Step 4: Decisions & Actions Therefore, we will… Aim to build on this burst of dopamine that users will experience when responding to people on the app, potentially by adding things like celebration animations when users respond.

This screenshot from our response form shows the moods that participants had before and after responding to a text. On a general level, it’s clear that participants experienced heightened moods after responding to a text.

 

Test Name: Avatar Presence Test

In this test, we attempted to figure out whether people are more responsive on messaging platforms that use avatars. To test this, we mocked up two messaging screens, which were identical except for the fact that the first had avatar icons and the second did not. We then presented this to participants, who rated which screen they would be more likely to respond to messages on. We once again recruited participants from different dorm lounges, which helped us make sure that the participants had diverse visual preferences.

Test Name: Avatar Presence Test  
Assigned to: Sumedha Kethini  
Step 1: Hypothesis We believe that… When a user sees their friend’s avatar next to a message, this increases their inclination to respond.
Step 2: Test To verify that, we will… Show users two message screens which are identical, except one has an avatar next to the message and the other doesn’t.
Step 3: Metric And measure…  Their indicated likelihood to respond to the given message
Step 4: Criteria We are right if… All three of the participants indicated that they are more likely to respond to the message with an avatar.

Learning Card

Insight Name: Avatar Presence Effect  
Person Responsible: Sumedha Kethini
Step 1: Hypothesis We believed that… When a user sees their friend’s avatar next to a message, this increases their inclination to respond.
Step 2: Observation We observed that… When presented a messages screen with vs without avatar icons, 7 out of 7 participants reported that they would be more likely to respond on the screen with avatars.
Step 3: Learnings & Insights From that we learned that… When a user sees their friend’s avatar next to a message, this does increase their inclination to respond.
Step 4: Decisions & Actions Therefore, we will… Take advantage of the effectiveness that avatars have on responsiveness, by requiring all users to set avatars.

This screenshot from our response form shows that all 7 participants preferred Option 1, which is the screen with avatars.

Test Name: Survey of Comfort during Nudges

We tested with five students – all undergraduates. We conducted with random people at coffee shops and mutual connections, they were all based in Stanford. Experiments were conducted in person. They were all young adults aged 20-23 because they align with our target market (busy young adults who want to maintain their social relationships whilst also not compromising on their work). 4 of them had an easy time bumping texts for friends, one out of these bumped a text for a teammate and rated it a 5 on the scale of comfort mentioning that it was easy because it was for work and the deadline was tonight. One of the users mentioned that they did not have any unresponded texts from friends and did not feel comfortable bumping an acquaintance.

Test Name: Survey of Comfort during Nudges  
Assigned to: Buffalo Team Duration: 20 minutes per participant 
Step 1: Hypothesis We believe that… Users are willing to participate in nudging others and keep their friends accountable when prompted by a list of people who they are waiting for responses from.
Step 2: Test To verify that, we will… Ask users to find at least one text of theirs that their friend hasn’t yet responded to. We then ask the users to nudge this text by either sending a follow up text or by ‘emphasizing’ the former text. Finally, we ask them if this felt comfortable and natural to them.
Step 3: Metric And measure…  Whether the participant successfully sent a nudge, how comfortable they felt on a scale of 1-5 after nudging, and qualitative feedback on what made the nudge feel natural or awkward.  
Step 4: Criteria We are right if… A significant majority of participants (3 out of 5) report feeling comfortable (3 out of 5 report feeling 4 or 5 on a 1-5 scale) when nudging a friend, and describe the action as feeling natural and socially acceptable. 

 

Insight Name: Sender Comfort While Nudging   
Person Responsible: Buffalo
Step 1: Hypothesis We believed that… Users are willing to participate in nudging others and feel comfortable keeping friends accountable when shown a list of people who haven’t responded to them. 
Step 2: Observation We observed that… Most participants sent a follow-up nudge when shown a screen of contacts that hadn’t responded to them, while 2 of them hesitated saying it might be too much. 
Step 3: Learnings & Insights From that we learned that… Most users are willing to nudge close friends but feel social friction when nudging people they are less close with. The framing of the nudge also matters – if it’s someone close it can be like a more aggressive or demanding follow-up but if it’s like a colleague or acquaintance it can be like a gentle reminder. 
Step 4: Decisions & Actions Therefore, we will… Let users choose between the kind of nudge they want to send – a soft nudge for acquaintances and a more direct follow-up for closer friends. 

 

Avatar

About the author

Leave a Reply