Intervention Study & Synthesis – Phone Scrolling

Intervention Study: Overview and Starting Out

In our intervention study, we focused on scrolling as the behavior we wanted to address. For many of our participants, scrolling was an inadvertent behavior that caused them to lose track of time and distract them from work or other preferred activities. The goal of this intervention was to interrupt user feeds with spam containing relevant content, which could both encourage people to stop scrolling and do something else with their time. If we were to fully develop this idea, it would look like a bot that users can personalize to post relevant and eye-catching content. However, for the purposes of our study, we acted as the bot by creating an Instagram account called “@stopscrollingstanford” and posted images to encourage followers to get off of Instagram. 

Over the course of 5 days we posted 87 times, and after receiving feedback from day 1, we shared our posts to our Instagram story multiple times each day to interrupt both the post feed and the story feed. The image below shows the posts that were made during the test period.

The posts typically consisted of a meme or gif with accompanying text. We aimed to include posts that provided suggestions for alternative activities as well as standard “stop scrolling messages”. You can see an example of both types of posts below.

We started the study with a short interview, providing instructions and by asking participants to do three things:

1. Follow @stopscrollingstanford
2. Share a screenshot of their activity stats on Instagram from the previous week
3. Create a shortcut to a Google form with our survey.

During our study, we asked our participants to like all posts they saw from our account and fill in a short survey twice a day (around lunch, and just before going to bed) to reflect on the previous time period. We reminded participants to fill in the survey via a text message sent at lunch and near bedtime each day. The survey is outlined below:

1. Name
2. Since the last time you filled out this form, how many times did you go on IG?
3. What were you doing before you went on IG? (list as many situations as you remember)
4. From the last few times you started scrolling, what prompted you to start?
5. Similarly, what made you stop scrolling?
6. For about how many minutes total did you scroll since the last time you filled out this form?
7. How do you feel about the amount of time you spent on IG (since the last time you filled in this form)?
8. How frequently did you see our posts? (From a scale of 0 – 5)
9. If you saw our post, did it impact your decision to stop scrolling?
10. How so? (optional)
11. How did you feel afterwards?

After the study we requested feedback about the experience (phone call or text depending on participants preference) and a screenshot of the previous week’s Instagram usage.

 

Our Study Participants

To simplify the intervention we only tested with people who regularly scroll Instagram and wished to reduce their scrolling habit. This requirement disqualified some of the original participants from the baseline study. We also had two participants who no longer had the time for the study.  This led us to recruiting four new participants to replace those from the baseline study. 

The demographics of participants range in age from 21 – 27, with the median and mode at 26. We measured 4 women and 2 men, with the common thread being Stanford association. We had 3 Design graduate students, Sandesh (26 M), Kevin (26 M) and Camille (27F). In addition, we had 2 working professionals in the health sector, Catinca (26 F) and Anika (22F). Lastly, we studied an undergraduate student on the east coast, Zelda (22F).  In terms of our personas, we had a few who were Social Sallys, (Sandesh, Anika)  one Curious Carl (Camille), and the remaining were Dobby personas (Zelda and Kevin).

Our baseline study asked participants to record what they were doing before and after scrolling on social media, and found that the most often reported activities were the following (this is also visually reflected in our connection circle diagram below): 

1. Eating
2. Using the bathroom
3. Working or attending class
4. Lying in bed (either just after waking up or just before sleeping)
5. Talking with friends (scrolling was reduced, but lulls in conversation increased the frequency of scrolling)
6. Being bored
7. Procrastinating
8. Hearing news that they wished to see online reactions to
9. Receiving notifications
10. Most often, idling for short periods of time between activities.

A primary insight that came from this study was that the most common instigator of scrolling was brief periods of downtime, which then had the potential to become long scrolling sessions if the participant was not mindful. Our idea for an intervention, then, was to flood each participants’ feed with suggestions of other ways in which to spend their downtime. By placing suggestions directly within their feed, they would be certain to encounter our posts whenever they began to scroll; and by focusing on other ways to fill time, we could address the root cause of the scrolling behavior. 

By flooding their feed (as opposed to offering only a suggestion or two now and then), we also aimed to reverse the polarity of the scrolling-boredom relationship; if a participant’s feed is saturated with our posts, they may quickly become bored of looking at it. Our hope was that in this way, scrolling could increase boredom, and boredom could instead instigate other activities besides scrolling.

 

Learnings From Our Intervention Study

The results from our 5-day intervention study were unexpected. As we synthesized our findings, we noticed that out of the 27 responses we received that answered the question, “If you saw our Instagram post, did it impact your decision to stop scrolling?” 13 responses said that it did not help them in stopping their scrolling, while 8 responses were a clear “yes” and the remaining only reported a partial effect. Looking into those responses more, we noticed that the “yes” responses only consistently came from two of our participants, and so we wanted to direct attention to the responses of those who found our intervention study to be ineffective. 

From our analysis and synthesis of these responses, we created the following fishbone diagram to illustrate the causes of why the posts in our study turned out to be ineffective:

We noticed that there were four main categories the causes fell into: timing, ability, content, and being a participant in the study. The biggest learning from looking at our results and creating this model is that our posts eventually blended into our participants’ Instagram feeds. This caused participants to either skip over them without realizing, or eventually become immune to seeing them. In some instances, our posts turned out to be counterproductive, as some participants enjoyed seeing the posts and were not motivated to leave the app. 

Another learning was that even if a user did see our post and wanted to heed its suggestion, they were generally not in the right time or place to actually do the activity. For instance, when they were only taking a quick 5-10 minute break from homework, it wouldn’t make sense for them to go out on a walk. We also heard from some participants that constantly seeing “stop scrolling” in their feed made them feel guilty, yet nothing was done to change their behavior.

Despite the intervention’s many flaws, we identified a few elements that seem to have been successful. We learned that it was possible to consistently be seen on some users’ Instagram feeds, as we were initially concerned about how Instagram’s algorithm would affect the visibility of our posts. Aside from one participant, nearly all participants marked that they saw our posts. Across all participants, the average for the question “How frequently did you see our posts? (From a scale of 0 – 5)”  was 3.3. We believe the visibility of our posts was due to participants “liking” them on instagram. One participant didn’t start seeing our posts until they went to our profile and liked all of our previous posts. 

Participants did stop scrolling for a variety of reasons, including seeing our posts. We broke these down into boredom, no new/good content, reminded of study, and having something else they needed to do. Our intervention was most successful in situations where the post reminded participants of their intention to scroll less.  It wasn’t clear if the constant stream of posts (approx. 1 every 45-60 minutes) impacted their perception of their feed as either interesting or boring. 

To further evaluate the effectiveness of the intervention, we compared the participants’ Instagram activities from the week before the study to that of the 5-day study after it concluded. As can be seen in the graphic below, all 4 participants who saw our posts and had baseline data reduced or maintained average Instagram time*. Unfortunately, many things can impact Instagram usage so we can’t draw clear conclusions from this data. With that said, it provides a baseline for this intervention’s potential to have a small impact on scrolling behavior. 

Plans Moving Forward

Moving forward, we would like to focus on Instagram stories (24 hour posts) in addition to standard posts since some participants mentioned that the majority of their scrolling occurred within Instagram stories. We would also like to offer suggestions that are relevant and in sync with their personal preferences. For example, if someone is trying to read more, the “bot” post would suggest reading. By connecting the participants with goals they actually want and can act on instead of scrolling, we can increase the relevancy of our posts to the users. Lastly, we would reduce the “stop scrolling” posts to limit the feeling of guilt in participants. This stemmed from our findings where one individual said that the posts made them feel negatively towards the bot, which didn’t help them reduce scrolling behavior. Moving forward, we can focus our product design on the most successful aspect of our study, which was when posts reminded participants of their intention of scrolling less.