Experiment Synthesis

Read the full analysis and description here!

Experiment 1:

Test Card:

Hypothesis: We believe that users are more motivated to go outside for narrative and collection purposes rather than for purposes tied to socialization.

Test: To verify this, we conducted a test with four participants that would get a reminder to go outside at a particular location, verifying via photo. The participants were split evenly into two groups. One group would be given the task with the purpose of engaging or socializing with another person at the location. Another group of participants would be given the purpose of collecting a sticker reward wrapped in narrative language. We framed the task as optional and only nudged participants once to reduce the possibility of them performing the task only because they felt required to by the experiment.

Metric: We measured the engagement rate for the two groups based on successful completion of the task.

Criteria: We are right if the engagement rate is higher for the group with the purpose of collecting a sticker reward wrapped in narrative language compared to the group with the purpose of engaging or socializing with another person at the location.

Learning Card:

Observations: During the experiment, the group of participants given the collection-based reward went to the goal location with a higher success rate than those who had the social-based reward. Both participants in the collection-based group went to the goal location and received a sticker reward, while only one participant in the social-based group went to the goal location.

Learning and Insights: The collection-based participants expressed curiosity as a strong motivator for them, indicating that fun through discovery can be leveraged to encourage users to go outside. Additionally, the social-based reward can be a motivator or demotivator depending on the individual person.

Decisions and Actions: Based on the findings, our team will prioritize the gamified and collection-based features of our solution rather than leaning further into the social aspects. The approach will allow the team to leverage the curiosity and fun through discovery while balancing the unique interests of participants who liked or disliked social interaction.

Experiment 2:

Test Card:

Hypothesis: We believe that the distance from a user to a location will impact their engagement with the activity. Then further the distance, the less likely for them to engage.

Test: To verify this, we will ask two groups of three users each to visit a location. One group had a location further and out of the way, and the other group had a nearby location.

Metric: We will measure engagement by recording the number of successful completions of the activity by each group.

Criteria: We are right if the group with a closer location has a higher rate of successful completion and engagement with the activity than the group with a farther location.

Learning Card:

Observations: We found that participants who were closer to their check-in locations had a higher conversion rate of visiting their assigned location. Many of the test users mentioned how rain and distance played a factor in their decision.

Learning and Insights: We learned that proximity plays an important role in user engagement. When users have to travel further distances, it can act as a barrier to engagement. Furthermore the rain highlighted how weather can also play a role in engagement.

Decisions and Actions: Based on our findings, we concluded that having multiple locations instead of just one can increase the likelihood of users attending their check-in locations. This approach not only provides more opportunities for engagement but also increases the chances of users being in close proximity to their check-in locations, thereby reducing barriers to engagement. Overall, our experiment highlights the importance of considering location when designing engagement strategies.

Messages with Group 1 during Experiment 2

Choosing Recruits:

For experiment one, the test subjects did not have to be local to Stanford. We used a number of friends who were local to the area but not necessarily Stanford students. For experiemnt two we wanted to test with locations on campus so we could gauge how out of the way a location was relative to the users. We chose two pods of people, 3 in Govco and 3 in Narnia, and sent the groups to different locations.