We focused on the following 2 assumptions for our assumption tests:
Assumption #2
For our first experiment, we focused on the assumption that users can quickly understand anchoring and successfully create a high-quality anchor statement during onboarding without friction. To run this, we created an onboarding flow prototype. In the onboarding flow, users first note how often they want to microlearn, then how long they want each session to be, identify an anchor, identify a learning topic, set up notifications, and then review their anchor statement.
We recruited participants without prior knowledge of anchoring but with a desire to micro-learn, so we pooled participants who participated in our baseline study and new participants. We made sure that participants weren’t previously a part of our intervention study, since they were already familiar with the idea of anchoring through the study. We recruited a total of 3 participants, 0 old participants and 3 new participants. New participants confirmed they had an interest in micro-learning, and fit our original study requirements (young adults).
For our design, we briefly explained to users what micro-learning is. As a reminder, micro-learning includes reading books, the news, articles/newsletters, listening to podcasts, and watching educational videos (short-form and long-form included). In particular, this includes any form of active, intentional learning (it cannot just pop up on your feed). After reminding users what micro-learning is, we told them that they downloaded our app to help them become more consistent with their micro-learning, and prompted them to begin the onboarding process to start using the app. After users finished the onboarding process, we asked them to rate how easy-to-use and understandable the onboarding flow was, with 1 being extremely unclear and difficult and 5 being extremely clear and easy.
The results of our experiment can be found in this spreadsheet. We measured the percentage of participants who completed onboarding, the average completion time for the onboarding form, which steps users dropped off at, and asked users if they understood their anchor statement.
Participants:
Synthesis:
We learned that users enjoyed our onboarding process, with an average completion time of 32 seconds, full completion of the process, an average understanding of 4.33/5 for anchor statements, and an average ease rating of 3.66/5. Due to a bug, some participants were confused why the notification setting couldn’t be customized according to their learning frequency preferences, which we realized we needed to fix after the tests were conducted. However, the assumption tests validated our assumption that users can quickly understand anchoring and set up notifications, goals, and an anchor statement through our onboarding process, and we feel fairly confident that the majority of users will find this process frictionless, and will be able to quickly get started on micro-learning!
Assumption #3
For our second experiment, we focused on the assumption that users see enough value in the app to connect external tools (calendar, Spotify, other learning platforms) to our platform. To run this, at the end of the onboarding process, we prompt users to connect at least one external app to our platform, such as Spotify.
We attached this experiment to the end of our first experiment, since users are supposed to have completed onboarding before reaching this step, so our recruited participants are the same as above. The results of the first experiment do not affect this experiment, since connecting an external app is a separate decision (and optional) from completing onboarding steps, so one does not influence the other. So, our participants were the same as our previous assumption test.
For our design, after completing the onboarding process (described earlier), we asked users to integrate a learning platform of their choice into the app, such as Spotify. We measured the how many users connected at least 1 learning platforms, how many connected at least 2, if any users dropped off, and if any users immediately started a micro-learning session. We also measured the length of time it took them to integrate a platform. The results of our experiment can be found in this spreadsheet.
Synthesis:
We found that users experienced a lot of friction integrating a learning platform into our app with our current design. One user commented that integrating platforms as links added a lot of friction, and she was “too lazy” to go find the links for her language learning materials. Another user noted that he was more spontaneous in terms of finding what he wanted to learned, so he didn’t have a link in mind when asked to integrate a learning platform. We also noted that adding a link takes a decently long time (53 seconds). These results suggest that users might see value in connecting external tools, but the current design fails to make that action easy enough to complete. In addition, not as many users as we thought needed an external learning platform integration, raising the question of whether we truly need this feature in the app (it might cause more friction than reducing friction).
Participants:
- Adi (didn’t complete)
- Yerim (completed)
- yerim integration 2 yerim integration (videos)
- Karina (dropped off)
- karina integration (video)
Learning Cards
