Arctic Fox – Usability Testing: Designing a Solution

Designing a Solution

Story Maps

Our project aims to help high schoolers who have difficulty focusing while studying. Our story map (see above) illustrates the daily routine of a high schooler who wishes to study more effectively but faces distractions from technological devices and social media. We assemble the timeline with stickies at the top of the diagram, and identify the following critical moments along with their MVPs and ideal outcomes.

  1. Feeling overwhelmed by her daunting workload, the student delays starting her homework by scrolling social media.
  • MVP: A function to schedule work sessions, and to clock in / clock out.
  • Ideal outcome: The barrier to starting work is lowered.

 

  1. The student gets stuck on a homework problem, texts her friends for help and ends up being distracted by her phone notifications. These distractions delay her work.
  • MVP: An awareness chime that goes off regularly. The student logs whether she is focused, and what distracted her (if she isn’t focused).
  • Ideal outcome: The student is reminded that she is distracted, thinks about why, and re-focuses.

 

  1. At the end of the day, having failed to accomplish all the tasks she planned to complete, the student is exhausted and discouraged.
  • MVP: A daily statistics page, including the level of focus throughout the day, sources of distractions, etc.
  • Ideal outcome: The student reflects on when she is distracted and what commonly distracts her, and works to improve her focus and eliminate distractions.

 

MVP Features 

As derived from the story map above, our MVP features are as follows.

  1. A function to schedule work sessions, and to clock in / clock out.
  2. An awareness chime that goes off regularly. The student logs whether she is focused, and what distracted her (if she isn’t focused).
  3. A daily statistics page, including the level of focus throughout the day, sources of distractions, etc.

 

System Paths

In our system path diagram (see above), we identify 2 types of users. The first is a high school student who is often distracted by technological devices. The second is a high school student who often procrastinates when overwhelmed with work and relies on time pressure to get work done. Both types of users follow the same workflow. 

Users start at the home page, beginning their work session. At regular intervals, the awareness chime will sound, prompting the user to indicate whether he is focused (and what is distracting him if he isn’t); this continues until the work session ends, upon which the user is shown the statistics page and exits the workflow.

 

Bubble Map

Our bubble map diagram (see above) illustrates the various components of our solution, with larger bubbles indicating greater importance. The largest bubble is the awareness chime sounding, which is our service’s most central feature. Components that are more closely related are placed closer together on the diagram.

 

Assumption Map

In light of our solution design, we compiled a list of assumptions related to our product and mapped them on a 2×2 diagram (see above), with the amount of evidence on the x-axis and the relative importance of the assumptions on the y-axis. The assumptions that are the most important and lacking in evidence will be the most crucial to our solution, and are given below.

  1. Students will hear the chime.
  2. Students will not find the chime excessively distracting.
  3. The chime will nudge students to re-focus on their studies if they are distracted.
  4. Students will accurately log what they believe distracted them if they are distracted.

 

Assumption Testing

Since we are basing our product around the key assumptions above, it is important to test that these assumptions are in fact true. As such, we came up with a test for each of our top four assumptions.

Assumption Test 1

We believe that… Students will hear the chime
To verify that, we will… Play chimes of varying  audible volumes to a test participant, in 2 different environments: quite & noisy
And measure… Whether or not the participant hears the chime at each volume and in each environment
We are right if… The participant hears every chime

Assumption Test 2

We believe that… Students will not find the chime excessively distracting
To verify that, we will… Play a chime for a test participant doing homework or work, and have them simply turn the chime off
And measure… The time it takes for them to return to what they were doing before (i.e an assignment) after the chime
We are right if… The participant returns to work quickly (i.e < 10 seconds)

Assumption Test 3

We believe that… The chime will nudge students to re-focus on their studies if they are distracted
To verify that, we will… Have a test participant use their favorite entertaining app, such as a mobile game or social media, and instruct them to start working on their assignment when they hear the chime
And measure… The time it takes for them to work on their assignment  after the chime
We are right if… The participant starts work quickly, rather than taking excessive time to exit their app and start  (i.e < 30 seconds)

Assumption Test 4

We believe that… Students will accurately log what they believe distracted them if they are distracted
To verify that, we will… Have a test participant note every 30 minutes while doing homework if they were distracted and if so, what they were distracted by. Meanwhile have a researcher (someone in our group) observe the student every 2-3 minutes and note what distractions the student has.
And measure… The difference between what the participant logged as distractions and what the researcher logged. 
We are right if… The majority of distractions logged by the researcher were also logged by the participant themself, meaning they accurately and honestly logged their distractions

 

Intervention Study

Our intervention study allowed us to gather insights from 5 individuals over 5 days. Over the course of the study, we asked participants to set consistent timers – either spanning 30-minute or 1-hour intervals – prior to starting a work session. We also asked them to place a sticky note next to their laptop or workstation. Each time the timer (with a sound/alarm) went off, we asked that the participant jotted down on the sticky note whether they were focused on their task or distracted in just a word or so.

Overall, the intervention study re-affirmed our assumption that regular alarms can serve as a positive guiding force to improve focus levels among students during work sessions. Over the course of the study, we saw improvements through students indicating that they were focused on the task more frequently as the alarm went off, both within a work session and between work sessions and days. This is likely due to our other assumption, that reflecting upon focus and distraction levels at the end of the day can serve as an introspective motivator for students, making them cognizant and helping them improve their habits as a result. The distractions that were indicated included social media, YouTube, texting, external characters (such as family members or friends who were serving as an in-person distraction), jumping between tasks inefficiently, and bathroom / snack breaks, with a heavy skew towards social media and text. 

Nonetheless, there are key considerations behind these insights that inform our solution design. First and foremost, our solution operates on an honor code system – we do not know if the students were honest about whether they were focused or not at each alarm as we did not monitor their work sessions. Thus, the progress observed may not be entirely accurate. Further, we do not know how long it took for students to get back on task after a focus alarm, which is a core consideration in evaluating the effectiveness of our product. 

Another insight is that those on half-hour intervals indicated more frequent focus levels than those on 1-hour intervals. It is important to note that all participants reported becoming consciously and later subconsciously aware of the alarm. They would anticipate the time it would go off and would return to work for the sake of reporting a focused state on their sticky note. This makes us wonder how we should quantify a successful solution. 

In theory, we planned that charting the rate at which a student is focused and distracted at each alarm should help create reflection about their work session. However, if students are re-focusing before the alarm as they become more sensitive to it’s expected times, then our solution is effective in the sense of bringing student attention back to work, but ineffective and a misrepresentation of sorts if our chart of a work session implies that the student was focused throughout the work session. This is a question we should consider deeply in the solution design. We are considering whether we should also, upon the alarm, ask whether students were focused during the entirety of the duration since the previous alarm. We also considered the implications of operating on an honor code, and decided this holds most true to the values of our design and using extrinsic interventions to foster intrinsic motivation.

Avatar

About the author

Leave a Reply