[NILE CROCODILE] Usability Testing: Designing a Solution

Team Nile Crocodiles: Adrian Rivas, Anchal Sayar, Andrew Franks, Jennifer Fung, Maggie Skortcheva

Story Maps

This is our first attempt at creating a story map for our target audience, couples where at least one partner is currently learning a language. Our proposed solution is to develop a centralized digital platform that acts as a repository for a couple’s act of love: language learning to build a deeper connection with their partner. This attempt laid the groundwork for our more robust story map by allowing us to brainstorm how, when, and why notifications would be sent to each partner and a way to communicate feedback to the language learner. We had a difficult time generating many features the first time since half of our group was unable to participate in the activity.

Upon receiving feedback that we should try to focus on more “rainy day” scenarios, we decided to make a story map focused on where our solution may fall short and how different features may remedy any shortcomings. Here, we came up with some of our dark horse ideas like AI assistance and “memory jars” which may require additional development and consideration to be sustainable. Nevertheless, it would help us fulfill the role of giving preliminary feedback to a language learner before sending their love notes without fully developing a language-learning curriculum like Duolingo. An alternative that may be able to fill this gap would be voice transcription built into the app, but this may introduce other issues like discouraging text-based interaction with the language and requiring more computing resources.

After reviewing our attempts as a complete group, we condensed our previous two attempts, generated more features, and organized them in chronological order (from top-to-bottom and left-to-right) with our MVP features lying above the thick, magenta line. In this step, we realized that we needed to incorporate a way for couples to link their accounts together so that the love notes would go to the right person. We decided to ask users to make a personal account to save their love notes, drafts, and interaction history with their partner at the beginning of the process. To avoid confusion if two or more users share the same name, we believe that a unique share code would be the best way to establish connections between couples. To encourage couples to incorporate language learning in more than a purely academic setting, we think it is a good idea to allow both partners to share multimedia content to make their messages more creative, interactive, and personal.

MVP Features

Our product focuses on facilitating language learners to exchange notes and receive feedback from native speakers. We’ve designed it as a mobile app to accommodate users’ time constraints. To get started, both language learners and native speakers will download the app and log in using their email accounts. They’ll connect by sharing a code generated in the language learners’ profiles. On the app’s main page, users will find a button to create a new note. Clicking this button will open a window where language learners can compose their notes, including text and other media. They’ll have various template options to enhance their note’s appearance. Once completed, the note will be sent to the native speaker’s inbox. The native speaker can then read the note and provide feedback. They’ll also have a similar note composition window to write their reaction, including text, photos, videos, or audio clips, and send it back to the language learner’s inbox. This simple process ensures effective communication between language learners and native speakers.

System Paths

To design our system maps, we tried to put ourselves in the shoes of our personas and visualize the different paths our users can take. This exercise also allowed us to explore and quickly iterate over user path variations.

Our product’s main focus is Motivated Moe and his partner.  Moe is trying to learn his partner’s native language with his help. They will use our app to send each other love notes in their target language.

We explored ways in which we can gently insert a language-learning element in a love note app. We contemplated Moe’s partner correcting his love note and decided against it since this would go against the spirit of a love note. After some more discussions, we decided to use an AI agent that would correct Moe’s note. This way we can also provide additional support for beginner language learners who may struggle to express themselves through a limited vocabulary. Our system paths also allowed us to explore ways in which we can keep users engaged.

Bubble Map

In designing this bubble map, we started by thinking about the four main components of our platform, these include: home, create note, account, and inbox. If our solution were to be designed into an app or website, these would likely be the four clickable tabs on the navigation bar. The Home component of the app would include prompt(s), specifically the ability to view prompt options for a given day and external links relevant to the language that the user is learning. Next, the Account page, much like a profile page, would be where the user can view all their notifications to date, update any profile information, and view a gallery of the past love notes they have written/received. When the user begins to draft a note, they will engage with the Create Note component. This portion of the app will allow them to create a message (voice recording, picture, text), select a template for their love note, and engage with an AI assistant that will provide feedback on their language usage. Last is the inbox, where the user can view their partner’s notes, add annotations (native speaker), and receive translation help. Ultimately, we found that the map helped us imagine how our solution could begin to translate into a graphical user interface and the different pages a user can transverse on our platform.

Assumption Map

Most Crucial Assumption: Couples are willing to write love notes to each other regularly.

Insights: 

  • We are asking partners to critique their partner’s heartfelt attempt at communicating their love towards them, so partners might not be very critical of their partner’s work despite the feedback’s value in making the language learner more proficient.
  • Constructive criticism may be a point of friction between couples, so if couples were not effectively communicating before this practice, critiques may be taken as a personal attack, causing rifts in the relationship.
  • We are not directly providing language-learning resources or curricula for language learners to use, so we are assuming that they have an adequate learning resource outside of our solution. We may help them find free third-party resources for their desired language through a series of hyperlinks.

Assumption Testing

 

Intervention Study

Summary

During the intervention Study on MA (language learner) and MZ (native speaker), we found a few notable behavior changes and trends. First, in the pre-interview study, Maddie noted that her language learning primarily consisted of using Duolingo as often as possible–averaging about 3 days per week, 20 minutes per session. However, during the intervention study, she submitted a love note in German every day, increasing her overall language learning from 3 to 5 days per week. However, the largest finding was her reflection that: “writing stories is much more helpful than just writing words of affirmation”. Maddie explained that the first few notes were easy to write, as they were simple love notes. However, when prompted to write a story about her favorite memory with her partner, she had to combine language from different Duolingo exercises and recall vocabulary across topics: “Most of my experience so far has been about remembering stuff in the context of lessons (transportation, fruits, etc.), but combining them to tell a larger story was kinda hard.”  However, after completing the story, she felt really accomplished and even learned new words as she used google translate to round out details of the story.

Key Insights

  • Language learners feel accomplished when they can recall vocabulary across topic areas.
  • Words of affirmation may be too limited in scope and therefore may not be sufficient for a long-term solution.
  • Telling stories allows language learners to engage with a much larger set of vocabulary and grammar in their target language.
  • Reliance on LLMs: for grammar and looking up phrases
  • Time spent on creating the note ranged from 30 seconds to 10 minutes
  • Anecdote: “Today I was with F and I talked with him in Spanish while playing a card game so that nobody else could understand, so I was able to get in the flow. Then, I wrote my email right after and also asked F for help while writing it for some Mexican Spanish slang to use in the email
  • Anecdote: “Most of my experience so far has been about stuff in context (transportation, topics in school, etc.) Combining them to tell a big story is kinda hard.
  • Anecdote: “What I found frustrating about the task was that I had to use a little bit of Google translate to help me articulate myself.”
  • Composing notes or longer-form texts can aid language learners in communicating reflectively.
  • The native speaker can also benefit from language practice. One of our participants grew up speaking Spanish with her family but not outside. She feels insecure about speaking Spanish with her extended family in Mexico. This also leads to some feelings of insecurity when speaking Spanish with her partner.
  • Native speakers who grew up speaking their native language only within their family might be less comfortable with their writing skills. Writing a love note gives them the space to practice their writing skills.
  • Communicating in English is faster and easier. This is not a new insight but something that continuously came up in our studies
  • Writing a love note is an enjoyable experience even when the user is tired. Thinking of their partner brings feelings of joy and happiness to the writer.

Design Considerations

  • The note construction part has to be quick and smooth since language learners do not have a lot of time to spend on writing a note. 
  • Native speakers can create challenges for language learners to motivate them to engage with the language in fun ways.
  • Native speakers might also need language practice, especially writing. 
Avatar

About the author

Leave a Reply