Usability Report
We uncovered the following issues during our usability testing:
SEVERE
-
Transparency of rubric
Our most pressing issue/point of critique was the lack of transparency of the “rubric” that each LLM intentionality score was graded by. This caused major friction and frustration for the user, as some prompts that they deemed “intentional” were marked as “low intentionality” by the product.
We plan to address this by providing a dropdown list of the rubric, including “high intentionality” and “low intentionality” scoring examples. This will be always-accessible to the user whenever they play the game.
MODERATE
-
Always having prompt templates to use
Another issue one of our users encountered was the lack of available templates, unless they had submitted a prompt with “low intentionality”. The dropdown with prompt templates only appears when they lose points. When they gain points, it disappears.
We plan to address this by having the dropdown available at all times.
-
Music
A moderate issue was that our prototype didn’t have background music, even though there was a sliding bar in the settings to adjust music volume. We originally just had this as a “placeholder”.
We plan to address this by adding music to our MVP for a more memorable experience.
-
More cohesive graphics
Our MVP lacked custom-made graphics, which made the visual cohesiveness of the product lacking.
We plan to address this by implementing our hand drawn graphics to the prototype, which will make the overall experience more outstanding.
TRIVIAL
-
Dark/light mode
One of our users pointed out the lack of a dark/light mode toggle, which is common for extensions to match the mode of the user’s personal device. This was one of the features we discussed, but decided not to implement because it was lower priority and would require much more work on the graphics end (given that we have hand-drawn graphics).
We do not plan to address this.
-
Avatar customization
Our MVP had very basic avatar customization, and it made the product feel much more naive. This was before we implemented the hand-drawn graphics.
This issue was resolved when we finished implementing the custom graphics.
