Target user
People in the United States aged 18-40 who use LLMs at least 5 times a week for work- and school-related tasks
Screener: https://forms.gle/BgugUeh1SR36E7g27
Weekly use of Chatbots (e.g. ChatGPT, Claude, Gemini, etc.)
- What is your age? [short answer]
- What’s your occupation? [short answer]
- How many times do you use chatbots per week?
- Don’t use at all (disqualifying factor)
- 1-5 times
- 5-10 times
- 10-15 times
- 15+ times
- What do you usually use chatbots for? (check all that apply)
- School
- Work
- Emotional companionship
- Medical advice
- Day-to-day questions
- Other (please specify)
- How much do you think you rely on chatbots in your daily life? (on a scale of 10, with 10 being extremely reliant)
———————— scale ————————–
- Are you willing to log every chatbot session you start for 5 consecutive days (~5 min/day)?
Baseline Study Outline / Diary Study Outline
We will track participant data by having participants submit a form with answers to the following questions every time they use a chatbot for a task. All data collected will be content data.
- How confident would you feel doing this task without a chatbot?
- Task type:
- School
- Work
- Emotional companionship
- Medical advice
- Day-to-day questions
- Other (please specify)
- Stakes: low/medium/high (used in order to more fairly compare baseline and intervention studies, can essentially help provide weighted average)
Introduction Document
This study aims to understand people’s reliance on chatbots by making participants answer questions about the nature of their tasks and their confidence in completing them prior to starting chatbot sessions.
Given recent developments in chatbots in the past few years, more and more students and professionals have resorted to using such technologies to complete work and assignments, search for information, and make decisions. However, while this has led to demonstrable improvements and productivity gains in the short term (particularly when it comes to tasks like copywriting, coding, and data analysis), many have concerns that people are beginning to form an unhealthy overreliance on AI tools, which can be harmful by limiting users’ own creativity and independent judgment. According to recent research from MIT that compared the essay writing process in three groups of 18 to 39 year olds using ChatGPT, Google search, and nothing respectively, ChatGPT users showed the weakest brain connectivity and underperformance on social and behavioral benchmarks (Kosmyna et al., 2025).
As a result, we believe it is necessary to investigate the extent to which users of these chatbots are reliant on their functionality to gain an understanding of ways to promote personal development and learning alongside chatbot use. To participate, people must meet the qualifications set by the screener, and they must be willing to participate in a diary study spanning five consecutive days. Over the course of this study, participants will fill out a form to submit data every time they begin a chatbot session, right before they start to use the chatbot tool. This form will ask for three things: 1) the type of task they are about to do, 2) the importance/stakes of the task they are about to do, and 3) how confident they are in completing the task they are about to do without the assistance of a chatbot. The times and frequency of chatbot use will be automatically logged with the submission of the form, but participants are responsible for logging every chatbot session they begin. In this case, a chatbot session represents a continuous string of prompting and interaction with the chatbot all related to one primary task. When participants switch to another task, even if they are still using the same chatbot, they must submit another log for that other task.
Over the course of the five-day diary study we expect participants to be diligent about what consists as chatbot use (covering any type of Large Language Model including but not limited to ChatGPT, Claude, Gemini, Grok, and others) and making sure to promptly identify when they are going to begin a chatbot session and making sure to submit a separate form entry every time they do so.
Data Collection Plan
As a participant in the study, you must identify when you are about to start a new chatbot session and fill out the data collection form prior to engaging with the chatbot. For the purposes of this study, a chatbot “session” represents the continuous string of prompts and interaction with the chatbot related to one primary task. You must log the following information for EACH chatbot session you begin over the course of the five days. You may (and will likely) fill out the form more than once per day.
- Before your session begins, you must fill out the form with the following information:
- Task type (as listed above)
- Confidence level on your ability to do the task on a scale of 1 to 10
- Stakes of your task
- Low: Essentially no stakes (day to day minor question, doing project for fun, etc.)
- Medium: Moderate stakes (school or work related question, but perhaps not an intermediate deadline, emotional advice, inability to understand academic / work concept etc.)
- High: Very high stakes (assignment due with immediate deadline, medical advice for a family member, incoming interview, etc.)
- If you are not totally sure on the stakes, just give your best judgement.
Example: On Saturday, a student is using a chatbot to help with an intro coding assignment that is due the following day. They are using a chatbot for help as they are confused by syntax in Python. Later, they have another brief session where they ask the chatbot a simple day to day question (“when does Green Library open on Sundays?”). For Saturday, this student’s entries could look like this for each session:
- Session 1:
- Task Type: School
- Stakes: High
- Confidence Level: 5
- Session 2:
- Task Type: Day to Day
- Stakes: Low
- Confidence Level: 9
Intro Emails (and data collection)
Hi XXX,
We’re running a short research study to better understand how people use AI chatbots (e.g., ChatGPT, Gemini, Claude). You’ll be logging your AI chatbot use over the next 5 days, including the task and confidence you feel with its use. We’re currently inviting participants to complete a brief screener survey (about 1-2 minutes).
If you are interested, you can get started from here: [SCREENER FORM]
Thank you for your time.
Best,
Hi XXX,
Thank you for agreeing to participate in our study on everyday AI chatbot use. Over the next 5 days, we will ask you to log each time you use an AI chatbot (e.g., ChatGPT, Gemini, Claude). Each log entry for a day will be short and take about 3-5 minutes total to complete.
The goal of this study is not to change how you use an AI chatbot, and it’s not an evaluation of your skills. We are simply interested in understanding how people decide to use these tools, what kinds of tasks they use them for, and how confident they feel using them. Please use an AI chatbot as you normally would. There is no right or wrong usage.
Here is the link to the logging form, which you should fill out each time you start a chat session. If you have multiple sessions a day, please log each one separately.
If you have any questions during the study, feel free to reach out at any time.
Thank you again for your participation. We really appreciate your time.
Best,
Closing Email
Hi XXX,
Thank you for completing the 5-day logging for our study on everyday AI chatbot use. We really appreciate the time and thought you put into each entry.
If you have any additional comments about your experience with the logging process or anything you think we should know, feel free to reply to this email. Otherwise, no further action is needed.
Thank you again for your participation.
Best,
