Designing Effective ESM Questions: Content and Phrasing

Jordi Quoidbach
Oct 11
3 min read

Experience Sampling Method (ESM)—also known as Ecological Momentary Assessment (EMA)—is designed to capture people’s lives as they happen. Participants respond to short surveys on their phones, often several times a day, across days or weeks. This produces intensive longitudinal data, giving us unique insight into how feelings, thoughts, and behaviors unfold in real time (Trull & Ebner-Priemer, 2014; Hamaker & Wichers, 2017).

But once we’ve decided that ESM is the right approach, a crucial next step remains: how do we phrase the questions themselves?

This post is inspired by Fritz, J., Piccirillo, M. L., Cohen, Z. D., et al. (2024). So You Want to Do ESM? 10 Essential Topics for Implementing the Experience-Sampling Method. Advances in Methods and Practices in Psychological Science, 7(3). https://doi.org/10.1177/25152459241267912.

Clarity is key

Participants may be answering surveys on the move, during work, or while winding down in the evening. If items are confusing or overly technical, frustration rises and compliance drops.

Consider the difference:

“To what extent did you experience anhedonia since the last notification?”
“Since the last notification, how much have you enjoyed what you were doing?”

Both aim to measure enjoyment, but only the second is easy to understand in daily life. Simple, concrete language improves data quality. Piloting with small groups—or running “think-aloud” studies where participants explain their reasoning—can help spot problems before data collection begins.

Keep it short—but not too short

ESM surveys are repeated multiple times per day, so brevity matters. Evidence suggests shorter questionnaires improve compliance and quality (Eisele et al., 2022). Still, constructs like collaboration or decision-making under pressure may require more than one item.

The challenge is finding the sweet spot: single items for simple states like happiness or stress, but additional items where depth is essential. Co-creation with participants—asking them which items feel most relevant—can guide these choices.

Time frame makes all the difference

Unlike traditional surveys that ask about the past week or month, ESM items must anchor to shorter, precise time frames that match the study design.

For momentary states: “How energetic do you feel right now?”
For experiences that may not occur every hour: “Since the last notification, have you felt stressed?”
For rare events: ask participants to complete a survey only when the event occurs (event-contingent).

Simply adding “right now” to a standard survey item is not enough. Items must be designed specifically for real-time data collection (Myin-Germeys et al., 2018).

⭐ 3 Golden Rules for Writing ESM Items

Keep it clear. Use simple, everyday words participants can answer quickly.
Keep it short. Limit each prompt to a single, focused idea.
Keep it time-anchored. Match the time frame (“right now,” “since the last beep,” etc.) to the process you want to capture.

Moving toward better standards

Currently, there are no universally accepted “gold standards” for ESM items. Several groups have published guidance (Eisele et al., 2021; Palmier-Claus et al., 2019), and the ESM Item Repository (Kirtley et al., 2024) is building a shared, open-science database of validated items.

Until then, the best approach is transparency: clearly report why items were chosen or adapted, pilot them thoroughly, and share validated items with the community. Over time, this will accelerate the creation of stronger, more consistent measures.

Takeaway

Designing strong ESM questions is about balance: clarity, brevity, and time sensitivity. Items must be easy to answer in everyday life, short enough to avoid fatigue, and anchored to the right time frame. By piloting, co-creating with participants, and sharing validated items, researchers can improve both the reliability of their data and the participant experience.

Starting Points for In-Depth Reading

Eisele, G., Kasanova, Z., & Houben, M. (2021). Questionnaire design and evaluation. In I. Myin-Germeys & P. Kuppens (Eds.), The open handbook of experience sampling methodology (pp. 71–89). KU Leuven. https://www.kuleuven.be/samenwerking/real/real-book
Kirtley, O. J., Eisele, G., Kunkels, Y. K., Hiekkaranta, A. P., Van Heck, L., Pihlajamäki, M., Kunc, B., Schoefs, S., Kemme, N. D. F., Biesemans, T., & Myin-Germeys, I. (2024). The experience sampling method (ESM) item repository. PsyArXiv. https://doi.org/10.17605/OSF.IO/KG376
Fritz, J., Piccirillo, M. L., Cohen, Z. D., et al. (2024). So You Want to Do ESM? 10 Essential Topics for Implementing the Experience-Sampling Method. Advances in Methods and Practices in Psychological Science, 7(3). https://doi.org/