Sign-in Changes | VA Identity Flows | Terms of use accessibility | Observer meta-research | Experience research
VA.gov Platform | Observer guidelines meta-research
Objective: Offer future participants more empowerment around observers in order to enhance VA.gov’s research practice, specifically concerning our trauma informed approach
My Role: I led this research as part of my duties on a federal contract position on the Platform Governance Team for the VA digital experience. I didn’t work alone, however, as this research required collaboration with several other researchers and stakeholders to accomplish. I closely consulted with VA stakeholders about guidance and improvements to our research practice, and other researchers were needed to test out the proposed protocol with participants in their studies
Starting Point - current guidelines
At the time of this study, there was existing research ops guidance for VA.gov research observers. Observers were asked to join a remote session without audio or video and with “observer” in place of their name. I talked with researchers and other stakeholders at VA, and discovered a shared interest in considering a more trauma-informed research design. I focused this work on how observers join our remote research sessions. Did nameless, faceless people on a video call make participants feel less more unease with the process?
Plans take shape
Several researchers were eager to try a new method of observer protocols during their next round of sessions. I also asked them to pose three questions about observers to their participants at the end of the sessions, after observers left. Asking questions at the end allowed researchers to focus on their study, and it meant less chance for a bias since observers, who were the subject of the questions, were off the call.
Updated guidelines
The four participating researchers collect data across their studies. I then collected the data from each researcher. I compiled and analyzed the data, then drafted findings, and offered recommendations for how to proceed. My recommended changes included asking participant’s consent before admitting observers, having observers display their name, and having observers offer a short greeting before going off camera.
I implemented these changes by updating the guidance and templates. I also presented the research to the UX community of practice, and continued working with individual researchers to make sure they were comfortable with the changes.
Participant experience enhanced
This important work studied changing the protocol for including observers in research sessions. I found it allowed participants to feel more comfortable. Since the new protocol proved itself, I worked with other researchers to expand the trial. During my time on the VA Platform contract, I updated various guidance for researchers who work on the VA digital experience, and I worked with them to make sure they understood and were comfortable using updated guidance.