Working group established at the 2022 ICRA meeting in Salamanca, Spain. Co-ordinated by Karolina Smeds. Members: Michael Akeroyd, Louise Hickson, Inga Holube, Gitte Keidser, Graham Naylor.
The goal is to publish our work during 2026.
Our aim is to develop a systematic method for assessing the ecological validity of study outcomes (EVO) to help create awareness of the importance of EVO, and facilitate study designs that support high EVO.
Our work builds on the definition of ecological validity presented in an earlier consensus paper (Keidser et al. 2020).
“In hearing science, ecological validity refers to the degree to which research findings reflect real-life hearing-related function, activity, or participation.”
The definition focuses on research findings, highlighting the importance of how well the results of a research study are indicative of what would happen in real or everyday life. To further emphasize this focus on research findings, we have introduced the new term “Ecological Validity of Outcomes” (EVO). In the Keidser et al. paper, it was pointed out that ecological validity “is not a binary phenomenon that is either present or absent… but each study represents a certain level of ecological validity”. This implies that ecological validity can be quantified, and that experiments can have a greater or lesser degree of ecological validity. The working group is focusing on the assessment of EVO.
How can this assessment be performed? If we knew about for instance the “real-life hearing-related function”, we could simply compare our results of the study with the function in real life. However, we seldom know about that real-life function, and if we did, we would perhaps not have performed the study.
Instead, the basic idea for the suggested assessment tool for EVO is to compare
● What everyday situation a study is designed to investigate
with
● how the study design may support or limit EVO
Based on five methodological dimensions presented in the paper of Keidser et al. 2020, two main design areas are investigated (1. Stimuli, Environment, and Task, 2. Individual and Context of participation). In these areas, design factors that may support EVO and design factors that may limit EVO are listed. The risks associated with the limiting factors are also rated in two steps. A “Low-risk” limitation is one that probably does not invalidate the results. A “High-risk” limitation is strong enough to potentially invalidate the results in relation to the stated purpose of the study.