California Tinkering Afterschool Network | Resarch Brief: Relevance as Rigor
Many research interventions may show initial positive results, but studies show that these results tend to fade when research structures and supports are removed from the local contexts. In this paper, Gutierrez and Penuel make the case for rethinking what is meant by “rigor” in educational research. They argue that current definitions are too narrow, and may be linked to the consistent failure of studies to sustain and scale.
Rigor, the authors posit, requires attention to the specific contexts and strategies in which change is implemented. Research must focus on the pressing and persistent problems in practice. Furthermore, to drive truly meaningful and sustainable educational improvement efforts, the authors write that there is a need for jointly negotiated research that integrates the perspectives, ideas, work, practical considerations, and analysis of educational practitioners.
While applauding new programs and approaches that call for more collaborative research models, the authors argue that standards for quality must also include evidence that the research is relevant to practice. Methodologically, this means that research designs must include joint work among researchers and practitioners on defining questions, designing and testing interventions, and making sense of results. The authors provide examples of such research approaches in the field, where researchers, as collaborative partners and reflective “observant participants,” help to surface the practices, meanings, and contradictions that may be invisible to educational practitioners due to lack of time or structure to support reflective practice.
The authors describe challenges to generalizing research results. Most notably these are funding to sustain results; developing buy-in across the system in which results would be implemented; and accounting for local variations and adaptations and learning from them in order to theorize and identify important design and implementation principles.
The paper reflects sociocultural and cultural historical theories of learning in which teaching and learning are understood as cultural practices that emerge and evolve in local contexts. In other words, they are not cookie cutter activities independent of real people and places. This view challenges the idea that solutions developed in one setting would work in exactly the same way with exactly the same outcomes in another setting. Instead, it suggests a need for research to closely attend to the educational ecosystem (the local situations, people, needs, and resources) and to design interventions that can take root in those ecosystems. At the same time, it suggests a need to deeply theorize the practices and programs, so that key principles can be surfaced and adapted, sometimes in new forms, in new contexts.
Implications for Practice
The ISE field is challenged to articulate learning outcomes in rigorous and generalizable ways. Part of this challenge is due to the temporal nature of any particular institutional intervention (such as a single field trip or a single game context) and how it supports or even shifts a person’s larger learning trajectory over time. This paper argues for a need to rethink what rigor means in ISE contexts. If rigor is relevance, what is most relevant to ISE programs? How can ISE research be designed to both study and enrich local conditions and also to deeply theorize practice in order to support adaptation and scaling of key findings?
Gutiérrez, K.D. & Penuel, W.R. (2014). Relevance to practice as a criterion for rigor. Educational Researcher, 43(19), 19-23. DOI: 10.3102/0013189X13520289
See other related research briefs in the Connected Collection: Research+Practice Partnerships in Informal Settings.