As identified in the first piece in this series Newsmonth #4 2018, there are many claims made about the positive impact that data walls can have on student academic achievement (Sharratt & Fullan, 2012; Renshaw et al 2013).
However, it is important to carefully examine the evidence that underpins such claims. To investigate the research on data walls, we undertook a systematic review of published articles in professional journals and selected books, using many different combinations of key words (data wall, assessment wall, public display of data). We also consulted with 16 international experts from a range of countries in the fields of assessment and data based decision making. While we initially located 48 potentially relevant sources, only 21 of these presented new empirical data relating to data wall use and details of research design, methodology and analysis. This pool of papers was reviewed to search for evidence based claims as distinct from opinion or advocacy.
Our review found that there is scant evidence on how data walls impact student learning and academic achievement. This observation similarly holds for students’ own uses of data walls for self monitoring and improvement purposes. The review found only one study which presented significant student achievement data and a related methodology as a measure of impact (Singh, Märtsin, & Glasswell, 2015). Within this three year study, data walls were used as one strategy within a larger intervention aimed at helping teachers to use data to drive pedagogical change. While some teachers in this study did eventually choose to implement versions of data walls within their classrooms, the intervention focused on teacher use. Student use of information on the data walls was not focal.
Typically, the data walls were located in staff rooms and, in an effort to protect student privacy, student identifying information was placed on the back of tiles. Throughout the duration of the study, the teachers had the support of a school based researcher, as well as access to a range of professional development opportunities. The study found that NAPLAN results and growth in TORCH reading tests outpaced expected growth, taking the student population into account. While the authors suggested that data walls were an effective tool for facilitating staff reflection, they also acknowledged that the walls were one element within a larger multi-faceted approach to interpreting and using data for improvement purposes.
We recognise that there is a body of additional research, commentary and blogs on data walls. This will be considered in the next piece in this series. Most are primarily qualitative in nature, offering small scale case studies beyond which no generalisation is possible. Typically, these rely heavily on participants’ perspectives of the impact data walls had on teaching and learning rather than quantitative data showing learning improvement over time. We acknowledge that the kind of data needed to substantiate claims of positive impact is difficult to collect, particularly because data walls do not directly cause student academic improvement. Impacts are mediated by teacher and student understandings of and conversations about the data, alongside stakeholder abilities to act upon such data in ways which will potentially benefit learning. Overall, our finding is that, to date, only scant research attention has been given to examining the impact of data walls on student learning and teaching practice. There has been no large scale longitudinal study of data wall impact. Until such research is undertaken, data walls represent an experimental strategy. This observation has clear implications for schools. If schools choose to use data walls, their impact should be carefully monitored, considering achievement growth as well as other factors. These additional factors may include staff time and other costs of setting up and maintaining data walls, along with potential affective impact on staff, students and parents/carers.
We encourage school leaders and teachers to become involved in researching data walls in their schools. Contact Dr Elizabeth Heck, Administrative Officer for the Schools Data Network Project (SDN) in the Institute for Learning Sciences and Teacher Education (Elizabeth.heck@acu.edu.au) if you are interested in joining our network.
Claire Wyatt-Smith, Lois Harris, Lenore Adie
Institute for Learning Sciences and Teacher Education, Australian Catholic University
References
Renshaw P, Baroutsis A, van Kraayenoord C, Goos M, & Dole S 2013 Teachers using classroom data well: Identifying key features of effective practices University of Queensland
Sharratt L & Fullan M 2012 Putting FACES on the data: What great leaders do! Thousand Oaks, CA: Corwin and the Ontario Principals' Council
Singh P, Märtsin M & Glasswell K 2015 Dilemmatic spaces: High-stakes testing and the possibilities of collaborative knowledge work to generate learning innovations Teachers and Teaching, 21(4), 379–399
