There is a gap between what we measure in schools and what matters to us as a society, writes Sara Ratner, a PhD canidate at the University of Sydney’s Centre for Educational Measurement and Assessment.
The COVID-related school closures of the past two years have taught us a great deal about the Australian education system. As homes became virtual classrooms and Public Health Orders prevailed, the interplay of the state and federal political systems was evident and the impact of policy decisions on educational experience was clear.
The pandemic has highlighted our education system’s agility, its inequity and, perhaps most critically, its value. While reflecting on this, it is perhaps timely to consider what learnings from the pandemic may influence our education system moving forward. In particular, how can we best measure system-wide success? What are we measuring and are those results empowering educators to make the right decisions?
Defining event
School communities were in a state of prolonged ambiguity as the new school year began. Recently, a school principal from Cairns shared a story of the ‘Stockdale Paradox’ (Collins, 2001). This refers to Admiral Stockdale, imprisoned in the ‘Hanoi Hilton’ [a prison] during the Vietnam war, who described the struggle of maintaining faith that you will prevail in the end (which one must never lose) while also facing a bleak reality.
This paradox echoes the plight of every educator as they commenced the new academic year. Pertinently, Admiral Stockdale said that not only was he determined to survive the ordeal, he was also determined for it to become a defining event in his life.
It is easy to imagine the same for our schools. Together, educators must hold on to the faith that we will get
through this while facing the bleak reality of staff shortages and student absences. Yet, this moment can become a defining event, a turning point for Australian schools.
Current framework
Australia’s system-wide success in education is judged using the National Assessment Program (NAP) which determines whether young Australians are meeting important educational outcomes (Australian Curriculum, Assessment and Reporting Authority (ACARA), 2018).
The NAP is overseen by the Education Council and managed by ACARA. It includes NAPLAN (Literacy and Numeracy); NAP Sample Assessments; Trends in International Mathematics and Science Study (TIMSS); Progress in International Reading Literacy Study (PIRLS); and the Programme for International Student Assessment (PISA).
Clearly, the Measurement Framework for Schooling (ACARA, 2015) relies heavily upon data generated through standardised testing and national surveys designed to provide comparability with other education systems around the world, or state and territory performance over time.
This framework – which reflects the purpose, values and identity of the Australian school system – measures system-wide performance.
What we value
It is imperative that Australia’s goals for schooling align with what matters to us as a society.
In the opening statement of the Alice Springs (Mparntwe) Education Declaration (Council of Australian Governments, 2019), Australia’s Education Ministers express the need for every child to realise their potential by providing them with the skills they need to participate in the
economy and in society, and by contributing to every aspect of their wellbeing.
Yet while our goals for schooling may address our desires for healthy, active and well-informed global citizens capable of creative thinking and problem solving, it appears we do not currently have a measurement framework that provides evidence of how we are tracking against these critical goals.
The success of a school system is far more than its scores on a test (Masters, 2019). Currently, we use three key performance measures: participation; achievement; and attainment (Ministerial Council on Education, Employment, Training and Youth Affairs, 2008). However, it is evident our schools are achieving on much broader outcomes than those currently measured.
True success
It is the stories behind the learning achievements and the growth students have achieved that truly indicate a school’s success. Yet we have failed to capture this adequately when discussing Australian schools at the macro level.
It is time to examine the intersection between what we value and what we measure in Australian schools in order to locate what really matters. We need to move beyond discussing the tests we employ to measure our success, and to unpack the means, motivation and manner behind them.
Every assessment is laden with cultural, political and social context that must be openly acknowledged. Unpacking the power at play is essential in order to attest to the reliability and applicability of the data it generates. This means not only examining the values we openly attest to in Australian schools, but the inherent values laden in the test construction and design.
Owing to the gap between what we measure and what matters to us as a society, we find ourselves in a challenging situation where what we are measuring begins to matter most. Too often we are busy relying on standardised scores as the measure where it is really our equity and excellence goals that ought to matter.
Tognolini and Mowbray (2020) argue that researchers must investigate all available evidence relating to any issue so as to produce defensible decision making. This has never been more important than when assessing the success of the Australian school system.
In examining the toolkit currently used to attest to system-wide success, let’s consider assessment tools that reflect the creativity, character and critical thinking that our students develop during their years of compulsory schooling.
Shifting our focus away from academic success will enable students to demonstrate their skills and competencies in preparation for active participation as citizens of the world (Fullan, 2021).
Any school system deemed to be excellent must be measured in a manner that encompasses not only academic test scores but also by the development of human qualities and interpersonal skills needed to thrive in society (Levin, 2012).
Pandemic’s impact
Many papers currently address ‘learning lost’ as a result of the pandemic (Brown et al., 2020; Drane et al., 2020; Gore et al., 2021). Yet readers of such articles could be forgiven for correlating this with a narrow view of learning that equates to academic success and grades. When addressing ‘learning loss’ as a result of the pandemic, it implies that the only kind of learning is that gained through the traditional classroom environment (Fullan, 2021).
However, there is much learningthat occurs as a result of the
social contact in the classroom and playground context too. This is often referred to as the ‘hidden curriculum’ and is vital in understanding one’s role in society (Hafferty, 1998).
The kindergarten child learning to stand in line, wait their turn or raise their hand to speak is mastering important life skills that are challenging to replicate in a virtual classroom.
It is imperative that when speaking of the pandemic’s impact on learning we look beyond the grades and form an integrated view of wellbeing and academic learning that best represents the complex, volatile and evolving world our students are destined to occupy (Fullan, 2021).
Agency and voice
There is a gap in the National Assessment Program when it comes to the emotional wellbeing of students, teachers and school leaders. If we are to be sincere in our appraisal of the education system it is imperative that we re-imagine a National Assessment Program that affords its stakeholders agency and voice to generate a holistic view of what success looks like.
Looking to the future and seeking to make the lessons of the pandemic a defining moment in our education system, let’s embrace the opportunity to gain ground on measures that matter and generate a picture of success that reflects the culture, character and competency of Australian schools.