Beyond numbers: Assessment data in the classroom

The quest for using data to inform educational decision making and practice is not new (Mandinach, 2012), write Professor James Tognolini and Rayanne Shakra of the Centre for Educational Measurement and Assessment, University of Sydney

This quest for data was heightened during COVID-19 when there was a real probability that there would not be Year 12 external examinations in several states; the question that was being considered was what can we do to produce scores for secondary certification if it is not possible to run the Year 12 HSC examination (or equivalent)?

The solution, in every case, was that we can use the judgement of teachers because they know their students and the content of the courses better than anyone else and would give a more valid estimate than any algorithm can generate.

The focus for this article is on how teachers interact with data on a day-to-day basis to produce the evidence that is integral to the process of teaching and learning.

“Assessment involves professional judgement about student performance with respect to a continuum of development and is based upon the image formed of the student by the collection of evidence” (Tognolini & Shakra, 2021, p17).

The key part to this definition is that teachers build images of what their students know and can do based upon data or evidence they collect from various assessments. This image is then used to track student progress through syllabi that have been designed to describe growth through an areaof learning.

Teachers constantly assess and process data in their day-to-day interactions with students. Data are “any information that helps educators know more about their students” (Wayman, 2013, p29) and which can be codified in some manner to facilitate systematic analysis (Schildkamp & Kuiper, 2010).

We can see that data are prevalent in every classroom situation. Data are more than test scores (Lai & Schildkamp, 2012). They help teachers make connections that lead to insights and improved student learning.

Typical classroom

Figure 1 (above) shows a snapshot which is replicated in most classrooms around the world at any one time resonating with every teacher of every subject.

Any teacher in the situation presented in Figure 1 would notice that the class seem very engaged; that one student in the left-hand corner seems to be distracted, that some of the students have their hands up, but are looking down at their books (which probably means they do not want to be asked) etc.

We would argue that teachers process these data constantly by focusing on an aspect, taking the resulting data back to the ‘image’, processing these in terms of their expectation based on the image and then making an informed judgment about what they are going to do.

For example, the teacher might focus on the student in the left-hand corner of the photograph. This piece of data we would call information. It is processed something like ‘he is a good student (based on all the evidence you have about this student. I will not draw attention to him on this occasion because I know he knows the answer and he is’.

This is an instance where teachers assess their students’ engagement levels. Teachers might have already formed an image of who will be excited to tell her the answer to a particular question and who might not.

Data that are in Figure 1 might or might not be new information to the teacher. The image unfolds in the classroom through the interactions and ongoing informal data that teachers choose to interact with. It is this contextual aspect of assessment that makes it highly influential in the teaching and learning process. We would argue that this is formative assessment; that teachers are formatively assessing and processing such information all the time. Therefore, formative assessment is closely integrated with teaching.

Teachers in every classroom are at the centre of the assessment process. They use their professional judgement when they interpret data, as well as to decide when and what data to collect and analyse. An important point here is that the data need to provide evidence to inform the image from a wide range of data sources.

Figure 2

Professional judgement

Figure 2 (above) is a representation of how teachers use their own professional judgement of what their students know and can do by collecting assessment data, then turning these data into information about the students. The compilation of this information is what forms the evidence base that informs a teacher’s professional judgement.

The teacher is responsible for using professional judgement constantly to reconcile data from sources. At certain times more formal assessments (NAPLAN, HSC, check-in assessments, etc) will also contribute to the image. In most cases, they will confirm what it is the teacher already knows. However, where they do not, the question is what has caused the difference? It is this search for understanding the difference between ‘expected’ and ‘most recent information’ that drives teaching.

We can use the judgement of teachers because they know their students and the content of the courses better than anyone else and would give a more valid estimate than any algorithm can generate.

It also shows how we answer one of the most common questions that we are asked “How do you bring summative and formative data together to produce a single image that is supported by evidence that gives teachers confidence in as being a faithful summary of the students’ location (progress) within the curriculum?”

Through data teachers bring together a student’s summative and formative assessments to build an accurate image of this student’s performance. As an example, the teacher can take data from summative assessments (eg NAPLAN) and process them through a comparison of what the teacher believed each child should have obtained on the assessment based upon the image of that child.

To do this, the teacher makes a judgement at the time of a summative test of what level grade the student should get based on the image. If the student performs as expected, then it means that the summative assessment provides some further confirmatory evidence of the image that the teacher has of the student. If it varies, then the teacher should want to know why. This means that summative assessments are being used in the same way that formative assessments are in the teaching and learning process. It is the way that teachers bring summative and formative together.

References

Lai, M. K., & Schildkamp, K. (2012). Data-based Decision Making: An Overview. In Data-based Decision Making in Education (pp. 9–21). Springer Netherlands. https://doi.org/10.1007/978-94-007-4816-3_2

Mandinach, E. B. (2012). A Perfect Time for Data Use: Using Data-Driven Decision Making to Inform Practice. Educational Psychologist, 47(2), 71–85. https://doi.org/10.1080/00461520.2012.667064

Schildkamp, K., & Kuiper, W. (2010). Data-informed curriculum reform: Which data, what purposes, and promoting and hindering factors. Teaching and Teacher Education, 26(3), 482–496. https://doi.org/10.1016/j.tate.2009.06.007

Tognolini, J., & Shakra, R. (2021). Give teachers a stronger voice in assessment. Independent Education, 51(1), 16-18.

Wayman, J. C. (2013). Leading data use: Pre-service courses for principals and superintendents. The Journal of Educational Research & Policy Studies, 13(2), 6-13.