Combining school-based and external assessment
1. Introduction
Year 12 students’ marks obtained from their school-based assessments and external examinations will be reported as WACE course scores for each pair of Stage 2 and Stage 3 course units completed. These scores give an indication of how each student performed in relation to:
- the standards for the pair of units studied, and
- other students in the state who studied the same pair of units.
In the same way that it is not possible to compare amounts of money in different currencies until they are converted into a common currency, or to compare distances travelling in kilometres and miles until one is changed to the scale of the other, it is not possible to compare marks from schools or from different courses until they are on a common scale.
This lack of comparability of scales means that, in the same way it is not possible to directly add 300 roubles with 600 lire to calculate monetary wealth, it is also not possible to add the marks from different courses and or schools to calculate intellectual wealth.
For example, it is not possible to compare or directly add marks from the following sources because the marks are not on the same scale:
- a mark of 80 in Stage 3 (units 3A/3B) Visual Arts at one school and a mark of 80 in Stage 3 Visual Arts at another school
- a mark of 60 in Stage 3 in English at one school and a mark of 60 in Stage 2 (units 2C/2D) in English at the same or another school
- a mark of 75 in Stage 3 Chemistry and a mark of 75 in Stage 3 Modern History at the same or another school.
Furthermore, it also is not possible to directly compare school or examination marks from one year to the next.
There are procedures used by the Curriculum Council to bring marks onto a common scale so that they can be combined. These are
- statistical moderation of school marks
- standardisation of marks.
In addition, the Curriculum Council, in collaboration with the Tertiary Institutions Service Centre, scale marks from different courses so that they can be combined to allow the calculation of the Tertiary Entrance Rank.
This information paper explains what happens to your school and examination marks as they undergo standardisation and statistical moderation and are used to calculate a combined score for a course and a WACE score for the course. Information on Scaling can be found on the TISC website at www.tisc.edu.au/
2. Purpose of marks
The main purpose of assessment is to inform students about how well they are achieving in an overall sense and to provide diagnostic feedback on how they performed on each assessment task or question. Assessment also helps teachers to analyse what their students have learnt, how well they have learnt it, and helps them decide how to adjust their teaching programs if needed. This assessment for learning is an essential feature of quality teaching and learning. At the level of the particular task or question, the marks tell a student what knowledge and skills have been mastered. Marks also show areas of weakness so these can be the focus of future learning.
A second and very important purpose of marks is to report achievement in each unit to students and their parents at the end of each semester and at the end of the year.
In Year 11 and 12, the marks allow teachers to rank their students’ achievements and then decide on the grade to be awarded. This is achieved by comparing their students’ performances with examples of work that are typical of ‘A’, ‘B’, ‘C’, ‘D’ and ‘E’ grade performances for the pair of units studied.
In addition to reporting school assessments to pupils and their parents/guardians, schools report to the Curriculum Council a mark out of 100 for each unit a student has studied. These school-based marks are combined, on a 50:50 basis, with their examination marks to produce a combined mark. This is used by the Tertiary Institutions Service Centre to calculate and report to each student a scaled score for each course studied and a Tertiary Entrance Rank if a the student has applied for university entrance.
The combined mark is also used to calculate the WACE course score which is reported on the WACE course report for each pair of Stage 2 or 3 units completed. The WACE score is a slight variation of the combined score which ensures year to year consistency in the relationship between the WACE score and the five different standards described in the course report.
3. External examination marks
At the end of Year 12, the Curriculum Council conducts examinations in all Stage 2 and Stage 3 unit pairs (e.g. Units 2A/2B and Units 3A/3B) in all courses except Workplace Learning.
Written examinations of 2½ or 3 hour duration are set in all of these. In addition, practical examinations will be held in about 11 courses.
3.1 Examination marking
Examinations are separately marked by at least two qualified markers under the supervision of a chief marker. Student and school identity is not known by markers as they mark the papers.
If the two markers disagree on a total mark, or even a mark for one part of the examination, they will either work together to decide which mark is correct or a third marker will be used to help establish a correct mark.
Generally, the overall mark out of 100 for the examination is called the ‘raw examination mark’. However, for each course which has a practical component, there will be a ‘raw examination mark for the written component’ and a ‘raw examination mark for the practical component’. The following processes of standardisation and statistical moderation are each applied to the raw examination marks for the written component and to the raw examination marks for the practical component before they are combined to form the course ‘combined score’.
3.2 Standardising raw examination marks to ensure fairness
Each year, examiners aim to set papers for a course (at Stage 2 and Stage 3) that have an average raw examination mark of about 60, with only the most outstanding students scoring in the high 90s or 100. Generally, the examiners succeed in this aim, but until the students sit the examination and it is marked, the actual average and marks spread will be unknown.
Standardisation is a process which takes care of a situation where the distribution of examination marks is not as required. Standardisation takes account of differences in the difficulty of examinations from year to year and ensures that students are not disadvantaged if an examination is harder than usual in a particular year.
If the examination is harder than usual, the standardised examination marks in that unit pair may be higher then the raw marks. If, on the other hand, the examination is easier than usual, the standardised examination marks may be lower than the raw examination marks.
All raw examination marks for a stage are standardised. This means that they are brought onto a common scale which has a mean (average) of 60 and a standard deviation (spread) of approximately 15. This distribution of marks is represented in Figure 1.
3.3 Standardisation procedure
Figure 2 shows how the raw examination marks at a stage are standardised to achieve the distribution shown in figure 1.
The top 2% of raw examination marks are adjusted to fit into the standardised mark range of 90 to100 marks, with the top raw examination mark being awarded 100 standardised examination marks.
The next lowest 6% of raw examination marks are adjusted to fit in the range of 80 to 90 standardised marks.
The next lowest 15.1% of raw examination marks are adjusted to fit in the range of 70 to 80 standardised marks.
At the bottom end of the scale, a raw examination mark of zero will be given a raw standardised mark of zero, and the lowest 0.2% of raw examination marks are adjusted to fit into the range of zero to 10 standardised marks.
3.4 The effect of Standardisation
The effect of standardisation can be shown in pictorial form. Five cases are shown below.
Case 1
The examination marks have the same spread (standard deviation of 13 marks) as the standardised distribution, but the average raw examination mark is greater than that of the standardised distribution: the standardised examination marks generally will be less than the raw examination marks.
Case 2
The examination marks have the same spread (standard deviation of 13 marks) as the standardised distribution, but the average raw examination mark is less than that of the standardised distribution - this means that the standardised examination marks generally will be greater than the raw examination marks.
Case 3
The examination marks have the same average (60 marks) as the standardised distribution, but the spread (standard deviation) of the raw examination marks is less than that of the standardised distribution - this means that the standardised marks will be more spread out than the raw examination marks. As a consequence, for candidates in the top half of the distribution, standardised examination marks generally will be greater than the raw examination marks, but for candidates in the lower half of the distribution, the standardised examination marks generally will be less than the raw examination marks.
Case 4
The examination marks have the same average (60 marks) as the standardised distribution, but the spread (standard deviation) of the raw examination marks is greater than that of the standardised distribution - this means that the standardised marks will be less spread out than the raw examination marks. As a consequence, for candidates in the top half of the distribution, standardised examination marks generally will be less than the raw examination marks, but for candidates in the lower half of the distribution, the standardised examination marks generally will be greater than the raw examination marks.
Case 5
The examination marks have a higher average (greater than 60 marks) than the standardised distribution, and the spread (standard deviation) of the raw examination marks is greater than that of the standardised distribution - this means that the standardised marks will be less spread out than the raw examination marks, and they generally will be less than the raw examination marks.
4. Statistical moderation of school marks
4.1 Statistical moderation to ensure fairness
In developing their school assessments for each unit, teachers must assess students’ knowledge and skills of the syllabus content, and use the range of assessment types, and their weightings, as outlined in the syllabus. Within these requirements, teachers have a high level of flexibility in the designing of assessments for their students.
Because of the flexibility available, different schools will use different sets of tests, assignments, investigations, projects etc. for assessment of the same unit.
The result of this is that school marks for that unit are on a different scale in every school. In a school where tests are easy and marking is generous, a student could get higher marks than students from a different school where the tests and marking is more difficult.
The use of unadjusted, raw school marks for the calculation of combined scores would almost certainly result in some students at particular schools gaining an unfair advantage over others.
4.2 Obtaining a school course mark
Teachers who teach and assess two units separately (e.g. Unit 2A English and 2B English) will obtain two marks out of 100; one for each unit. When these two results are submitted to the Curriculum Council they are simple averaged to give a raw school mark for a course.
Teachers who teach and assess two units in an integrated manner may obtain a single mark out of 100 for the pair of units. In this instance, the single mark will be the raw school mark for the course. This mark will be reported to the Curriculum Council for each of the two units.
4.3 Within school comparability of marks
If schools have several classes studying a particular pair of units (e.g. 3A and 3B English) the Curriculum Council requires schools to have procedures in place to guarantee the comparability of the marks being awarded in the different classes.
Some of the procedures that schools put in place are: common assessment outlines; common tests and examinations; common marking keys; and cross marking.
4.4 Statistical moderation procedure
For each stage of a course, statistical moderation is the procedure used to convert all state-wide sets of raw school marks out of 100 onto a common scale 1 . This needs to be done because, as indicated before, comparing marks from different schools would be unfair unless they are brought onto a common scale.
The common scale used in statistical moderation is the scale of the standardised examination marks for the particular stage of the course. The reason for choosing this scale is that all students who complete a Stage 2 or Stage 3 unit pair sit the Stage 2 or Stage 3 examination (unless they are exempt).
In other words, the WACE examination results for a stage, once standardised, provide a common scale onto which the marks from different schools can be placed. It is like the currency conversion example. Amounts in several different currencies cannot be compared until they are converted to a common currency such as Australian dollars.
The first step in moderating school marks is to identify the moderation population for a particular pair of units for a course at a school. The moderation population comprises students who sat the examination for that pair of units and performed at about the same level as they performed at school throughout the year in that unit pair.
Students who are not included in the moderation population are those who
- do not have a mark out of 100 for each of the units and the WACE examination;
- have successfully applied under the Council’s sickness/misadventure procedure and whose WACE examination score was lower than the estimated (generated) mark; or
- have not made a genuine attempt at the WACE examination.
In addition, for language courses, students who have a background in the language are not included in the moderation population.
The second step is to use the standardised WACE examination marks of the moderation population from the school to adjust the school marks. This is done by calculating the average and the standard deviation of the standardised WACE examination marks of the moderation population at the school.
Following this, the raw school marks are adjusted so that their average and standard deviation are the same as the average and standard deviation of the standardised WACE examination marks of the moderation population at the school 2.
4.5 The effect of statistical moderation
The effect of statistical moderation can be shown in pictorial form. Two sets of diagrams are shown below, each contrasting the effects of different combinations of averages and standard deviations of marks.
Cases 1 and 2
If school marks for a course are on average lower, and more tightly bunched (smaller standard deviation) than the standardised WACE course examination marks of these students, then the school marks are moderated upwards and are more spread out.
Conversely, if school marks for a course are on average greater, and less tightly bunched (greater standard deviation) than the standardised WACE course examination marks of these students, then the school marks are moderated downwards and are less spread out.
Use your mouse to click and slide the standardised exam marks (oranges) and the school marks (apples) up and down the 0-100 scale to see the effects of statistical moderation.
Final mark is the average of the standardised moderated school mark and the exam mark.
There is a misconception that exam scores 'override' school-based assessments. This is wrong as shown in the following scenarios. The final students' mark is always the average of the exam score and the moderated school assessment
Student A has the highest school mark and the fourth highest examination mark for the group. Student A's moderated school mark remains the top mark for the group. His/her final mark will be the average of the standardised examination mark and the standardised, moderated school mark. In the example it would be considered the highest combined score for the group.
Student B has the second highest school mark and the lowest examination mark. Student B's moderated school mark remains the second highest mark for the group. His/her final mark will be the average of the standardised examination mark and the standardised, moderated school mark and will be the sixth highest combined score for the group.
Student D had the fourth highest school mark for the group and the fifth highest examination mark. Student D's moderated school mark remains the fourth highest mark for the group. His/her final mark will be the average of the standardised examination mark and the standardised, moderated school mark and would be the fourth highest combined score for the group.
Student J has the lowest school mark for the group and the second highest examination mark. Student J's moderated school mark remains the lowest mark for the group. His/her final mark will be the average of the standardised examination mark and the standardised, moderated school mark and would be the fifth highest combined score for the group.
Cases 3 and 4
If school marks for a course are on average higher, and more tightly bunched (smaller standard deviation) than the standardised WACE course examination marks of these students, then the school marks are moderated downwards and spread out further. If school marks for a course are on average lower, and have a standard deviation which is similar to that of the standardised WACE course examination marks, then the school marks are moderated upwards, and no change to the spread of the marks is required.
Use your mouse to click and slide the standardised exam marks (oranges) and the school marks (apples) up and down the 0-100 scale to see the effects of statistical moderation.
Standardised exam marks relate directly to school marks after statistical moderation. If school marks are lower than the standardised exam marks, the school marks will be moderated upwards. If school marks are higher than the standardised exam marks, the school marks will be moderated downwards.
1. This also applies to separated written and practical components where they exist.
2. In some cases it is necessary to make further adjustments to ensure that no moderated marks are lower than zero or greater than 100.
