Chapter 4
Reaching Out: International Benchmarks for Performance Assessment
Beginning in the 1990s, the United States launched a set of wide-ranging “standards-based” reforms intended to better prepare all children for the higher educational demands of life and work in the twenty-first century. All fifty states developed standards for learning and tests to evaluate student progress. No Child Left Behind (NCLB) reinforced the use of test-based accountability to raise achievement, and scores have climbed on state tests used for accountability purposes, yet the United States has not shown comparable gains on international assessments of student learning since the law was passed in 2001.
On the Program in International Student Assessment (PISA) tests in 2012, the United States ranked at about the median of sixty-five participating countries in reading and science, and below the median in mathematics, with scores that were almost unchanged since PISA began in 2000 (Organization for Economic Cooperation and Development, 2012). Across this time period, US students have traditionally scored lower on the problem-solving test than these content tests. The United States also had a much wider achievement gap than the highest-ranked jurisdictions: Canada, Hong Kong, Macau, Shanghai, Estonia, Finland, Japan, Korea, the Netherlands, Switzerland, and Singapore.
It is worth noting that PISA assessments focus explicitly on twenty-first-century skills, going beyond the question posed by most US standardized tests, “Did students learn what we taught them?” to ask, “What can students do with what they have learned?” (Stage, 2005). PISA defines literacy in mathematics, science, and reading as the ability to apply knowledge to new problems and situations. This kind of higher-order learning is increasingly emphasized in other nations’ assessment systems, but often discouraged by the multiple-choice tests most US states have adopted.
Policy discussions in Washington often refer to these international rankings when emphasizing the need to create more “internationally competitive” standards by benchmarking expectations in the United States to those in high-performing nations. Typically the focus is on identifying the topics that are taught at each grade level in these countries. The analyses reveal that higher-achieving countries teach fewer topics more deeply each year, focus on applications of knowledge rather than recall of facts, and have a more thoughtful sequence of expectations based on developmental learning progressions within and across domains (Schmidt, Wang, & McKnight, 2005; Valverde & Schmidt, 2000).
It is also important to examine how these topics are taught and assessed so that we understand how the top education systems shape what students actually learn and can do. European and Asian nations where student learning has improved dramatically have created curriculum guidance and assessments focused explicitly on twenty-first-century skills: the abilities to find and organize information to solve problems, frame and conduct investigations, analyze and synthesize data, apply learning to new situations, self-monitor and improve one’s own learning and performance, communicate well in multiple forms, work in teams, and learn independently.
The forms of testing used in the United States and those used in higher-achieving countries reinforce the sharp divergences between curricula. Whereas US tests rely primarily on multiple-choice items that evaluate recall and recognition of discrete facts, most high-achieving countries primarily rely on open-ended items that require students to analyze, apply knowledge, and write extensively. Furthermore, these nations’ growing emphasis on project-based, inquiry-oriented learning has prompted increased use of school-based tasks, which include research projects, science investigations, development of products, and related reports or presentations. These assessments, which are incorporated into the overall examination scoring system, help focus the day-to-day work of teaching and learning on the development of higher-order skills and use of knowledge to solve problems. (See table 4.1 for a summary of select systems.)
Table 4.1 Examples of International Assessment Systems
| Country/State | Core System | Assessments Used | Design and Grading |
|---|---|---|---|
| Finland | Student performance is evaluated for a sample of students at the end of second and ninth grades to inform curriculum and school investments. All other assessments are designed and managed locally, based on the national curriculum. |
National: Problems and written tasks that ask students to apply their thinking. School based: Research tasks, presentations, demonstrations. |
National: Designed by teachers through the Finnish Ministry of Education. Graded by teachers. School based: Teachers design and grade tasks based on recommended assessment criteria and benchmarks for each subject and grade within the national core curriculum. |
| Most students take a voluntary national matriculation examination to provide information to colleges. Students choose which subjects they will sit for (usually at least four), with the test in the student’s native tongue being compulsory. | The exam uses mostly open-ended questions to evaluate skills such as problem solving, analysis, and writing. Students answer a prescribed number of questions out of a larger group of questions offered. | The exam is administered, organized, and evaluated by the national Matriculation Exam Board appointed by the Ministry of Education. Teachers grade the exams locally, using the official guidelines. Samples are regraded by professional raters hired by the Exam Board. | |
| Sweden | Students take faculty-designed, nationally approved examinations in year 9 and in the last two years of upper secondary school in Swedish, English, and mathematics. Teachers use these assessments as one factor in determining students’ grades at year 9, along with course work and local assessments. Some localities require schools to give an examination in year 5 in these same subjects. All other assessments are managed locally by teachers. They are designed to determine whether students have met the objectives of the national syllabus. |
National: Open-ended tasks requiring analysis of materials or problems, and written responses; materials may be given in advance of the test. School based: Course work, research projects, diagnostic tasks, essays, problem sets. |
National: University faculty, with secondary teachers, design the national exams at year 9 and the upper secondary level. Teachers grade the assessments during time set aside by regional authorities to calibrate grading practices and minimize variation across the region. Scores are factored into course grades. School based: Teachers design, administer, and grade tasks based on the national curriculum and syllabi, which outline objectives to achieve in each subject. Diagnostic materials for assessing syllabus goals are made available on an optional basis. |
| England | Nationally provided assessments are offered as guidance for school-based formative assessments conducted by teachers. Teachers choose which tasks and tests to use and when to use them, within certain parameters. Assessments for primary school are designed and managed locally. |
National: Observation scales completed by teachers regarding pupils’ work and performance on specific kinds of tasks; written, oral, and performance tasks and tests. School based: Course work, tests, projects, essays. |
National: The Qualifications and Curriculum Authority develops the national assessments, which are scored by teachers, and a range of guidance and supports for in-school assessment. School based: Teachers evaluate student performance and work samples based on the national curriculum and syllabi. Regional authorities support teacher training for assessment and in-school moderation. |
| Most students voluntarily take the General Certificate of Secondary Education exams at age sixteen. If they take advanced courses, they may later take A-level exams, which inform universities. Students choose the exams they will take based on their interests. Twenty-five percent to 60 percent of the exam grade is based on classroom assessments. | National: Essays and open-ended problem solutions, oral language assessments. School based: Course work, tests, projects. |
National: External exams are designed and graded by different examining groups (e.g., Oxford, Cambridge, Ed Excel, the Assessments and Qualifications Alliance), which are selected by different schools. School based: Teachers develop and score school-based components based on the syllabus. |
|
| Australia | At the national level, a literacy and numeracy assessment is given at grades 3, 5, 7, and 9. States and localities manage their own assessment systems. | National: Multiple-choice, short-answer, and extended written responses. | National: Designed, administered, and scored by the Curriculum Corporation with questions and prompts contributed by state education agencies. |
| Queensland, Australia | All additional assessments are school based, developed by teachers based on the national curriculum guidelines and state syllabi. On an optional basis, schools may draw on a bank of “rich tasks” from the New Basics project that can be administered across grade levels and scored at the local level, with moderation. |
School based: Open-ended papers, projects, and inquiries. Rich tasks are complex, interdisciplinary tasks requiring research, writing, and the development of multifaceted products. |
School based: Assessments are developed, administered, and scored by teachers. Scoring is moderated by regional panels of teachers and professors that examine scored portfolios of student work representing each score point from each grade level from each school. A state panel also looks at specimens across schools as well. Based on these moderation processes, schools are given instructions to adjust grades for comparability. Rich tasks are developed by teachers with assessment developers; they are accompanied by scoring rubrics and moderation processes by which the quality of student work and scoring can be evaluated. |
| Victoria, Australia | Beyond the national test, all additional assessments are school based until grades 11 and 12, when students choose to take exams in different subject areas as part of the Victorian Certificate of Education (VCE), used to provide information to universities and employers. The exams have both external and school-based components. At least 50 percent of the examination score comprises required classroom-based assessments given throughout the school year. | State VCE: Multiple-choice (25 percent) and open-ended (75 percent) written, oral, and performance elements. School based: Lab experiments, essay, research papers, and presentations. |
The Victoria Curriculum and Assessment Authority (VCAA) oversees the development of the external examinations by teachers and university faculty and ensures the quality of the school-assessed component of the VCE. Teachers score the external exam and design and score the classroom-based assessments in response to syllabus guidelines. Schools are audited and given feedback through an inspection system that examines the quality of the tasks assigned by teachers, the work done by students, and the appropriateness of the grades and feedback given to students. In addition, the VCAA uses statistical moderation based on the external exam scores to adjust the level and spread of each school’s assessments to match that on the common exam. |
| Singapore | External examinations are given at the end of primary school (grade 6) in mathematics, science, English, and native tongue (Malay, Chinese, or Tamil). Results are used to guide course placements in secondary school. All other assessments are school based. |
National: Short and long open-ended responses. School based: Course work, research projects, investigations. |
National: The Singapore Education Assessment Board designs the assessments and manages the assessment system. School based: Designed and graded by the classroom teacher in response to the syllabus. |
| At grade 10, students take the GCE N- or O-level examinations. Students choose the subject areas in which they want to be examined. School-based components comprise up to 20 percent of the final score. Results inform postsecondary choices. GCE A-level (Advanced) examinations are taken after two more years of school to inform university admissions. | National: Short and long open-ended responses and multiple-choice items. School based: Research projects, laboratory investigations and experiments. |
National: The Singapore Education Assessment Board manages the assessment system. The GCE examinations are developed by the Cambridge International Examinations Group. School based: Teachers develop, implement, and score projects and other products that complement the external examinations. |
|
| International Baccalaureate | The International Baccalaureate (IB) Diploma Program, used worldwide for students in grades 11 and 12, assesses students using school-based assessments throughout the two-year program and externally developed exams at the end of the two-year program. School-based assessments comprise 20 to 50 percent of the examination score for each subject. | External (IB developed): Essays, open-ended problem solutions, short-answer, and multiple-choice items. School based: Speeches, projects, portfolio, presentations, investigations, labs, artistic performances. |
External: Designed, administered, and graded by trained IB examiners (usually current or former teachers). School based: Designed and graded by the classroom teacher based on a common syllabus and scoring criteria. |
Smaller countries often have a system of national standards that are sometimes accompanied by national tests in the upper grades. Top-ranking Finland uses local assessments almost exclusively to evaluate its national standards and manages a voluntary national assessment for college admissions at only one grade level. Larger nations—such as Canada, Australia, and China—have state- or provincial-level standards, and their assessment systems are typically a blend of state and local assessments. Managing assessment at the state rather than national level, where it remains relatively close to the schools, turns out to be an important way of enabling strong teacher participation and ensuring high-quality local assessments that can be moderated to ensure consistency in scoring.
In many cases, school-based assessments complement centralized on-demand tests and may constitute up to 60 percent of the final examination score. Tasks are mapped to the standards or syllabus for the subject and are selected because they represent critical skills, topics, and concepts. They are often outlined in the curriculum guide, but they are generally designed, administered, and scored locally, based on common specifications and evaluation criteria. Whether these assessments are locally or centrally developed, decisions about when to undertake these tasks are made at the classroom level, so they are used when appropriate for students’ learning process. Teachers can get information and provide feedback as needed, something that traditional standardized tests do not enable them to do. In addition, as teachers use and evaluate these tasks, they become more knowledgeable about both the standards and how to teach to them. They also become more aware of their students’ learning needs. Thus, the process can increase curriculum equity, both shaping what students are taught and improving the quality of teaching and learning.
Like the behind-the-wheel test given for all new drivers, these performance assessments evaluate what students can actually do, not just what they know. The road test not only reveals some important information about drivers’ skills, preparation for the test helps improve those skills as novice drivers practice to get better. In the same way, performance assessments set a standard toward which everyone must work. The task and the standards are not secret, so teachers and students know what skills they need to develop and how they will need to be demonstrated.
Finally, these countries typically do not use their examination systems to sanction schools or deny diplomas to students. Following the problems that resulted from the Thatcher government’s use of test-based school rankings in England, which caused a narrowing of the curriculum and widespread exclusions of low-performing students from school (Rustique-Forrester, 2005), several countries enacted legislation precluding the use of test results for school rankings. Instead, high school examinations provide information for higher education, vocational training, and employment. Students often choose the areas in which they will be examined to demonstrate their qualifications. The systems are focused on using information for users of the system and for curriculum improvement rather than sanctions. Thus, governments can set higher standards and work with schools to achieve them rather than devising tests and setting cut scores at a minimal level to avoid dysfunctional side effects.
In this chapter, we examine the assessment systems of several education systems: Finland and Sweden, plus a group of English-speaking jurisdictions that have some shared approaches to assessment, as well as some interesting variations: Australia, Singapore, and the United Kingdom. In addition, we describe the International Baccalaureate system, which is now used in many US schools as well as schools around the world. (Table 4.1 summarizes these assessment system features.) These examples provide interesting lessons about how assessments can be linked to curriculum and integrated into the instructional process to shape and improve learning for students and teachers alike.
FINLAND
Finland has been a poster child for school improvement since it rapidly climbed to the top of the international rankings after it emerged from the Soviet Union’s shadow. It has ranked at the top of the OECD nations on the PISA assessments in mathematics, science, and reading in every year since 2000, with scores climbing each year. Leaders in Finland attribute these gains to their intensive investments in teacher education and major overhaul of the curriculum and assessment system. Prospective teachers are competitively selected from the pool of college graduates and enter a graduate-level teacher preparation program that is entirely free of charge and comes with a living stipend. The master’s degree program includes both extensive course work on how to teach, with a dual focus on inquiry-oriented teaching and teaching that meets the needs of diverse learners, plus at least a full year of clinical experience in a school associated with the university. Preparation includes a strong focus on how to use formative performance assessments in the service of student learning (Laukkanen, 2008; Buchberger & Buchberger, 2004).
Policymakers decided that if they invested in very skillful teachers, they could allow local schools more autonomy to decide what and how to teach—a reaction against the highly centralized system they sought to overhaul. Finland’s national core curriculum is now a much leaner document, reduced from hundreds of pages of highly specific prescriptions to descriptions of a small number of skills and core concepts. (For example, about ten pages describe the full set of math standards for all grades.) This guides teachers in collectively developing local curricula and assessments that encourage students to be active learners who can find, analyze, and use information to solve problems in novel situations.
Finland has no external standardized tests to rank students or schools. Although it may seem counterintuitive to Americans accustomed to external testing as a means of accountability, Finland’s leaders point to its use of school-based, student-centered, open-ended tasks embedded in the curriculum as an important reason for the nation’s extraordinary success on international exams (Lavonen, 2008; Finnish National Board of Education, 2007). School-level samples of student performance are evaluated periodically by the Finnish education authorities, generally at the end of grades 2 and 9, to inform curriculum and school investments. All other assessments are designed and managed locally.
The national core curriculum provides teachers with recommended assessment criteria for specific grades in each subject and in the overall final assessment of student progress each year (Finnish National Board of Education, 2008b). Local schools and teachers then use those guidelines to craft a more detailed curriculum and set of learning outcomes at each school, as well as approaches to assessing benchmarks in the curriculum (Finnish National Board of Education, 2008b). Teachers are treated as pedagogical experts who have extensive decision-making authority in the areas of curriculum and assessment in addition to other areas of school policy and management (Finnish National Board of Education, 2008a).
According to the Finnish National Board of Education (2008b), the main purpose of assessing students is to guide and encourage students’ own reflection and self-assessment. Consequently, ongoing feedback from the teacher is very important. Teachers give students formative and summative reports through verbal feedback and on a numerical scale based on students’ level of performance in relation to the objectives of the curriculum. All Finnish schools use a grading scale of 4 to 10, where 5 is “adequate” and 10 is “excellent.” The recommended assessment criteria are shaped around the grade of 8, or “good.” Teachers’ reports must be based on multiple forms of assessment, not only exams. Schools are responsible for giving basic education certificates as students complete the milestones of comprehensive school up to ninth grade and additional classes prior to university (European Commission, 2007/2008).
Most Finnish students take a set of voluntary matriculation examinations that provide information for university admissions based on their abilities to apply problem-solving, analytical, and writing skills. University and high school faculty members construct the examinations, composed of open-ended essays and problem solutions, under the guidance of the Matriculation Exam Board, which is appointed by the Finnish Ministry of Education to organize, manage, and administer the exam (Finnish Matriculation Examination, 2008). The board members (about forty in number) are faculty and curriculum experts in the subject areas tested, nominated by universities and the National Board of Education. More than three hundred associate members, also typically high school and college faculty, help develop and review the tests. High school teachers grade the matriculation exams locally using official guidelines, and samples of the grades are reexamined by professional raters hired by the board (Kaftandjieva & Takala, 2002).
Students take at least four exams, with the test in the students’ mother tongue (Finnish, Swedish, or Saami) being compulsory. These tests have a textual skills section, which evaluates students’ analytical skills and linguistic expression, and an essay that focuses on the development of thinking, linguistic expression, and coherency. They then choose three other tests from among the following: the test in the second national language, a foreign language test, the mathematics test, and one or more tests from the general studies battery of tests in the sciences and humanities (e.g., religion, ethics, philosophy, psychology, history, social studies, physics, chemistry, biology, geography, and health education). The tests also incorporate questions that cross disciplinary boundaries. The system assumes that all students aiming for college (a majority of Finnish students) will be at least bilingual, and many will be trilingual. The language tests evaluate listening and reading comprehension as well as writing.
In addition to the choice of which tests to take, students choose which items to answer within the exams. In the general battery, they are typically given a set of questions or prompts from which they must respond to six or eight of their choice. The mathematics test has about fifteen problems from which students choose ten to answer. Problems require critical thinking and modeling, as well as straightforward problem-solving. For example, the basic mathematics exam poses this problem:
The advanced mathematics exam poses this one:
Assessment is used in Finland to cultivate students’ active learning skills by posing complex problems and helping students address these problems. It is rare to see a teacher standing at the front of a classroom lecturing students for fifty minutes. Instead, students are likely to determine their own weekly targets with their teachers in specific subject areas and choose the tasks they will work on. A description of a Finnish school (Korpela, 2004) illustrates how students may be completing independent or group projects or writing articles for their own magazine. The cultivation of independence and active learning allows students to focus on broad knowledge with emphasis on analytical thinking, problem solving, and metacognitive skills. These types of skills are increasingly emphasized on tests such as the PISA, which seek to evaluate students’ capacities to think independently and creatively in applying knowledge (Lavonen, 2008).
SWEDEN
Over the past forty years, Sweden’s national assessment system has, like Finland’s, shifted from a centralized system based on one test to a more localized system based on multiple forms of assessments. Around 1970, Sweden abolished its studentexamen, a nationally administered exit exam that ranked upper secondary students and placed them in higher education programs (European Commission, 2006/2007). With this change, Sweden hoped to increase upper secondary school enrollment and provide more open access to higher education, reducing social and educational inequities (Eckstein & Noah, 1993).
Sweden’s national curriculum, which includes nationally approved syllabi for individual subjects, is adapted in each district to address local conditions (Swedish National Agency for Education, 2005). Sweden pairs its nationally outlined and locally implemented curriculum with multiple layers of assessment controlled by schools and teachers. Assessments in compulsory school consist of several components. During each school term, the teacher, student, and parent meet to discuss the student’s learning and social development (Swedish National Agency for Education, 2005). Schools can use a number of optional diagnostic materials to assess students in Swedish, Swedish as a second language, mathematics, and English. The diagnostic materials help teachers assess students and support their learning. The diagnostic materials in years 6 through 9 assess where students stand in relation to the goals set by the syllabi (Swedish National Agency for Education, 2005). Teachers also design school-based assessments based on the syllabi, which outline the content of the course work students complete in their classes (Qualifications and Curriculum Authority, 2008a; O’Donnell, 2004; Eckstein & Noah, 1993).
Students take nationally approved examinations in year 9. The exams assess the subjects of Swedish, Swedish as a second language, English, and mathematics. Teachers use these assessments as one factor in determining students’ grades. The exam at year 9 is compulsory for schools, but not for students. Sweden uses the scores from the test to ensure the grades given by teachers compare to the national standards (Qualifications and Curriculum Authority, 2008a). In some jurisdictions, schools give an examination at the end of year 5 in these same subjects.
Toward the end of their upper secondary schooling, Swedish students receive a grade in each course and a learning certificate that acts as a compilation of all grades awarded for courses and projects they completed. Teachers keep extensive records of student progress using course work, assessments designed by teachers based on the course syllabi, and nationally approved examinations when grading the core subjects of Swedish, English, and mathematics and selected other areas (Swedish National Agency for Education, 2005). Regional education officials and schools provide time for teachers to calibrate their grading practices to minimize variation across the schools and the region (Eckstein & Noah, 1993).
While university personnel produce the National School Board examinations at year 9 and at the upper secondary level, teachers help design the tasks and questions and grade the assessments (Eckstein & Noah, 1993; O’Donnell, 2004). The exam questions are grounded in real-world contexts, asking students to use analytical skills and draw on content knowledge learned during their classes. For example, Sweden’s native language test at the upper elementary school level asks students about a broad theme. One year, the exam used the theme of travel and provided students with contemporary poem, prose, and poetry extracts from a variety of authors; a practical description of how to plan a trip; and data about travel presented in a set of texts, charts, and statistical tables. Schools gave students materials a week in advance of the exam so students had time to review the materials. Students then had five hours to write an essay on the topic of their choice that was evaluated on specific criteria emphasized in the syllabus from their course. The skills assessed included using appropriate language in certain circumstances, comprehending the different purposes of language, persuasive mechanisms, presenting information, as well as creative self-expression, word choice, and grammar (Eckstein & Noah, 1993, p. 119).
The following examples from math assessments illustrate how questions are embedded in real-world contexts. Here is an on-demand sample question from the grade 5 exam that asks students (ages eleven to twelve) to grapple with a problem that they might have in their own lives, both weighing and balancing decisions as well as applying math knowledge:
The mathematics exam from the third year of the upper secondary level also frames the questions in real-world, tangible topics, and formats. Students have almost four hours to answer fifteen questions. The first ten questions require short answers and the last five questions require longer answers for which students show their work.
ENGLAND
The British examination tradition has influenced assessment in nearly all of the English-speaking countries around the world. Assessments have typically been open-ended essay and constructed-response examinations, but the nature of the tasks and their administration has changed over the past two decades to include more school-based tasks and projects.
School-based assessment is the norm prior to high school. At ages fifteen to sixteen, the national qualification framework includes multiple measures of student achievement associated with four pathways that are based on students’ aspirations after graduation: apprenticeship (often based on a National Vocational Qualification assessment), diploma, the General Certificate of Secondary Education (GCSE), and the A-level examinations for university-bound students.
Most students take the GCSE, a two-year course of study evaluated by assessments both within and at the end of courses or units. Students choose the assessments they would like to take based on their interests and areas of expertise. The exams involve constructed-response items and structured, extended classroom-based tasks called “controlled assessments,” which comprise 25 to 60 percent of the final examination score, depending on the subject area.
These new tasks, called controlled assessments, are either designed by the awarding body and graded by teachers or designed by teachers and marked by the awarding body. Either way teachers determine the timing of controlled assessments. New tasks for the GCSE increasingly emphasize functional skills like problem solving, team building, and communication, as well as personal learning and thinking skills across subjects. These examples are of classroom-based tasks in English. The number of total “marks” or points for each task is noted in the left-hand column.
The next example is from a GCSE assessment task in interactive computer technology (ICT).
Most students take five or more GCSE exams. Their performance determines the level of the diploma they receive, and whether they will go on to advanced studies, which are evaluated by A-level exams that qualify students for university admissions. England offers forty-five A-level exams. The exam questions require extended answers aimed at assessing deeper levels of understanding and applications of knowledge to real-world problems, as illustrated in the example from a probability and statistics exam.
Most of the exams take the form of essay questions. The mathematics exams include questions that ask students to show the reasoning behind their answers. Foreign language exams require oral presentations. The A-level exam in English literature asks students to show their skills and knowledge in four sections: poetry, drama, prose, and general. Students analyze works of literature they have read as part of their curriculum in terms of their meaning and interpretation as well as literary devices and writing strategies. Course work accounts for 25 to 30 percent of the A-level score, depending on the course. Students must now also complete an independently designed extended research project as part of the A-level assessments. Assessments are scored by teachers in a moderated process managed by the five examination agencies that organize sets of examinations.
AUSTRALIA
In contrast to smaller countries such as England, Finland, and Sweden, which have long had national curriculum guidance, each state in Australia has traditionally had its own curriculum and assessment program. Recently Australia created a national curriculum framework and a national test at grades 3, 5, 7, and 9, but states continue to maintain their own assessment systems as well. As in England, many states have developed a blended model of centralized and school-based assessments that include performance components. Victoria is a well-known example of this approach. In most states, local school-based performance assessment is a well-developed part of the system. Perhaps the most highly developed example of a highly localized system is the state of Queensland. We describe each of these next.
Victoria
In Victoria, a mixed system of centralized and decentralized assessment combines school-based assessment practices with a set of state exams guided by the Victoria Essential Learning Standards. The Victoria system is not unlike the British system that combines on-demand tests with prescribed classroom assessments that contribute to the overall examination score. The Victoria Curriculum and Assessment Authority (VCAA) establishes courses in a wide range of studies, develops the external examinations, and ensures the quality of the school-assessed component of the Victorian Certificate of Education (VCE), which provides information that guides pathways to further study at the university, technical institutes, and the world of work.
VCAA conceptualizes assessment as “of,” “for,” and “as” learning. Teachers are involved in developing assessments, along with university faculty in the subject area. All prior-year assessments are public in an attempt to make the standards and means of measuring them as transparent as possible. Before the external examinations are given to students, teachers and academics sit and take the exams themselves, as if they were students. The external subject-specific examinations, given in grades 11 and 12, include about 25 percent machine-scored items; the remaining items are open-ended and are scored by the classroom teacher. The exams may include written, oral, and performance elements.
In addition, classroom-based tasks that are given throughout the school year comprise at least half of the total examination score. Teachers design these required assignments and assessments—lab experiments and investigations on central topics as well as research papers and presentations—in response to syllabus expectations. The required classroom tasks ensure that students are getting the kind of learning opportunities that prepare them for the assessments they will later take, that they are getting feedback they need to improve, and that they will be prepared to succeed not only on these very challenging tests but in college and in life.
The following example from the Victoria biology test describes a particular virus to students, asks them to design a drug to kill the virus and, in several pages, explain how the drug operates (complete with diagrams), and then to design an experiment to test the drug:
In preparation for this on-demand test, students taking biology will have been assessed on six pieces of work during the school year covering specific outcomes in the syllabus. For example, they will have conducted practical tasks such as using a microscope to study plant and animal cells by preparing slides of cells, staining them, and comparing them in a variety of ways, resulting in a written product with visual elements. They also will have conducted practical tasks on enzymes and membranes and on the maintenance of stable internal environments for animals and plants. Finally, they will have completed and presented a research report on characteristics of pathogenic organisms and mechanisms by which organisms can defend against disease. These tasks link directly to the expectations that students will encounter on the external examination but go well beyond what that examination can measure in terms of how students can apply their knowledge.
These tasks are graded according to criteria set out in the syllabus and count toward the examination score. The quality of the tasks assigned by teachers, the work done by students, and the appropriateness of the grades and feedback given to students are audited through an inspection system, and schools are given feedback on all of these elements. In addition, the VCAA uses statistical moderation to ensure that the same assessment standards are applied to students across schools. The external exams are used as the basis for this moderation, which adjusts the level and spread of each school’s assessments of its students to match the level and spread of the same students’ collective scores on the common external test score. The result is a rich curriculum for students with extensive teacher participation and a comparable means for gauging student performance.
Queensland
In Queensland, there has been no assessment system external to schools for forty years. Until the early 1970s, a traditional postcolonial examination system controlled the curriculum. When it was eliminated, all assessments became school based. Teachers develop, administer, and score the assessments in relation to the national curriculum guidelines and state syllabi (also developed by teachers), and panels that include teachers from other schools as well as at least one professor from the tertiary education system moderate the assessments.
To create the standards used throughout the province, the central authority gathers groups of teachers and subject experts to write standards that specify different levels of achievement and describe the characteristics of student work at each level. In these documents, the objectives convey the knowledge or skill expected at each standard. The standard descriptors detail the expected characteristics and quality of the work. The teachers and experts also develop samples of work used as exemplars for the different levels. These standards guide the assessments teachers develop and their scoring.
Syllabi are developed to show a clear alignment between the General Objectives that state what students should be able to achieve as a result of completing the course and the standards associated with exit criteria. The syllabi seek to strike a balance between “informed prescription” and “informed professionalism.” They spell out a small number of key concepts and skills to be learned in each course and what kinds of projects or activities (including minimum assessment requirements) students should be engaged in. Each school designs its program to fit the needs and experiences of its own students, choosing specific texts and topics with this in mind. However, all schools must use shared, specific criteria for evaluation of student work, based on the course objectives and specific standards for grades of A, B, C, D, and E.
As the criteria from the physics syllabus in table 4.2 indicate in detail, in the category of knowledge and conceptual understanding, work that meets an A standard demonstrates interpretation, comparison, and explanation of complex concepts, theories, and principles, whereas work at an E standard is characterized by reproduction of isolated facts and application of simple, given algorithms. In this course, objectives also include investigative processes, and evaluating and concluding, with indicators spelled out for all of these objectives.
Table 4.2 General Objectives and Standards for Physics in Queensland
| General Objectives | Standards | ||||
|---|---|---|---|---|---|
| Knowledge and conceptual understanding | Students should acquire knowledge and construct understanding of facts, theories, concepts, and principles of physics. To work scientifically, students need to have an understanding of underlying scientific knowledge, including the associated mathematical skills. They need to engage with the processes and phenomena observed in Physics through characteristics of data analysed. Students need to make informed judgments based on sound reasoning in order to direct them in their scientific endeavours and to engage with problem solving. By the end of the course, students should be able to: Recall and interpret concepts, theories and principles of Physics—this includes the abilities to remember, reproduce and interpret subject matter such as facts, definitions, formulas, terminology, concepts, theories, principles, laws, procedures, sequences, events, diagrams, symbols, figures, systems and patterns Describe and explain processes and phenomena of Physics—this includes the abilities to compare and classify the concepts, theories and principles being explored, based on primary and secondary data Link and apply algorithms, concepts, theories and schema of Physics—this includes the abilities to adapt, translate and reconstruct understandings in order to find solutions. |
||||
| Standard A | Standard B | Standard C | Standard D | Standard E | |
| The student work has the following characteristics: |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| Investigative processes | Students need to recognise the methodologies available to them to investigate scientifically. They need to be able to judge the worth of quantitative and qualitative data and interpret and apply the outcomes of such data. Students require the skills to manipulate and review data and scientific techniques so that they may improve their scientific knowledge. They need to synthesise the research that they have generated and be able to discuss the outcomes in relation to their initial purpose. By the end of the course, students should be able to: Conduct and appraise Physics research tasks—this includes the abilities to formulate questions, hypothesise, plan, manage, evaluate, refine and justify decisions made during investigations, as well as the critical reflection required to fulfill research goals Operate scientific equipment and technology safely—this includes the abilities to safely select, adapt and apply technological, laboratory and fieldwork equipment, and consider its limitations; it also incorporates the ability to do this individually and in groups Use primary and secondary data—this includes the abilities to analyse and extrapolate from data, and to identify relationships, patterns and anomalies in primary and secondary data. |
||||
| Standard A | Standard B | Standard C | Standard D | Standard E | |
| The student work has the following characteristics: |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| Evaluating and concluding | Students who are working scientifically need to be able to make decisions about the knowledge they have gained and generated. They need to distinguish between a plausible conclusion and one based on pure supposition. Students need to be able to synthesise their thoughts and the thinking of others into a coherent whole, from which they can make judgments and propose future possibilities. They need to reach conclusions and explain the world in which they live, using science. They need to be able to adhere to communication and scientific conventions in communicating their decisions to selected audiences. By the end of the course, students should be able to: Determine, analyse and evaluate the interrelationships involved in applications of Physics—this includes the abilities to identify the physics involved, to determine the simple and complex relationships that exist between concepts, principles, theories and schema and then to critically examine the associated implications Predict outcomes and justify conclusions and recommendations—this includes the abilities to explore scenarios and consider possible outcomes, and then to provide justifications of conclusions and recommendations Communicate information in a variety of ways—this includes the abilities to select, use and present data and ideas to convey meaning, an argument or a case to selected audiences in a range of formats. |
||||
| Standard A | Standard B | Standard C | Standard D | Standard E | |
| The student work has the following characteristics: |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
The expectations of work quality are challenging, as shown in the following example of the required extended experimental investigation. Students are expected to design and conduct their own investigation, collect and display data, draw valid conclusions, and explain them in a thoughtful paper that resembles a shorter version of a scientific journal article:
In one example (see Darling-Hammond & Wentworth, 2010), a student investigated a problem entitled, “The Air Pocket.” The paper starts with a picture of a vertical air jet from a straw producing a cavity on a water surface. The student investigated the parameters that would affect the volume of the cavity, preparing a thirty-two-page paper that evaluated the problem theoretically and empirically, presented data through tables and charts, analyzed findings by summarizing individual results and developing a regression to evaluate the combined effects of several variables on the volume of the cavity, and evaluating the results, along with the potential errors, means for improving the investigation, and additional research needed. Overall, the paper more closely resembles a research report from a scientific laboratory than a traditional high school physics test.
These kinds of extended responses are demanded in all subject areas, shaped by the core concepts and modes of inquiry of the disciplines. Student reflection is also a common element of the assessments. Consistent scoring of such intellectually ambitious work is made possible in part by internal and external moderation processes and in part by the clear guidance of the syllabi and rubrics that set standards for the work.
At the end of the year, teachers collect a portfolio of each student’s work, which includes the specific assessment tasks, and grade it on a five-point scale. To calibrate these grades, teachers put together a selection of portfolios from each grade level—one from each of the five score levels plus borderline cases—and send these to a regional panel for moderation. The panel of five teachers rescores the portfolios and confers about whether the grade is warranted, making a judgment on the spread. State review panels also look at a sample of student work from each district to ensure that schools implement the standards across all districts. Based on this analysis and a twelfth-grade standardized statewide test, the Queensland Core Skills Test, the Queensland authority confirms the levels of achievement proposed by school programs and may adjust it if it does not calibrate to the standards.
Aiming for even more applied, interdisciplinary work, Queensland developed a “rich tasks” approach to standards and assessment, which was introduced as a pilot in 2003. Part of the New Basics project, this effort created extended multidisciplinary tasks that are developed centrally and used locally when teachers determine the time is right and they can be integrated with locally oriented curricula (Queensland Government, 2001). These are “specific activities that students undertake that have real-world value and use, and through which students are able to display their grasp and use of important ideas and skills.” Rich tasks are defined as:
A culminating performance or demonstration or product that is purposeful and models a life role. It presents substantive, real problems to solve and engages learners in forms of pragmatic social action that have real value in the world. The problems require identification, analysis and resolution, and require students to analyze, theorize and engage intellectually with the world. As well as having this connectedness to the world beyond the classroom, the tasks are also rich in their application: they represent an educational outcome of demonstrable and substantial intellectual and educational value. And, to be truly rich, a task must be transdisciplinary. Transdisciplinary learnings draw upon practices and skills across disciplines while retaining the integrity of each individual discipline. (Queensland Government, 2001)
The following example is a task description in a science course:
A bank of these tasks now exists across grade levels, along with scoring rubrics and moderation processes by which the quality of the tasks, the student work, and the scoring can be evaluated. Extensively researched, this system has had excellent success as a tool for school improvement. Studies found that students are more engaged in learning in schools using the rich tasks. On traditional tests, these New Basics students scored about the same as students who did not use the tasks, but they performed notably better on assessments designed to gauge higher-order thinking.
SINGAPORE
The Singapore government has employed the developers of the Queensland system to expand and refine its system of performance assessments. Singapore’s examination system is inherited from Great Britain, and Singapore is also increasingly emphasizing the integration of school-based assessment into large-scale testing systems. These classroom-embedded components include some ambitious projects that resemble the more extended kinds of student work seen in Queensland and Hong Kong.
Policy analysts have been intensely interested in Singapore’s education system since its students took first place in the TIMSS (Trends in International Mathematics and Science Study) assessments in mathematics and science in 1995, 1999, and 2003 and then maintained this record on the PISA assessments which Singapore joined in 2009. Singapore’s scores are based on high achievement by all of the country’s students, including the Malay and Tamil minorities, who have been rapidly closing what was once a yawning achievement gap (Dixon, 2005). About 90 percent of Singapore’s students scored above the international median on the TIMSS tests. This accomplishment is even more remarkable given that fewer than half of Singapore’s students routinely speak English, the language of the test, at home. Most speak one of the other four official national languages of the country—Mandarin, Malay, or Tamil—and some speak one of several dozen other languages or dialects.
Since the prime minister introduced the Thinking Schools, Learning Nation initiative in 1997, Singapore’s explicit focus in its reforms of curriculum, assessment, and teaching has been to develop a creative and critical thinking culture within schools. The goal is to teach and assess these skills for students and create an inquiry culture among teachers as well. Teachers are encouraged to conduct action research on their teaching and revise their strategies in response to what they learn. This initiative was married to commitments to integrate technology in all aspects of education, a mission nearly fully accomplished a decade later, and to dramatically open up college and university admissions.
Higher education is now available to virtually all Singaporeans. Based on their interests, labor force needs, and the results of their grades, O-level exams, and other accomplishments, students pursue one of three pathways after tenth grade, when secondary school ends. About 25 percent attend junior college for two years, followed by university, which leads to professional paths such as teaching, science, engineering, medicine, law, and the civil service; about 60 percent attend a polytechnic college for three years, after which about half go on to university while the others go into jobs in technical and engineering fields; the remainder—about 15 percent—attend an institute of technical education for two years, after which some continue on to college or university. Virtually everyone finishes one of these pathways.
Historically Singapore has operated a modified British-style system of examinations. Students sit for national exams administered by the Singapore Examinations and Assessment Board (SEAB). At the end of year 6 (age twelve), students take the Primary School Leaving Examinations. These are open-ended written and oral examinations in four core subject areas—mathematics, science, English, and a native language—that are administered and scored by teachers in moderated scoring sessions. The exams in the English and native languages include four components: two written essays of at least 150 words, listening comprehension, language comprehension, and an oral exam that requires students to engage in a conversation on a set topic for fifteen minutes. Two examiners observe the candidates and grade the oral proficiency of the student. In math, students must demonstrate the steps in solving a problem.
Students take the General Certificate of Examinations Normal or Ordinary Level (GCE N/O-Level) at the end of year 10 (age sixteen). The GCE O-level examinations are based on common course syllabi that outline what is to be taught; they require short and long open-ended responses and essays across a wide range of content areas. Students choose the areas in which they want to be examined. Although the results are used to guide postsecondary admissions and not to determine graduation from high school, they exert substantial influence on the high school curriculum. Recent reforms are changing the curriculum and assessment system to make it more explicitly focused on creativity and independent problem solving.
Students attending junior college (grades 11 and 12) en route to university take the GCE Advanced Level (A-level) exams at the end of year 12 (age eighteen). A new A-level curriculum and examination system was introduced in 2002. The exams encourage multidisciplinary learning by requiring that students “select and draw together knowledge and skills they have learned from across different subject areas, and apply them to tackle new and unfamiliar areas or problems” (Singapore Examinations and Assessment Board, 2006, p. 2).
The A-level curricular framework includes core content areas in which students take courses and associated exams: humanities, mathematics and sciences, and languages. It also includes life skills, emphasizing leadership, enrichment, and service to others, and knowledge skills, evaluated through a general paper, project work, and a course in knowledge and inquiry. A typical A-level student is evaluated in three compulsory subjects with a general paper, project work, and a native language assessment, along with four content subjects.
The newer areas of life skills and knowledge skills are intended to develop the more advanced thinking skills thought to be underrepresented in the traditional content-based curriculum and examinations system. They represent the goals of reforms launched in 1997 as part of the Thinking Schools, Learning Nation initiative, which created a number of changes:
Syllabi, examinations and university admission criteria were changed to encourage thinking out of the box and risk-taking. Students are now more engaged in project work and higher order thinking questions to encourage creativity, independent, and inter-dependent learning. (Ng, 2008, p. 6)
The content courses are also evolving to include more critical thinking, inquiry, and investigation, along with mastery of content. A number of the high school content tests are accompanied by school-based tasks, such as research projects and experiments designed and conducted by students. Each of the science courses now includes a component called the “school-based science practical assessment.” These school-based components, which teachers manage and score according to specifications provided by the Examinations Board, count for up to 20 percent of the examination grade. Scoring is both internally and externally moderated. The goal is for students to be able to:
- Follow a detailed set or sequence of instructions and use techniques, apparatus, and materials safely and effectively
- Make and record observations, measurements, methods, and techniques with precision and accuracy
- Interpret and evaluate observations and experimental data
- Identify a problem, design and plan investigations, evaluate methods and techniques, and suggest possible improvements in the design
The projects can be submitted to the university as part of the application, and universities are encouraged to examine evidence about student accomplishments beyond examination scores.
In carrying out the PW assessment task, students are intended to acquire self-directed inquiry skills as they propose their own topic, plan their time lines, allocate individual areas of work, interact with teammates of different abilities and personalities, and gather and evaluate primary and secondary research material. These PW processes reflect life skills and competencies such as knowledge application, collaboration, communication, and independent learning, which prepare students for the future workplace.
About twelve thousand students complete this task annually. Assessment is school based and criterion referenced. While the SEAB externally specifies task setting, conditions, assessment criteria, achievement standards, and marking processes, classroom teachers carry out the assessment of all three components of PW using a set of assessment criteria provided by the board. All schools are given exemplar material that illustrates the expected marking standards. The board provides training for assessors and internal moderators. Like all other assessments, the grading is both internally and externally moderated.
Intellectually challenging school-based assessments are also encouraged in the earlier grades. The curriculum and assessment guidelines that accompany the national standards suggest that teachers engage in continual assessment in the classroom using a variety of assessment modes, such as classroom observations, oral communication, written assignments and tests, and practical and investigative tasks. The ministry has developed a number of curriculum and assessment supports for teachers. For example, SAIL (Strategies for Active and Independent Learning) aims to support more learner-centered project work in classrooms and provides assessment rubrics to clarify learning expectations. All schools have received training for using these tools. The Institute of Education has held workshops to support learning about the new assessments and has integrated the new strategies into teacher development programs.
HONG KONG
High-scoring Hong Kong has also begun to expand its ambitious school-based assessment system, replacing its previous highly centralized examination system. As outlined in Hong Kong’s Learning to Learn plan, the goal of the reforms is to shape curriculum and instruction around critical thinking, problem solving, self-management skills, metacognitive thinking skills, and collaboration (Education Bureau, 2008; Chan, Kennedy, Yu, & Fok, 2008).
The Hong Kong Examinations and Assessment Authority (2009) explains the rationale for growing use of school-based assessments (SBA) this way:
The primary rationale for SBA is to enhance the validity of the assessment, by including the assessment of outcomes that cannot be readily assessed within the context of a one-off public examination. SBA can also reduce dependence on the result of public examinations, which may not always provide the most reliable indication of the actual abilities of candidates. Obtaining assessments based on student performance over an extended period of time and developed by those who know the students best—their subject teachers—provides a more reliable assessment of each student. Another reason for including SBA is to promote a positive impact on teaching and learning. It can serve to motivate students by engaging them in meaningful activities; and for teachers, it can reinforce curriculum aims and good teaching practices, and provide structure and significance to an activity that they are in any case involved in on a daily basis, namely assessing their own students. . . . Teachers know that SBA, which typically involves students in activities such as making oral presentations, developing a portfolio of work, undertaking fieldwork, carrying out an investigation, doing practical laboratory work or completing a design project, help students to acquire important skills, knowledge and work habits that cannot readily be assessed or promoted through paper-and-pencil testing. Not only are they outcomes that are essential to learning within the disciplines, they are also outcomes that are valued by tertiary institutions and by employers. Moreover, they are activities that students find meaningful and enjoyable.
This captures the reasoning of many governments that have been seeking to expand embedded performance assessments. In Hong Kong, these assessments are scored by teachers who are trained as assessors. Tests are allocated randomly to scorers, and essay responses are typically rated by two independent scorers (Dowling, n.d.). Results of the new school-based assessments are statistically moderated to ensure comparability within the province. The assessments are internationally benchmarked, through the evaluation of sample student papers, to peg the results to those in other countries. Many of the new assessments are also to be scored online, which the Examinations and Assessment Authority notes is now the common practice in twenty of China’s mainland provinces, as well as in the United Kingdom.
To guide the process of assessment reform, the Hong Kong Education Bureau has implemented a school development and accountability framework that emphasizes school self-evaluation, plus external peer evaluation, using a set of performance indicators. The bureau promotes the use of multiple forms of assessment in schools, including projects, portfolios, observations, and examinations, and looks for a variety of assessments in the performance indicators used for school evaluation (Chan et al., 2008). For example, the performance indicators ask, “Is the school able to adopt varied modes of assessment and effectively assess students’ performance in respect of knowledge, skills, and attitude?” and, “How does the school make use of curriculum evaluation data to inform curriculum planning?” (Education Bureau, 2008). This practice of examining school practices and the quality of assessments through an inspection or peer review process is also used in Australia and Great Britain to improve teaching by using standards as a tool for sharing knowledge and reflecting on practice.
INTERNATIONAL BACCALAUREATE DIPLOMA PROGRAM
The International Baccalaureate (IB) Diploma Program is offered in sixteen hundred schools serving 100,000 students in 125 countries. Like other systems in many high-achieving nations, the IB high school curriculum is standards based and syllabus based, integrating assessment within the curriculum in a tightly constructed teaching and learning system that blends classroom-based and external examinations. The IB program generally serves students in grades 11 and 12, assessing students using school-based assessments throughout the two-year program and externally based exams at the end of the course of study. Both types of assessments measure students’ individual performance on the objectives outlined by syllabi, or “subject outlines” written by the International Baccalaureate Organization (IBO), the nonprofit educational foundation that provides services and support to IB schools worldwide.
In almost all of the subjects, teachers conduct school-based assessments by grading individual pieces of course work based on the objective set out by the IB subject outlines. School-based assessments contribute between 20 and 30 percent of the total grade in most subjects and as much as 50 percent in arts courses like music, theater arts, and visual arts. Course work graded by teachers includes such assessments as oral exercises in language subjects, projects, student portfolios, class presentations, practical laboratory work, mathematical investigations, and artistic performances (International Baccalaureate Organization, 2008).
The externally based exams usually consist of essays, structured problems, short-response questions, data-response questions, text-response questions, case study questions, and a limited use of multiple-choice questions. There are a limited number of externally assessed pieces of work (i.e., a theory of knowledge essay, extended essay, and world literature assignment) that students complete over an extended period of time under teacher supervision and are graded by external evaluators, or “IB examiners,” personnel trained and organized by the IBO.
IB externally based exams ask students to apply the analytical and problem-solving skills and content knowledge they gain in their IB course work to specific problems or analyses. For example, a sample English A1—Higher Level essay asks students to answer one essay question and base their answer on at least two of three works studied in class. Students pick from five categories: drama, poetry, prose: the novel and short story; prose: other than the novel and short story; or general questions on literature. On the English exam, students may choose from essay questions like the following examples:
A typical Mathematics Standard Level essay asks students to show their work and support their answers with work and explanations. It also asks students to draw any graphs they create on their graphing calculator, noting that students may receive credit for using the correct method even if an error produces an incorrect final answer. The exam includes a number of multistep questions. The following question addresses multiple math skills, including probability, proportions, and algebra:
In addition to these kinds of assessments of specific course-based learning, the IB course of studies, like the British A levels and the Singaporean exams, requires students to complete an extended essay. This paper is an independent, self-directed piece of research, culminating in a four-thousand-word paper. According to the IB organization, it is intended to provide practical preparation for undergraduate research required in college and is an opportunity for students to engage in an in-depth study of a topic of interest within a chosen subject. Emphasis is placed on the research process: formulating an appropriate research question, engaging in a personal exploration of the topic, communicating ideas, and developing an argument. Participation in this process is intended to develop students’ capacity to analyze, synthesize, and evaluate knowledge.
CONCLUSION
An examination of assessment practices in a number of high-achieving nations and rigorous programs such as the IB illustrates the importance of assessment of, for, and as learning, rather than as a separate disjointed element of the education enterprise. High-quality assessments provide feedback to students, teachers, and schools about what has been learned, and they feed forward information that can shape future learning, as well as guide college and career-related decision making.
These systems closely align curriculum expectations, subject and performance criteria, and desired learning outcomes. They engage teachers in assessment development and scoring as a way to improve their professional practice and their capacity to support student learning and achievement. They engage students in authentic assessments to improve their motivation and learning. They seek to advance student learning in higher-order thinking skills and problem solving by using a wider range of instructional and assessment strategies. And they privilege quality over quantity of standardized testing—moving systems from “accounting” to more useful “accountability” for learning.