APPENDIX C: A FRAMEWORK FOR MEASURING THE COSTS, EXPENDITURES, AND BENEFITS OF PERFORMANCE ASSESSMENT

The table text preceded by an asterisk represents the more important cost and expenditure components; the rest of the table text represents a lesser impact on overall costs.

Table C.1 A Framework for Measuring the Costs, Expenditures, and Benefits of Performance Assessment—Formative Assessment

Formative Factors Costs Expenditures Benefits
Development *These costs are largely for local teacher time both to develop instruments and establish a system to monitor student progress. This would be classified as a cost, not an expenditure, because the teacher time is already paid for through the contract. There may be a few expenditures for materials and supplies beyond what would be used otherwise, but these seem minor. The benefits from development of assessments (formative, benchmark, or summative) would come from insights about how to assess student learning more effectively that help teachers improve instruction and student learning. Measuring such benefits in terms of dollars is probably impossible.
Production These are simply the costs of duplicating any tests/quizzes and so on that the teacher chooses to use. They could be somewhat higher if more sophisticated materials are used.
Training *Odden and Picus (2014) offer a professional development (PD) model that includes ten days of pupil-free PD for teachers—which could include time for development and use of formative assessments. The model also includes the extensive use of coaches who could help teachers analyze formative assessment results. To the extent district budgets include these resources for PD, this would represent a cost; to the extent that additional PD time is needed for teachers (additional days in the contract, for example) or additional staff to serve in the role of coaches, there would be additional expenditures. Expenditures would accrue in instances where additional days are added to teacher contracts for PD, for substitute time during the school year, and to pay trainers who work with teachers on issues of assessment. The complexity of estimating these expenditures results from separating expenditures explicitly for assessment from other PD expenditures. *The benefits from the training component accrue from additional knowledge, skills, and teaching strategies learned by the teachers as part of the PD program. Time spent with coaches in activities that improve instruction could also provide benefits in terms of student learning.
*Teachers also need time for collaboration, either paid for as part of the pupil-free PD days or as part of the existing teacher contract through planning and collaboration time during the school day.
Instruction *The concept here is not teaching students or teachers, but rather adjusting instruction based on the results of the formative assessment. Under that assumption, this effort would fall under the concept of a cost as teachers would use their preparation time to modify instruction and teaching based on the results of the assessments. Theoretically, there are no additional expenditures assuming teachers use the same instructional time, only to do different things based on the information gleaned from formative assessments. *The benefits from these costs accrue through improved student learning, as well as enhanced teacher skills and, it is hoped, improved teacher satisfaction with their work.
Test administration *This is mostly a cost, as the assessments would take place during class time and thus replace other activities in the classroom. Expenditures are likely to be minimal as most formative assessments could be done as part of the classroom activities. If a school or district elects to use commercially available formative assessments or materials from a particular website, then there may be some actual expenditure required.
Management *These costs would be mostly transferring teacher and site leader activities from other activities to the design, implementation, and analysis of formative assessment results and would be paid for through the existing contracts. Potential benefits come from improved operation of a school resulting from better communication among staff and between site leadership and teachers.
Scoring *This is mostly time teachers spend scoring and analyzing the outcomes of the assessments and would link closely with the instruction category above, where the adjustments in instructional activities and strategies would take place. *To the extent that individuals are paid extra for work scoring exams (i.e., writing exams), there would be direct expenditures in this category as well. *Benefits are the added knowledge that teachers gain about student knowledge and skills enabling them to focus instruction on areas of need in making classroom learning time more effective and efficient.
Reporting *Since formative assessments are mainly for the use of individual teachers, these costs would be minimal and part of a teacher’s contracted time on the job. Expenditures would be only the costs of materials used in conducting the assessment. *Benefits are similar to those identified above for scoring specifically the knowledge that teachers gain about student knowledge and skills enabling them to focus instruction on areas of need in making classroom learning time more effective and efficient. Another potential benefit of formative assessments would be the ability to meet student needs during initial instruction rather than reverting to substantial interventions to ensure the student masters important material.
Program evaluation *This represents the time teachers spend evaluating the helpfulness of the formative assessments in designing instruction to help improve student learning. Based on how well the instruments used by the teacher worked in helping them better understand student needs, formative assessment instruments would be modified to improve their ability to predict student needs. The latter would be a benefit as well as a “cost” based on the time they spent on the task. *Based on how well the instruments used by teachers worked in helping them better understand student needs, formative assessment instruments would be modified to improve their ability to predict student needs. The latter would be a benefit as well as a “cost” based on the time they spent on the task.

Table C.2 A Framework for Measuring the Costs, Expenditures, and Benefits of Performance Assessment—Benchmark Assessment

Benchmark Factors Costs Expenditures Benefits
Development *Benchmark assessments are generally commercial products. In this instance, it is assumed that private companies that develop these assessment instruments will capitalize development costs into the price of the assessment to districts. In this analysis, focus is on the district/school costs, which are thus assumed to be mostly the time devoted to choosing specific assessment instruments.
*If a district develops its own benchmark assessment instruments (and Boudett, City, & Murnane, 2008, describe the challenges for doing this), there are likely to be substantial personnel costs associated with the development of the benchmark tests.		 Expenditures relate to the type and frequency of assessments as well as to the scale of the assessment used. Districts that develop their own benchmark-assessment instruments are likely to spend more per pupil for benchmark assessments than are those districts or schools that use existing benchmark assessments developed by consortia, a state, group of states, or a national provider. Consistent data across a school/district regarding student learning offers benefits in terms of the design of effective instruction as well as enabling teachers and site leadership to assess both student performance and, if possible, teacher performance as well.
Production *If developed in-house, the costs would be time taken away from other learning and school/district management activities. If developed commercially, the costs would be absorbed by the publisher and passed on to districts and schools that purchase the assessments. *These expenditures would be absorbed by the publisher and shared across all clients if purchased from a commercial vendor, but would likely be substantial if developed in-house.
Training *To properly administer, evaluate, and use the results of any benchmark assessment, teachers will require time (and probably the help of a trainer—at least initially) to learn how to use the benchmark exams to better improve student performance. Expenditures would accrue in instances where additional days are added to teacher contracts for PD, for substitute time during the school year, and to pay trainers who work with teachers on issues of assessment. The complexity of estimating these expenditures results from separating expenditures explicitly for assessment from other PD expenditures. *These costs could also be a benefit if, as suggested above, the training leads to improved teaching and increased access to good instruction for students.
Instruction *As above, this represents time for adjusting instruction based on the results of the benchmark assessment. *As above, benefits accrue through improved student learning, as well as enhanced teacher skills and, it is hoped, improved teacher satisfaction with their work.
Test administration *Costs and expenditures will depend on the form of the assessment, but the time invested in the testing is a cost, while the material (e.g., paper, pencil, test booklets, or online access) represents expenditures for the administration of the assessment. *Costs and expenditures will depend on the form of the assessment, but the time invested in the testing is a cost, while the material (e.g., paper, pencil, test booklets, or online access) represents expenditures for the administration of the assessment.
Management *This is likely a cost in terms of the time of teachers, site, and district leaders to ensure the assessment is administered as intended when intended. Expenditures would accrue for district-level assessment and evaluation staff, particularly if the benchmark assessments required additional staff, and at the school site if there is an assessment coordinator who is paid or receives some form of teaching relief for his or her work as the assessment coordinator.
Scoring *Depending on the form of the assessment, this could be time-consuming and represent either a cost of personnel time (as it is in lieu of other things they might do) or an expenditure to pay teachers (or others) to do the scoring. The costs will vary considerably depending on if the test is online or uses Scantron-type forms, or if the assessment includes writing samples that must be individually read and graded with appropriate checks for reliability and consistency in the scoring. *Depending on the form of the assessment, this could be time-consuming and represent either a cost of personnel time (as it is in lieu of other things they might do) or an expenditure to pay teachers (or others) to do the scoring. The costs will vary considerably depending on if the test is online or uses Scantron-type forms, or if the assessment includes writing samples that must be individually read and graded with appropriate checks for reliability and consistency in the scoring. Benefits accrue to the extent that the results of the assessments are used to evaluate instructional practice and then to improve teaching and instruction and evaluate curriculum.
Reporting *The costs and expenditures will vary with the extent to which the results of benchmark assessments are distributed. If the findings stay within a school, reporting costs are mostly minor production costs and the time it takes to explain the meaning of the results to each teacher and, as appropriate, to students and their parents. The costs and expenditures will vary with the extent to which the results of benchmark assessments are distributed. If the findings stay within a school, reporting costs are mostly minor production costs, and the time it takes to explain the meaning of the results to each teacher and, as appropriate, to students and their parents. Benefits result from discussion of testing outcomes and how students, parents, teachers, and site leaders use the data to improve teaching and learning.
Program evaluation *As with any other program, evaluation of its value in meeting goals is critical. It will take time of leaders and teachers to review the impact that continued benchmark assessments will offer in terms of focusing instruction in ways that lead to improved student performance. If a formal evaluation of alternative benchmark-assessment instruments were conducted, there would be expenditures for either a third-party evaluation or expenditures for on-site personnel charged with evaluation responsibilities. Less formal evaluations by district/site personnel more likely would be opportunity costs for the time spent on the evaluation. Benefits are improved assessments and better understanding of how assessment data can lead to improved instruction.

Table C.3 A Framework for Measuring the Costs, Expenditures, and Benefits of Performance Assessment—Summative Assessment

Summative Factors Costs Expenditures Benefits
Development *If districts participate in helping assessment companies or states develop tests by piloting test items and providing review of items and test procedures without compensation from the developer (a likely outcome for state developed tests), these would be opportunity costs facing the school or district. *Summative, or standardized, assessments are generally commercial products or instruments developed in an individual state. It is assumed that private companies that develop these assessment instruments will capitalize development costs into the price of the assessment to districts. In states that develop their own testing systems, the costs of development must be considered, as well as the ongoing costs of replacing test items to keep the instruments valid and reliable. To the extent that schools/districts participate in the development of assessments, they will benefit from enhanced knowledge about what is tested and how it relates to state standards.
Production *To the extent that a standardized test is a paper-and-pencil exam, there are substantial expenditures associated with the production of text booklets, proctor instructions, and packaging of the tests and materials for the return of the completed test forms and booklets.
Training *Costs are associated with the time devoted to learning how to administer the tests and, more important, the time teachers spend teaching such things as test-taking strategies. *There would be expenditures for training individuals to administer assessments.
Instruction *The cost here is time spent on direct instruction focused on the test. To the extent that assessments are linked to state standards, this may not be a large “cost” or, more correctly, the benefits may outweigh the costs. *To the extent that assessments are linked to state standards, there may be considerable benefits as instruction focuses on what is assessed.
Test administration *Costs and expenditures include time to prepare for, administer, and return the test materials and are potentially extensive, particularly if each school has an assessment coordinator who gets time relief from teaching or other responsibilities to administer the test at his or her school/district. If the school/district has a person who would not otherwise be on staff for this function, it would be an expenditure. *Costs and expenditures include time to prepare for, administer, and return the test materials and are potentially extensive, particularly if each school has an assessment coordinator who gets time relief from teaching or other responsibilities to administer the test at his or her school/district. If the school/district has a person who would not otherwise be on staff for this function, it would be an expenditure.
Management *Similar to those identified in administration above. *Similar to those identified in administration above.
Scoring *This is an expenditure that is either part of the contract for the test itself or incurred by the state/district to score the assessments.
Reporting *The costs and expenditures will vary with the extent to which the results of summative assessments are distributed, although one would anticipate they are widely available at the school, district, and probably state level.
*Reporting costs are mostly minor production costs and the time it takes to explain the meaning of the results to each teacher and, as appropriate, to students and their parents.
*There may also be costs associated with the reporting of test results in the press, as school leaders need to react to the news that is published.		 The costs and expenditures will vary with the extent to which the results of summative assessments are distributed, although one would anticipate they are widely available at the school, district, and probably state level. Information on student performance will help focus instruction and learning in the school/district and help each teacher better focus his or her teaching.
Program evaluation *Mostly time to evaluate the usefulness of the test as well as to identify if the assessment is providing data that are useful to policymakers given the alignment of the assessment to state standards.
..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset
3.142.35.75