Contents

List of Programs

About This Book

About These Authors

Acknowledgments

Chapter 1: What This Book Is About

Introduction

An Overview of Item Analysis and Test Reliability

A Brief Introduction to SAS

Chapter 2: Reading Test Data and Scoring a Test

Introduction

Reading Data from a Text File and Scoring a Test

Explanation of Program 2.1

Reading Space-Delimited Data

Reading Comma-Delimited Data (CSV File)

Reading Data Directly from an Excel Workbook

Reading an Answer Key from a Separate File

Modifying the Program to Score a Test of an Arbitrary Number of Items

Displaying a Histogram of Test Scores

Matching Student Names with Student IDs

Creating a Fancier Roster Using PROC REPORT

Exporting Your Student Roster to Excel

Conclusion

Chapter 3: Computing and Displaying Answer Frequencies

Introduction

Displaying Answer Frequencies (in Tabular Form)

Modifying the Program to Display the Correct Answer in the Frequency Tables

Developing an Automated Program to Score a Test and Produce Item Frequencies

Displaying Answer Frequencies in Graphical Form

Conclusion

Chapter 4: Checking Your Test Data for Errors

Introduction

Detecting Invalid IDs and Answer Choices

Checking for ID Errors

Using “Fuzzy” Matching to Identify an Invalid ID

Checking for and Eliminating Duplicate Records

Conclusion

Chapter 5: Classical Item Analysis

Introduction

Point-Biserial Correlation Coefficient

Making a More Attractive Report

The Next Step: Restructuring the Data Set

Displaying the Mean Score of the Students Who Chose Each of the Multiple Choices

Combining the Mean Score per Answer Choice with Frequency Counts

Computing the Proportion Correct by Quartile

Combining All the Item Statistics in a Single Table

Interpreting the Item Statistics

Conclusion

Chapter 6: Adding Special Features to the Scoring Program

Introduction

Modifying the Scoring Program to Accept Alternate Correct Answers

Deleting Items and Rescoring the Test

Analyzing Tests with Multiple Versions (with Correspondence Information in a Text File)

Analyzing Tests with Multiple Versions (with Correspondence Information in an Excel File)

Analyzing Tests with Multiple Versions (with Correspondence Information and Student Data in an Excel File)

Conclusion

Chapter 7: Assessing Test Reliability

Introduction

Computing Split-Half Reliability

Computing Kuder-Richardson Formula 20 (KR-20)

Computing Cronbach’s Alpha

Demonstrating the Effect of Item Discrimination on Test Reliability

Demonstrating the Effect of Test Length on Test Reliability

Conclusion

Chapter 8: An Introduction to Item Response Theory - PROC IRT

Introduction

IRT basics

Looking at Some IRT Results

What We Aren’t Looking At!

Preparing the Data Set for PROC IRT

Running PROC IRT

Running Other Models

Classical Item Analysis on the 30-Item Physics Test

Conclusion

References

Chapter 9: Tips on Writing Multiple-Choice Items

Introduction

Getting Started/Organized

Types of Items for Achievement Tests

Conclusion

References

Chapter 10: Detecting Cheating on Multiple- Choice Tests

Introduction

How to Detect Cheating: Method One

How to Detect Cheating: Method Two

Searching for a Match

Conclusion

References

Chapter 11: A Collection of Test Scoring, Item Analysis, and Related Programs

Introduction

Scoring a Test (Reading Data from a Text File)

Scoring a Test (Reading Data From an Excel File)

Printing a Roster

Data Checking Program

Item Analysis Program

Program to Delete Items and Rescore the Test

Scoring Multiple Test Versions (Reading Test Data and Correspondence Data from Text Files)

Scoring Multiple Test Versions (Reading Test Data from a Text File and Correspondence Data from an Excel File)

Scoring Multiple Test Versions (Reading Test Data and Correspondence Data from Excel Files)

KR-20 Calculation

Program to Detect Cheating (Method One)

Program to Detect Cheating (Method Two)

Program to Search for Possible Cheating

Conclusion

Index