Examining Differences in Examinee Performance in Paper and Pencil and Computerized Testing

  • Gautam Puhan ETS
  • Keith A Boughton CTB
  • Sooyeon Kim ETS
Keywords: PPT, CBT, differential item functioning, item impact, standardized mean difference, paper, pencil, computer, assessment, test, testing, item, technology

Abstract

The study evaluated the comparability of two versions of a certification test: a paper-and-pencil test (PPT) and computer-based test (CBT). An effect size measure known as Cohen’s d and differential item functioning (DIF) analyses were used as measures of comparability at the test and item levels, respectively. Results indicated that the effect sizes were small (d < 0.20) and not statistically significant (p > 0.05), suggesting no substantial difference between the two test versions. Moreover, DIF analysis revealed that reading and mathematics items were comparable for both versions. However, three writing items were flagged for DIF. Substantive reviews failed to identify format differences that could explain the performance differences, so the causes of DIF could not be identified.
Published
2007-11-20
How to Cite
Puhan, G., Boughton, K. A., & Kim, S. (2007). Examining Differences in Examinee Performance in Paper and Pencil and Computerized Testing. The Journal of Technology, Learning and Assessment, 6(3). Retrieved from https://ejournals.bc.edu/index.php/jtla/article/view/1633
Section
Articles