Wanjing Anya Ma

All Publications

• ROAR-CAT: Rapid Online Assessment of Reading ability with Computerized Adaptive Testing. Behavior research methods Ma, W. A., Richie-Halford, A., Burkhardt, A. K., Kanopka, K., Chou, C., Domingue, B. W., Yeatman, J. D. 2025; 57 (1): 56

  Abstract

  The Rapid Online Assessment of Reading (ROAR) is a web-based lexical decision task that measures single-word reading abilities in children and adults without a proctor. Here we study whether item response theory (IRT) and computerized adaptive testing (CAT) can be used to create a more efficient online measure of word recognition. To construct an item bank, we first analyzed data taken from four groups of students (N = 1960) who differed in age, socioeconomic status, and language-based learning disabilities. The majority of item parameters were highly consistent across groups (r = .78-.94), and six items that functioned differently across groups were removed. Next, we implemented a JavaScript CAT algorithm and conducted a validation experiment with 485 students in grades 1-8 who were randomly assigned to complete trials of all items in the item bank in either (a) a random order or (b) a CAT order. We found that, to achieve reliability of 0.9, CAT improved test efficiency by 40%: 75 CAT items produced the same standard error of measurement as 125 items in a random order. Subsequent validation in 32 public school classrooms showed that an approximately 3-min ROAR-CAT can achieve high correlations (r = .89 for first grade, r = .73 for second grade) with alternative 5-15-min individually proctored oral reading assessments. Our findings suggest that ROAR-CAT is a promising tool for efficiently and accurately measuring single-word reading ability. Furthermore, our development process serves as a model for creating adaptive online assessments that bridge research and practice.

  View details for DOI 10.3758/s13428-024-02578-y

  View details for PubMedID 39810042

  View details for PubMedCentralID PMC11732908

• Development and validation of a rapid and precise online sentence reading efficiency assessment FRONTIERS IN EDUCATION Yeatman, J. D., Tran, J. E., Burkhardt, A. K., Ma, W., Mitchell, J. L., Yablonski, M., Gijbels, L., Townley-Flores, C., Richie-Halford, A. 2024; 9

  View details for DOI 10.3389/feduc.2024.1494431

  View details for Web of Science ID 001383682800001

• Rapid online assessment of reading and phonological awareness (ROAR-PA). Scientific reports Gijbels, L., Burkhardt, A., Ma, W. A., Yeatman, J. D. 2024; 14 (1): 10249

  Abstract

  Phonological awareness (PA) is at the foundation of reading development: PA is introduced before formal reading instruction, predicts reading development, is a target for early intervention, and is a core mechanism in dyslexia. Conventional approaches to assessing PA are time-consuming and resource intensive: assessments are individually administered and scoring verbal responses is challenging and subjective. Therefore, we introduce a rapid, automated, online measure of PA-The Rapid Online Assessment of Reading-Phonological Awareness-that can be implemented at scale without a test administrator. We explored whether this gamified, online task is an accurate and reliable measure of PA and predicts reading development. We found high correlations with standardized measures of PA (CTOPP-2, r = .80) for children from Pre-K through fourth grade and exceptional reliability (α = .96). Validation in 50 first and second grade classrooms showed reliable implementation in a public school setting with predictive value of future reading development.

  View details for DOI 10.1038/s41598-024-60834-9

  View details for PubMedID 38704429

  View details for PubMedCentralID 2752890