國立虎尾科技大學 |

Linking Errors Introduced by Rapid Guessing Responses When Employing Multigroup Concurrent IRT Scaling.

Record Type:	Language materials, manuscript : Monograph/item
Title/Author:	Linking Errors Introduced by Rapid Guessing Responses When Employing Multigroup Concurrent IRT Scaling./
Author:	Deng, Jiayi.
Description:	1 online resource (169 pages)
Notes:	Source: Dissertations Abstracts International, Volume: 85-12, Section: A.
Contained By:	Dissertations Abstracts International85-12A.
Subject:	Educational psychology. -
Online resource:	click for full text (PQDT)
ISBN:	9798383163399

Linking Errors Introduced by Rapid Guessing Responses When Employing Multigroup Concurrent IRT Scaling.
Deng, Jiayi.

Linking Errors Introduced by Rapid Guessing Responses When Employing Multigroup Concurrent IRT Scaling. - 1 online resource (169 pages)

Source: Dissertations Abstracts International, Volume: 85-12, Section: A.

Thesis (Ph.D.)--University of Minnesota, 2024.

Includes bibliographical references

Test score comparability in international large-scale assessments (LSA) is of utmost importance in measuring the effectiveness of education systems and understanding the impact of education on economic growth. To effectively compare test scores on an international scale, score linking is widely used to convert raw scores from different linguistic version of test forms into a common score scale. An example is the multigroup concurrent IRT calibration method, which is used for estimating item and ability parameters across multiple linguistic groups of test-takers. The method uses common item parameters to most items and groups, with a select few items allowed to have group-specific parameters. Although prior researchers used empirical data from international LSAs to demonstrate the effectiveness of multigroup concurrent IRT calibration in offering greater global comparability in score scales, it is important to note that they assumed comparable test-taking efforts across cultural and linguistic populations. This assumption may not hold true due to differential rapid guessing (RG) rates, potentially biasing item parameter estimation. To address this gap, I proposed a real data analysis and simulation to explore this area. The objective of the current study is to investigate the linking errors introduced by RG responses when employing multigroup concurrent IRT calibration.In the real data analysis, data from the Arabic and Chinese groups in the PISA 2018 Form 18 science module were linked, with RG responses flagged using response time information. Despite observed differential RG, the linking procedure proved robust to anchor identification and ability estimation. In the simulation, data was generated for two groups with varying motivation levels. These groups were administered two linguistic versions of a test form comprising multiple-choice items. Factors such as differential RG rate, association between ability and RG propensity, group impact, sample size, and model fit criteria were considered. The assessment focused on anchor identification accuracy, item parameter estimation accuracy, and ability parameter estimation accuracy and precision. The findings showed that multigroup concurrent IRT calibration was robust against differential RG, with sample size and group impact being primary factors influencing errors. However, differential RG could affect ability estimation precision and item parameter estimation accuracy.

Electronic reproduction.
Ann Arbor, Mich. :
ProQuest,
2024

Mode of access: World Wide Web

ISBN: 9798383163399Subjects--Topical Terms:

555103
Educational psychology.
Subjects--Index Terms:

Large-scale assessmentsIndex Terms--Genre/Form:

554714
Electronic books.

Linking Errors Introduced by Rapid Guessing Responses When Employing Multigroup Concurrent IRT Scaling.
LDR:03913ntm a22004097 4500 001 1150467
005 20241028051457.5
006 m o d
007 cr bn ---uuuuu
008 250605s2024 xx obm 000 0 eng d
020 $a 9798383163399
035 $a (MiAaPQ)AAI31144267
035 $a AAI31144267
040 $a MiAaPQ $b eng $c MiAaPQ $d NTU
100 1 $a Deng, Jiayi. $3 1476954
245 1 0 $a Linking Errors Introduced by Rapid Guessing Responses When Employing Multigroup Concurrent IRT Scaling.
264 0 $c 2024
300 $a 1 online resource (169 pages)
336 $a text $b txt $2 rdacontent
337 $a computer $b c $2 rdamedia
338 $a online resource $b cr $2 rdacarrier
500 $a Source: Dissertations Abstracts International, Volume: 85-12, Section: A.
500 $a Advisor: Davison, Mark L.;Rodriguez, Michael.
502 $a Thesis (Ph.D.)--University of Minnesota, 2024.
504 $a Includes bibliographical references
520 $a Test score comparability in international large-scale assessments (LSA) is of utmost importance in measuring the effectiveness of education systems and understanding the impact of education on economic growth. To effectively compare test scores on an international scale, score linking is widely used to convert raw scores from different linguistic version of test forms into a common score scale. An example is the multigroup concurrent IRT calibration method, which is used for estimating item and ability parameters across multiple linguistic groups of test-takers. The method uses common item parameters to most items and groups, with a select few items allowed to have group-specific parameters. Although prior researchers used empirical data from international LSAs to demonstrate the effectiveness of multigroup concurrent IRT calibration in offering greater global comparability in score scales, it is important to note that they assumed comparable test-taking efforts across cultural and linguistic populations. This assumption may not hold true due to differential rapid guessing (RG) rates, potentially biasing item parameter estimation. To address this gap, I proposed a real data analysis and simulation to explore this area. The objective of the current study is to investigate the linking errors introduced by RG responses when employing multigroup concurrent IRT calibration.In the real data analysis, data from the Arabic and Chinese groups in the PISA 2018 Form 18 science module were linked, with RG responses flagged using response time information. Despite observed differential RG, the linking procedure proved robust to anchor identification and ability estimation. In the simulation, data was generated for two groups with varying motivation levels. These groups were administered two linguistic versions of a test form comprising multiple-choice items. Factors such as differential RG rate, association between ability and RG propensity, group impact, sample size, and model fit criteria were considered. The assessment focused on anchor identification accuracy, item parameter estimation accuracy, and ability parameter estimation accuracy and precision. The findings showed that multigroup concurrent IRT calibration was robust against differential RG, with sample size and group impact being primary factors influencing errors. However, differential RG could affect ability estimation precision and item parameter estimation accuracy.
533 $a Electronic reproduction. $b Ann Arbor, Mich. : $c ProQuest, $d 2024
538 $a Mode of access: World Wide Web
650 4 $a Educational psychology. $3 555103
650 4 $a Educational tests & measurements. $3 1180442
650 4 $a Education policy. $3 1107727
650 4 $a Linguistics. $3 557829
653 $a Large-scale assessments
653 $a Multigroup concurrent
653 $a Rapid guessing
653 $a Score linking
653 $a IRT calibration
655 7 $a Electronic books. $2 local $3 554714
690 $a 0525
690 $a 0288
690 $a 0290
690 $a 0458
710 2 $a ProQuest Information and Learning Co. $3 1178819
710 2 $a University of Minnesota. $b Educational Psychology. $3 1184998
773 0 $t Dissertations Abstracts International $g 85-12A.
856 4 0 $u http://pqdd.sinica.edu.tw/twdaoapp/servlet/advanced?query=31144267 $z click for full text (PQDT)