國立虎尾科技大學 |

Data Sharing in Peer-Assessment Systems for Education.

Record Type:	Language materials, manuscript : Monograph/item
Title/Author:	Data Sharing in Peer-Assessment Systems for Education./
Author:	Song, Yang.
Description:	1 online resource (101 pages)
Notes:	Source: Dissertation Abstracts International, Volume: 79-07(E), Section: B.
Contained By:	Dissertation Abstracts International79-07B(E).
Subject:	Computer science. -
Online resource:	click for full text (PQDT)
ISBN:	9780355635454

Data Sharing in Peer-Assessment Systems for Education.
Song, Yang.

Data Sharing in Peer-Assessment Systems for Education. - 1 online resource (101 pages)

Source: Dissertation Abstracts International, Volume: 79-07(E), Section: B.

Thesis (Ph.D.)--North Carolina State University, 2017.

Includes bibliographical references

Fifty years of research has found great potential for peer assessment as a pedagogical approach. With peer assessment, not only do students receive more copious assessments; they also learn to become assessors. In recent decades, more educational peer assessments have been facilitated by online systems. Those online systems are designed differently to suit different class settings and student groups; therefore, their designs are all different from each other: rating-based or ranking-based, reviews assigned randomly or to fixed groups, anonymous or onymous review, etc. Though there are different systems and a large number of users for each, there is a dearth of comparisons between different designs. This is mainly caused by the fact that the data generated by peer assessment systems is stored and analyzed separately; there is no standard for data sharing in this research community.

Electronic reproduction.
Ann Arbor, Mich. :
ProQuest,
2018

Mode of access: World Wide Web

ISBN: 9780355635454Subjects--Topical Terms:

573171
Computer science.
Index Terms--Genre/Form:

554714
Electronic books.

Data Sharing in Peer-Assessment Systems for Education.
LDR:04278ntm a2200373Ki 4500 001 918215
005 20181022132747.5
006 m o u
007 cr mn||||a|a||
008 190606s2017 xx obm 000 0 eng d
020 $a 9780355635454
035 $a (MiAaPQ)AAI10758969
035 $a AAI10758969
040 $a MiAaPQ $b eng $c MiAaPQ $d NTU
100 1 $a Song, Yang. $3 1182253
245 1 0 $a Data Sharing in Peer-Assessment Systems for Education.
264 0 $c 2017
300 $a 1 online resource (101 pages)
336 $a text $b txt $2 rdacontent
337 $a computer $b c $2 rdamedia
338 $a online resource $b cr $2 rdacarrier
500 $a Source: Dissertation Abstracts International, Volume: 79-07(E), Section: B.
500 $a Adviser: Edward F. Gehringer.
502 $a Thesis (Ph.D.)--North Carolina State University, 2017.
504 $a Includes bibliographical references
520 $a Fifty years of research has found great potential for peer assessment as a pedagogical approach. With peer assessment, not only do students receive more copious assessments; they also learn to become assessors. In recent decades, more educational peer assessments have been facilitated by online systems. Those online systems are designed differently to suit different class settings and student groups; therefore, their designs are all different from each other: rating-based or ranking-based, reviews assigned randomly or to fixed groups, anonymous or onymous review, etc. Though there are different systems and a large number of users for each, there is a dearth of comparisons between different designs. This is mainly caused by the fact that the data generated by peer assessment systems is stored and analyzed separately; there is no standard for data sharing in this research community.
520 $a In this work, we focus on the data sharing between educational peer assessment systems. We designed a Peer-Review Markup Language (PRML) as a generic data schema to modeling and sharing data generated by different educational peer assessment systems. Based on PRML, a data warehouse can be built and different systems can ETL (Extract, Transform and Load) their data, contribute the data to the common data warehouse and share the data with other researchers.
520 $a Making use of data shared by different peer assessment systems can help researchers to answer more general research questions, e.g. are reviewers more reliable in ranking-based or rating-based peer assessment? To answer this question, we designed algorithms to evaluate assessors' reliabilities based on their rating/ranking against the global ranks of the artifacts they have reviewed. These algorithms are suitable for data from both rating-based and ranking-based peer assessment systems. The experiments were done based on more than 15,000 peer assessments from multiple peer assessment systems. We found that the assessors in ranking-based peer assessments are more reliable than the assessors in rating-based peer assessments. Further analysis also demonstrated that the assessors in ranking-based assessments tend to assess the more differentiable artifacts correctly, but there no such pattern for rating-based assessors.
520 $a Another research question that can be answered with this shared data is, how do collusions harm the peer review process? Ideally, if only a small number of students try to "game" the peer assessment process, the overall validity will not be affected much. However, one researcher found from his experience that more students became colluders through a semester -- they gave each other high scores, or, even worse, gave high scores to every artifact they reviewed. In the worst case, a big number of colluders may make the honest reviewers outliers, which harms the validity of peer assessment. We have defined two different patterns of possible collusions and apply graph mining algorithms to detect the colluders in the data shared with us.
533 $a Electronic reproduction. $b Ann Arbor, Mich. : $c ProQuest, $d 2018
538 $a Mode of access: World Wide Web
650 4 $a Computer science. $3 573171
650 4 $a Educational technology. $3 556755
650 4 $a Curriculum development. $3 1148494
655 7 $a Electronic books. $2 local $3 554714
690 $a 0984
690 $a 0710
690 $a 0727
710 2 $a ProQuest Information and Learning Co. $3 1178819
710 2 $a North Carolina State University. $b Computer Science. $3 1182345
773 0 $t Dissertation Abstracts International $g 79-07B(E).
856 4 0 $u http://pqdd.sinica.edu.tw/twdaoapp/servlet/advanced?query=10758969 $z click for full text (PQDT)