國立虎尾科技大學 |

Interactive IR user study design, evaluation, and reporting /

Record Type:	Language materials, printed : Monograph/item
Title/Author:	Interactive IR user study design, evaluation, and reporting // Jiqun Liu and Chirag Shah.
remainder title:	Interactive information retrieval user study design, evaluation, and reporting.
Author:	Liu, Jiqun,
other author:	Shah, Chirag,
Description:	1 PDF (xvii, 75 pages) :color illustrations. :
Notes:	Part of: Synthesis digital library of engineering and computer science.
Subject:	Information storage and retrieval systems - Use studies. -
Online resource:	https://doi.org/10.2200/S00923ED1V01Y201905ICR067
Online resource:	https://ieeexplore.ieee.org/servlet/opac?bknumber=8732801
ISBN:	9781681735801

Interactive IR user study design, evaluation, and reporting /
Liu, Jiqun,

Interactive IR user study design, evaluation, and reporting /Interactive information retrieval user study design, evaluation, and reporting.Jiqun Liu and Chirag Shah. - 1 PDF (xvii, 75 pages) :color illustrations. - Synthesis lectures on information concepts, retrieval, and services,#671947-9468 ;. - Synthesis digital library of engineering and computer science..

Part of: Synthesis digital library of engineering and computer science.

Includes bibliographical references (pages 63-74).

1. Introduction -- 1.1. Background : user study in IIR research -- 1.2. Aim of the work -- 1.3. Systematic review and user study evaluation -- 1.4. Implications of the faceted framework

Abstract freely available; full-text restricted to subscribers or individual document purchasers.

Compendex

Since user study design has been widely applied in search interactions and information retrieval (IR) systems evaluation studies, a deep reflection and meta-evaluation of interactive IR (IIR) user studies is critical for sharpening the instruments of IIR research and improving the reliability and validity of the conclusions drawn from IIR user studies. To this end, we developed a faceted framework for supporting user study design, reporting, and evaluation based on a systematic review of the state-of-the-art IIR research papers recently published in several top IR venues (n=462). Within the framework, we identify three major types of research focuses, extract and summarize facet values from specific cases, and highlight the under-reported user study components which may significantly affect the results of research. Then, we employ the faceted framework in evaluating a series of IIR user studies against their respective research questions and explain the roles and impacts of the underlying connections and "collaborations" among different facet values. Through bridging diverse combinations of facet values with the study design decisions made for addressing research problems, the faceted framework can shed light on IIR user study design, reporting, and evaluation practices and help students and young researchers design and assess their own studies.

Mode of access: World Wide Web.

ISBN: 9781681735801

Standard No.: 10.2200/S00923ED1V01Y201905ICR067doiSubjects--Topical Terms:

1253108
Information storage and retrieval systems
--Use studies.Subjects--Index Terms:

interactive information retrieval

LC Class. No.: Z667 / .L585 2019eb

Dewey Class. No.: 025.04

Interactive IR user study design, evaluation, and reporting /
LDR:04983nam 2200661 i 4500 001 959765
003 IEEE
005 20190702190106.0
006 m eo d
007 cr cn |||m|||a
008 201209s2019 caua fob 000 0 eng d
020 $a 9781681735801 $q electronic
020 $z 9781681735818 $q hardcover
020 $z 9781681735795 $q paperback
024 7 $a 10.2200/S00923ED1V01Y201905ICR067 $2 doi
035 $a (CaBNVSL)thg00979209
035 $a (OCoLC)1104725763
035 $a 8732801
040 $a CaBNVSL $b eng $e rda $c CaBNVSL $d CaBNVSL
050 4 $a Z667 $b .L585 2019eb
082 0 4 $a 025.04 $2 23
100 1 $a Liu, Jiqun, $e author. $3 1253106
245 1 0 $a Interactive IR user study design, evaluation, and reporting / $c Jiqun Liu and Chirag Shah.
246 3 $a Interactive information retrieval user study design, evaluation, and reporting.
264 1 $a [San Rafael, California] : $b Morgan & Claypool, $c 2019.
300 $a 1 PDF (xvii, 75 pages) : $b color illustrations.
336 $a text $2 rdacontent
337 $a electronic $2 isbdmedia
338 $a online resource $2 rdacarrier
490 1 $a Synthesis lectures on information concepts, retrieval, and services, $x 1947-9468 ; $v #67
500 $a Part of: Synthesis digital library of engineering and computer science.
504 $a Includes bibliographical references (pages 63-74).
505 0 $a 1. Introduction -- 1.1. Background : user study in IIR research -- 1.2. Aim of the work -- 1.3. Systematic review and user study evaluation -- 1.4. Implications of the faceted framework
505 8 $a 2. Interactive information retrieval -- 2.1. Background -- 2.2. Understanding user behavior and experience -- 2.3. System/interface features evaluation -- 2.4. Meta-evaluation of evaluation metrics -- 2.5. Summary
505 8 $a 3. Methodology : paper selection and coding scheme -- 3.1. Paper selection : SIGIR, CHIIR, CHI, JASIST, IP&M, and TOIS -- 3.2. Coding scheme
505 8 $a 4. Faceted framework of IIR user studies -- 4.1. Structure of the faceted framework -- 4.2. Under-reported facets and factors -- 4.3. Summary
505 8 $a 5. Evaluating IIR user studies of different types -- 5.1. Understanding user behavior and experience -- 5.2. System/interface features evaluation -- 5.3. Meta-evaluation of evaluation measures -- 5.4. Summary
505 8 $a 6. Implications and limitations of the faceted framework -- 6.1. Implications for interaction IR user study design and evaluation -- 6.2. Implications for IIR user study reporting practices -- 6.3. Limitations and challenges -- 7. Conclusion and future directions.
506 $a Abstract freely available; full-text restricted to subscribers or individual document purchasers.
510 0 $a Compendex
510 0 $a INSPEC
510 0 $a Google scholar
510 0 $a Google book search
520 $a Since user study design has been widely applied in search interactions and information retrieval (IR) systems evaluation studies, a deep reflection and meta-evaluation of interactive IR (IIR) user studies is critical for sharpening the instruments of IIR research and improving the reliability and validity of the conclusions drawn from IIR user studies. To this end, we developed a faceted framework for supporting user study design, reporting, and evaluation based on a systematic review of the state-of-the-art IIR research papers recently published in several top IR venues (n=462). Within the framework, we identify three major types of research focuses, extract and summarize facet values from specific cases, and highlight the under-reported user study components which may significantly affect the results of research. Then, we employ the faceted framework in evaluating a series of IIR user studies against their respective research questions and explain the roles and impacts of the underlying connections and "collaborations" among different facet values. Through bridging diverse combinations of facet values with the study design decisions made for addressing research problems, the faceted framework can shed light on IIR user study design, reporting, and evaluation practices and help students and young researchers design and assess their own studies.
530 $a Also available in print.
538 $a Mode of access: World Wide Web.
538 $a System requirements: Adobe Acrobat Reader.
588 $a Title from PDF title page (viewed on June 26, 2019).
650 0 $a Information storage and retrieval systems $x Use studies. $3 1253108
650 0 $a Information services $x Use studies. $3 804663
650 0 $a Research $x Evaluation. $3 661102
653 $a interactive information retrieval
653 $a user studies
653 $a faceted approach
653 $a evaluation
653 $a controlled laboratory study
653 $a field study
653 $a human-information interaction
653 $a information seeking and retrieval
653 $a user study reporting practice
700 1 $a Shah, Chirag, $e author. $3 1253107
776 0 8 $i Print version: $z 9781681735795 $z 9781681735818
830 0 $a Synthesis digital library of engineering and computer science. $3 598254
830 0 $a Synthesis lectures on information concepts, retrieval, and services ; $v #22. $3 866817
856 4 0 $3 Abstract with links to full text $u https://doi.org/10.2200/S00923ED1V01Y201905ICR067
856 4 2 $3 Abstract with links to resource $u https://ieeexplore.ieee.org/servlet/opac?bknumber=8732801