國立虎尾科技大學 |

The Roads Not Taken : = Model Multiplicity in Machine Learning.

Record Type:	Language materials, manuscript : Monograph/item
Title/Author:	The Roads Not Taken :/
Reminder of title:	Model Multiplicity in Machine Learning.
Author:	Watson-Daniels, Jamelle D.
Description:	1 online resource (138 pages)
Notes:	Source: Dissertations Abstracts International, Volume: 85-12, Section: B.
Contained By:	Dissertations Abstracts International85-12B.
Subject:	Computer science. -
Online resource:	click for full text (PQDT)
ISBN:	9798382786476

The Roads Not Taken : = Model Multiplicity in Machine Learning.
Watson-Daniels, Jamelle D.

The Roads Not Taken :Model Multiplicity in Machine Learning. - 1 online resource (138 pages)

Source: Dissertations Abstracts International, Volume: 85-12, Section: B.

Thesis (Ph.D.)--Harvard University, 2024.

Includes bibliographical references

In machine learning, model multiplicity is the existence of multiple models that perform equally well for a given prediction task (also known as the "Rashomon effect" ). The set of near-optimal models is referred to as the "Rashomon set." Predictive multiplicity examines how predictions change over this set of near-optimal models. If model outputs vary significantly across similar models, this information can offer insight into predictive arbitrariness. In this thesis, I introduce frameworks for evaluating and leveraging predictive multiplicity in different settings.First, I present methods to measure predictive multiplicity in probabilistic classification (predicting the probability of a positive outcome) and develop optimization-based methods to compute these measures efficiently and reliably for convex empirical risk minimization problems. Empirical results show that real-world probabilistic classification tasks can in fact admit competing models that assign substantially different risk estimates. Additionally, I provide insight into how predictive multiplicity arises by analyzing dataset characteristics.Second, I formulate predictive multiplicity analysis in a resource constrained setting recognizing that predictive allocation tasks are governed by a resource budget. I also extend the multiplicity framing, outlining the concept of multi-target multiplicity for quantifying the impact of choices made in regard to target specification for a given predictive allocation task. With this framework, I demonstrate how to fit separate models that are useful for predicting the three outcomes of interest independently and arriving at a way of ranking patients that results in a more equitable allocation.Third, I investigate the connections between predictive multiplicity and predictive churn which is the change in predictions pre- and post- model update in response to a change in training data. I present empirical and theoretical results on characterizing churn in terms of the Rashomon set. Results show that churn unstable points overlap by more than 50 percent with ambiguity points. This points to similarities in the two concepts. Theoretical results to characterize predictive churn between two Rashomon sets as well as churn between models within one Rashomon set hinges on the type of Rashomon set.I focus on predictive multiplicity to advocate for transparency in the prediction model training procedure. These methods to evaluate predictive multiplicity, as well as connections with predictive churn, contribute to a larger effort for machine learning researchers to be accountable to the individuals affected by model predictions. Similar to a person deciding between roads to take while travelling, insight into alternative options (i.e., roads not taken) may provide insight into the significance of the decisions made.

Electronic reproduction.
Ann Arbor, Mich. :
ProQuest,
2024

Mode of access: World Wide Web

ISBN: 9798382786476Subjects--Topical Terms:

573171
Computer science.
Subjects--Index Terms:

FairnessIndex Terms--Genre/Form:

554714
Electronic books.

The Roads Not Taken : = Model Multiplicity in Machine Learning.
LDR:04187ntm a22003857 4500 001 1148250
005 20240916070038.5
006 m o d
007 cr bn ---uuuuu
008 250605s2024 xx obm 000 0 eng d
020 $a 9798382786476
035 $a (MiAaPQ)AAI31294605
035 $a AAI31294605
040 $a MiAaPQ $b eng $c MiAaPQ $d NTU
100 1 $a Watson-Daniels, Jamelle D. $3 1474178
245 1 4 $a The Roads Not Taken : $b Model Multiplicity in Machine Learning.
264 0 $c 2024
300 $a 1 online resource (138 pages)
336 $a text $b txt $2 rdacontent
337 $a computer $b c $2 rdamedia
338 $a online resource $b cr $2 rdacarrier
500 $a Source: Dissertations Abstracts International, Volume: 85-12, Section: B.
500 $a Advisor: Parkes, David C.
502 $a Thesis (Ph.D.)--Harvard University, 2024.
504 $a Includes bibliographical references
520 $a In machine learning, model multiplicity is the existence of multiple models that perform equally well for a given prediction task (also known as the "Rashomon effect" ). The set of near-optimal models is referred to as the "Rashomon set." Predictive multiplicity examines how predictions change over this set of near-optimal models. If model outputs vary significantly across similar models, this information can offer insight into predictive arbitrariness. In this thesis, I introduce frameworks for evaluating and leveraging predictive multiplicity in different settings.First, I present methods to measure predictive multiplicity in probabilistic classification (predicting the probability of a positive outcome) and develop optimization-based methods to compute these measures efficiently and reliably for convex empirical risk minimization problems. Empirical results show that real-world probabilistic classification tasks can in fact admit competing models that assign substantially different risk estimates. Additionally, I provide insight into how predictive multiplicity arises by analyzing dataset characteristics.Second, I formulate predictive multiplicity analysis in a resource constrained setting recognizing that predictive allocation tasks are governed by a resource budget. I also extend the multiplicity framing, outlining the concept of multi-target multiplicity for quantifying the impact of choices made in regard to target specification for a given predictive allocation task. With this framework, I demonstrate how to fit separate models that are useful for predicting the three outcomes of interest independently and arriving at a way of ranking patients that results in a more equitable allocation.Third, I investigate the connections between predictive multiplicity and predictive churn which is the change in predictions pre- and post- model update in response to a change in training data. I present empirical and theoretical results on characterizing churn in terms of the Rashomon set. Results show that churn unstable points overlap by more than 50 percent with ambiguity points. This points to similarities in the two concepts. Theoretical results to characterize predictive churn between two Rashomon sets as well as churn between models within one Rashomon set hinges on the type of Rashomon set.I focus on predictive multiplicity to advocate for transparency in the prediction model training procedure. These methods to evaluate predictive multiplicity, as well as connections with predictive churn, contribute to a larger effort for machine learning researchers to be accountable to the individuals affected by model predictions. Similar to a person deciding between roads to take while travelling, insight into alternative options (i.e., roads not taken) may provide insight into the significance of the decisions made.
533 $a Electronic reproduction. $b Ann Arbor, Mich. : $c ProQuest, $d 2024
538 $a Mode of access: World Wide Web
650 4 $a Computer science. $3 573171
650 4 $a Applied mathematics. $3 1069907
653 $a Fairness
653 $a Machine learning
653 $a Predictive multiplicity
653 $a Rashomon set
653 $a Rashomon effect
655 7 $a Electronic books. $2 local $3 554714
690 $a 0984
690 $a 0364
710 2 $a ProQuest Information and Learning Co. $3 1178819
710 2 $a Harvard University. $b Engineering and Applied Sciences - Applied Math. $3 1474179
773 0 $t Dissertations Abstracts International $g 85-12B.
856 4 0 $u http://pqdd.sinica.edu.tw/twdaoapp/servlet/advanced?query=31294605 $z click for full text (PQDT)