國立虎尾科技大學 |

Entropic Regularization in Wasserstein Gans : = Robustness, Generalization and Privacy.

Record Type:	Language materials, manuscript : Monograph/item
Title/Author:	Entropic Regularization in Wasserstein Gans :/
Reminder of title:	Robustness, Generalization and Privacy.
Author:	Reshetova, Daria.
Description:	1 online resource (124 pages)
Notes:	Source: Dissertations Abstracts International, Volume: 85-06, Section: B.
Contained By:	Dissertations Abstracts International85-06B.
Subject:	Mathematical functions. -
Online resource:	click for full text (PQDT)
ISBN:	9798381019605

Entropic Regularization in Wasserstein Gans : = Robustness, Generalization and Privacy.
Reshetova, Daria.

Entropic Regularization in Wasserstein Gans :Robustness, Generalization and Privacy. - 1 online resource (124 pages)

Source: Dissertations Abstracts International, Volume: 85-06, Section: B.

Thesis (Ph.D.)--Stanford University, 2023.

Includes bibliographical references

In recent years, GANs have emerged as a powerful method for learning distributions from data by modeling the target distribution as a function of a known distribution. The function, often referred to as the generator, is optimized to minimize a chosen distance measure between the generated and target distributions. One commonly used measure for this purpose is the so-called Wasserstein distance. However, Wasserstein distance is hard to compute and optimize, and in practice, entropic regularization techniques are used to facilitate its computation and improve numerical convergence. Introducing regularization, however, changes the problem we are trying to solve, and hence the learned solution. While the computational advantages of entropic regularization have been well-established in the recent literature, the influence of the regularization on the learned generative model and the distribution it generates has remained poorly understood. In this thesis, we study the consequences of regularizing Wasserstein GANs with entropic regularization and show three important impacts of the regularization: 1) we shed light on how entropic regularization impacts the learned GAN solution; 2) we show that it improves sample complexity by removing the curse of dimensionality; 3) we show that entropic regularization can be used to e↵ectively train GANs from di↵erentially privatized data.First, we mathematically derive the learned distribution of the GAN to show the e↵ects of the regularization in a benchmark setting. Prior works focus primarily on evaluating GANs on real data, typically images, and although clearly valuable, such evaluations are often subjective due to a lack of clear baselines for benchmarking. The target distribution of images is too complex to get a precise form of the generated distribution even in the population setting. However, in a benchmark setting where the generator is linear and the target distribution is high-dimensional Gaussian, we are able to characterize the learned solution and prove that regularization leads to the robustness of the learned distribution as well as feature selection. We additionally show that debiasing the Entropy-regularized Wasserstein GAN with the autocorrective term (using Sinkhorn divergence as a loss function) leads to an unbiased solution.Second, we study the e↵ect of regularization on the sample complexity. Without regularization it is known that Wasserstein GANs su↵er from the curse of dimensionality, the number of samples needed for the empirical solution to approximate the population solution with some given error scales exponentially in the dimension of the problem. This dissertation shows that entropic regularization can resolve the curse of dimensionality enabling convergence at the parametric rate for a large class of generators and distributions, namely Lipschitz generators and sub-gaussian real distributions. These conditions are commonly satisfied in practice (for example, for images and sigmoid or tanh activations in the last layer of the generator). We present a theorem that quantifies the benefits of using Entropy-regularized Wasserstein Distance as a loss function. Our findings indicate that this regularization technique can effectively mitigate the challenges posed by high-dimensional data, offering a more robust and ecient learning process.Finally, we show that entropic regularization can extend the application of GANs to the realm of sensitive data privatized with differential privacy.

Electronic reproduction.
Ann Arbor, Mich. :
ProQuest,
2024

Mode of access: World Wide Web

ISBN: 9798381019605Subjects--Topical Terms:

1474744
Mathematical functions.
Index Terms--Genre/Form:

554714
Electronic books.

Entropic Regularization in Wasserstein Gans : = Robustness, Generalization and Privacy.
LDR:04786ntm a22003377 4500 001 1148751
005 20240930100136.5
006 m o d
007 cr bn ---uuuuu
008 250605s2023 xx obm 000 0 eng d
020 $a 9798381019605
035 $a (MiAaPQ)AAI30726895
035 $a (MiAaPQ)STANFORDbm099mt8200
035 $a AAI30726895
040 $a MiAaPQ $b eng $c MiAaPQ $d NTU
100 1 $a Reshetova, Daria. $3 1474795
245 1 0 $a Entropic Regularization in Wasserstein Gans : $b Robustness, Generalization and Privacy.
264 0 $c 2023
300 $a 1 online resource (124 pages)
336 $a text $b txt $2 rdacontent
337 $a computer $b c $2 rdamedia
338 $a online resource $b cr $2 rdacarrier
500 $a Source: Dissertations Abstracts International, Volume: 85-06, Section: B.
500 $a Advisor: Ozgur, Ayfer;Gamal, Abbas El;Weissman, Tsachy.
502 $a Thesis (Ph.D.)--Stanford University, 2023.
504 $a Includes bibliographical references
520 $a In recent years, GANs have emerged as a powerful method for learning distributions from data by modeling the target distribution as a function of a known distribution. The function, often referred to as the generator, is optimized to minimize a chosen distance measure between the generated and target distributions. One commonly used measure for this purpose is the so-called Wasserstein distance. However, Wasserstein distance is hard to compute and optimize, and in practice, entropic regularization techniques are used to facilitate its computation and improve numerical convergence. Introducing regularization, however, changes the problem we are trying to solve, and hence the learned solution. While the computational advantages of entropic regularization have been well-established in the recent literature, the influence of the regularization on the learned generative model and the distribution it generates has remained poorly understood. In this thesis, we study the consequences of regularizing Wasserstein GANs with entropic regularization and show three important impacts of the regularization: 1) we shed light on how entropic regularization impacts the learned GAN solution; 2) we show that it improves sample complexity by removing the curse of dimensionality; 3) we show that entropic regularization can be used to e↵ectively train GANs from di↵erentially privatized data.First, we mathematically derive the learned distribution of the GAN to show the e↵ects of the regularization in a benchmark setting. Prior works focus primarily on evaluating GANs on real data, typically images, and although clearly valuable, such evaluations are often subjective due to a lack of clear baselines for benchmarking. The target distribution of images is too complex to get a precise form of the generated distribution even in the population setting. However, in a benchmark setting where the generator is linear and the target distribution is high-dimensional Gaussian, we are able to characterize the learned solution and prove that regularization leads to the robustness of the learned distribution as well as feature selection. We additionally show that debiasing the Entropy-regularized Wasserstein GAN with the autocorrective term (using Sinkhorn divergence as a loss function) leads to an unbiased solution.Second, we study the e↵ect of regularization on the sample complexity. Without regularization it is known that Wasserstein GANs su↵er from the curse of dimensionality, the number of samples needed for the empirical solution to approximate the population solution with some given error scales exponentially in the dimension of the problem. This dissertation shows that entropic regularization can resolve the curse of dimensionality enabling convergence at the parametric rate for a large class of generators and distributions, namely Lipschitz generators and sub-gaussian real distributions. These conditions are commonly satisfied in practice (for example, for images and sigmoid or tanh activations in the last layer of the generator). We present a theorem that quantifies the benefits of using Entropy-regularized Wasserstein Distance as a loss function. Our findings indicate that this regularization technique can effectively mitigate the challenges posed by high-dimensional data, offering a more robust and ecient learning process.Finally, we show that entropic regularization can extend the application of GANs to the realm of sensitive data privatized with differential privacy.
533 $a Electronic reproduction. $b Ann Arbor, Mich. : $c ProQuest, $d 2024
538 $a Mode of access: World Wide Web
650 4 $a Mathematical functions. $3 1474744
650 4 $a Linear programming. $3 528157
650 4 $a Mathematicians. $3 890815
650 4 $a Privacy. $3 575491
650 4 $a Entropy. $3 596653
650 4 $a Neural networks. $3 1011215
650 4 $a Mathematics. $3 527692
655 7 $a Electronic books. $2 local $3 554714
690 $a 0800
690 $a 0405
710 2 $a ProQuest Information and Learning Co. $3 1178819
710 2 $a Stanford University. $3 1184533
773 0 $t Dissertations Abstracts International $g 85-06B.
856 4 0 $u http://pqdd.sinica.edu.tw/twdaoapp/servlet/advanced?query=30726895 $z click for full text (PQDT)