國立虎尾科技大學 |

Minimum Divergence Methods in Statistical Machine Learning = From an Information Geometric Viewpoint /

紀錄類型:	書目-語言資料,印刷品 : Monograph/item
正題名/作者:	Minimum Divergence Methods in Statistical Machine Learning/ by Shinto Eguchi, Osamu Komori.
其他題名:	From an Information Geometric Viewpoint /
作者:	Eguchi, Shinto.
其他作者:	Komori, Osamu.
面頁冊數:	X, 221 p. 18 illus., 15 illus. in color.online resource. :
Contained By:	Springer Nature eBook
標題:	Statistics . -
電子資源:	https://doi.org/10.1007/978-4-431-56922-0
ISBN:	9784431569220

Minimum Divergence Methods in Statistical Machine Learning = From an Information Geometric Viewpoint /
Eguchi, Shinto.

Minimum Divergence Methods in Statistical Machine LearningFrom an Information Geometric Viewpoint /[electronic resource] :by Shinto Eguchi, Osamu Komori. - 1st ed. 2022. - X, 221 p. 18 illus., 15 illus. in color.online resource.

Information geometry -- Information divergence -- Maximum entropy model -- Minimum divergence method -- Unsupervised learning algorithms -- Regression model -- Classification. .

This book explores minimum divergence methods of statistical machine learning for estimation, regression, prediction, and so forth, in which we engage in information geometry to elucidate their intrinsic properties of the corresponding loss functions, learning algorithms, and statistical models. One of the most elementary examples is Gauss's least squares estimator in a linear regression model, in which the estimator is given by minimization of the sum of squares between a response vector and a vector of the linear subspace hulled by explanatory vectors. This is extended to Fisher's maximum likelihood estimator (MLE) for an exponential model, in which the estimator is provided by minimization of the Kullback-Leibler (KL) divergence between a data distribution and a parametric distribution of the exponential model in an empirical analogue. Thus, we envisage a geometric interpretation of such minimization procedures such that a right triangle is kept with Pythagorean identity in the sense of the KL divergence. This understanding sublimates a dualistic interplay between a statistical estimation and model, which requires dual geodesic paths, called m-geodesic and e-geodesic paths, in a framework of information geometry. We extend such a dualistic structure of the MLE and exponential model to that of the minimum divergence estimator and the maximum entropy model, which is applied to robust statistics, maximum entropy, density estimation, principal component analysis, independent component analysis, regression analysis, manifold learning, boosting algorithm, clustering, dynamic treatment regimes, and so forth. We consider a variety of information divergence measures typically including KL divergence to express departure from one probability distribution to another. An information divergence is decomposed into the cross-entropy and the (diagonal) entropy in which the entropy associates with a generative model as a family of maximum entropy distributions; the cross entropy associates with a statistical estimation method via minimization of the empirical analogue based on given data. Thus any statistical divergence includes an intrinsic object between the generative model and the estimation method. Typically, KL divergence leads to the exponential model and the maximum likelihood estimation. It is shown that any information divergence leads to a Riemannian metric and a pair of the linear connections in the framework of information geometry. We focus on a class of information divergence generated by an increasing and convex function U, called U-divergence. It is shown that any generator function U generates the U-entropy and U-divergence, in which there is a dualistic structure between the U-divergence method and the maximum U-entropy model. We observe that a specific choice of U leads to a robust statistical procedure via the minimum U-divergence method. If U is selected as an exponential function, then the corresponding U-entropy and U-divergence are reduced to the Boltzmann-Shanon entropy and the KL divergence; the minimum U-divergence estimator is equivalent to the MLE. For robust supervised learning to predict a class label we observe that the U-boosting algorithm performs well for contamination of mislabel examples if U is appropriately selected. We present such maximal U-entropy and minimum U-divergence methods, in particular, selecting a power function as U to provide flexible performance in statistical machine learning. .

ISBN: 9784431569220

Standard No.: 10.1007/978-4-431-56922-0doiSubjects--Topical Terms:

1253516
Statistics .

LC Class. No.: QA276-280

Dewey Class. No.: 519

Minimum Divergence Methods in Statistical Machine Learning = From an Information Geometric Viewpoint /
LDR:05013nam a22003855i 4500 001 1090446
003 DE-He213
005 20220314204554.0
007 cr nn 008mamaa
008 221228s2022 ja | s |||| 0|eng d
020 $a 9784431569220 $9 978-4-431-56922-0
024 7 $a 10.1007/978-4-431-56922-0 $2 doi
035 $a 978-4-431-56922-0
050 4 $a QA276-280
072 7 $a PBT $2 bicssc
072 7 $a MAT029000 $2 bisacsh
072 7 $a PBT $2 thema
082 0 4 $a 519 $2 23
100 1 $a Eguchi, Shinto. $e author. $4 aut $4 http://id.loc.gov/vocabulary/relators/aut $3 1308401
245 1 0 $a Minimum Divergence Methods in Statistical Machine Learning $h [electronic resource] : $b From an Information Geometric Viewpoint / $c by Shinto Eguchi, Osamu Komori.
250 $a 1st ed. 2022.
264 1 $a Tokyo : $b Springer Japan : $b Imprint: Springer, $c 2022.
300 $a X, 221 p. 18 illus., 15 illus. in color. $b online resource.
336 $a text $b txt $2 rdacontent
337 $a computer $b c $2 rdamedia
338 $a online resource $b cr $2 rdacarrier
347 $a text file $b PDF $2 rda
505 0 $a Information geometry -- Information divergence -- Maximum entropy model -- Minimum divergence method -- Unsupervised learning algorithms -- Regression model -- Classification. .
520 $a This book explores minimum divergence methods of statistical machine learning for estimation, regression, prediction, and so forth, in which we engage in information geometry to elucidate their intrinsic properties of the corresponding loss functions, learning algorithms, and statistical models. One of the most elementary examples is Gauss's least squares estimator in a linear regression model, in which the estimator is given by minimization of the sum of squares between a response vector and a vector of the linear subspace hulled by explanatory vectors. This is extended to Fisher's maximum likelihood estimator (MLE) for an exponential model, in which the estimator is provided by minimization of the Kullback-Leibler (KL) divergence between a data distribution and a parametric distribution of the exponential model in an empirical analogue. Thus, we envisage a geometric interpretation of such minimization procedures such that a right triangle is kept with Pythagorean identity in the sense of the KL divergence. This understanding sublimates a dualistic interplay between a statistical estimation and model, which requires dual geodesic paths, called m-geodesic and e-geodesic paths, in a framework of information geometry. We extend such a dualistic structure of the MLE and exponential model to that of the minimum divergence estimator and the maximum entropy model, which is applied to robust statistics, maximum entropy, density estimation, principal component analysis, independent component analysis, regression analysis, manifold learning, boosting algorithm, clustering, dynamic treatment regimes, and so forth. We consider a variety of information divergence measures typically including KL divergence to express departure from one probability distribution to another. An information divergence is decomposed into the cross-entropy and the (diagonal) entropy in which the entropy associates with a generative model as a family of maximum entropy distributions; the cross entropy associates with a statistical estimation method via minimization of the empirical analogue based on given data. Thus any statistical divergence includes an intrinsic object between the generative model and the estimation method. Typically, KL divergence leads to the exponential model and the maximum likelihood estimation. It is shown that any information divergence leads to a Riemannian metric and a pair of the linear connections in the framework of information geometry. We focus on a class of information divergence generated by an increasing and convex function U, called U-divergence. It is shown that any generator function U generates the U-entropy and U-divergence, in which there is a dualistic structure between the U-divergence method and the maximum U-entropy model. We observe that a specific choice of U leads to a robust statistical procedure via the minimum U-divergence method. If U is selected as an exponential function, then the corresponding U-entropy and U-divergence are reduced to the Boltzmann-Shanon entropy and the KL divergence; the minimum U-divergence estimator is equivalent to the MLE. For robust supervised learning to predict a class label we observe that the U-boosting algorithm performs well for contamination of mislabel examples if U is appropriately selected. We present such maximal U-entropy and minimum U-divergence methods, in particular, selecting a power function as U to provide flexible performance in statistical machine learning. .
650 0 $a Statistics . $3 1253516
650 0 $a Computer science—Mathematics. $3 1253519
650 0 $a Mathematical statistics. $3 527941
650 1 4 $a Statistics in Engineering, Physics, Computer Science, Chemistry and Earth Sciences. $3 1366002
650 2 4 $a Statistical Theory and Methods. $3 671396
650 2 4 $a Probability and Statistics in Computer Science. $3 669886
700 1 $a Komori, Osamu. $e author. $4 aut $4 http://id.loc.gov/vocabulary/relators/aut $3 1308400
710 2 $a SpringerLink (Online service) $3 593884
773 0 $t Springer Nature eBook
776 0 8 $i Printed edition: $z 9784431569206
776 0 8 $i Printed edition: $z 9784431569213
856 4 0 $u https://doi.org/10.1007/978-4-431-56922-0
912 $a ZDB-2-SMA
912 $a ZDB-2-SXMS
950 $a Mathematics and Statistics (SpringerNature-11649)
950 $a Mathematics and Statistics (R0) (SpringerNature-43713)