國立虎尾科技大學 |

Explorations on High Dimensional Landscapes : = Spin Glasses and Deep Learning.

紀錄類型:	書目-語言資料,手稿 : Monograph/item
正題名/作者:	Explorations on High Dimensional Landscapes :/
其他題名:	Spin Glasses and Deep Learning.
作者:	Sagun, Levent.
面頁冊數:	1 online resource (171 pages)
附註:	Source: Dissertation Abstracts International, Volume: 78-12(E), Section: B.
Contained By:	Dissertation Abstracts International78-12B(E).
標題:	Mathematics. -
電子資源:	click for full text (PQDT)
ISBN:	9780355127867

Explorations on High Dimensional Landscapes : = Spin Glasses and Deep Learning.
Sagun, Levent.

Explorations on High Dimensional Landscapes :Spin Glasses and Deep Learning. - 1 online resource (171 pages)

Source: Dissertation Abstracts International, Volume: 78-12(E), Section: B.

Thesis (Ph.D.)

Includes bibliographical references

This thesis deals with understanding the structure of high-dimensional and non-convex energy landscapes. In particular, its focus is on the optimization of two classes of functions: homogeneous polynomials and loss functions that arise in machine learning.

Electronic reproduction.
Ann Arbor, Mich. :
ProQuest,
2018

Mode of access: World Wide Web

ISBN: 9780355127867Subjects--Topical Terms:

527692
Mathematics.
Index Terms--Genre/Form:

554714
Electronic books.

Explorations on High Dimensional Landscapes : = Spin Glasses and Deep Learning.
LDR:04374ntm a2200409Ki 4500 001 909709
005 20180426091044.5
006 m o u
007 cr mn||||a|a||
008 190606s2017 xx obm 000 0 eng d
020 $a 9780355127867
035 $a (MiAaPQ)AAI10259815
035 $a (MiAaPQ)nyu:12869
035 $a AAI10259815
040 $a MiAaPQ $b eng $c MiAaPQ
099 $a TUL $f hyy $c available through World Wide Web
100 1 $a Sagun, Levent. $3 1180622
245 1 0 $a Explorations on High Dimensional Landscapes : $b Spin Glasses and Deep Learning.
264 0 $c 2017
300 $a 1 online resource (171 pages)
336 $a text $b txt $2 rdacontent
337 $a computer $b c $2 rdamedia
338 $a online resource $b cr $2 rdacarrier
500 $a Source: Dissertation Abstracts International, Volume: 78-12(E), Section: B.
500 $a Advisers: Gerard Benarous; Yann Lecun.
502 $a Thesis (Ph.D.) $c New York University $d 2017.
504 $a Includes bibliographical references
520 $a This thesis deals with understanding the structure of high-dimensional and non-convex energy landscapes. In particular, its focus is on the optimization of two classes of functions: homogeneous polynomials and loss functions that arise in machine learning.
520 $a In the first part, the notion of complexity of a smooth, real-valued function is studied through its critical points. Existing theoretical results predict that certain random functions that are defined on high dimensional domains have a narrow band of values whose pre-image contains the bulk of its critical points. This section provides empirical evidence for convergence of gradient descent to local minima whose energies are near the predicted threshold justifying the existing asymptotic theory. Moreover, it is empirically shown that a similar phenomenon may hold for deep learning loss functions. Furthermore, there is a comparative analysis of gradient descent and its stochastic version showing that in high dimensional regimes the latter is a mere speedup.
520 $a The next study focuses on the halting time of an algorithm at a given stopping condition. Given an algorithm, the normalized fluctuations of the halting time follow a distribution that remains unchanged even when the input data is sampled from a new distribution. Two qualitative classes are observed: a Gumbel-like distribution that appears in Google searches, human decision times, and spin glasses and a Gaussian-like distribution that appears in conjugate gradient method, deep learning with MNIST and random input data.
520 $a Following the universality phenomenon, the Hessian of the loss functions of deep learning is studied. The spectrum is seen to be composed of two parts, the bulk which is concentrated around zero, and the edges which are scattered away from zero. Empirical evidence is presented for the bulk indicating how over-parametrized the system is, and for the edges that depend on the input data. Furthermore, an algorithm is proposed such that it would explore such large dimensional, degenerate landscapes to locate a solution with decent generalization properties. Finally, a demonstration of how the new method can explain the empirical success of some of the recent methods that have been proposed for distributed deep learning.
520 $a In the second part, two applied machine learning problems are studied that are complementary to the machine learning problems of part I. First, US asylum applications cases are studied using random forests on the data of past twenty years. Using only features up to when the case opens, the algorithm can predict the outcome of the case with 80% accuracy. Next, a particular question and answer system has been studied. The questions are collected from Jeopardy! show and they fed to Google, then the results are parsed into a recurrent neural network to come up with a system that would outcome the answer to the original question. Close to 50% accuracy is achieved where human level benchmark is just a little above 60%.
533 $a Electronic reproduction. $b Ann Arbor, Mich. : $c ProQuest, $d 2018
538 $a Mode of access: World Wide Web
650 4 $a Mathematics. $3 527692
650 4 $a Computer science. $3 573171
650 4 $a Physics. $3 564049
655 7 $a Electronic books. $2 local $3 554714
690 $a 0405
690 $a 0984
690 $a 0605
710 2 $a ProQuest Information and Learning Co. $3 1178819
710 2 $a New York University. $b Mathematics. $3 1180487
773 0 $t Dissertation Abstracts International $g 78-12B(E).
856 4 0 $u http://pqdd.sinica.edu.tw/twdaoapp/servlet/advanced?query=10259815 $z click for full text (PQDT)