國立虎尾科技大學 |

Inspecting and Directing Neural Language Models.

Record Type:	Language materials, manuscript : Monograph/item
Title/Author:	Inspecting and Directing Neural Language Models./
Author:	Noraset, Thanapon.
Description:	1 online resource (92 pages)
Notes:	Source: Dissertation Abstracts International, Volume: 79-11(E), Section: B.
Contained By:	Dissertation Abstracts International79-11B(E).
Subject:	Computer science. -
Online resource:	click for full text (PQDT)
ISBN:	9780438116856

Inspecting and Directing Neural Language Models.
Noraset, Thanapon.

Inspecting and Directing Neural Language Models. - 1 online resource (92 pages)

Source: Dissertation Abstracts International, Volume: 79-11(E), Section: B.

Thesis (Ph.D.)--Northwestern University, 2018.

Includes bibliographical references

The ability of a machine to synthesize textual output in a form of human language is a long-standing goal in a field of artificial intelligence and has wide-range of applications such as spell correction, speech recognition, machine translation, abstractive summarization, etc. The statistical approach to enable such ability mainly involves defining representations of textual inputs and computation of likelihood of the outputs text sequence. With the recent advancement in a neural network or deep learning research, machine learning models can construct general vector representations of words, also known as word embeddings, that are useful for many natural language processing tasks. Furthermore, neural language models can accurately assign a probability to a sequence of text, and become a default choice of researchers and developers. One of the key advantages of these deep learning models is that they require little to none human heuristics in feature engineering, and instead, are optimized using a large amount of data.

Electronic reproduction.
Ann Arbor, Mich. :
ProQuest,
2018

Mode of access: World Wide Web

ISBN: 9780438116856Subjects--Topical Terms:

573171
Computer science.
Index Terms--Genre/Form:

554714
Electronic books.

Inspecting and Directing Neural Language Models.
LDR:03712ntm a2200373Ki 4500 001 916915
005 20180928111503.5
006 m o u
007 cr mn||||a|a||
008 190606s2018 xx obm 000 0 eng d
020 $a 9780438116856
035 $a (MiAaPQ)AAI10816819
035 $a (MiAaPQ)northwestern:14142
035 $a AAI10816819
040 $a MiAaPQ $b eng $c MiAaPQ $d NTU
100 1 $a Noraset, Thanapon. $3 1190785
245 1 0 $a Inspecting and Directing Neural Language Models.
264 0 $c 2018
300 $a 1 online resource (92 pages)
336 $a text $b txt $2 rdacontent
337 $a computer $b c $2 rdamedia
338 $a online resource $b cr $2 rdacarrier
500 $a Source: Dissertation Abstracts International, Volume: 79-11(E), Section: B.
500 $a Adviser: Douglas C. Downey.
502 $a Thesis (Ph.D.)--Northwestern University, 2018.
504 $a Includes bibliographical references
520 $a The ability of a machine to synthesize textual output in a form of human language is a long-standing goal in a field of artificial intelligence and has wide-range of applications such as spell correction, speech recognition, machine translation, abstractive summarization, etc. The statistical approach to enable such ability mainly involves defining representations of textual inputs and computation of likelihood of the outputs text sequence. With the recent advancement in a neural network or deep learning research, machine learning models can construct general vector representations of words, also known as word embeddings, that are useful for many natural language processing tasks. Furthermore, neural language models can accurately assign a probability to a sequence of text, and become a default choice of researchers and developers. One of the key advantages of these deep learning models is that they require little to none human heuristics in feature engineering, and instead, are optimized using a large amount of data.
520 $a Despite the performance improvement and convenience of the neural language models, they lack a useful property found in the previous statistical model such as an n-gram language model---the parameters of neural language models are not directly interpretable. This leads to a set of challenges when using such models. For instance, the word embeddings do not directly convey what information is captured, hence which semantic gaps exist in the embeddings is uncertain. Furthermore, the likelihoods of a sequence of text are defined by a series of non-linear functions, making the behavior of the models, even for a short phrase, difficult to ascertain or change to the desired direction.
520 $a This dissertation addresses these two shortcomings by exploiting the models' ability to generate interpretable text. First, we propose a more transparent view of the information captured by a word embedding, and introduce the Definition Modeling, the task of generating a definition for a given word and its embedding. Second, we study an explicit approach to adjust the model's behavior, and present Dynamic KL Regularization, a method for training neural language models to follow a given set of statistical constraints. Finally, we explore efficient solutions to a fundamental, yet lacking the capability of the start-of-the-art neural language model: accurately computing a likelihood of a short phrase that it will produce.
533 $a Electronic reproduction. $b Ann Arbor, Mich. : $c ProQuest, $d 2018
538 $a Mode of access: World Wide Web
650 4 $a Computer science. $3 573171
650 4 $a Artificial intelligence. $3 559380
650 4 $a Language. $3 571568
655 7 $a Electronic books. $2 local $3 554714
690 $a 0984
690 $a 0800
690 $a 0679
710 2 $a ProQuest Information and Learning Co. $3 1178819
710 2 $a Northwestern University. $b Electrical and Computer Engineering. $3 1186892
773 0 $t Dissertation Abstracts International $g 79-11B(E).
856 4 0 $u http://pqdd.sinica.edu.tw/twdaoapp/servlet/advanced?query=10816819 $z click for full text (PQDT)