國立虎尾科技大學 |

Learning Toward Object Invariance Across Modalities.

Record Type:	Language materials, manuscript : Monograph/item
Title/Author:	Learning Toward Object Invariance Across Modalities./
Author:	Ho, Chih-Hui.
Description:	1 online resource (155 pages)
Notes:	Source: Dissertations Abstracts International, Volume: 85-10, Section: B.
Contained By:	Dissertations Abstracts International85-10B.
Subject:	Electrical engineering. -
Online resource:	click for full text (PQDT)
ISBN:	9798382224602

Learning Toward Object Invariance Across Modalities.
Ho, Chih-Hui.

Learning Toward Object Invariance Across Modalities. - 1 online resource (155 pages)

Source: Dissertations Abstracts International, Volume: 85-10, Section: B.

Thesis (Ph.D.)--University of California, San Diego, 2024.

Includes bibliographical references

Humans possess a remarkable ability to identify and pinpoint objects within complex environments, even from partial views or descriptions. For example, when we are asked to "grab a green mug next to coffee machine", we can easily identify the green mug regardless of what view of the green mug is presented to us and distinguish it from other objects. This requires the model to comprehend the instruction, pick up the correct object, and be invariant with object views.In this thesis, I start investigating object invariance from systematically analyzing what are the challenging views of an object such that the existing models fail to recognize the object. We show that existing models cannot recognize the object from challenging object views. To address this problem, I proposed a training method "PIE" to encourage the model to learn a structured feature space that is view-invariant. However, PIE learns the structured feature space in supervised manner, which requires annotation from human. To overcome this limitation, I further proposed VISPE to learn the view-invariant feature in a self-supervised manner for Multiview classification and retrieval tasks.In addition to learning invariance with images, the use of language as an abstraction of invariant object representation is investigated. I first discover that the existing visual language models fails to produce a consistent prediction when the object class name changes to different granularity (e.g. from Bengal tiger to tiger). A novel prompt tuning methods is proposed to regularize model prediction and to better align language and vision. However, it remains a question whether these visual language models can realize the existence of an object. This is investigated by curating a visual question answering dataset that contains "unanswerable questions", where the referred object is not in the image. An unsupervised approach is proposed to encourage the model to be aware of unanswerable questions. These efforts have improved the understanding of how to learn object invariance across modalities.

Electronic reproduction.
Ann Arbor, Mich. :
ProQuest,
2024

Mode of access: World Wide Web

ISBN: 9798382224602Subjects--Topical Terms:

596380
Electrical engineering.
Subjects--Index Terms:

Computer visionIndex Terms--Genre/Form:

554714
Electronic books.

Learning Toward Object Invariance Across Modalities.
LDR:03482ntm a22004097 4500 001 1146258
005 20240812064354.5
006 m o d
007 cr bn ---uuuuu
008 250605s2024 xx obm 000 0 eng d
020 $a 9798382224602
035 $a (MiAaPQ)AAI30993164
035 $a AAI30993164
040 $a MiAaPQ $b eng $c MiAaPQ $d NTU
100 1 $a Ho, Chih-Hui. $3 1471617
245 1 0 $a Learning Toward Object Invariance Across Modalities.
264 0 $c 2024
300 $a 1 online resource (155 pages)
336 $a text $b txt $2 rdacontent
337 $a computer $b c $2 rdamedia
338 $a online resource $b cr $2 rdacarrier
500 $a Source: Dissertations Abstracts International, Volume: 85-10, Section: B.
500 $a Advisor: Vasconcelos, Nuno.
502 $a Thesis (Ph.D.)--University of California, San Diego, 2024.
504 $a Includes bibliographical references
520 $a Humans possess a remarkable ability to identify and pinpoint objects within complex environments, even from partial views or descriptions. For example, when we are asked to "grab a green mug next to coffee machine", we can easily identify the green mug regardless of what view of the green mug is presented to us and distinguish it from other objects. This requires the model to comprehend the instruction, pick up the correct object, and be invariant with object views.In this thesis, I start investigating object invariance from systematically analyzing what are the challenging views of an object such that the existing models fail to recognize the object. We show that existing models cannot recognize the object from challenging object views. To address this problem, I proposed a training method "PIE" to encourage the model to learn a structured feature space that is view-invariant. However, PIE learns the structured feature space in supervised manner, which requires annotation from human. To overcome this limitation, I further proposed VISPE to learn the view-invariant feature in a self-supervised manner for Multiview classification and retrieval tasks.In addition to learning invariance with images, the use of language as an abstraction of invariant object representation is investigated. I first discover that the existing visual language models fails to produce a consistent prediction when the object class name changes to different granularity (e.g. from Bengal tiger to tiger). A novel prompt tuning methods is proposed to regularize model prediction and to better align language and vision. However, it remains a question whether these visual language models can realize the existence of an object. This is investigated by curating a visual question answering dataset that contains "unanswerable questions", where the referred object is not in the image. An unsupervised approach is proposed to encourage the model to be aware of unanswerable questions. These efforts have improved the understanding of how to learn object invariance across modalities.
533 $a Electronic reproduction. $b Ann Arbor, Mich. : $c ProQuest, $d 2024
538 $a Mode of access: World Wide Web
650 4 $a Electrical engineering. $3 596380
650 4 $a Computer science. $3 573171
653 $a Computer vision
653 $a Deep learning
653 $a Machine learning
653 $a Multiview recognition
653 $a Object invariance
653 $a Visual language model
655 7 $a Electronic books. $2 local $3 554714
690 $a 0544
690 $a 0984
690 $a 0800
710 2 $a ProQuest Information and Learning Co. $3 1178819
710 2 $a University of California, San Diego. $b Electrical and Computer Engineering. $3 1465765
773 0 $t Dissertations Abstracts International $g 85-10B.
856 4 0 $u http://pqdd.sinica.edu.tw/twdaoapp/servlet/advanced?query=30993164 $z click for full text (PQDT)