國立虎尾科技大學 |

Unlocking Artificial Intelligent Video Understanding Through Object-Centric Relational Reasoning.

Record Type:	Language materials, manuscript : Monograph/item
Title/Author:	Unlocking Artificial Intelligent Video Understanding Through Object-Centric Relational Reasoning./
Author:	Zhou, Honglu.
Description:	1 online resource (188 pages)
Notes:	Source: Dissertations Abstracts International, Volume: 85-05, Section: B.
Contained By:	Dissertations Abstracts International85-05B.
Subject:	Computer science. -
Online resource:	click for full text (PQDT)
ISBN:	9798380845830

Unlocking Artificial Intelligent Video Understanding Through Object-Centric Relational Reasoning.
Zhou, Honglu.

Unlocking Artificial Intelligent Video Understanding Through Object-Centric Relational Reasoning. - 1 online resource (188 pages)

Source: Dissertations Abstracts International, Volume: 85-05, Section: B.

Thesis (Ph.D.)--Rutgers The State University of New Jersey, School of Graduate Studies, 2023.

Includes bibliographical references

Modern visual systems have achieved high precision in tasks such as face detection, scene segmentation, and object localization, through their emphasis on analyzing static images and low-level pixel data. However, our experience as humans is deeply shaped by our perception of objects, their relations, and what happens to the objects in the visual world. Therefore, rather than building a machine that attempts to attain artificial visual intelligence solely from the static image or directly from the low-level pixels, we may need to accomplish that from dynamic relational reasoning about objects in videos.Relational reasoning reasons the saliency of objects and their dynamic interactions. In this context, object refers to an entity, such as a discrete concept, a physical object, or a person. Towards intelligent video understanding, which is crucial for machines to interpret and interact with the dynamic visual world that we humans live in, we propose using relational reasoning about objects in videos to address the key challenges in video understanding: (i) spatiotemporal reasoning, (ii) high-level semantic understanding, and (iii) rudimentary video representation learning.In this dissertation, we begin with considering the problem of spatiotemporal reasoning, and the proxy task to evaluate on is Snitch Localization, where multiple objects move and are frequently occluded by one another. We propose Hopper, which uses a Multi-hop Transformer for reasoning object permanence through multi-step relational modeling. Hopper demonstrates the effectiveness of multi-step relational reasoning for spatiotemporal reasoning. Moving towards the non-trivial higher-level semantic understanding of videos, we tackle the task of human Group Activity Recognition that detects the activity collectively performed by a group of actors in a video. We propose COMPOSER, a Multiscale Transformer based architecture that performs multi-scale relational reasoning to learn group activity compositionally. COMPOSER shows relational reasoning additionally armed with multi-scale modeling that explicitly models hierarchy and compositionality, facilitates an accurate high-level semantic understanding of videos. Last but foremost, video representations are the basis of video understanding; improved video representations could potentially enhance video system performance. We propose Paprika, a pre-training procedure that leverages graph-based relational reasoning to learn a generalizable video representation, targeting an improved downstream procedural understanding ability in instructional videos, and thus paving the way for robot learning from human-provided demonstrations to provide humans with better assistance and interactions. Contributions: Our solutions outperform the current state-of-the-art methods on several video datasets. Our results indicate that object-centric relational reasoning is the key towards intelligent video understanding. By introducing inductive biases such as multi-step reasoning, multi-hierarchy, graph-structured knowledge, suitable modalities, and external sources of information, relational reasoning can be further enhanced, ultimately leading to a machine-learning-based video system that is more efficient to develop, a more comprehensive understanding of videos, and a more effective ability to solve a variety of video-related tasks.

Electronic reproduction.
Ann Arbor, Mich. :
ProQuest,
2024

Mode of access: World Wide Web

ISBN: 9798380845830Subjects--Topical Terms:

573171
Computer science.
Subjects--Index Terms:

Computer VisionIndex Terms--Genre/Form:

554714
Electronic books.

Unlocking Artificial Intelligent Video Understanding Through Object-Centric Relational Reasoning.
LDR:04867ntm a22003977 4500 001 1143797
005 20240517104955.5
006 m o d
007 cr mn ---uuuuu
008 250605s2023 xx obm 000 0 eng d
020 $a 9798380845830
035 $a (MiAaPQ)AAI30526139
035 $a AAI30526139
040 $a MiAaPQ $b eng $c MiAaPQ $d NTU
100 1 $a Zhou, Honglu. $3 1468588
245 1 0 $a Unlocking Artificial Intelligent Video Understanding Through Object-Centric Relational Reasoning.
264 0 $c 2023
300 $a 1 online resource (188 pages)
336 $a text $b txt $2 rdacontent
337 $a computer $b c $2 rdamedia
338 $a online resource $b cr $2 rdacarrier
500 $a Source: Dissertations Abstracts International, Volume: 85-05, Section: B.
500 $a Advisor: Kapadia, Mubbasir.
502 $a Thesis (Ph.D.)--Rutgers The State University of New Jersey, School of Graduate Studies, 2023.
504 $a Includes bibliographical references
520 $a Modern visual systems have achieved high precision in tasks such as face detection, scene segmentation, and object localization, through their emphasis on analyzing static images and low-level pixel data. However, our experience as humans is deeply shaped by our perception of objects, their relations, and what happens to the objects in the visual world. Therefore, rather than building a machine that attempts to attain artificial visual intelligence solely from the static image or directly from the low-level pixels, we may need to accomplish that from dynamic relational reasoning about objects in videos.Relational reasoning reasons the saliency of objects and their dynamic interactions. In this context, object refers to an entity, such as a discrete concept, a physical object, or a person. Towards intelligent video understanding, which is crucial for machines to interpret and interact with the dynamic visual world that we humans live in, we propose using relational reasoning about objects in videos to address the key challenges in video understanding: (i) spatiotemporal reasoning, (ii) high-level semantic understanding, and (iii) rudimentary video representation learning.In this dissertation, we begin with considering the problem of spatiotemporal reasoning, and the proxy task to evaluate on is Snitch Localization, where multiple objects move and are frequently occluded by one another. We propose Hopper, which uses a Multi-hop Transformer for reasoning object permanence through multi-step relational modeling. Hopper demonstrates the effectiveness of multi-step relational reasoning for spatiotemporal reasoning. Moving towards the non-trivial higher-level semantic understanding of videos, we tackle the task of human Group Activity Recognition that detects the activity collectively performed by a group of actors in a video. We propose COMPOSER, a Multiscale Transformer based architecture that performs multi-scale relational reasoning to learn group activity compositionally. COMPOSER shows relational reasoning additionally armed with multi-scale modeling that explicitly models hierarchy and compositionality, facilitates an accurate high-level semantic understanding of videos. Last but foremost, video representations are the basis of video understanding; improved video representations could potentially enhance video system performance. We propose Paprika, a pre-training procedure that leverages graph-based relational reasoning to learn a generalizable video representation, targeting an improved downstream procedural understanding ability in instructional videos, and thus paving the way for robot learning from human-provided demonstrations to provide humans with better assistance and interactions. Contributions: Our solutions outperform the current state-of-the-art methods on several video datasets. Our results indicate that object-centric relational reasoning is the key towards intelligent video understanding. By introducing inductive biases such as multi-step reasoning, multi-hierarchy, graph-structured knowledge, suitable modalities, and external sources of information, relational reasoning can be further enhanced, ultimately leading to a machine-learning-based video system that is more efficient to develop, a more comprehensive understanding of videos, and a more effective ability to solve a variety of video-related tasks.
533 $a Electronic reproduction. $b Ann Arbor, Mich. : $c ProQuest, $d 2024
538 $a Mode of access: World Wide Web
650 4 $a Computer science. $3 573171
650 4 $a Information technology. $3 559429
653 $a Computer Vision
653 $a Object-centric relational reasoning
653 $a Multi-hop Transformer
653 $a Video representation learning
653 $a Video understanding
655 7 $a Electronic books. $2 local $3 554714
690 $a 0984
690 $a 0800
690 $a 0489
710 2 $a ProQuest Information and Learning Co. $3 1178819
710 2 $a Rutgers The State University of New Jersey, School of Graduate Studies. $b Computer Science. $3 1241263
773 0 $t Dissertations Abstracts International $g 85-05B.
856 4 0 $u http://pqdd.sinica.edu.tw/twdaoapp/servlet/advanced?query=30526139 $z click for full text (PQDT)