國立虎尾科技大學 |

Advancing Vision Intelligence Through the Development of Efficiency, Interpretability and Fairness in Deep Learning Models.

Record Type:	Language materials, manuscript : Monograph/item
Title/Author:	Advancing Vision Intelligence Through the Development of Efficiency, Interpretability and Fairness in Deep Learning Models./
Author:	Kong, Fanjie.
Description:	1 online resource (109 pages)
Notes:	Source: Dissertations Abstracts International, Volume: 85-11, Section: B.
Contained By:	Dissertations Abstracts International85-11B.
Subject:	Computer engineering. -
Online resource:	click for full text (PQDT)
ISBN:	9798382729022

Advancing Vision Intelligence Through the Development of Efficiency, Interpretability and Fairness in Deep Learning Models.
Kong, Fanjie.

Advancing Vision Intelligence Through the Development of Efficiency, Interpretability and Fairness in Deep Learning Models. - 1 online resource (109 pages)

Source: Dissertations Abstracts International, Volume: 85-11, Section: B.

Thesis (Ph.D.)--Duke University, 2024.

Includes bibliographical references

Deep learning has demonstrated remarkable success in developing vision intelligence across a variety of application domains, including autonomous driving, facial recognition, medical image analysis, etc. However, developing such vision systems poses significant challenges, particularly in relation to ensuring efficiency, interpretability, and fairness. Efficiency requires a model to leverage the least possible computational resources while preserving performance relative to more computationally-demanding alternatives, which is essential for the practical deployment of large-scale models in real-time applications. Interpretability demands a model to align with the domain-specific knowledge of the task it addresses while having the capability for case-based reasoning. This characteristic is especially crucial in high-stakes areas such as healthcare, criminal justice, and financial investment. Fairness ensures that computer vision models do not perpetuate or exacerbate societal biases in downstream applications such as web image search, text-guided image generation, etc. In this dissertation, I will discuss the contributions that I have made in advancing vision intelligence regarding to efficiency, interpretability and fairness in computer vision models.The first part of this dissertation will focus on how to design computer vision models to efficiently process very large images. We propose a novel CNN architecture termed Zoom-In Network that leverages a hierarchical attention sampling mechanisms to select important regions of images to process. Such approach without processing the entire image yields outstanding memory efficiency while maintaining classification accuracy on various tiny object image classification datasets.The second part of this dissertation will discuss how to build post-hoc interpretation method for deep learning models to obtain insights reasoned from the predictions. We propose a novel image and text insight-generation framework based on attributions from deep neural nets. We test our approach on an industrial dataset and demonstrate our method outperforms competing methods.Finally, we study fairness in large vision-language models. More specifically, we examined gender and racial bias in text-based image retrieval for neutral text queries. In an attempt to address bias in the test-time phase, we proposed post-hoc bias mitigation to actively balance the demographic group in the image search results. Experiments on multiple datasets show that our method can significantly reduce bias while maintaining satisfactory retrieval accuracy at the same time.My research in enhancing vision intelligence via developments in efficiency, interpretability, and fairness, has undergone rigorous validation using publicly available benchmarks and has been recognized at leading peer-reviewed machine learning conferences. This dissertation has sparked interest within the AI community, emphasizing the importance of improving computer vision models through these three critical dimensions, namely, efficiency, interpretability and fairness.

Electronic reproduction.
Ann Arbor, Mich. :
ProQuest,
2024

Mode of access: World Wide Web

ISBN: 9798382729022Subjects--Topical Terms:

569006
Computer engineering.
Subjects--Index Terms:

Deep learningIndex Terms--Genre/Form:

554714
Electronic books.

Advancing Vision Intelligence Through the Development of Efficiency, Interpretability and Fairness in Deep Learning Models.
LDR:04495ntm a22003977 4500 001 1149687
005 20241022112615.5
006 m o d
007 cr bn ---uuuuu
008 250605s2024 xx obm 000 0 eng d
020 $a 9798382729022
035 $a (MiAaPQ)AAI30992500
035 $a AAI30992500
040 $a MiAaPQ $b eng $c MiAaPQ $d NTU
100 1 $a Kong, Fanjie. $3 1476013
245 1 0 $a Advancing Vision Intelligence Through the Development of Efficiency, Interpretability and Fairness in Deep Learning Models.
264 0 $c 2024
300 $a 1 online resource (109 pages)
336 $a text $b txt $2 rdacontent
337 $a computer $b c $2 rdamedia
338 $a online resource $b cr $2 rdacarrier
500 $a Source: Dissertations Abstracts International, Volume: 85-11, Section: B.
500 $a Advisor: Henao, Ricardo;Li, Hai.
502 $a Thesis (Ph.D.)--Duke University, 2024.
504 $a Includes bibliographical references
520 $a Deep learning has demonstrated remarkable success in developing vision intelligence across a variety of application domains, including autonomous driving, facial recognition, medical image analysis, etc. However, developing such vision systems poses significant challenges, particularly in relation to ensuring efficiency, interpretability, and fairness. Efficiency requires a model to leverage the least possible computational resources while preserving performance relative to more computationally-demanding alternatives, which is essential for the practical deployment of large-scale models in real-time applications. Interpretability demands a model to align with the domain-specific knowledge of the task it addresses while having the capability for case-based reasoning. This characteristic is especially crucial in high-stakes areas such as healthcare, criminal justice, and financial investment. Fairness ensures that computer vision models do not perpetuate or exacerbate societal biases in downstream applications such as web image search, text-guided image generation, etc. In this dissertation, I will discuss the contributions that I have made in advancing vision intelligence regarding to efficiency, interpretability and fairness in computer vision models.The first part of this dissertation will focus on how to design computer vision models to efficiently process very large images. We propose a novel CNN architecture termed Zoom-In Network that leverages a hierarchical attention sampling mechanisms to select important regions of images to process. Such approach without processing the entire image yields outstanding memory efficiency while maintaining classification accuracy on various tiny object image classification datasets.The second part of this dissertation will discuss how to build post-hoc interpretation method for deep learning models to obtain insights reasoned from the predictions. We propose a novel image and text insight-generation framework based on attributions from deep neural nets. We test our approach on an industrial dataset and demonstrate our method outperforms competing methods.Finally, we study fairness in large vision-language models. More specifically, we examined gender and racial bias in text-based image retrieval for neutral text queries. In an attempt to address bias in the test-time phase, we proposed post-hoc bias mitigation to actively balance the demographic group in the image search results. Experiments on multiple datasets show that our method can significantly reduce bias while maintaining satisfactory retrieval accuracy at the same time.My research in enhancing vision intelligence via developments in efficiency, interpretability, and fairness, has undergone rigorous validation using publicly available benchmarks and has been recognized at leading peer-reviewed machine learning conferences. This dissertation has sparked interest within the AI community, emphasizing the importance of improving computer vision models through these three critical dimensions, namely, efficiency, interpretability and fairness.
533 $a Electronic reproduction. $b Ann Arbor, Mich. : $c ProQuest, $d 2024
538 $a Mode of access: World Wide Web
650 4 $a Computer engineering. $3 569006
650 4 $a Computer science. $3 573171
650 4 $a Electrical engineering. $3 596380
653 $a Deep learning
653 $a Computer vision models
653 $a Fairness
653 $a Machine learning
655 7 $a Electronic books. $2 local $3 554714
690 $a 0464
690 $a 0800
690 $a 0984
690 $a 0544
710 2 $a ProQuest Information and Learning Co. $3 1178819
710 2 $a Duke University. $b Electrical and Computer Engineering. $3 845695
773 0 $t Dissertations Abstracts International $g 85-11B.
856 4 0 $u http://pqdd.sinica.edu.tw/twdaoapp/servlet/advanced?query=30992500 $z click for full text (PQDT)