國立虎尾科技大學 |

3D Semantic Scene Understanding and Reconstruction.

Record Type:	Language materials, manuscript : Monograph/item
Title/Author:	3D Semantic Scene Understanding and Reconstruction./
Author:	Feng, Qiaojun.
Description:	1 online resource (163 pages)
Notes:	Source: Dissertations Abstracts International, Volume: 85-07, Section: B.
Contained By:	Dissertations Abstracts International85-07B.
Subject:	Robotics. -
Online resource:	click for full text (PQDT)
ISBN:	9798381432688

3D Semantic Scene Understanding and Reconstruction.
Feng, Qiaojun.

3D Semantic Scene Understanding and Reconstruction. - 1 online resource (163 pages)

Source: Dissertations Abstracts International, Volume: 85-07, Section: B.

Thesis (Ph.D.)--University of California, San Diego, 2023.

Includes bibliographical references

Semantic understanding and reconstruction of the surrounding 3D environment is a necessary requirement for intelligent robots to autonomously fulfill various tasks like environment exploration, surveillance, autonomous driving, indoor household and healthcare service, to name a few. Although the progress on semantic understanding of 2D images is impressive, building a 3D consistent, meaningful yet compact and scalable semantic map for robotics applications is still challenging. In this dissertation, we develop 3D semantic map approaches for robots in different form, including the dense semantic map and the object-level semantic map. We propose TerrainMesh, a dense semantic map in the form of 3D mesh, for the terrain reconstruction from the aerial images and the sparse depth measurements. The joint 2D-3D and geometric-semantic learning framework is proposed to reconstruct the local semantic meshes and the global semantic mesh can be constructed by merging the local meshes with the help of the SLAM algorithm. We investigate the object-level semantic map constructed from 3D measurements. We propose CORSAIR, a retrieval and registration algorithm for point cloud objects. A 3D sparse convolution neural network model is trained to extract the global features for similar shape retrieval and the local per-point features for correspondence generation for pose registration with the help of symmetry. We develop ELLIPSDF, a bi-level object shape model including a coarse level of 3D ellipsoid and a fine level of 3D continuous signed distance function (SDF). We also design the approach to initialize and optimize the object pose together with the bi-level shape from the multiple depth image observations. We also propose the object-level semantic map from the 2D images and investigate its connections with the localization task. We introduce the object mesh model and its observation model of semantic instance segmentation and semantic keypoints. We derive the observation residual function and minimize it to optimize both the object states and the camera poses. We develop OrcVIO, object residual constrained visual-inertial odometry with object ellipsoid and semantic keypoints model. We implement the observation residual model between the ellipsoid and its 2D semantic bounding box and semantic keypoints and connect this with MSCKF framework to implement the online tightly coupled estimation of object and IMU-camera states. The object-level semantic map also provides a meaningful yet efficient representation of the environment. Finally we discuss the potential directions to further extend the 3D semantic understanding technique for robotics.

Electronic reproduction.
Ann Arbor, Mich. :
ProQuest,
2024

Mode of access: World Wide Web

ISBN: 9798381432688Subjects--Topical Terms:

561941
Robotics.
Subjects--Index Terms:

Semantic understandingIndex Terms--Genre/Form:

554714
Electronic books.

3D Semantic Scene Understanding and Reconstruction.
LDR:04016ntm a22003857 4500 001 1146413
005 20240812064609.5
006 m o d
007 cr bn ---uuuuu
008 250605s2023 xx obm 000 0 eng d
020 $a 9798381432688
035 $a (MiAaPQ)AAI30694632
035 $a AAI30694632
040 $a MiAaPQ $b eng $c MiAaPQ $d NTU
100 1 $a Feng, Qiaojun. $3 1471800
245 1 0 $a 3D Semantic Scene Understanding and Reconstruction.
264 0 $c 2023
300 $a 1 online resource (163 pages)
336 $a text $b txt $2 rdacontent
337 $a computer $b c $2 rdamedia
338 $a online resource $b cr $2 rdacarrier
500 $a Source: Dissertations Abstracts International, Volume: 85-07, Section: B.
500 $a Advisor: Atanasov, Nikolay.
502 $a Thesis (Ph.D.)--University of California, San Diego, 2023.
504 $a Includes bibliographical references
520 $a Semantic understanding and reconstruction of the surrounding 3D environment is a necessary requirement for intelligent robots to autonomously fulfill various tasks like environment exploration, surveillance, autonomous driving, indoor household and healthcare service, to name a few. Although the progress on semantic understanding of 2D images is impressive, building a 3D consistent, meaningful yet compact and scalable semantic map for robotics applications is still challenging. In this dissertation, we develop 3D semantic map approaches for robots in different form, including the dense semantic map and the object-level semantic map. We propose TerrainMesh, a dense semantic map in the form of 3D mesh, for the terrain reconstruction from the aerial images and the sparse depth measurements. The joint 2D-3D and geometric-semantic learning framework is proposed to reconstruct the local semantic meshes and the global semantic mesh can be constructed by merging the local meshes with the help of the SLAM algorithm. We investigate the object-level semantic map constructed from 3D measurements. We propose CORSAIR, a retrieval and registration algorithm for point cloud objects. A 3D sparse convolution neural network model is trained to extract the global features for similar shape retrieval and the local per-point features for correspondence generation for pose registration with the help of symmetry. We develop ELLIPSDF, a bi-level object shape model including a coarse level of 3D ellipsoid and a fine level of 3D continuous signed distance function (SDF). We also design the approach to initialize and optimize the object pose together with the bi-level shape from the multiple depth image observations. We also propose the object-level semantic map from the 2D images and investigate its connections with the localization task. We introduce the object mesh model and its observation model of semantic instance segmentation and semantic keypoints. We derive the observation residual function and minimize it to optimize both the object states and the camera poses. We develop OrcVIO, object residual constrained visual-inertial odometry with object ellipsoid and semantic keypoints model. We implement the observation residual model between the ellipsoid and its 2D semantic bounding box and semantic keypoints and connect this with MSCKF framework to implement the online tightly coupled estimation of object and IMU-camera states. The object-level semantic map also provides a meaningful yet efficient representation of the environment. Finally we discuss the potential directions to further extend the 3D semantic understanding technique for robotics.
533 $a Electronic reproduction. $b Ann Arbor, Mich. : $c ProQuest, $d 2024
538 $a Mode of access: World Wide Web
650 4 $a Robotics. $3 561941
650 4 $a Computer science. $3 573171
653 $a Semantic understanding
653 $a Terrain reconstruction
653 $a TerrainMesh
653 $a Signed distance function
655 7 $a Electronic books. $2 local $3 554714
690 $a 0771
690 $a 0984
690 $a 0800
710 2 $a ProQuest Information and Learning Co. $3 1178819
710 2 $a University of California, San Diego. $b Electrical and Computer Engineering. $3 1465765
773 0 $t Dissertations Abstracts International $g 85-07B.
856 4 0 $u http://pqdd.sinica.edu.tw/twdaoapp/servlet/advanced?query=30694632 $z click for full text (PQDT)