國立虎尾科技大學 |

Context-Aware Worker Intent Interpretation for Human-Robot Collaboration in Construction.

Record Type:	Language materials, manuscript : Monograph/item
Title/Author:	Context-Aware Worker Intent Interpretation for Human-Robot Collaboration in Construction./
Author:	Wang, Xin.
Description:	1 online resource (123 pages)
Notes:	Source: Dissertations Abstracts International, Volume: 85-12, Section: B.
Contained By:	Dissertations Abstracts International85-12B.
Online resource:	click for full text (PQDT)
ISBN:	9798383001301

Context-Aware Worker Intent Interpretation for Human-Robot Collaboration in Construction.
Wang, Xin.

Context-Aware Worker Intent Interpretation for Human-Robot Collaboration in Construction. - 1 online resource (123 pages)

Source: Dissertations Abstracts International, Volume: 85-12, Section: B.

Thesis (Ph.D.)--The University of Wisconsin - Madison, 2024.

Includes bibliographical references

With years of technical development, construction robots and/or autonomous machines have shown the potential to enhance productivity and safety in the construction industry. However, robots and autonomous machines have not been widely adopted on construction sites. There are various reasons contributing to the low adoption of robots in construction including operational and personnel barriers. An intuitive and accurate human intent representation can help contribute to resolving the above barriers. On one hand, the establishment of such representation can greatly enhance the operability of robots in dynamic construction environments. On the other hand, such representation helps to build a safe environment for worker-robot collaboration. So far, there are much related work about human intent interpretation, such as recognizing human actions, identifying hand gestures, automating eye tracking, and understanding speech language. These references show that the recent advance in technologies (e.g., computer vision, wearable sensor) has built a solid foundation to interpret and predict the intentions of onsite construction workers and support their collaborations with robotic machines. However, two research questions need to be answered before making such interpretations and predictions work well on construction sites: (1) How to capture and interpret worker intents accurately for worker-robot collaborations in construction? (2) How to extract useful context information to facilitate representation on construction sites? To answer these two questions, a context-aware human intent representation is proposed in this study to support human-robot collaboration on construction sites. It consists of three components: recognition building, object-enhanced interaction and machine-aware collaboration. In the first component recognition building, a novel vision-based method is developed to achieve gesture recognition. Since computer vision technologies may be easily affected by the construction environment (e.g., diverse dust and light conditions), a novel wearable sensors-based method is then developed. Through a comparison between these two methods, the sensor-based method is found to have the advantages of early triggering and robust anti-interference capabilities, but may incur higher communication costs in human-robot interactions.In the second component object-enhanced interaction, a novel object-aware method is proposed for human-robot collaboration in construction, integrating first-person vision and gesture recognition. An end-to-end two-stream network which includes a first-person view-based stream and a motion sensory data-based stream is designed. The first-person view-based stream models the user's gaze using an attention module to concentrate on the important spatiotemporal regions of first-person video for context extraction. The motion sensory data-based stream is used to process the motion sensory data to extract features related to the hand motions. Finally, the feature maps coming from these two streams are fused to achieve the hand gesture recognition.In the third component machine-aware collaboration, a novel machine-aware hand gesture recognition method is developed as a human-robot interface for use on construction sites having multiple types of machines. The developed method firstly relies on an eye tracker to visually detect and track construction machines in the first-person view. Then, the machine-of-interest is determined based on the bounding boxes of machines and gaze points. Finally, a hand gesture recognition architecture is incorporated with the machine information for conveying messages to the machine-of-interest. The above methods have been evaluated and tested by experiments on different construction sites. The evaluation results have demonstrated that the proposed methods can capture and interpret the worker intents accurately with context awareness to support human-robot collaborations on construction sites. The expected contributions of the proposed methods include improving interaction efficiency with construction robots, decreasing onsite safety issues, refining the design and implementation of construction robots, promoting the adoption of robots in construction, etc. Future work will focus on the following aspects. First, expanding the dataset to encompass a wider range of construction sites, subjects, equipment, tasks, and types of hand gestures will enhance the robustness of the proposed methods. Second, efforts will be made to address the technological challenges associated with automation, including electronics, computation, and communication. Third, the integration of sensor and data fusion techniques will be explored to enhance the reliability of message communication between workers and machines. Fourth, gathering feedback from workers on novel human-robot interfaces will be prioritized to evaluate their trust in construction robots and potential safety concerns during human-robot interactions on construction sites.

Electronic reproduction.
Ann Arbor, Mich. :
ProQuest,
2024

Mode of access: World Wide Web

ISBN: 9798383001301Subjects--Index Terms:

Construction automationIndex Terms--Genre/Form:

554714
Electronic books.

Context-Aware Worker Intent Interpretation for Human-Robot Collaboration in Construction.
LDR:06339ntm a22003497 4500 001 1150221
005 20241022111617.5
006 m o d
007 cr bn ---uuuuu
008 250605s2024 xx obm 000 0 eng d
020 $a 9798383001301
035 $a (MiAaPQ)AAI31332167
035 $a AAI31332167
040 $a MiAaPQ $b eng $c MiAaPQ $d NTU
100 1 $a Wang, Xin. $3 1019411
245 1 0 $a Context-Aware Worker Intent Interpretation for Human-Robot Collaboration in Construction.
264 0 $c 2024
300 $a 1 online resource (123 pages)
336 $a text $b txt $2 rdacontent
337 $a computer $b c $2 rdamedia
338 $a online resource $b cr $2 rdacarrier
500 $a Source: Dissertations Abstracts International, Volume: 85-12, Section: B.
500 $a Advisor: Zhu, Zhenhua.
502 $a Thesis (Ph.D.)--The University of Wisconsin - Madison, 2024.
504 $a Includes bibliographical references
520 $a With years of technical development, construction robots and/or autonomous machines have shown the potential to enhance productivity and safety in the construction industry. However, robots and autonomous machines have not been widely adopted on construction sites. There are various reasons contributing to the low adoption of robots in construction including operational and personnel barriers. An intuitive and accurate human intent representation can help contribute to resolving the above barriers. On one hand, the establishment of such representation can greatly enhance the operability of robots in dynamic construction environments. On the other hand, such representation helps to build a safe environment for worker-robot collaboration. So far, there are much related work about human intent interpretation, such as recognizing human actions, identifying hand gestures, automating eye tracking, and understanding speech language. These references show that the recent advance in technologies (e.g., computer vision, wearable sensor) has built a solid foundation to interpret and predict the intentions of onsite construction workers and support their collaborations with robotic machines. However, two research questions need to be answered before making such interpretations and predictions work well on construction sites: (1) How to capture and interpret worker intents accurately for worker-robot collaborations in construction? (2) How to extract useful context information to facilitate representation on construction sites? To answer these two questions, a context-aware human intent representation is proposed in this study to support human-robot collaboration on construction sites. It consists of three components: recognition building, object-enhanced interaction and machine-aware collaboration. In the first component recognition building, a novel vision-based method is developed to achieve gesture recognition. Since computer vision technologies may be easily affected by the construction environment (e.g., diverse dust and light conditions), a novel wearable sensors-based method is then developed. Through a comparison between these two methods, the sensor-based method is found to have the advantages of early triggering and robust anti-interference capabilities, but may incur higher communication costs in human-robot interactions.In the second component object-enhanced interaction, a novel object-aware method is proposed for human-robot collaboration in construction, integrating first-person vision and gesture recognition. An end-to-end two-stream network which includes a first-person view-based stream and a motion sensory data-based stream is designed. The first-person view-based stream models the user's gaze using an attention module to concentrate on the important spatiotemporal regions of first-person video for context extraction. The motion sensory data-based stream is used to process the motion sensory data to extract features related to the hand motions. Finally, the feature maps coming from these two streams are fused to achieve the hand gesture recognition.In the third component machine-aware collaboration, a novel machine-aware hand gesture recognition method is developed as a human-robot interface for use on construction sites having multiple types of machines. The developed method firstly relies on an eye tracker to visually detect and track construction machines in the first-person view. Then, the machine-of-interest is determined based on the bounding boxes of machines and gaze points. Finally, a hand gesture recognition architecture is incorporated with the machine information for conveying messages to the machine-of-interest. The above methods have been evaluated and tested by experiments on different construction sites. The evaluation results have demonstrated that the proposed methods can capture and interpret the worker intents accurately with context awareness to support human-robot collaborations on construction sites. The expected contributions of the proposed methods include improving interaction efficiency with construction robots, decreasing onsite safety issues, refining the design and implementation of construction robots, promoting the adoption of robots in construction, etc. Future work will focus on the following aspects. First, expanding the dataset to encompass a wider range of construction sites, subjects, equipment, tasks, and types of hand gestures will enhance the robustness of the proposed methods. Second, efforts will be made to address the technological challenges associated with automation, including electronics, computation, and communication. Third, the integration of sensor and data fusion techniques will be explored to enhance the reliability of message communication between workers and machines. Fourth, gathering feedback from workers on novel human-robot interfaces will be prioritized to evaluate their trust in construction robots and potential safety concerns during human-robot interactions on construction sites.
533 $a Electronic reproduction. $b Ann Arbor, Mich. : $c ProQuest, $d 2024
538 $a Mode of access: World Wide Web
653 $a Construction automation
653 $a Human-robot collaboration
655 7 $a Electronic books. $2 local $3 554714
690 $a 0543
690 $a 0800
710 2 $a ProQuest Information and Learning Co. $3 1178819
710 2 $a The University of Wisconsin - Madison. $b Civil & Environmental Engr. $3 1182853
773 0 $t Dissertations Abstracts International $g 85-12B.
856 4 0 $u http://pqdd.sinica.edu.tw/twdaoapp/servlet/advanced?query=31332167 $z click for full text (PQDT)