A platform for research: civil engineering, architecture and urbanism
A platform for research: civil engineering, architecture and urbanism
Vision–based framework for automatic interpretation of construction workers' hand gestures
Abstract Construction robots have been recently developed to improve construction productivity and safety. One of the critical steps to make the robots work with human workers as teams is to provide a user-friendly interface to support their mutual interactions on construction sites. Compared with existing interfaces, hand gestures are easy to use, natural, and intuitive. This paper proposed a novel vision-based framework to capture and interpret the worker's hand gestures as a human-robot interface in construction. The framework consists of three components: worker detection and tracking, recognition queues formulation, and hand gesture recognition. Its effectiveness on the hand gesture recognition was tested with field experiments and achieved the overall precision and recall of 87.0% and 66.7%. Also, a laboratory study was conducted to illustrate the use of the framework to interact with a robotic dump truck. Future work will integrate the proposed framework into robotic construction machines.
Highlights Create a vision-based framework that integrates worker detection, tracking, and hand gesture recognition. Build a hierarchical hand gesture recognition architecture. Evaluate the effectiveness of the framework in fields with the overall precision and recall of 87.0% and 66.7% Investigate the feasibility of using hand gestures to control construction machines in laboratory environments.
Vision–based framework for automatic interpretation of construction workers' hand gestures
Abstract Construction robots have been recently developed to improve construction productivity and safety. One of the critical steps to make the robots work with human workers as teams is to provide a user-friendly interface to support their mutual interactions on construction sites. Compared with existing interfaces, hand gestures are easy to use, natural, and intuitive. This paper proposed a novel vision-based framework to capture and interpret the worker's hand gestures as a human-robot interface in construction. The framework consists of three components: worker detection and tracking, recognition queues formulation, and hand gesture recognition. Its effectiveness on the hand gesture recognition was tested with field experiments and achieved the overall precision and recall of 87.0% and 66.7%. Also, a laboratory study was conducted to illustrate the use of the framework to interact with a robotic dump truck. Future work will integrate the proposed framework into robotic construction machines.
Highlights Create a vision-based framework that integrates worker detection, tracking, and hand gesture recognition. Build a hierarchical hand gesture recognition architecture. Evaluate the effectiveness of the framework in fields with the overall precision and recall of 87.0% and 66.7% Investigate the feasibility of using hand gestures to control construction machines in laboratory environments.
Vision–based framework for automatic interpretation of construction workers' hand gestures
Wang, Xin (author) / Zhu, Zhenhua (author)
2021-07-30
Article (Journal)
Electronic Resource
English