A platform for research: civil engineering, architecture and urbanism
A platform for research: civil engineering, architecture and urbanism
Automatic construction site hazard identification integrating construction scene graphs with BERT based domain knowledge
Abstract Hazards often arise from interactions of different parties and can cost great loss. Proactively identification can prevent it from happening. Computer vision currently can identify the entities and attributes inside the construction scenes but fail to generate interaction-level scene descriptions and integrate them with domain knowledge for hazards inference. This paper proposed an automatic hazard inference method using construction scene graphs and C-BERT network. First, computer vision was utilized to build the construction scene graphs with interaction-level scene descriptions including entities, attributes, and their interactions. Second, C-BERT network was designed to infer hazards by integrating scene graphs with domain knowledge like construction regulations. 5 different working scenes were used to demonstrate the validity of the proposed approach and reached 97.82% of hazard identification accuracy. It provided an efficient method for combining visual information and domain knowledge for automated safety monitoring, and path for the industry's massive multimodal information fusion.
Highlights An automatic hazards inference method based on interaction level scene descriptions and domain knowledge is proposed. Construction scene graphs generate interaction level scene descriptions. BERT is used to integrate domain knowledge into construction scene information.
Automatic construction site hazard identification integrating construction scene graphs with BERT based domain knowledge
Abstract Hazards often arise from interactions of different parties and can cost great loss. Proactively identification can prevent it from happening. Computer vision currently can identify the entities and attributes inside the construction scenes but fail to generate interaction-level scene descriptions and integrate them with domain knowledge for hazards inference. This paper proposed an automatic hazard inference method using construction scene graphs and C-BERT network. First, computer vision was utilized to build the construction scene graphs with interaction-level scene descriptions including entities, attributes, and their interactions. Second, C-BERT network was designed to infer hazards by integrating scene graphs with domain knowledge like construction regulations. 5 different working scenes were used to demonstrate the validity of the proposed approach and reached 97.82% of hazard identification accuracy. It provided an efficient method for combining visual information and domain knowledge for automated safety monitoring, and path for the industry's massive multimodal information fusion.
Highlights An automatic hazards inference method based on interaction level scene descriptions and domain knowledge is proposed. Construction scene graphs generate interaction level scene descriptions. BERT is used to integrate domain knowledge into construction scene information.
Automatic construction site hazard identification integrating construction scene graphs with BERT based domain knowledge
Zhang, Lite (author) / Wang, Junjie (author) / Wang, Yanbo (author) / Sun, Hai (author) / Zhao, Xuebing (author)
2022-08-18
Article (Journal)
Electronic Resource
English