Multi‐defect segmentation from façade images using balanced copy

Multi‐defect segmentation from façade images using balanced copy–paste method

Li, Jiajun / Wang, Qian / Ma, Jun / Guo, Jingjing

Façade defect is an unavoidable and considerable problem to existing buildings and can cause great influence to building owners. The traditional manual façade inspection method is costly, inefficient, and unsafe. Although recent studies achieved the classification of façade defects from images, pixel‐level façade defect segmentation has not been tackled. Therefore, this study proposed and implemented a balanced copy–paste method with the Mask Region‐based Convolutional Neural Network (Mask R‐CNN) model to realize automatic detection and segmentation of façade defects. The proposed balanced copy–paste method was able to improve the recognition accuracy for minority classes and small objects. The proposed method was applied to a façade defect dataset with 2286 images that contained six common categories of defects. Comparisons with three other methods demonstrated that the proposed method could achieve the highest accuracy in defect detection (mean average precision (mAP) = 33.33) and segmentation (mAP = 27.26). Furthermore, compared to the original dataset, the proposed method could result in the greatest accuracy improvement for both minority classes (31% in detection and 32% in segmentation) and small objects (23% in detection and 19% in segmentation). Compared to the traditional over‐sampling method and other methods based on algorithm level, the proposed method also showed higher computational efficiency.