Digital twin-based test-time adaptation for robust visual perception in manufacturing

초록

Modern manufacturing is shifting toward high-mix, low-volume production, and the growing prevalence of human-robot collaboration has increased the complexity and variability of industrial robotic manipulators. In such dynamic and partially unstructured environments, accurate manipulator segmentation must remain reliable despite photometric and geometric domain shifts, including illumination changes, background variation, target displacement, and manipulator pose changes, to support calibration, collision avoidance, and human-robot safety. To address this need, we propose a digital twin based test-time adaptation (DT-TTA) framework for artificial intelligence (AI) driven manipulator segmentation in manufacturing. The implemented AI model is a frozen convolutional neural network based segmentation backbone, and the engineering application is robust manipulator visual perception for human-robot collaboration. The framework leverages simulated disturbances in a digital twin environment to construct priors stored in a hybrid memory bank and matched with pre-trained encoder features through frequency domain aware dynamic prompting, where low-frequency components compensate for global illumination and appearance shifts while high-frequency components preserve manipulator boundaries and semantic structure. Batch normalization statistics are aligned using mixed source-digital twin priors to enable stable adaptation at inference time. On the Human-UR10 manufacturing benchmark, where UR10 denotes the Universal Robots UR10 manipulator, DT-TTA improves the Dice score from 91.53% to 96.15% under moderately dim illumination and from 81.27% to 85.52% under complex backgrounds, while maintaining 19.5 frames per second without updating backbone weights. These results support digital twin guided, frequency-aware test-time adaptation for manipulator visual perception under the evaluated manufacturing domain shifts.

키워드