An augmented reality-based wearable system for handheld-free and intuitive robot programming

Abstract

Industrial robots have been programmed to optimize mass production by executing predefined trajectories with their high speed, precision, and repeatability over long production cycles. However, the rise of Industry 4.0 and the increasing demand for diverse product varieties in small quantities have sparked the need for new programming methodologies that enable rapid reprogramming and short production cycles. Recently, augmented reality (AR)-based robot programming has gained attention for overlaying virtual images into real working environments, making programming intuitive and fast. However, the discomfort from prolonged use of AR-handheld devices, as well as the accuracy and latency issues associated with vision-based hand tracking, limit their usability, and impede effective human-robot cooperation. To overcome these limitations, this paper introduces an augmented reality (AR)-based wearable robot programming system using a haptic glove, allowing users to intuitively program the robot in a handheld-free and wearable manner. The use of a haptic glove enables accurate measurements of finger joint angles and handheld-free haptic feedback, eliminating discomfort and occlusion problem. The system leverages measured hand posture and finger joint angles, combined with voice commands, to enable the simultaneous input of various types of commands for intuitive programming. A practical human-robot collaboration scenario demonstration with a 6-degrees of freedom (DOF) hydraulic manipulator to disassemble a ladder for repair shows the programming flexibility of our system. The proposed system addresses the evolving trends of the manufacturing industry by enhancing programming flexibility and intuitiveness, as well as promoting seamless cooperation between humans and robots.