A human-in-the-loop digital-twin continuous- improvement framework integrating virtual reality and human performance models

Abstract

Despite recent advances in disruptive technologies like automation, machine learning, artificial intelligence, and virtual reality, humans remain a precious resource for manufacturing assembly. Paradigms such as Industry 4.0, IoT, digital twin, and cyber-physical systems emphasize the convergence of connectivity, integration, high-fidelity simulation, and heterarchical systems architectures to develop complex integrated systems. More recently, Industry 5.0 places the human operator at the center of system design, but due to human-machine interfaces, developing such systems is challenging. The introductory chapters describe the changing role of the operator in manufacturing and motivate the development of human-centric solutions. This thesis investigates the development of human-centric systems by leveraging virtual reality for digital workstation prototyping. To this end, the first core chapter reports a study confirming that a virtual workstation can measure data crucial to manufacturing assembly, specifically, the throughput rate, risk of defective assemblies, and assembly errors. This study aims to increase confidence in data acquired from the simulation and serves to convince skeptics. After confirming that we can acquire meaningful data, the second study applies this technique to a decision framework for suggesting the best manufacturing design. This illustrates the usefulness of this technique in digital prototyping and product design. The results also illustrate that this technique can be used to plan workstation and factory layouts to meet production requirements. These two studies revealed that human trials are costly and time-inefficient, as experiment designers often acquire ample data. The third study addresses this issue by developing a data-efficient experimental framework. This framework uses an active machine learning model that adapts the design experiments online. This illustrates that VR simulation can include an intelligent system and move from a passive framework for acquiring data to an intelligent one with adaptive control. A deeper inspection revealed that the active model can be used for sample-efficient modeling and control simultaneously. This presents an ethical issue as controlling human systems removes the operator's free will. Such a system could optimize for production and the operator's health. To investigate using the active model for control, a fourth study applies this technique to the control of non-human systems, showing it can be extended with application-specific constraints. In conclusion, the framework for developing human assembly systems using virtual reality reduces capital and technical investment risk. Confirmation of measurement via simulation, application of data acquisition, and data-based control applications appear as logical steps. Control of human systems presents some ethical challenges, and we suggest a theoretical approach. The methods described herein will facilitate the development of modern human manufacturing systems by providing a framework that can be applied to the development of new systems and the retrofitting of existing ones. This work is also valid for other physical fields like medicine, mining, and engineering.