Towards Advanced Drilling Defect Prediction: Multimodal Data Integration and Machined Surface Image Generation

Abstract

The drilling process remains a fundamental machining operation across various industries, such as aerospace, automotive, and energy, where precision and reliability are critical for product performance. Among engineering materials for the drilling process, carbon fiber-reinforced plastic (CFRP) has emerged as a preferred choice for its exceptional strength-to-weight ratio, making it indispensable in lightweight and high-performance applications. However, CFRP drilling presents distinct challenges, particularly in managing process-induced defects such as delamination and uncut fibers, which significantly compromise product quality and structural integrity. While rapid and accurate defect prediction coupled with real-time monitoring is essential for ensuring manufacturing reliability, achieving this goal faces substantial technical barriers. A primary challenge lies in effectively processing the heterogeneous data generated during drilling operations, which includes diverse sensor signals—such as force, vibration, current, voltage, and sound—with varying sampling rates and formats. This complexity is further compounded by the need for machined surface inspection, traditionally performed through time-consuming and costly optical measurement systems. This dissertation explores advanced approaches for drilling defect prediction, focusing on multimodal data integration and machined surface image generation. The research is built upon three fundamental multimodal learning strategies: (1) data fusion, (2) sensor-to-image translation, and (3) multimodal coordination. For multimodal data integration, the first strategy-data fusion-effectively combines signals from diverse sources to improve real-time defect prediction, addressing the challenges of sensor data heterogeneity inherent in drilling processes. Meanwhile, machined surface image generation encompasses the latter two strategies: sensor-to-image translation and multimodal coordination. Sensor-to-image translation transforms time-series sensor data into high-resolution surface images, offering a detailed visual representation of machined quality. Through generative modeling, multimodal coordination integrates multimodal sensor data with machined surface images to capture nuanced machining characteristics, such as delamination and uncut fibers. These proposed approaches overcome limitations in existing methods and demonstrate enhanced defect prediction performance in drilling processes. The research provides adaptable approaches that ensure accurate predictions by effectively utilizing multimodal data as inputs or both inputs and outputs, depending on specific drilling requirements. Furthermore, all approaches maintain real-time processing capabilities directly applicable to industrial manufacturing environments. The contributions of this work establish a robust foundation for advancing defect prediction technologies in smart manufacturing, offering practical solutions for quality control and process optimization. These innovations can save significant industrial costs by reducing manual inspection labor and eliminating time-consuming inspection procedures.