A deep learning-based fault detection model, which can be implemented for plastic injection molding for car parts companies, is presented in this research. Compared to conventional fault detection approaches, it is known that recent deep learning-based fault detection approaches are more useful for real-world problems with a large number of variables. In addition, due to the development of advanced sensors generating types of data in multiple modalities, fault detection prompts the need for multimodal deep learning, which is able to facilitate information from various modalities in an end-to-end learning fashion. The presented deep learning-based model opts for an early fusion scheme in which the low-level feature representations of modalities are combined. A case study involving real-world car parts company data, which are related to a car window side molding process, proves that the presented deep learning-based fault detection model outperforms late fusion methods and conventional fault detection models.