Differential attention transformer-enhanced graph neural network for accurate material property prediction

Abstract

The accurate prediction of material properties is critical for accelerating materials discovery and design. Machine Learning methods, particularly graph neural networks, have recently demonstrated promising results in modeling the structural and electronic characteristics of crystalline materials. However, performance can degrade in deep architectures and structurally complex materials and capturing subtle structural dependencies remains challenging. In this work, we introduce a Differential Attention Transformer-enhanced Graph Neural Network that integrates chemically informed structural features with a dual-attention mechanism. Our approach follows a structure framework, beginning with collection and preprocessing of crystal structures further incorporates a learning architecture combining self- and differential-attention, residual graph convolutions and chemically-informed positional encodings to model complex atomic interactions. When evaluated on benchmark Materials Project datasets for bandgap and formation energy prediction, proposed method up to an 8% reduction in mean absolute error for formation energy and a 5% reduction for bandgap compared with recent graph neural network-based approaches. These results indicate that proposed method provides an accurate and scalable framework for crystal-structure-informed property prediction, contributing to continued progress in data-driven materials informatics.