A Deep Reinforcement Learning Framework with Edge-Aware Encoding for the Traveling Salesman Problem with Drone Duong Thi Thuy Ngan Ulsan National Institute of Science and Technology

Abstract

The coordination between trucks and drones for last-mile delivery has recently emerged as a promis- ing research direction in intelligent transportation systems due to its potential to improve delivery ef- ficiency and flexibility. In this setting, one or more drones cooperate with a ground truck to deliver parcels directly to customers, leveraging the complementary strengths of both vehicles. Despite signif- icant research efforts, existing optimization and heuristic approaches often face scalability and runtime limitations as the problem size increases, which restricts their applicability in real-world or dynamic en- vironments. To overcome these challenges, this thesis introduces a deep reinforcement learning frame- work designed to efficiently generate high-quality routing solutions for the Traveling Salesman Problem with Drone (TSP-D). The proposed framework integrates an edge-aware graph attention network as the encoder with a transformer-based decoder, enabling the model to capture both node and edge-level re- lationships in the delivery network. The TSP-D is formulated as a sequential decision-making problem, where a probabilistic policy is learned to iteratively construct delivery routes. Training and evalua- tion are conducted using real-world road network data to ensure the practical relevance of the proposed framework. Experimental results show that our model achieves near-optimal solutions with signifi- cantly lower computational time compared to optimal dynamic programming and traditional heuristic algorithms. Overall, this research contributes a scalable, edge-aware learning framework that enhances cooperative truck–drone routing and demonstrates the effectiveness of graph-based deep reinforcement learning in solving complex combinatorial optimization problems in transportation logistics.