Game Theoretic Approaches to Decentralized Optimal Control of Unmanned Vehicles

Abstract

In this thesis, we consider game theoretic approaches to decentralized optimal control of unmanned vehicles (UVs). First, we consider the problem of decentralized optimal control for a group of large-population unmanned two-wheeled vehicles under partial observation. In particular, each individual vehicle is controlled solely by local noisy information measured from a local sensor to track the average behavior (mean field) of the entire vehicle group while achieving overall optimal control performance. We obtain decentralized optimal controls for two-wheeled vehicles by solving the partially-observed linear-quadratic mean field game. These controls are decentralized since they are functions of the local estimated state from the local Kalman filter. We show that the set of the decentralized optimal controls constitutes an $\epsilon$-Nash equilibrium, where $\epsilon$ converges to zero as the number of vehicles $N$ becomes arbitrarily large. Then, the theoretical results are validated through the results of simulations and experiments under various operation scenarios for a group of large population two-wheeled vehicles.

Next, we consider the problem of leader-follower decentralized optimal control for a hexarotor group, which consists of a single leader and large population followers. The unit-quaternion framework is used to construct a mathematical model of our hexarotor vehicle to overcome the singularity problem of the rotation matrix expressed in Euler angles. Furthermore, our hexarotor has 6-degree of freedom (DoF), which allows to control its translation and attitude simultaneously, since it has six tilted propellers. In the problem setup, the leader hexarotor is coupled with the follower hexarotors through the average behavior of the followers (mean field term), and the followers are coupled with each other through their average behavior and the control of the leader. Based on the mean field Stackelberg game framework, we can get a set of decentralized optimal controls for the leader and $N$ follower hexarotors, where each control is a function of its local information. We show that the set of decentralized optimal controls constitutes an $\epsilon$-Stackelberg equilibrium for the leader and $N$ followers, where $\epsilon$ converges to zero as $N$ becomes arbitrarily large. Through simulations with two different operating scenarios, we show that the leader-follower hexarotors follow their desired position and attitude references, and the followers are controlled by the leader while effectively tracking their approximated average behavior. Furthermore, we show the nonsingularity and 6-DoF control performance of the leader-follower hexarotor group due to the novel modeling technique of the hexarotor presented in the chapter.

Finally, we propose a new approach to solve the backward reachability problem for nonlinear dynamical systems. Previously, this class of problems has been studied within frameworks of optimal control and zero-sum differential games, where the backward reachable set can be expressed as the zero sublevel set of the value function that can be characterized by solving the Hamilton-Jacobi-Bellman (HJB) partial differential equation (PDE). In many cases, however, high computational cost is essential to numerically solve such HJB PDEs due to curse of dimensionality. We use the pseudospectral method to convert the associated optimal control problem into nonlinear programs (NLPs), and show that the zero sublevel set obtained by the optimal cost of the NLP is the corresponding backward reachable set. Our approach does not require to solve complex HJB PDEs, and therefore it can reduce computation time and handle high-dimensional dynamical systems compared with the numerical software package developed by I. Mitchell, which has been used widely in the literature to obtain backward reachable sets by solving HJB equations. We provide several examples to validate effectiveness of the proposed approach.