Events Calendar

TITLE: OPTIMAL CONTROL FOR DATA HARVESTING AND SIGNAL MODEL ESTIMATION

ABSTRACT: Over the last decade, the application of Wireless Sensor Networks (WSNs) has surged in fields such as environmental monitoring, human health, and smart cities. With this wealth of technologies comes the challenge of how to extract volumes of data collected by such sensor nodes distributed over large, often remote, geographical regions. Data harvesting is the problem of extracting measurements from the remote nodes of WSNs using mobile agents such as ground vehicles or drones. Using mobile agents can significantly reduce the energy consumption of sensor nodes relative to other modes of extracting the data, extending the lifetime and capabilities of the WSN. Moreover, in remote areas where GPS may not be feasible due to limited power resources on the sensor nodes, the need for accurate sensor node localization and signal broadcasting model estimation becomes critical. Therefore, designing the trajectory of mobile agents is crucial for rapid data collection and information gathering while adhering to vehicle constraints such as dynamics and energy usage. In this thesis, we focus on the application of optimal control methods to design trajectories for mobile agents in data harvesting. This thesis makes contributions in three areas: the creation of a parameterized optimal control policy, the application of a Deep Reinforcement Learning (DRL) based control, and the use of Fisher Information (FI) as a cost matrix in a Receding Horizon Control (RHC) method. Parameterized Optimal Control Policy: Our contributions in this area begin by considering a data harvesting problem in 1-D space. We use a Hamiltonian analysis to show that the optimal control can be described using a parameterized policy and then develop a gradient descent scheme using Infinitesimal Perturbation Analysis (IPA) to calculate the gradients of the cost function with respect to the control parameters. We also consider this problem in a multi-agent setting. To avoid collisions between agents, we apply a Control Barrier Function (CBF) technique to ensure the agents closely track the desired optimal trajectory to complete their mission while avoiding any collisions. Finally, we extend the problem to a mobile sensor scenario. In this more complicated setting, we demonstrate that the optimization problem for the control policy parameters can be effectively solved using a heuristic approach. Deep Reinforcement Learning based Control: The parametric optimal control approach cannot be easily extended from the 1-D setting to 2-D space. For this reason, we turn to DRL techniques. We utilize the Hamiltonian analysis again to get the necessary conditions for optimal control and then translate the problem to a Markov Decision Process (MDP) in discrete time. We apply reinforcement learning techniques, including double deep Q-learning and Proximal Policy Optimization (PPO), to find high-performing solutions across different scenarios. We demonstrate the effectiveness of these methods in 2-D simulations. Fisher Information based Receding Horizon Control: For the data harvesting problem in large scale unknown environments, estimating the parameters defining the broadcast model and the location of all the nodes in the environment is critical for efficient extraction of the data. To address that, we start with a Received Signal Strength (RSS) model that relies on a Line-of-Sight (LoS) path-loss model with power measurements that are corrupted by Gaussian distributed noise. We first consider a single agent tasked with estimating these unknown parameters in discrete time, and then develop a Fisher Information Matrix (FIM) Receding Horizon (RH) controller for agent motion planning in real time. We also design a Neural Network (NN)-based controller to approximate the optimal solution to the Hamilton-Jacobi-Bellman (HJB) problem, maximizing information gain along a continuous time trajectory. Additionally, a two-stage formation-based RH controller is designed for multi-agent scenarios. The experiments demonstrate that the FI-based RHC methods enhance the estimation accuracy in various simulation environments.

COMMITTEE: ADVISOR Professor Sean Andersson, ME/SE; CHAIR Professor Sheryl Grace, ME; Professor Christos Cassandras, ECE/SE; Professor Roberto Tron, ME/SE; Professor Alyssa Pierson, ME/SE