SE PhD Final Defense of Vittorio Giammarino

  • Starts: 11:00 am on Monday, July 8, 2024

TITLE: On the use of Expert Data to imitate behavior and accelerate Reinforcement Learning

ADVISOR: Ioannis Paschalidis ECE, BME, MSE

COMMITTEE: Ehsed Ohn-Bar ECE; Wenchao Li ECE, SE; Roberto Tron ME, SE

CHAIR: Pirooz Vakili SE, ME

ABSTRACT: This dissertation examines the integration of expert datasets to enhance the data efficiency of online Deep Reinforcement Learning (DRL) algorithms in large state and action space problems. The focus is on effectively integrating real-world data, including data from biological systems, to accelerate the learning process within the online DRL pipeline.

The motivation for this work is twofold. First, the internet provides access to a vast amount of data, such as videos, that demonstrate various tasks of interest but are not necessarily designed for use in the DRL framework. Leveraging these data to enhance DRL algorithms presents an exciting and challenging opportunity. Second, biological systems exhibit numerous inductive biases in their behavior that enable them to be highly efficient and adaptable learners. Incorporating these mechanisms for efficient learning remains an open question in DRL, and this work considers the use of human and animal data as a possible solution to this problem.

Throughout this dissertation, important questions are addressed, such as how prior knowledge can be distilled into RL agents, the benefits of leveraging offline datasets for online RL, and the algorithmic challenges involved. Five original works are presented that investigate the use of animal videos to enhance RL learning performance, develop a framework to learn bio-inspired foraging policies using human data, propose an online algorithm for performing hierarchical imitation learning in the options framework, and formulate and theoretically motivate novel algorithms for imitation from videos in the presence of visual mismatch.

This research demonstrates the effectiveness of utilizing offline datasets to improve the efficiency and performance of online DRL algorithms, providing valuable insights into accelerating the learning process for complex tasks.

Location:
CDS 1646
Registration:
https://bostonu.zoom.us/j/93567332214?pwd=NnB3QW1vZlFnb3BiaUNCc21lcUFodz09
Hosting Professor
Ioannis Paschalidis ECE, BME, MSE