CISE/AIR Initiative Seminar: Rogerio Schmidt Feris, Principal Scientist and Manager, MIT-IBM Watson AI Lab

December 3, 2021
3:00PM-4:00PM
Location: PHO 906

Dynamic Neural Networks for Efficient Multimodal Video Understanding

The significant growth of multimodal video data in recent years has increased the demand for efficient deep neural network models, particularly in domains where real-time inference is essential. While significant progress has been made on model compression and acceleration for video understanding, most existing methods rely on one-size-fits-all models, which apply the exact same amount of computation for all video segments across all modalities, regardless of their complexity. In this talk, I will cover methods that adaptively change computation depending on the content of the input. First, I will describe a method that dynamically selects the right video frames, at the right level of detail (resolution), to make video understanding more efficient. Then, in the context of audio-visual action recognition, I will present a method that adaptively decides which modality to use for each video segment, with the goal of improving both accuracy and efficiency. Finally, I will discuss ongoing work on adaptive learning for synthetic training data generation.

Rogerio Schmidt Feris is a principal scientist and manager at the MIT-IBM Watson AI lab. He joined IBM in 2006 after receiving a Ph.D. from the University of California, Santa Barbara. He has also worked as an Affiliate Associate Professor at the University of Washington and as an Adjunct Associate Professor at Columbia University. He has authored over 150 technical papers and has over 40 issued patents in the areas of computer vision, multimedia, and machine learning. Rogerio’s work has been covered by the New York Times, ABC News, and CBS 60 minutes, among other media outlets. He is an Associate Editor of TPAMI, has served as a Program Chair of WACV 2017, and frequently serves as an Area Chair of top premiere computer vision and machine learning conferences, such as NeurIPS, CVPR, ICLR, ICML, ECCV, and ICCV. See more at his website. 

Faculty Host: Venkatesh Saligrama
Student Host: Saeed Mohammadzadeh