ECE MS Thesis Defense: Yuhan Chen

Title: Distilled Model for Contextual Soft Moderation on Social Media

Presenter: Yuhan Chen

Advisor: Professor Gianluca Stringhini

Committee: Professor Ashok Cutkosky and Professor Alex Olshevsky

Abstract: Automated soft moderation on social media platforms must accurately differentiate posts supporting or refuting false claims. This capability is critical to mitigating contextual false positives—situations where moderation incorrectly flags content merely referencing a false claim, even if the content actually disputes or corrects it. Recent studies have addressed this issue by integrating stance detection with the Contrastive Textual Deviation (CTD) task, significantly reducing contextual false positives within state-of-the-art systems such as Lambretta. However, the considerable computational requirements of large language models hinder their practical deployment. Additionally, knowledge distillation methods tailored specifically for fine-tuned encoder-decoder architectures like T5 remain largely unexplored.

This thesis systematically explores various knowledge distillation techniques, including soft-label transfer and intermediate-layer representation alignment. We present a distilled, compact version of the original teacher model that preserves nuanced stance detection capabilities while significantly improving computational efficiency. Extensive experimental evaluations demonstrate that the distilled model effectively maintains comparable accuracy in reducing contextual false positives, achieving performance nearly equivalent to the original model on benchmark datasets but with substantially lower computational resources. Specifically, we compress an 11-billion-parameter model to just 770 million parameters—a 93% size reduction—while incurring less than a 7% accuracy drop.

The proposed distilled model provides a scalable, efficient solution for deploying reliable automated soft moderation tools on social media platforms, successfully balancing accuracy with practical resource constraints.