IS&T RCS Boot Camp - GPT & Transformers for Natural Language Processing (Hands-on)

Human communication is rich and complex, and one of the main ways we encode it computationally is through Natural Language Processing (NLP). We’ll explore recent advances in NLP, building from the ground up over the course of three sections. First, we’ll look at generating random first names of people using a simple character level “bigram” model. Then we’ll dive into word embeddings, a technique for encoding words as vectors that captures their semantic meanings. Second, we’ll look at the popular word2vec method and explore how to perform linguistic operations using simple vector arithmetic. Finally, we’ll look at transformer models and see how we can use a pre-trained SentenceTransformer model to do a range of classification on real-world data.