Supplemental Reading

The papers for the supplemental reading assignments have been selected for a number of reasons, including being fairly approachable without substantial background knowledge and being relevant to what we are discussing in class. Some papers have been selected do to their relevance to the responsible computing component of the course.

I strongly recommend completing your supplemental reading assignments on Perusall. All of the supplemental assignments on Perusall have some basic information about background knowledge you will need to complete them1 and have been annotated with information about models or tasks that we have not discussed in class. Readings will be available on Perusall at least 2 weeks before their presentation date.

There are two ways to engage with the supplemental reading during the semester. The links below have the details for those assignments:

Supplemental Reading List

Fill out this form to share your supplemental reading preferences

Topic: N-Gram Language Models

Presentation/Response Date: 2/22

  • Shane Bergsma, Dekang Lin, Randy Goebel (2009), Web-scale N-gram models for lexical disambiguation
  • Rogelio Nazar, Irene Renau (2012), Google Books N-gram Corpus used as a Grammar Checker

    Topic: Naive Bayes, Classifier Evaluation

    Presentation/Response Date: 2/27

  • Marco Tulio Ribeiro, Tongshuang Wu, Carlos Guestrin, Sameer Singh (2020), Beyond Accuracy: Behavioral Testing of NLP Models with CheckList

    Topic: Part of Speech Tagging with Hidden Markov Models

    Presentation/Response Date: 2/29

  • Aparna Garimella, Carmen Banea, Dirk Hovy, Rada Mihalcea (2019), Women's Syntactic Resilience and Men's Grammatical Luck: Gender-Bias in Part-of-Speech Tagging and Dependency Parsing
  • Thorsten Brants (2000), TnT -- A Statistical Part-of-Speech Tagger

    Topic: Logistic Regression

    Presentation/Response Date: 3/5

  • Bo Pang, Lillian Lee, Shivakumar Vaithyanathan (2002), Thumbs up? Sentiment Classification using Machine Learning Techniques

    Topic: Vector Semantics

    Presentation/Response Date: 3/7

  • Manaal Faruqui, Yulia Tsvetkov, Pushpendre Rastogi, Chris Dyer (2016), Problems With Evaluation of Word Embeddings Using Word Similarity Tasks
  • Hila Gonen, Ganesh Jawahar, Djamé Seddah, Yoav Goldberg (2020), Simple, Interpretable and Stable Method for Detecting Words with Usage Change across Corpora

    Topic: Word2Vec

    Presentation/Response Date: 3/12

  • Aylin Caliskan, Joanna J. Bryson, Arvind Narayanan (2017), Semantics derived automatically from language corpora contain human-like biases
  • David Bamman, Chris Dyer, Noah A. Smith (2014), Distributed Representations of Geographically Situated Language

    Topic: Neural Networks for Text Classification

    Presentation/Response Date: 3/14

  • Mohit Iyyer, Varun Manjunatha, Jordan Boyd-Graber, Hal Daumé III (2015), Deep Unordered Composition Rivals Syntactic Methods for Text Classification

    Topic: BERT for Text Classification (II)

    Presentation/Response Date: 4/2

  • Christopher D. Manning, Kevin Clark, John Hewitt, Urvashi Khandelwal, Omer Levy (2020), Emergent linguistic structure in artificial neural networks trained by self-supervision (❗)
  • Victor Sanh, Lysandre Debut, Julien Chaumond, Thomas Wolf (2019), DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter

    Topic: Neural Language Modeling

    Presentation/Response Date: 4/4

  • Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever (2018), Language Models are Unsupervised Multitask Learners
  • Emily Sheng, Kai-Wei Chang, Premkumar Natarajan, Nanyun Peng (2019), The Woman Worked as a Babysitter: On Biases in Language Generation

    Topic: Machine Translation

    Presentation/Response Date: 4/9

  • Rico Sennrich, Barry Haddow, Alexandra Birch (2016), Improving Neural Machine Translation Models with Monolingual Data (❗)
  • Ilya Sutskever, Oriol Vinyals, Quoc V. Le (2014), Sequence to Sequence Learning with Neural Networks (❗)

    Topic: Ethics in NLP: Data

    Presentation/Response Date: 4/11

  • Chris Callison-Burch, Mark Dredze (2010), Creating Speech and Language Data With Amazon's Mechanical Turk
  • Veniamin Veselovsky, Manoel Horta Ribeiro, Robert West (2023), Artificial Artificial Artificial Intelligence: Crowd Workers Widely Use Large Language Models for Text Production Tasks
  • Amandalynne Paullada, Inioluwa Deborah Raji, Emily M. Bender, Emily Denton, Alex Hanna (2021), Data and its (dis)contents: A survey of dataset development and use in machine learning research

    Topic: Ethics in NLP: Models

    Presentation/Response Date: 4/16

  • Maarten Sap, Dallas Card, Saadia Gabriel, Yejin Choi, Noah A. Smith (2019), The Risk of Racial Bias in Hate Speech Detection
  • Emily M. Bender, Timnit Gebru, Angelina McMillan-Major, Margaret Mitchell (2021), On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? 🦜

    Topic: Language Understanding

    Presentation/Response Date: 4/18

  • Alex Wang, Yada Pruksachatkun, Nikita Nangia, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, Samuel R. Bowman (2019), SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systems

    1. typically, this will just be textbook chapters that you should make sure to read before completing the supplemental reading, but in a few cases I will recommend a paper only if you have completed courses like machine learning, deep learning, or linear algebra. That includes papers marked with ❗. ↩