Choose a topic related to the general theme of the course and review the findings of at least 6 papers related to that topic. A review paper should not be just a summary of the individual findings of each paper – instead, you should identify a larger research question, discuss how the findings relate to this research question, what the commonalities and differences in methodologies are, and discuss whether the papers provide convergent evidence (ie., they all point into the same direction) or whether the papers provide contradictory findings and a discussion of why this may be the case. Optionally, it would also be great to discuss what might be necessary to make further progress on answering the research question that you have chosen.
To illustrate this, you could choose the question of how language models deal with negation and discuss the two papers that we discussed as part of the seminar and on top of that, you would discuss several other papers that look at how models deal with negation. You could then discuss what kind of methodologies (e.g., behavioral experiments, probing, using specialized NLI tasks, …) the papers employ and what the general findings are, and whether there seem to be trends such as different architectures showing different levels of negation processing or the effect of scaling models.
Choose a topic related to the general theme of the course and run experiments with one or several existing language models. Document the experiments that you ran and analyze the results that you got in a technical report. A technical report is similar to the methods and results sections of the papers that we read but you are not required to discuss your results in relation to existing results (but you are welcome to do so).
This type of paper is much more open-ended and may take on many different forms. You could, for example, take an existing evaluation set and evaluate newer models that have been released since the evaluation data has been released; you could translate an existing dataset into another language and see whether the findings also hold for other languages; you could design your own evaluation set for a phenomenon that has not been investigated in the past; you could apply a different evaluation method that has been used in the past; …
A proposal: The title and 1-3 paragraph proposal of what topic you plan to work on and how you plan to approach the topic. This will be ungraded but the more details you provide, the better feedback I can give you. If you are writing a review paper, focus primarily on the research question; if you are doing your own project, also describe what kind of experiments you plan to run.
The final paper
Both the proposal and the final paper should be uploaded on the CMS.
Proposal due date: July 2, 2023
Final paper due date: October 16, 2023