来源：medium
作者：huggingface 编辑：肖琴
【新智元导读】
NLP领域发展迅速，初入坑者阅读哪些论文才能快速跟上现代NLP的最新趋势？HuggingFace团队近日发布这份论文列表和资源清单，紧跟研究最前沿，必备收藏。
在过去的两年中，NLP在各种不同任务和应用上的进展十分迅速。这些进展是由于构建NLP系统的经典范式发生了转变带来的：很长一段时间以来，研究人员都使用预训练的词嵌入(如word2vec或GloVe)来初始化神经网络，然后使用一个特定于任务的架构，该架构使用单个数据集以监督方法训练。
最近，一些研究证明，我们可以利用非监督(或自监督)信号，如语言建模，在web规模的数据集上学习分层上下文表示，并将这种预训练转移到下游任务(迁移学习)。令人兴奋的是，这种转变带来了下游应用领域的重大进展，从问题回答到通过句法分析进行自然语言推理……
“我可以读哪些论文来跟上现代NLP的最新趋势?”
几周前，我的一位朋友决定入坑NLP。他已经有机器学习和深度学习的背景，所以他真诚地问我：“我可以阅读哪些论文来跟上现代NLP的最新趋势?”
这是一个非常好的问题，尤其是考虑到NLP会议(以及普遍的ML会议)收到的论文投稿呈指数级增长时：NAACL 2019收到投稿比2018增加了80%， ACL 2019收到的投稿比2018年增加了90%……
因此，我为他整理了这份论文列表和资源清单，并与大家分享。
免责声明：本列表并非详尽无遗，也无法涵盖NLP中的所有主题(例如，没有涵盖语义解析、对抗性学习、NLP强化学习等)。所选择的论文主要是过去几年/几个月最具影响力的工作。
一般而言，开始进入一个新领域的好方法是阅读介绍性或总结性的博客（比如这篇），可以让你在花时间阅读论文之前快速了解背景。
以下的参考文献涵盖了NLP迁移学习的基本思想：
Deep contextualized word representations (NAACL 2018)
Matthew E. Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, Luke Zettlemoyer
Universal Language Model Fine-tuning for Text Classification (ACL 2018)
Jeremy Howard, Sebastian Ruder
Improving Language Understanding by Generative Pre-Training
Alec Radford, Karthik Narasimhan, Tim Salimans, Ilya Sutskever
Language Models are Unsupervised Multitask Learners
Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding (NAACL 2019)
Jacob Devlin, Ming-Wei Chang, Kenton Lee, Kristina Toutanova
Cloze-driven Pretraining of Self-attention Networks (arXiv 2019)
Alexei Baevski, Sergey Edunov, Yinhan Liu, Luke Zettlemoyer, Michael Auli
Unified Language Model Pre-training for Natural Language Understanding and Generation (arXiv 2019)
Li Dong, Nan Yang, Wenhui Wang, Furu Wei, Xiaodong Liu, Yu Wang, Jianfeng Gao, Ming Zhou, Hsiao-Wuen Hon
MASS: Masked Sequence to Sequence Pre-training for Language Generation (ICML 2019)
Kaitao Song, Xu Tan, Tao Qin, Jianfeng Lu, Tie-Yan Liu
Transformer结构已经成为序列建模任务流行结构。Source: Attention is all you need
表示学习(Representation Learning)
What you can cram into a single vector: Probing sentence embeddings for linguistic properties (ACL 2018)
Alexis Conneau, German Kruszewski, Guillaume Lample, Loïc Barrault, Marco Baroni
No Training Required: Exploring Random Encoders for Sentence Classification(ICLR 2019)
John Wieting, Douwe Kiela
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding (ICLR 2019)
Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, Samuel R. Bowman
SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systems (arXiv 2019)
Alex Wang, Yada Pruksachatkun, Nikita Nangia, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, Samuel R. Bowman
Linguistic Knowledge and Transferability of Contextual Representations (NAACL 2019)
Nelson F. Liu, Matt Gardner, Yonatan Belinkov, Matthew E. Peters, Noah A. Smith
To Tune or Not to Tune? Adapting Pretrained Representations to Diverse Tasks(arXiv 2019)
Matthew Peters, Sebastian Ruder, Noah A. Smith
A Neural Conversational Model (ICML Deep Learning Workshop 2015)
Oriol Vinyals, Quoc Le
A Persona-Based Neural Conversation Model (ACL 2016)
Jiwei Li, Michel Galley, Chris Brockett, Georgios P. Spithourakis, Jianfeng Gao, Bill Dolan
A Simple, Fast Diverse Decoding Algorithm for Neural Generation (arXiv 2017)
Jiwei Li, Will Monroe, Dan Jurafsky
Neural Approaches to Conversational AI (arXiv 2018)
Jianfeng Gao, Michel Galley, Lihong Li
TransferTransfo: A Transfer Learning Approach for Neural Network Based Conversational Agents (NeurIPS 2018 CAI Workshop)
Thomas Wolf, Victor Sanh, Julien Chaumond, Clement Delangue
Wizard of Wikipedia: Knowledge-Powered Conversational agents (ICLR 2019)
Emily Dinan, Stephen Roller, Kurt Shuster, Angela Fan, Michael Auli, Jason Weston
Learning to Speak and Act in a Fantasy Text Adventure Game (arXiv 2019)
Jack Urbanek, Angela Fan, Siddharth Karamcheti, Saachi Jain, Samuel Humeau, Emily Dinan, Tim Rocktäschel, Douwe Kiela, Arthur Szlam, Jason Weston
