Sebastian Ruder

DECEMBER 6, 2021

2016 ), Natural Questions (NQ; Kwiatkowski et al., 2016 ), among many others. 2016 ), and BookTest ( Bajgar et al., 2016 ) or narratives written by crowd workers such as MCTest ( Richardson et al., 2016 ), and ROCStories ( Mostafazedh et al., 2016 ), and BookTest ( Bajgar et al., 2016 ; Chandu et al.,

BERT 52

BERT NLP Natural Language Processing Computational Linguistics 52

Sebastian Ruder

AUGUST 18, 2019

From shallow to deep  Over the last years, state-of-the-art models in NLP have become progressively deeper. Up to two years ago, the state of the art on most tasks was a 2-3 layer deep BiLSTM, with machine translation being an outlier with 16 layers ( Wu et al.,

NLP 75

NLP BERT Natural Language Processing Computational Linguistics 75

Sebastian Ruder

JANUARY 24, 2022

Transactions of the Association for Computational Linguistics, 9, 978–994. Transactions of the Association for Computational Linguistics, 9, 570–585. In Advances in Neural Information Processing Systems 29 (NIPS 2016). Transactions of the Association for Computational Linguistics, 9, 362–373.

NLP 52

NLP ML BERT Computational Linguistics 52

The Stanford AI Lab Blog

OCTOBER 13, 2021

In Association for Computational Linguistics (ACL), pp. In International Conference on Machine Learning (ICML), 2016. ↩ ↩ 2 Yonatan Geifman and Ran El-Yaniv. In World Wide Web (WWW), pp. 491–500, 2019. ↩ Adina Williams, Nikita Nangia, and Samuel Bowman. Selective classification for deep neural networks.

Neural Network 40

Neural Network Machine Learning Computational Linguistics Deep Learning 40