The Stanford AI Lab Blog

OCTOBER 13, 2021

In Proceedings of the IEEE International Conference on Computer Vision, pp. Distributionally robust neural networks for group shifts: On the importance of regularization for worst-case generalization. In Association for Computational Linguistics (ACL), pp. Selective classification for deep neural networks.

Neural Network 40

Neural Network Machine Learning Computational Linguistics Deep Learning 40

Sebastian Ruder

AUGUST 18, 2019

This goes back to layer-wise training of early deep neural networks ( Hinton et al., From shallow to deep  Over the last years, state-of-the-art models in NLP have become progressively deeper. In contrast, current models like BERT-Large and GPT-2 consist of 24 Transformer blocks and recent models are even deeper.

NLP 75

NLP BERT Natural Language Processing Computational Linguistics 75

Sebastian Ruder

JANUARY 24, 2022

Transactions of the Association for Computational Linguistics, 9, 978–994. Transactions of the Association for Computational Linguistics, 9, 570–585. In Advances in Neural Information Processing Systems 29 (NIPS 2016). link] ↩︎ Hendricks, L. Schneider, R., Alayrac, J. De Sa, C.,

NLP 52

NLP ML BERT Computational Linguistics 52