Sebastian Ruder

NOVEMBER 14, 2022

Research models such as BERT and T5 have become much more accessible while the latest generation of language and multi-modal models are demonstrating increasingly powerful capabilities. Models that allow interaction via natural language have become ubiquitious. The size of the gradient circle represents the number of languages in the class.

Natural Language Processing 52

Natural Language Processing NLP Computational Linguistics BERT 52

Sebastian Ruder

DECEMBER 6, 2021

Reading Comprehension assumes a gold paragraph is provided Standard approaches for reading comprehension build on pre-trained models such as BERT. Using BERT for reading comprehension involves fine-tuning it to predict a) whether a question is answerable and b) whether each token is the start and end of an answer span.

BERT 52

BERT NLP Natural Language Processing Computational Linguistics 52

Topbots

JUNE 15, 2023

70% of research papers published in a computational linguistics conference only evaluated English.[ In Findings of the Association for Computational Linguistics: ACL 2022 , pages 2340–2354, Dublin, Ireland. Association for Computational Linguistics. Are All Languages Created Equal in Multilingual BERT?

Natural Language Processing 59

Natural Language Processing Computational Linguistics NLP ChatGPT 59

Explosion

DECEMBER 28, 2019

The update fixed outstanding bugs on the tracker, gave the docs a huge makeover, improved both speed and accuracy, made installation significantly easier and faster, and added some exciting new features, like ULMFit/BERT/ELMo-style language model pretraining. Dec 9: Ines’ key thoughts on trends in AI from 2019 and looking into 2020.

NLP 52

NLP BERT Machine Learning Python 52

Sebastian Ruder

JANUARY 24, 2022

6] such as W2v-BERT [7] as well as more powerful multilingual models such as XLS-R [8]. For each input chunk, nearest neighbor chunks are retrieved using approximate nearest neighbor search based on BERT embedding similarity. Advances in Neural Information Processing Systems, 2020. What happened?   Why is it important?  

NLP 52

NLP ML BERT Computational Linguistics 52

The Stanford AI Lab Blog

JANUARY 21, 2022

Conference of the North American Chapter of the Association for Computational Linguistics. ↩ Devlin, J., BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. RoBERTa: A Robustly Optimized BERT Pretraining Approach. Florence: A New Foundation Model for Computer Vision. Neumann, M.,

Robotics 40

Robotics Computational Linguistics BERT Computer Vision 40

Applied Data Science

DECEMBER 23, 2022

In our review of 2019 we talked a lot about reinforcement learning and Generative Adversarial Networks (GANs), in 2020 we focused on Natural Language Processing (NLP) and algorithmic bias, in 202 1 Transformers stole the spotlight. Just wait until you hear what happened in 2022. What happened?

Neural Network 52

Neural Network Data Science AI AI 52