Sebastian Ruder

NOVEMBER 14, 2022

Models that allow interaction via natural language have become ubiquitious. Research models such as BERT and T5 have become much more accessible while the latest generation of language and multi-modal models are demonstrating increasingly powerful capabilities. Transfer learning in natural language processing.

Natural Language Processing 52

Natural Language Processing NLP Computational Linguistics BERT 52

Sebastian Ruder

AUGUST 18, 2019

In contrast, current models like BERT-Large and GPT-2 consist of 24 Transformer blocks and recent models are even deeper. The latter in particular finds that simply training BERT for longer and on more data improves results, while GPT-2 8B reduces perplexity on a language modelling dataset (though only by a comparatively small factor).

NLP 75

NLP BERT Natural Language Processing Computational Linguistics 75

Explosion

DECEMBER 28, 2019

The update fixed outstanding bugs on the tracker, gave the docs a huge makeover, improved both speed and accuracy, made installation significantly easier and faster, and added some exciting new features, like ULMFit/BERT/ELMo-style language model pretraining. ✨ Mar 20: A few days later, we upgraded Prodigy to v1.8

NLP 52

NLP BERT Machine Learning Python 52

Sebastian Ruder

JANUARY 24, 2022

2021) 2021 saw many exciting advances in machine learning (ML) and natural language processing (NLP). 6] such as W2v-BERT [7] as well as more powerful multilingual models such as XLS-R [8]. Advances in Neural Information Processing Systems, 2020. Credit for the title image: Liu et al. Why is it important?  

NLP 52

NLP ML BERT Computational Linguistics 52

Applied Data Science

DECEMBER 23, 2022

In our review of 2019 we talked a lot about reinforcement learning and Generative Adversarial Networks (GANs), in 2020 we focused on Natural Language Processing (NLP) and algorithmic bias, in 202 1 Transformers stole the spotlight. As humans we do not know exactly how we learn language: it just happens. What happened?

Neural Network 52

Neural Network Data Science AI AI 52

Sebastian Ruder

DECEMBER 6, 2021

Reading Comprehension assumes a gold paragraph is provided Standard approaches for reading comprehension build on pre-trained models such as BERT. Using BERT for reading comprehension involves fine-tuning it to predict a) whether a question is answerable and b) whether each token is the start and end of an answer span.

BERT 52

BERT NLP Natural Language Processing Computational Linguistics 52

Marek Rei

NOVEMBER 4, 2024

I have written short summaries of 68 different research papers published in the areas of Machine Learning and Natural Language Processing. Computational Linguistics 2022. link] Developing a system for the detection of cognitive impairment based on linguistic features. University of Szeged.

Machine Learning 80

Machine Learning NLP Large Language Models LLM 80

NLP People

JULY 24, 2019

The 57th Annual Meeting of the Association for Computational Linguistics (ACL 2019) is starting this week in Florence, Italy. Especially pre-trained word embeddings such as Word2Vec, FastText and BERT allow NLP developers to jump to the next level. Neural Network Methods in Natural Language Processing.

NLP 52

NLP Neural Network Deep Learning Natural Language Processing 52