Sebastian Ruder

AUGUST 18, 2019

This post expands on the NAACL 2019 tutorial on Transfer Learning in NLP. In the span of little more than a year, transfer learning in the form of pretrained language models has become ubiquitous in NLP and has contributed to the state of the art on a wide range of tasks. However, transfer learning is not a recent phenomenon in NLP.

NLP 75

NLP BERT Natural Language Processing Computational Linguistics 75

Topbots

JUNE 15, 2023

Language Disparity in Natural Language Processing This digital divide in natural language processing (NLP) is an active area of research. 70% of research papers published in a computational linguistics conference only evaluated English.[ Square One Bias in NLP: Towards a Multi-Dimensional Exploration of the Research Manifold.

Natural Language Processing 59

Natural Language Processing Computational Linguistics NLP ChatGPT 59

Sebastian Ruder

DECEMBER 6, 2021

2013 ), MCScript ( Modi et al., Among current key applications of NLP, QA has the lowest linguistic global utility, i.e. performance averaged across the world's languages ( Blasi et al., Linguistic and demographic utility of different NLP applications ( Blasi et al., 2016 ), and BookTest ( Bajgar et al.,

BERT 52

BERT NLP Natural Language Processing Computational Linguistics 52

Explosion

DECEMBER 17, 2013

I wrote this blog post in 2013, describing an exciting advance in natural language understanding technology. You should probably at least skim that post before reading this one, unless you’re very familiar with NLP research. The derivation for the transition system we’re using, Arc Hybrid, is in Goldberg and Nivre (2013).

Python 45

Python Algorithm NLP Computational Linguistics 45

ML Review

MARCH 5, 2019

vector: Probing sentence embeddings for linguistic properties. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) (Vol. Star our repo: ai-distillery And clap your little hearts out for MTank ! References Harris, Z. Distributional structure. Word, 10(2–3), 146–162.

AI 40

AI AI Computer Vision Computational Linguistics 40

NLP People

JULY 24, 2019

The 57th Annual Meeting of the Association for Computational Linguistics (ACL 2019) is starting this week in Florence, Italy. We took the opportunity to review major research trends in the animated NLP space and formulate some implications from the business perspective. But what is the substance behind the buzz?

NLP 52

NLP Neural Network Deep Learning Natural Language Processing 52