Ehud Reiter

AUGUST 25, 2024

” BERT/BART/etc can be used in data-to-text, but may not be best approach Around 2020 LSTMs got replaced by fine-tuned transformer language models such as BERT and BART. seemed to think that ACL was about neural language models, not about natural language processing in the wider sense.

BERT 135

BERT NLP Natural Language Processing Prompt Engineering 135

Mlearning.ai

APRIL 8, 2023

Charting the evolution of SOTA (State-of-the-art) techniques in NLP (Natural Language Processing) over the years, highlighting the key algorithms, influential figures, and groundbreaking papers that have shaped the field. Evolution of NLP Models To understand the full impact of the above evolutionary process.

NLP 98

NLP Neural Network Natural Language Processing Convolutional Neural Networks 98

Towards AI

JULY 20, 2023

Natural Language Processing Getting desirable data out of published reports and clinical trials and into systematic literature reviews (SLRs) — a process known as data extraction — is just one of a series of incredibly time-consuming, repetitive, and potentially error-prone steps involved in creating SLRs and meta-analyses.

Data Extraction 75

Data Extraction NLP Natural Language Processing Automation 75

Snorkel AI

MARCH 1, 2023

This process results in generalized models capable of a wide variety of tasks, such as image classification, natural language processing, and question-answering, with remarkable accuracy. BERT proved useful in several ways, including quantifying sentiment and predicting the words likely to follow in unfinished sentences.

BERT 83

BERT Natural Language Processing Large Language Models Neural Network 83

Sebastian Ruder

NOVEMBER 14, 2022

Models that allow interaction via natural language have become ubiquitious. Research models such as BERT and T5 have become much more accessible while the latest generation of language and multi-modal models are demonstrating increasingly powerful capabilities. The development cycle of a language model.

Natural Language Processing 52

Natural Language Processing NLP Computational Linguistics BERT 52

Towards AI

JULY 21, 2023

Photo by Kunal Shinde on Unsplash NATURAL LANGUAGE PROCESSING (NLP) WEEKLY NEWSLETTER NLP News Cypher | 08.09.20 Language diversity Estimate the language diversity of the sample of languages you are studying (Ponti et al., Research Work on methods that address the challenges of low-resource languages.

NLP 80

NLP Auto-complete Natural Language Processing BERT 80

The Stanford AI Lab Blog

MAY 31, 2022

Language Model Pretraining Language models (LMs), like BERT 1 and the GPT series 2 , achieve remarkable performance on many natural language processing (NLP) tasks. To achieve this, we first chunk each document into segments of roughly 256 tokens, which is half of the maximum BERT LM input length.

BERT 40

BERT Natural Language Processing NLP Neural Network 40

Sebastian Ruder

FEBRUARY 24, 2021

Fine-tuning a pre-trained language model (LM) has become the de facto standard for doing transfer learning in natural language processing. 2018 ) while pre-trained language models are favoured over models trained on translation ( McCann et al., 2018 ), natural language inference ( Conneau et al.,

Natural Language Processing 52

Natural Language Processing BERT NLP Computer Vision 52

Topbots

JUNE 15, 2023

Language Disparity in Natural Language Processing This digital divide in natural language processing (NLP) is an active area of research. 2 ] Multilingual models perform worse on several NLP tasks on low resource languages than on high resource languages such as English.[

Natural Language Processing 59

Natural Language Processing Computational Linguistics NLP ChatGPT 59

Viso.ai

FEBRUARY 25, 2023

Transformer models have become the de-facto status quo in Natural Language Processing (NLP). For example, the popular ChatGPT AI chatbot is a transformer-based language model. They are based on the transformer architecture, which was originally proposed for natural language processing (NLP) in 2017.

Convolutional Neural Networks 52

Convolutional Neural Networks Computer Vision Neural Network Natural Language Processing 52

Snorkel AI

MAY 25, 2023

A brief history of large language models Large language models grew out of research and experiments with neural networks to allow computers to process natural language. In the 2010s, this research intersected with the then-bustling field of neural networks, setting the ground for the first large language model.

Large Language Models 52

Large Language Models BERT Neural Network LLM 52

Snorkel AI

MAY 25, 2023

A brief history of large language models Large language models grew out of research and experiments with neural networks to allow computers to process natural language. In the 2010s, this research intersected with the then-bustling field of neural networks, setting the ground for the first large language model.

Large Language Models 52

Large Language Models BERT Neural Network LLM 52

Heartbeat

JUNE 6, 2023

In recent years, researchers have also explored using GCNs for natural language processing (NLP) tasks, such as text classification , sentiment analysis , and entity recognition. Once the GCN is trained, it is easier to process new graphs and make predictions about them. Richong, Z., Yongyi, M., & Xudong L.

NLP 59

NLP Convolutional Neural Networks Neural Network Natural Language Processing 59

Sebastian Ruder

DECEMBER 6, 2021

Reading Comprehension assumes a gold paragraph is provided Standard approaches for reading comprehension build on pre-trained models such as BERT. Using BERT for reading comprehension involves fine-tuning it to predict a) whether a question is answerable and b) whether each token is the start and end of an answer span.

BERT 52

BERT NLP Natural Language Processing Computational Linguistics 52

Viso.ai

JUNE 12, 2024

Their applications include various Natural Language Processing ( NLP ) tasks, including question answering, text summarization, sentiment analysis , etc. Previous ChatGPT models The GPT-1 version was introduced in June 2018 as a method for language understanding by using generative pre-training.

Large Language Models 52

Large Language Models ChatGPT Computer Vision Neural Network 52

Sebastian Ruder

JANUARY 6, 2020

 Unsupervised pretraining was prevalent in NLP this year, mainly driven by BERT ( Devlin et al., A whole range of BERT variants have been applied to multimodal settings, mostly involving images and videos together with text (for an example see the figure below). 2019 ; Tran, 2020 ), which can be seen below. 2019 ; Wu et al.,

NLP 52

NLP ML Neural Network BERT 52

Dlabs.ai

AUGUST 10, 2020

Despite 80% of surveyed businesses wanting to use chatbots in 2020 , how many do you think will implement them well? On principle, all chatbots work by utilising some form of natural language processing (NLP). That in itself is a vast field requiring expertise and cooperation between computer science and linguistics.

Natural Language Processing 52

Natural Language Processing Chatbots Neural Network Machine Learning 52

Lexalytics

APRIL 5, 2021

This is the sort of representation that is useful for natural language processing. ELMo would also be the first of the Muppet-themed language models that would come to include ERNIE [ 120 ], Grover [ 121 ]….and The base model of BERT [ 103 ] had 12 (!) layers of bidirectional Transformers.

Neural Network 52

Neural Network Convolutional Neural Networks Natural Language Processing BERT 52

Sebastian Ruder

JANUARY 24, 2022

2021) 2021 saw many exciting advances in machine learning (ML) and natural language processing (NLP). 6] such as W2v-BERT [7] as well as more powerful multilingual models such as XLS-R [8]. We have also seen new ways to integrate retrieval into pre-trained language models [94] [95]. What happened?  

NLP 52

NLP ML BERT Computational Linguistics 52

Explosion

DECEMBER 28, 2019

The update fixed outstanding bugs on the tracker, gave the docs a huge makeover, improved both speed and accuracy, made installation significantly easier and faster, and added some exciting new features, like ULMFit/BERT/ELMo-style language model pretraining. Dec 9: Ines’ key thoughts on trends in AI from 2019 and looking into 2020.

NLP 52

NLP BERT Machine Learning Python 52

Sebastian Ruder

FEBRUARY 12, 2024

5 years ago, around the time I finished my PhD, if you wanted to work on cutting-edge natural language processing (NLP), your choice was relatively limited. It’s crazy how the AI and NLP landscape has evolved over the last five years. Recently, I decided to go on the job market again, which has become much more diverse.

NLP 52

NLP LLM AI AI 52

Applied Data Science

DECEMBER 23, 2022

In our review of 2019 we talked a lot about reinforcement learning and Generative Adversarial Networks (GANs), in 2020 we focused on Natural Language Processing (NLP) and algorithmic bias, in 202 1 Transformers stole the spotlight. The debate was on again: maybe language generation is really just a prediction task?

Neural Network 52

Neural Network Data Science AI AI 52

Explosion

MAY 17, 2023

A lot of people are building truly new things with Large Language Models (LLMs), like wild interactive fiction experiences that weren’t possible before. But if you’re working on the same sort of Natural Language Processing (NLP) problems that businesses have been trying to solve for a long time, what’s the best way to use them?

LLM 135

LLM NLP Large Language Models OpenAI 135

Applied Data Science

MARCH 11, 2022

For machine learning researchers there are many reasons why this is more than a philosophical question: Natural Language Processing (NLP) is an area containing some of the most important machine learning applications, like translation and text summarisation. The topic is in red. The agents won this one!

Neural Network 52

Neural Network Data Science AI AI 52

Marek Rei

NOVEMBER 4, 2024

I have written short summaries of 68 different research papers published in the areas of Machine Learning and Natural Language Processing. They annotate a new test set of news data from 2020 and find that performance of certain models holds up very well and the field luckily hasn’t overfitted to the CoNLL 2003 test set.

Machine Learning 80

Machine Learning NLP Large Language Models LLM 80

AWS Machine Learning Blog

SEPTEMBER 11, 2024

The recent history of PBAs begins in 1999, when NVIDIA released its first product expressly marketed as a GPU, designed to accelerate computer graphics and image processing. In 2018, other forms of PBAs became available, and by 2020, PBAs were being widely used for parallel problems, such as training of NN.

ML 95

ML Deep Learning Algorithm Large Language Models 95

Unite.AI

OCTOBER 8, 2024

The paper states*: ‘Gender biases present in train data are known to bleed into natural language processing (NLP) systems, resulting in dissemination and potential amplification of those biases. For this approach, the authors made use of a lattice rescoring method from an earlier 2020 work.

LLM 173

LLM NLP Natural Language Processing BERT 173

deepsense.ai

NOVEMBER 14, 2023

Within Natural Language Processing (NLP), ‘pseudo-evaluation’ approaches that we call ‘Superficial Utility Comparison Kriterion’ ( SUCK ) methods, like BLEU [32], METEOR [33], ROUGE [34], or BLEURT [35], attempt to salvage the situation. 2023 [link] [link] [link] BERTScore: Evaluating text generation with BERT , Zhang T.,

Generative AI 111

Generative AI Prompt Engineering Prompt Engineer ML 111

Unite.AI

SEPTEMBER 26, 2024

The momentum continued in 2017 with the introduction of transformer models like BERT and GPT, which revolutionized natural language processing. By 2020, OpenAI's GPT-3 set new standards for AI capabilities, highlighting the high costs of training such large models.

AI 147

AI AI Neural Network Natural Language Processing 147

Unite.AI

AUGUST 8, 2023

These advanced AI deep learning models have seamlessly integrated into various applications, from Google's search engine enhancements with BERT to GitHub’s Copilot, which harnesses the capability of Large Language Models (LLMs) to convert simple code snippets into fully functional source codes.

Generative AI 243

Generative AI ChatGPT Neural Network Convolutional Neural Networks 243

Topbots

JUNE 29, 2023

4] In the open-source camp, initial attempts at solving the Text2SQL puzzle were focussed on auto-encoding models such as BERT, which excel at NLU tasks.[5, Content Enhanced BERT-based Text-to-SQL Generation [8] Torsten Scholak et al. Speak to your Parser: Interactive Text-to-SQL with Natural Language Feedback [16] Janna Lipenkova.

Auto-complete 52

Auto-complete Algorithm Data Scientist Auto-classification 52