Marek Rei

NOVEMBER 7, 2024

A recent survey paper by Calderon and Reichart [10] found that less than 10% of NLP interpretability papers consider inherently interpretable self-explaining models, with the authors advocating for more research on causal based interpretability methods. Weakly Supervised Explainable Phrasal Reasoning with Neural Fuzzy Logic.

Explainability 90

Explainability BERT Large Language Models NLP 90

Marek Rei

NOVEMBER 12, 2019

BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding Jacob Devlin, Ming-Wei Chang, Kenton Lee, Kristina Toutanova. ArXiv 2018. EMNLP 2018. NAACL 2018. NAACL 2018. At the end, I also include the summaries for my own published papers since the last iteration (papers 61-74). Here we go.

Machine Learning 52

Machine Learning NLP Neural Network BERT 52

Snorkel AI

MARCH 1, 2023

BERT BERT, an acronym that stands for “Bidirectional Encoder Representations from Transformers,” was one of the first foundation models and pre-dated the term by several years. BERT proved useful in several ways, including quantifying sentiment and predicting the words likely to follow in unfinished sentences.

BERT 83

BERT Natural Language Processing Large Language Models Neural Network 83

Unite.AI

AUGUST 1, 2023

Paper Title: "BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding" Key Takeaway: Introduced BERT, showcasing the efficacy of pre-training deep bidirectional models, thereby achieving state-of-the-art results on various NLP tasks. This demonstrates a classic case of ‘knowledge conflict'.

Prompt Engineer 246

Prompt Engineer Prompt Engineering ChatGPT Convolutional Neural Networks 246

Snorkel AI

MAY 25, 2023

BERT, the first breakout large language model In 2019, a team of researchers at Goole introduced BERT (which stands for bidirectional encoder representations from transformers). By making BERT bidirectional, it allowed the inputs and outputs to take each others’ context into account. BERT), or consist of both (e.g.,

Large Language Models 52

Large Language Models BERT Neural Network LLM 52

Snorkel AI

MAY 25, 2023

BERT, the first breakout large language model In 2019, a team of researchers at Goole introduced BERT (which stands for bidirectional encoder representations from transformers). By making BERT bidirectional, it allowed the inputs and outputs to take each others’ context into account. BERT), or consist of both (e.g.,

Large Language Models 52

Large Language Models BERT Neural Network LLM 52

Mlearning.ai

JANUARY 17, 2024

The foundations for today’s generative language applications were elaborated in the 1990s ( Hochreiter , Schmidhuber ), and the whole field took off around 2018 ( Radford , Devlin , et al.). The access to the GPT Store is for ChatGPT Plus users only (around $20 per month). You can create your own GPT and offer it to other users.

Generative AI 98

Generative AI Prompt Engineer Prompt Engineering Large Language Models 98

Sebastian Ruder

JANUARY 6, 2020

 Unsupervised pretraining was prevalent in NLP this year, mainly driven by BERT ( Devlin et al., A whole range of BERT variants have been applied to multimodal settings, mostly involving images and videos together with text (for an example see the figure below). 2019 ) and other variants. VideoBERT ( Sun et al., 2019 ; Wu et al.,

NLP 52

NLP ML Neural Network BERT 52

Explosion

DECEMBER 28, 2019

The update fixed outstanding bugs on the tracker, gave the docs a huge makeover, improved both speed and accuracy, made installation significantly easier and faster, and added some exciting new features, like ULMFit/BERT/ELMo-style language model pretraining. ✨ Mar 20: A few days later, we upgraded Prodigy to v1.8 to support spaCy v2.1.

NLP 52

NLP BERT Machine Learning Python 52

Sebastian Ruder

AUGUST 15, 2021

NLP is all about pre-trained Transformers This should come as no surprise but it's still interesting to see that among the 14 "hot" topics of 2021 (see below) were five pre-trained models (BERT, RoBERTa, BART, GPT-2, XLM-R) and one general "Language models" topic. ExplainaBoard: An Explainable Leaderboard for NLP.

NLP 52

NLP Explainability Neural Network Natural Language Processing 52

Sebastian Ruder

JANUARY 24, 2022

6] such as W2v-BERT [7] as well as more powerful multilingual models such as XLS-R [8]. For each input chunk, nearest neighbor chunks are retrieved using approximate nearest neighbor search based on BERT embedding similarity. W2v-BERT: Combining Contrastive Learning and Masked Language Modeling for Self-Supervised Speech Pre-Training.

NLP 52

NLP ML BERT Computational Linguistics 52

Explosion

MAY 17, 2023

In 2014 I started working on spaCy , and here’s an excerpt of how I explained the motivation for the library: Computers don’t understand text. In my opinion the best things to read on this are articles from when the developments were relatively fresh, such as Sebastian Ruder’s 2018 blog post NLP’s ImageNet moment has finally arrived.

LLM 135

LLM NLP Large Language Models OpenAI 135

Explosion

MARCH 17, 2019

In this post, we highlight some of the things we’re especially pleased with, and explain some of the most challenging parts of preparing this big release. Language model pretraining By far the biggest news in NLP research over 2018 was the success of language model pretraining. Version 2.1 Devlin et al.

NLP 52

NLP Python Neural Network Natural Language Processing 52

AWS Machine Learning Blog

AUGUST 7, 2023

It will explain the thought process and experimentation behind the solution, including the model training and development process. Part 2 will delve into the productionized solution, explaining the design decisions, data flow, and illustration of the model training and deployment architecture.

Large Language Models 82

Large Language Models LLM BERT NLP 82

Topbots

JUNE 29, 2023

4] In the open-source camp, initial attempts at solving the Text2SQL puzzle were focussed on auto-encoding models such as BERT, which excel at NLU tasks.[5, We will focus on the six requirements that seem most important for the task: accuracy, scalability, speed, explainability, privacy and adaptability over time.

Auto-complete 52

Auto-complete Algorithm Data Scientist Auto-classification 52