Snorkel AI

MAY 25, 2023

BERT, the first breakout large language model In 2019, a team of researchers at Goole introduced BERT (which stands for bidirectional encoder representations from transformers). By making BERT bidirectional, it allowed the inputs and outputs to take each others’ context into account. Next, OpenAI released GPT-3 in June of 2020.

Large Language Models 52

Large Language Models BERT Neural Network LLM 52

Snorkel AI

MAY 25, 2023

BERT, the first breakout large language model In 2019, a team of researchers at Goole introduced BERT (which stands for bidirectional encoder representations from transformers). By making BERT bidirectional, it allowed the inputs and outputs to take each others’ context into account. Next, OpenAI released GPT-3 in June of 2020.

Large Language Models 52

Large Language Models BERT Neural Network LLM 52

Sebastian Ruder

JANUARY 6, 2020

 Unsupervised pretraining was prevalent in NLP this year, mainly driven by BERT ( Devlin et al., A whole range of BERT variants have been applied to multimodal settings, mostly involving images and videos together with text (for an example see the figure below). 2019 ; Tran, 2020 ), which can be seen below. 2019 ; Wu et al.,

NLP 52

NLP ML Neural Network BERT 52

Sebastian Ruder

JANUARY 24, 2022

6] such as W2v-BERT [7] as well as more powerful multilingual models such as XLS-R [8]. For each input chunk, nearest neighbor chunks are retrieved using approximate nearest neighbor search based on BERT embedding similarity. Advances in Neural Information Processing Systems, 2020. Multiple templates are possible for each task.

NLP 52

NLP ML BERT Computational Linguistics 52

John Snow Labs

MAY 15, 2023

Explainability and Interpretability: Models should be capable of answer stakeholder questions about the decision-making processes of AI systems. Personal information leakage has been shown to be as high as 50-70% in popular word and sentence embeddings according to [ Song & Raghunathan 2020 ].

NLP 52

NLP Data Scientist Software Engineer Responsible AI 52

Explosion

DECEMBER 28, 2019

The update fixed outstanding bugs on the tracker, gave the docs a huge makeover, improved both speed and accuracy, made installation significantly easier and faster, and added some exciting new features, like ULMFit/BERT/ELMo-style language model pretraining. Dec 9: Ines’ key thoughts on trends in AI from 2019 and looking into 2020.

NLP 52

NLP BERT Machine Learning Python 52

Mlearning.ai

JANUARY 17, 2024

Major milestones in the last few years comprised BERT (Google, 2018), GPT-3 (OpenAI, 2020), Dall-E (OpenAI, 2021), Stable Diffusion (Stability AI, LMU Munich, 2022), ChatGPT (OpenAI, 2022). The access to the GPT Store is for ChatGPT Plus users only (around $20 per month). You can create your own GPT and offer it to other users.

Generative AI 98

Generative AI Prompt Engineering Prompt Engineer Large Language Models 98

Heartbeat

JUNE 6, 2023

Editorially independent, Heartbeat is sponsored and published by Comet, an MLOps platform that enables data scientists & ML teams to track, compare, explain, & optimize their experiments. References Paperwithcode | Graph Convolutional Network Kai, S., Richong, Z., Yongyi, M., & Xudong L. & Nie, JY.

NLP 59

NLP Convolutional Neural Networks Neural Network Natural Language Processing 59

Marek Rei

NOVEMBER 4, 2024

link] Proposes an explainability method for language modelling that explains why one word was predicted instead of a specific other word. Adapts three different explainability methods to this contrastive approach and evaluates on a dataset of minimally different sentences. UC Berkeley, CMU. EMNLP 2022. Imperial, Cambridge, KCL.

Machine Learning 80

Machine Learning NLP Large Language Models LLM 80

Dlabs.ai

AUGUST 10, 2020

Despite 80% of surveyed businesses wanting to use chatbots in 2020 , how many do you think will implement them well? The solution is based on a Transformer-type neural network, used in the BERT model as well, that has recently triumphed in the field of machine learning and natural language understanding.

Natural Language Processing 52

Natural Language Processing Chatbots Neural Network Machine Learning 52

Sebastian Ruder

DECEMBER 5, 2023

 Similar to  earlier years  where BERT was ubiquitous, instruction-tuned language models (LMs) and large language models (LLMs) are used in almost every paper. INSTRUCTSCORE: Explainable Text Generation Evaluation with Finegrained Feedback (Xu et al.).  This Evaluation based on LLMs is increasingly common.  While

NLP 52

NLP Large Language Models LLM Explainability 52

Applied Data Science

MARCH 11, 2022

This explain this statement at the NeurIPS 2017 Test-of-Time Award: It seems easier to train a bi-directional LSTM with attention than to compute the PCA of a large matrix. — Rahimi Each agent’s reward is related to the variance explained by its own eigenvector. The topic is in red. The agents won this one!

Neural Network 52

Neural Network Data Science AI AI 52

ML Review

DECEMBER 16, 2019

Hwang et a l present SQLova, a semantic parsing model that uses BERT to encode questions and table headers, and attention-based decoder that produces SQL constructs (for instance, SELECTs, WHERE clauses, aggregation functions, etc) that are later ranked and evaluated. Thank you for reading!

Machine Learning 52

Machine Learning Neural Network Explainability Convolutional Neural Networks 52

Explosion

MAY 17, 2023

In 2014 I started working on spaCy , and here’s an excerpt of how I explained the motivation for the library: Computers don’t understand text. In their experiments, OpenAI prompted GPT3 with 32 examples of each task, and found that they were able to achieve similar accuracy to the BERT baselines. The results in Section 3.7,

LLM 135

LLM NLP Large Language Models OpenAI 135

Topbots

JUNE 29, 2023

4] In the open-source camp, initial attempts at solving the Text2SQL puzzle were focussed on auto-encoding models such as BERT, which excel at NLU tasks.[5, We will focus on the six requirements that seem most important for the task: accuracy, scalability, speed, explainability, privacy and adaptability over time.

Auto-complete 52

Auto-complete Algorithm Data Scientist Auto-classification 52