Sebastian Ruder

AUGUST 18, 2019

This post expands on the NAACL 2019 tutorial on Transfer Learning in NLP. In the span of little more than a year, transfer learning in the form of pretrained language models has become ubiquitous in NLP and has contributed to the state of the art on a wide range of tasks. However, transfer learning is not a recent phenomenon in NLP.

NLP 75

NLP BERT Natural Language Processing Computational Linguistics 75

Explosion

NOVEMBER 9, 2016

Most NLP problems can be reduced to machine learning problems that take one or more texts as input. However, most NLP problems require understanding of longer spans of text, not just individual words. This has always been a huge weakness of NLP models. 2016) presented a model that achieved 86.8% Now we have a solution.

Deep Learning 52

Deep Learning NLP Convolutional Neural Networks Neural Network 52

Marek Rei

FEBRUARY 5, 2020

2019 was another record breaking year in machine learning and NLP research. Andrew started the process with a subset of 2018 conferences for our joint article on The Geographic Diversity of NLP Conferences. Out of the top 7, Cambridge and Edinburgh publish quite a bit in NLP, while the others are focusing mainly on ML.

NLP 52

NLP ML Machine Learning AI 52

Heartbeat

AUGUST 23, 2023

Enter Natural Language Processing (NLP) and its transformational power. This is the promise of NLP: to transform the way we approach legal discovery. The seemingly impossible chore of sorting through mountains of legal documents can be accomplished with astonishing efficiency and precision using NLP.

NLP 52

NLP Natural Language Processing Algorithm Categorization 52

Towards AI

SEPTEMBER 1, 2023

SpaCy is a language processing library written in Python and Cython that has been well-established since 2016. With little or no examples, they are able to perform language processing operations such as classification, sentiment analysis, concept extraction, named entity recognition, translation, and more.

LLM 116

LLM Large Language Models Deep Learning Python 116

Unite.AI

FEBRUARY 23, 2024

Back in 2016, after a couple of years working as a Product Engineer, I started to entertain the idea of a startup as a technical co-founder. We research their content, determine what heading structures they use, NLP keywords and phrases, and at what frequencies, among other things.

AI Tools 162

AI Tools Algorithm NLP AI 162

Unite.AI

AUGUST 5, 2024

Groq, founded in 2016 by Jonathan Ross, a former Google engineer, has been quietly developing specialized chips designed to accelerate AI workloads, particularly in the realm of language processing. This financial windfall, led by investment giant BlackRock, has catapulted Groq's valuation to an impressive $2.8

Large Language Models 162

Large Language Models Natural Language Processing AI AI 162

NYU Center for Data Science

JULY 18, 2023

The group was first launched in 2016 by Associate Professor of Computer Science, Data Science and Mathematics Joan Bruna , and Associate Professor of Mathematics and Data Science and incoming CDS Interim Director Carlos Fernandez-Granda with the goal of advancing the mathematical and statistical foundations of data science.

Data Science 98

Data Science Machine Learning Natural Language Processing Robotics 98

Hal Daumé III

JULY 12, 2016

I like this paper because the problem is great (I think NLP should really push in the direction of helping learners learn and teachers teach!), The approach is a reasonably standard ILP formulation, with some additional machinery to deal with the fact that the data is very sparse.

Neural Network 40

Neural Network NLP ML 40

Hal Daumé III

AUGUST 14, 2016

P16-1152 : Artem Sokolov; Julia Kreutzer; Christopher Lo; Stefan Riezler Learning Structured Predictors from Bandit Feedback for Interactive NLP This was perhaps my favorite paper of the conference because it's trying to do something new and hard and takes a nice approach. P16-2096 : Dirk Hovy; Shannon L.

Neural Network 40

Neural Network NLP Natural Language Processing 40

Sebastian Ruder

JANUARY 19, 2021

The selection of areas and methods is heavily influenced by my own interests; the selected topics are biased towards representation and transfer learning and towards natural language processing (NLP). This is less of a problem in NLP where unsupervised pre-training involves classification over thousands of word types.

NLP 52

NLP ML Computer Vision Natural Language Processing 52

AssemblyAI

NOVEMBER 8, 2023

Main Limitations : Inability of the neural network to handle mixed-case words (e.g., "AssemblyAI") and acronyms (e.g., “NLP”), and reliance on rule-based refinement via a lookup table for casing, which hampers efficiency, scalability, and generalization. 2016 is still at the forefront of the SOTA models.

Neural Network 147

Neural Network BERT Large Language Models Deep Learning 147

Explosion

OCTOBER 22, 2016

This is a guest post by Wah Loon Keng , the author of spacy-nlp , a client that exposes spaCy ’s NLP text parsing to Node.js (and other languages) via Socket.IO. Sat Oct 22 2016 17:36:14 GMT+0000 (UTC)] INFO Starting poly-socketio server on port: 6466, expecting 4 IO clients. We then install spaCy and Socket.IO CLI: 2.4.0,

NLP 52

NLP Python Natural Language Processing AI 52

Unite.AI

FEBRUARY 16, 2024

For example, see Face-to-Face Interaction with Pedagogical Agents, Twenty Years Later , a 2016 article that overviews the field and cites a lot of the relevant material. Natural language processing (NLP) , natural language understanding and dialogue management that processes the content of the student's speech and produces the next response.

Natural Language Processing 173

Natural Language Processing UX Design AI AI 173

Sebastian Ruder

JANUARY 24, 2022

2021) 2021 saw many exciting advances in machine learning (ML) and natural language processing (NLP). If CNNs are pre-trained the same way as transformer models, they achieve competitive performance on many NLP tasks [28].   Popularized by GPT-3 [32] , prompting has emerged as a viable alternative input format for NLP models.

NLP 52

NLP ML BERT Computational Linguistics 52

AWS Machine Learning Blog

FEBRUARY 21, 2025

Context (Snippet from PDF file) Question Answer THIS STRATEGIC ALLIANCE AGREEMENT (Agreement) is made and entered into as of November 6, 2016 (the Effective Date) by and between Dialog Semiconductor (UK) Ltd., His area of research is all things natural language (like NLP, NLU, and NLG). Outside of work, Yunfei enjoys reading and music.

LLM 84

LLM AI AI Data Scientist 84

Mlearning.ai

FEBRUARY 15, 2023

Introduction In natural language processing, text categorization tasks are common (NLP). Submission Suggestions Text Classification in NLP using Cross Validation and BERT was originally published in MLearning.ai Depending on the data they are provided, different classifiers may perform better or worse (eg. Uysal and Gunal, 2014).

BERT 52

BERT NLP Natural Language Processing Algorithm 52

AWS Machine Learning Blog

NOVEMBER 9, 2023

See Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks , REALM , kNN-LM and RETRO. In the past, she had worked on several NLP-based services such as Comprehend Medical, a medical diagnosis system at Amazon Health AI and Machine Translation system at Meta AI. She received her PhD from Virginia Tech in 2017.

Natural Language Processing 108

Natural Language Processing NLP Large Language Models LLM 108

Probably Approximately a Scientific Blog

AUGUST 28, 2016

Recently, I attended Chris Biemann's excellent crowdsourcing course at ESSLLI 2016 (the 28th European Summer School in Logic, Language and Information), and was inspired to write about the topic. I've collected data for most of my papers, but never thought of it as an interesting blog post topic.

NLP 52

NLP Explainability Natural Language Processing 52

Artificial Corner

JUNE 18, 2023

ChatGPT released by OpenAI is a versatile Natural Language Processing (NLP) system that comprehends the conversation context to provide relevant responses. Question Answering has been an active research area in NLP for many years so there are several datasets that have been created for evaluating QA systems.

ChatGPT 105

ChatGPT Natural Language Processing BERT NLP 105

Topbots

JUNE 22, 2023

SA is a very widespread Natural Language Processing (NLP). Hence, whether general domain ML models can be as capable as domain-specific models is still an open research question in NLP. Other articles in my line of research (NLP, RL) Lima Paiva, F. It has several applications and thus can be used in several domains (e.g.,

Machine Learning 111

Machine Learning ChatGPT Natural Language Processing Categorization 111

AWS Machine Learning Blog

MARCH 28, 2024

News CommonCrawl is a dataset released by CommonCrawl in 2016. News CommonCrawl SEC Filing Coverage 2016-2022 1993-2022 Size 25.8 His research interests are in NLP, Generative AI, and LLM Agents. Publicly listed companies are required to file various documents regularly. This creates a large number of documents over the years.

Large Language Models 123

Large Language Models LLM Machine Learning Generative AI 123

Explosion

JANUARY 12, 2018

We founded Explosion in October 2016, so this was our first full calendar year in operation. In August 2016, Ines wrote a post on how AI developers could benefit from better tooling and more careful attention to interaction design. spaCy’s Machine Learning library for NLP in Python. Here’s what we got done.

NLP 52

NLP Neural Network Machine Learning Python 52

Explosion

OCTOBER 2, 2016

We launched displaCy as a visualiser for our NLP library spaCy in 2015 and open-sourced the code in August 2016. It supports serving the visualizations in the browser, generating the raw markup or outputting the results in a Jupyter notebook. For more details, see the visualizers documentation.

NLP 40

NLP Robotics 40

Defined.ai blog

FEBRUARY 26, 2023

This has been a longstanding concern for large companies with distributed workforces, with companies like Apple acquiring startups like Emotient all the way back in 2016. Voci Technologies provides AI that can analyze meeting transcriptions and give you a sentiment report, helping you improve team dynamics and boost morale.

Categorization 98

Categorization AI AI Automation 98

Snorkel AI

MARCH 1, 2023

2016) This paper introduced DCGANs, a type of generative model that uses convolutional neural networks to generate images with high fidelity. The company is best known for NLP tools, but also enables the use of computer vision, audio, and multimodal models. Attention Is All You Need Vaswani et al.

BERT 83

BERT Natural Language Processing Large Language Models Neural Network 83

NLP People

JULY 31, 2017

Tay Microsoft unveiled Tay (short for “thinking about you”) in March 2016 on Twitter. Dallas’ Robocop In 2016, police in Dallas were tracking a man alleged to have fatally shot five men. According to Bloomberg , scientists even met in 2017 to discuss the apparently very real dangers behind AI. Is there any danger?

Artificial Intelligence 52

Artificial Intelligence Artificial Intelligence Robotics Algorithm 52

AWS Machine Learning Blog

APRIL 18, 2023

Large language models (LLMs) with billions of parameters are currently at the forefront of natural language processing (NLP). Although LLMs are capable of performing various NLP tasks, they are considered generalists and not specialists. He specializes in deep learning, especially in the area of NLP and CV.

Large Language Models 83

Large Language Models NLP LLM Deep Learning 83

Explosion

DECEMBER 30, 2021

Mar 17: March saw a new episode of Vincent Warmerdam’s “Intro to NLP with spaCy” series. Jun 4: At the beginning of June, Ines and Matt gave a talk at the Bay Area NLP group , one of the largest NLP communities in the world. ? Jun 10: Ines gave a keynote at the Teaching NLP Workshop at NAACL-2021. Mar 4: We released 1.0

NLP 52

NLP Machine Learning Python Natural Language Processing 52

AWS Machine Learning Blog

MARCH 31, 2023

He has been with the Transportation Cabinet since 2016 working in various IT roles. The contact center is powered by Amazon Connect, and Max, the virtual agent, is powered by Amazon Lex and the AWS QnABot solution. Amazon Connect directs some incoming calls to the virtual agent (Max) by identifying the caller number.

Automation 95

Automation Natural Language Processing NLP Conversational AI 95

AWS Machine Learning Blog

JULY 15, 2024

Before Mission Cloud, she worked as an ML and software engineer at Amazon for six years, specializing in recommender systems for Amazon fashion shopping and NLP for Alexa. She is also the recipient of the Best Paper Award at IEEE NetSoft 2016, IEEE ICC 2011, ONDM 2010, and IEEE GLOBECOM 2005. Cristian Torres is a Sr.

Big Data 121

Big Data Big Data Engineer Generative AI ML 121

ML Review

JUNE 4, 2018

Recent Intersections Between Computer Vision and Natural Language Processing (Part Two) This is the second instalment of our latest publication series looking at some of the intersections between Computer Vision (CV) and Natural Language Processing (NLP). Source : Britz (2016)[ 62 ] CNNs can encode abstract features from images.

Neural Network 52

Neural Network Convolutional Neural Networks Computer Vision Deep Learning 52

ODSC - Open Data Science

SEPTEMBER 22, 2023

Before founding Gretel, he launched Harvest.ai, a security startup that leveraged NLP and AI to protect cloud data, which was acquired by Amazon in 2016. His current mission is to leverage AI and NLP technology to create the enterprise solution that speaks the language of each individual workplace.

Software Engineer 52

Software Engineer Data Science Machine Learning NLP 52

Sebastian Ruder

DECEMBER 6, 2021

2016 ), Natural Questions (NQ; Kwiatkowski et al., 2016 ), among many others. 2016 ), and BookTest ( Bajgar et al., 2016 ) or narratives written by crowd workers such as MCTest ( Richardson et al., 2016 ), and ROCStories ( Mostafazedh et al., 2016 ), and BookTest ( Bajgar et al., 2019 ), DROP ( Dua et al.,

BERT 52

BERT NLP Natural Language Processing Computational Linguistics 52

AWS Machine Learning Blog

FEBRUARY 28, 2023

Use natural language processing (NLP) in Amazon HealthLake to extract non-sensitive data from unstructured blobs. We can see that Amazon HeathLake NLP interprets this as containing the condition “stroke” by querying for the condition record that has the same patient ID and displays “stroke.” mg/actuat / salmeterol 0.05

ML 90

ML Machine Learning Categorization NLP 90

John Snow Labs

MAY 23, 2023

Visual question answering (VQA), an area that intersects the fields of Deep Learning, Natural Language Processing (NLP) and Computer Vision (CV) is garnering a lot of interest in research circles. NLP is a particularly crucial element of the multi-discipline research problem that is VQA. is an object detection task.

Natural Language Processing 40

Natural Language Processing Deep Learning NLP Convolutional Neural Networks 40

Mlearning.ai

MARCH 2, 2023

Pre-training of Deep Bidirectional Transformers for Language Understanding BERT is a language model that can be fine-tuned for various NLP tasks and at the time of publication achieved several state-of-the-art results. Conclusion: BERT as Trend-Setter in NLP and Deep Learning References I. BERT Architecture and Training III.1

BERT 52

BERT NLP Deep Learning Neural Network 52