Sebastian Ruder

FEBRUARY 23, 2023

This post gives a brief overview of modularity in deep learning. For modular fine-tuning for NLP, check out our EMNLP 2022 tutorial. Fuelled by scaling laws, state-of-the-art models in machine learning have been growing larger and larger. We give an in-depth overview of modularity in our survey on Modular Deep Learning.

Deep Learning 52

Deep Learning NLP Computer Vision Metadata 52

Applied Data Science

DECEMBER 23, 2022

Just wait until you hear what happened in 2022. In our review of 2019 we talked a lot about reinforcement learning and Generative Adversarial Networks (GANs), in 2020 we focused on Natural Language Processing (NLP) and algorithmic bias, in 202 1 Transformers stole the spotlight. In 2022 we got diffusion models ( NeurIPS paper ).

Neural Network 52

Neural Network AI AI Data Science 52

Marek Rei

NOVEMBER 4, 2024

EMNLP 2022. EMNLP 2022. NeurIPS 2022. EMNLP 2022. EMNLP 2022. They show performance improvements in some settings and speed improvements in all evaluated settings, showing particular usefulness in settings where the LLM needs to retrieve information about multiple entities (e.g. UC Berkeley, CMU. Google Research.

Machine Learning 80

Machine Learning NLP Large Language Models LLM 80

Topbots

JANUARY 30, 2023

For instance, in April 2022, DeepMind introduced Flamingo, a visual language model that can seamlessly ingest text, images and video. [5] In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics , pages 5185–5198, Online. Association for Computational Linguistics. [2] 2212.08120.

Large Language Models 52

Large Language Models Computational Linguistics LLM Natural Language Processing 52

Sebastian Ruder

NOVEMBER 14, 2022

This post is partially based on a keynote I gave at the Deep Learning Indaba 2022. Linguistic and demographic global utility metrics for a number of language technology tasks. Overall, however, we observe very low linguistic utility numbers, showing an unequal performance distribution across the world's languages.

Natural Language Processing 52

Natural Language Processing NLP Computational Linguistics BERT 52

Topbots

JUNE 15, 2023

70% of research papers published in a computational linguistics conference only evaluated English.[ In Findings of the Association for Computational Linguistics: ACL 2022 , pages 2340–2354, Dublin, Ireland. Association for Computational Linguistics. Association for Computational Linguistics.

Natural Language Processing 59

Natural Language Processing Computational Linguistics NLP ChatGPT 59

ODSC - Open Data Science

APRIL 3, 2023

Iryna is co-director of the NLP program within ELLIS, a European network of excellence in machine learning. She is currently the president of the Association of Computational Linguistics.

Neural Network 117

Neural Network Natural Language Processing NLP Computational Linguistics 117