article thumbnail

Common Flaws in NLP Evaluation Experiments

Ehud Reiter

However I think journals such as Computational Linguistics and TACL could adjust reviewing procedures to check some of above. We only looked at human evaluations, but I suspect the problem may be just as bad with metric evaluations (eg see Arvan et al (2022) ). Computational Linguistics. Unfortunately.

NLP 259
article thumbnail

ACL 2022 Highlights

Sebastian Ruder

ACL 2022 took place in Dublin from 22nd–27th May 2022. Language diversity and multimodality Panelists and their spoken languages at the ACL 2022 keynote panel on supporting linguistic diversity. This was my first in-person conference since ACL 2019. This is also my first conference highlights post since NAACL 2019.

NLP 52
professionals

Sign Up for our Newsletter

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

article thumbnail

2022: We reviewed this year’s AI breakthroughs

Applied Data Science

Just wait until you hear what happened in 2022. Dall-e , and pre-2022 tools in general, attributed their success either to the use of the Transformer or Generative Adversarial Networks. In 2022 we got diffusion models ( NeurIPS paper ). This was one of the first appearances of an AI model used for Text-to-Image generation.

article thumbnail

Stanford AI Lab Papers and Talks at ACL 2022

The Stanford AI Lab Blog

The 60th Annual Meeting of the Association for Computational Linguistics (ACL) 2022 is taking place May 22nd - May 27th. We’re excited to share all the work from SAIL that’s being presented, and you’ll find links to papers, videos and blogs below.

article thumbnail

What are they thinking?

Allen AI

In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 1115–1127, Seattle, United States. Association for Computational Linguistics. Association for Computational Linguistics.

article thumbnail

ClarifyDelphi

Allen AI

2022; Levine et al., arXiv preprint arXiv:2201.07763 (2022). In Findings of the Association for Computational Linguistics: EMNLP 2020 , pp. 2020; Awad et al., Learning to ask the right questions We use Reinforcement Learning (PPO) to train our question generation system. When is it acceptable to break the rules?

article thumbnail

AI2 at ACL 2023

Allen AI

2022) tasks showcase a scenario where initial predictions by a learned model (yˆ) are incorrect. Two examples for action planning (Tandon et al., 2021) and summarization (Saunders et al., Human-written critiques © indicate errors in model outputs. While humans can reliably critique each other, machines lack such ability.