Remove BERT Remove Computational Linguistics Remove Software Engineer
article thumbnail

ML and NLP Research Highlights of 2021

Sebastian Ruder

6] such as W2v-BERT [7] as well as more powerful multilingual models such as XLS-R [8].   It is still an open question how much code generation models improve the workflow of software engineers in practice [85]. W2v-BERT: Combining Contrastive Learning and Masked Language Modeling for Self-Supervised Speech Pre-Training.

NLP 52