CANINE: Pre-training an Efficient Tokenization-Free Encoder for Language Representation Jonathan H. Clark, Dan Garrette, Iulia Turc, John Wieting Google Research, USA {jhclark,dhgarrette,iuliaturc,jwieting}@google.com Abstract Pipelined NLP systems have largely been su- perseded by end-to-end neural modeling,…
Browsing Categoryattrezzatura
Quality at a Glance:
Quality at a Glance: An Audit of Web-Crawled Multilingual Datasets Julia Kreutzer1,2, Isaac Caswell3, Lisa Wang3,4, Ahsan Wahab5,47, Daan van Esch6, Nasanbayar Ulzii-Orshikh7, Allahsera Tapo8,9, Nishant Subramani10,11, Artem Sokolov4, Claytone Sikasote12,13, Monang Setyawan14, Supheakmungkol Sarin14,…
Decomposing and Recomposing Event Structure
Decomposing and Recomposing Event Structure William Gantt University of Rochester, USA wgantt@cs.rochester.edu Lelia Glass Georgia Institute of Technology, USA lelia.glass@modlangs.gatech.edu Aaron Steven White University of Rochester, USA aaron.white@rochester.edu Abstract We present an event structure classification…
Word Acquisition in Neural Language Models
Word Acquisition in Neural Language Models Tyler A. Chang1,2, Benjamin K. Bergen1 1Department of Cognitive Science 2Halıcıo˘glu Data Science Institute University of California, San Diego, USA {tachang, bkbergen}@ucsd.edu Abstract We investigate how neural language mod-…
Word Representation Learning in Multimodal Pre-Trained
Word Representation Learning in Multimodal Pre-Trained Transformers: An Intrinsic Evaluation Sandro Pezzelle, Ece Takmaz, Raquel Fern´andez Institute for Logic, Language and Computation University of Amsterdam, The Netherlands {s.pezzelle|e.takmaz|raquel.fernandez}@uva.nl Abstract This study carries out a systematic…
Idiomatic Expression Identification using Semantic Compatibility
Idiomatic Expression Identification using Semantic Compatibility Ziheng Zeng and Suma Bhat Department of Electrical and Computer Engineering University of Illinois at Urbana-Champaign Champaign, IL USA {zzeng13, spbhat2}@illinois.edu Abstract Idiomatic expressions are an integral part of…
Quantifying Cognitive Factors in Lexical Decline
Quantifying Cognitive Factors in Lexical Decline David Francis1 Ella Rabinovich1 Farhan Samir1 David Mortensen2 Suzanne Stevenson1 1Department of Computer Science, University of Toronto, Canada 2Language Technologies Institute, Carnegie Mellon University, USA {dfrancis, ella, fsamir, suzanne}@cs.toronto.edu…
Explanation-Based Human Debugging of NLP Models: A Survey
Explanation-Based Human Debugging of NLP Models: A Survey Piyawat Lertvittayakumjorn and Francesca Toni Department of Computing Imperial College London, UK {pl1515, piedi}@imperial.ac.uk Abstract Debugging a machine learning model is hard since the bug usually involves…
Instance-Based Neural Dependency Parsing
Instance-Based Neural Dependency Parsing Hiroki Ouchi1,3 Jun Suzuki2,3 Sosuke Kobayashi2,4 Sho Yokoi2,3 Tatsuki Kuribayashi2,5 Masashi Yoshikawa2,3 Kentaro Inui2,3 1 NAIST, Japan 2 Tohoku University, Japan 3 RIKEN, Japan 4 Preferred Networks, Inc., Japan 5 Langsmith,…
Planning with Learned Entity Prompts for Abstractive Summarization
Planning with Learned Entity Prompts for Abstractive Summarization Shashi Narayan Google Research shashinarayan@google.com Yao Zhao Google Brain yaozhaoyz@google.com Joshua Maynez Google Research joshuahm@google.com Gonc¸alo Sim˜oes Google Research gsimoes@google.com Vitaly Nikolaev Google Research vitalyn@google.com Ryan McDonald∗…