Saturated Transformers are Constant-Depth Threshold Circuits William Merrill∗† Ashish Sabharwal∗ Noah A. Smith∗‡ ∗Allen Institute for AI, USA †New York University, USA ‡University of Washington, USA willm@nyu.edu {ashishs, noah}@allenai.org Abstract Transformers have become a standard…
浏览类别处理
Generate, Annotate, and Learn: NLP with Synthetic Text
Generate, Annotate, and Learn: NLP with Synthetic Text Xuanli He1 Islam Nassar1 Jamie Kiros2 Gholamreza Haffari1 Mohammad Norouzi2 1Monash University, Australia 2Google Research, Brain Team, 加拿大 {xuanli.he1, gholamreza.haffari}@monash.edu, mnorouzi@google.com Abstract This paper studies the use…
High Quality Rather than High Model Probability:
High Quality Rather than High Model Probability: Minimum Bayes Risk Decoding with Neural Metrics Markus Freitag, David Grangier, Qijun Tan, Bowen Liang Google Research, 美国 {freitag, grangier, qijuntan, bowenl}@google.com Abstract In Neural Machine Translation, 它…
Formal Language Recognition by Hard Attention Transformers:
Formal Language Recognition by Hard Attention Transformers: Perspectives from Circuit Complexity Yiding Hao, Dana Angluin, and Robert Frank Yale University New Haven, CT, 美国 {yiding.hao, robert.frank, dana.angluin}@yale.edu Abstract This paper analyzes three formal models of…
Minimum Description Length Recurrent Neural Networks
Minimum Description Length Recurrent Neural Networks Nur Lan1,2, Michal Geyer2, Emmanuel Chemla1,3∗, Roni Katzir2∗ 1Ecole Normale Sup´erieure, France 2Tel Aviv University, Israel 3EHESS, PSL University, 法国国家科学研究中心 {nlan,chemla}@ens.fr michalgeyer@mail.tau.ac.il rkatzir@tauex.tau.ac.il Abstract We train neural networks to…
Heterogeneous Supervised Topic Models
Heterogeneous Supervised Topic Models Dhanya Sridhar(西德:2) and Hal Daum´e III† and David Blei ♠ (西德:2)Universit´e de Montr´eal and Mila-Quebec AI Institute, Canada dhanya.sridhar@mila.quebec †University of Maryland and Microsoft Research, USA hal3@umd.edu ♠Columbia University, USA david.blei@columbia.edu…
Fact Checking with Insufficient Evidence
Fact Checking with Insufficient Evidence Pepa Atanasova Jakob Grue Simonsen Christina Lioma Isabelle Augenstein Department of Computer Science, University of Copenhagen, 丹麦 {pepa, simonsen, c.lioma, augenstein}@di.ku.dk Abstract Automating the fact checking (FC) process relies on…
True Few-Shot Learning with Prompts—A Real-World Perspective
True Few-Shot Learning with Prompts—A Real-World Perspective Timo Schick and Hinrich Sch ¨utze Center for Information and Language Processing (CIS), LMU Munich, Germany schickt@cis.lmu.de, inquiries@cislmu.org Abstract Prompt-based approaches excel at few-shot learning. 然而, Perez et…
Data-to-text Generation with Variational Sequential Planning
Data-to-text Generation with Variational Sequential Planning Ratish Puduppully and Yao Fu and Mirella Lapata Institute for Language, Cognition and Computation School of Informatics, 爱丁堡大学 10 Crichton Street, Edinburgh EH8 9AB, UK r.puduppully@sms.ed.ac.uk yao.fu@ed.ac.uk…
Uncertainty Estimation and Reduction of Pre-trained Models for
Uncertainty Estimation and Reduction of Pre-trained Models for Text Regression Yuxia Wang(西德:2) Daniel Beck(西德:2) Timothy Baldwin(西德:2) Karin Verspoor†(西德:2) (西德:2) The University of Melbourne, 墨尔本, 维多利亚, Australia †RMIT University, 墨尔本, 维多利亚, Australia yuxiaw@student.unimelb.edu.au d.beck@unimelb.edu.au tb@ldwin.net karin.verspoor@rmit.edu.au…