Computation and Language

Authors and titles for January 2021

[1] arXiv:2101.00010 [pdf, other]

Title: UnNatural Language Inference

Koustuv Sinha, Prasanna Parthasarathi, Joelle Pineau, Adina Williams

Comments: Accepted at ACL 2021 (Long Paper), 9 pages + Appendix

Subjects: Computation and Language (cs.CL); Machine Learning (cs.LG)

[2] arXiv:2101.00027 [pdf, other]

Title: The Pile: An 800GB Dataset of Diverse Text for Language Modeling

Leo Gao, Stella Biderman, Sid Black, Laurence Golding, Travis Hoppe, Charles Foster, Jason Phang, Horace He, Anish Thite, Noa Nabeshima, Shawn Presser, Connor Leahy

Subjects: Computation and Language (cs.CL)

[3] arXiv:2101.00036 [pdf, other]

Title: KART: Parameterization of Privacy Leakage Scenarios from Pre-trained Language Models

Yuta Nakamura (1), Shouhei Hanaoka (1 and 2), Yukihiro Nomura (3 and 4), Naoto Hayashi (3), Osamu Abe (1 and 2), Shuntaro Yada (5), Shoko Wakamiya (5), Eiji Aramaki (5) ((1) The University of Tokyo, (2) The Department of Radiology, The University of Tokyo Hospital, (3) The Department of Computational Diagnostic Radiology and Preventive Medicine, The University of Tokyo Hospital, (4) Chiba University, (5) Nara Institute of Science and Technology)

Subjects: Computation and Language (cs.CL)

[4] arXiv:2101.00056 [pdf, other]

Title: Towards Modelling Coherence in Spoken Discourse

Rajaswa Patil, Yaman Kumar Singla, Rajiv Ratn Shah, Mika Hama, Roger Zimmermann

Comments: 12 pages

Subjects: Computation and Language (cs.CL)

[5] arXiv:2101.00063 [pdf, other]

Title: EarlyBERT: Efficient BERT Training via Early-bird Lottery Tickets

Xiaohan Chen, Yu Cheng, Shuohang Wang, Zhe Gan, Zhangyang Wang, Jingjing Liu

Comments: Accepted at ACL-IJCNLP 2021

Subjects: Computation and Language (cs.CL); Artificial Intelligence (cs.AI)

[6] arXiv:2101.00078 [pdf, other]

Title: Controlled Analyses of Social Biases in Wikipedia Bios

Anjalie Field, Chan Young Park, Kevin Z. Lin, Yulia Tsvetkov

Comments: Accepted to the Web Conference 2022 (WWW '22)

Subjects: Computation and Language (cs.CL)

[7] arXiv:2101.00117 [pdf, other]

Title: Multi-task Retrieval for Knowledge-Intensive Tasks

Jean Maillard, Vladimir Karpukhin, Fabio Petroni, Wen-tau Yih, Barlas Oğuz, Veselin Stoyanov, Gargi Ghosh

Subjects: Computation and Language (cs.CL)

[8] arXiv:2101.00121 [pdf, other]

Title: WARP: Word-level Adversarial ReProgramming

Karen Hambardzumyan, Hrant Khachatrian, Jonathan May

Comments: Accepted ACL 2021 Long Paper

Subjects: Computation and Language (cs.CL)

[9] arXiv:2101.00123 [pdf, other]

Title: Intent Classification and Slot Filling for Privacy Policies

Wasi Uddin Ahmad, Jianfeng Chi, Tu Le, Thomas Norton, Yuan Tian, Kai-Wei Chang

Comments: ACL 2021 (camera ready)

Subjects: Computation and Language (cs.CL)

[10] arXiv:2101.00124 [pdf, other]

Title: Discourse-level Relation Extraction via Graph Pooling

I-Hung Hsu, Xiao Guo, Premkumar Natarajan, Nanyun Peng

Comments: 12 pages, page 10-12 appendix

Subjects: Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Machine Learning (cs.LG)

[11] arXiv:2101.00130 [pdf, other]

Title: Sensei: Self-Supervised Sensor Name Segmentation

Jiaman Wu, Dezhi Hong, Rajesh Gupta, Jingbo Shang

Subjects: Computation and Language (cs.CL)

[12] arXiv:2101.00133 [pdf, other]

Title: NeurIPS 2020 EfficientQA Competition: Systems, Analyses and Lessons Learned

Sewon Min, Jordan Boyd-Graber, Chris Alberti, Danqi Chen, Eunsol Choi, Michael Collins, Kelvin Guu, Hannaneh Hajishirzi, Kenton Lee, Jennimaria Palomaki, Colin Raffel, Adam Roberts, Tom Kwiatkowski, Patrick Lewis, Yuxiang Wu, Heinrich Küttler, Linqing Liu, Pasquale Minervini, Pontus Stenetorp, Sebastian Riedel, Sohee Yang, Minjoon Seo, Gautier Izacard, Fabio Petroni, Lucas Hosseini, Nicola De Cao, Edouard Grave, Ikuya Yamada, Sonse Shimaoka, Masatoshi Suzuki, Shumpei Miyawaki, Shun Sato, Ryo Takahashi, Jun Suzuki, Martin Fajcik, Martin Docekal, Karel Ondrej, Pavel Smrz, Hao Cheng, Yelong Shen, Xiaodong Liu, Pengcheng He, Weizhu Chen, Jianfeng Gao, Barlas Oguz, Xilun Chen, Vladimir Karpukhin, Stan Peshterliev, Dmytro Okhonko, Michael Schlichtkrull, Sonal Gupta, Yashar Mehdad, Wen-tau Yih

Comments: 26 pages; Published in Proceedings of Machine Learning Research (PMLR), NeurIPS 2020 Competition and Demonstration Track

Subjects: Computation and Language (cs.CL); Artificial Intelligence (cs.AI)

[13] arXiv:2101.00146 [pdf, other]

Title: De-identifying Australian Hospital Discharge Summaries: An End-to-End Framework using Ensemble of Deep Learning Models

Leibo Liu, Oscar Perez-Concha, Anthony Nguyen, Vicki Bennett, Louisa Jorm

Journal-ref: Journal of Biomedical Informatics 135 (2022) 104215

Subjects: Computation and Language (cs.CL); Information Retrieval (cs.IR)

[14] arXiv:2101.00148 [pdf, other]

Title: Bilingual Lexicon Induction via Unsupervised Bitext Construction and Word Alignment

Haoyue Shi, Luke Zettlemoyer, Sida I. Wang

Comments: ACL-IJCNLP 2021 camera-ready version, with full supplementary material

Subjects: Computation and Language (cs.CL)

[15] arXiv:2101.00153 [pdf, html, other]

Title: Graphmax for Text Generation

Liu Bin, Yin Guosheng

Journal-ref: Journal of Artificial Intelligence Research, vol. 78, pp.823-848, Nov. 2023

Subjects: Computation and Language (cs.CL)

[16] arXiv:2101.00154 [pdf, other]

Title: DISCOS: Bridging the Gap between Discourse Knowledge and Commonsense Knowledge

Tianqing Fang, Hongming Zhang, Weiqi Wang, Yangqiu Song, Bin He

Comments: WWW 2021 paper. 12 pages and 6 Figures

Subjects: Computation and Language (cs.CL); Artificial Intelligence (cs.AI)

[17] arXiv:2101.00160 [pdf, other]

Title: How Do Your Biomedical Named Entity Recognition Models Generalize to Novel Entities?

Hyunjae Kim, Jaewoo Kang

Comments: IEEE Access 2022

Subjects: Computation and Language (cs.CL)

[18] arXiv:2101.00167 [pdf, other]

Title: Unifying Discourse Resources with Dependency Framework

Yi Cheng, Sujian Li, Yueyuan Li

Subjects: Computation and Language (cs.CL)

[19] arXiv:2101.00178 [pdf, other]

Title: UnitedQA: A Hybrid Approach for Open Domain Question Answering

Hao Cheng, Yelong Shen, Xiaodong Liu, Pengcheng He, Weizhu Chen, Jianfeng Gao

Comments: ACL 2021 camera-ready

Subjects: Computation and Language (cs.CL); Artificial Intelligence (cs.AI)

[20] arXiv:2101.00180 [pdf, other]

Title: Transformer based Automatic COVID-19 Fake News Detection System

Sunil Gundapu, Radhika Mamidi

Comments: First Workshop on Combating Online Hostile Posts in Regional Languages during Emergency Situation, 12 pages

Subjects: Computation and Language (cs.CL)

[21] arXiv:2101.00190 [pdf, other]

Title: Prefix-Tuning: Optimizing Continuous Prompts for Generation

Xiang Lisa Li, Percy Liang

Subjects: Computation and Language (cs.CL)

[22] arXiv:2101.00196 [pdf, other]

Title: On Explaining Your Explanations of BERT: An Empirical Study with Sequence Classification

Zhengxuan Wu, Desmond C. Ong

Subjects: Computation and Language (cs.CL)

[23] arXiv:2101.00204 [pdf, other]

Title: BanglaBERT: Language Model Pretraining and Benchmarks for Low-Resource Language Understanding Evaluation in Bangla

Abhik Bhattacharjee, Tahmid Hasan, Wasi Uddin Ahmad, Kazi Samin, Md Saiful Islam, Anindya Iqbal, M. Sohel Rahman, Rifat Shahriyar

Comments: Findings of North American Chapter of the Association for Computational Linguistics, NAACL 2022 (camera-ready)

Subjects: Computation and Language (cs.CL)

[24] arXiv:2101.00234 [pdf, other]

Title: Subformer: Exploring Weight Sharing for Parameter Efficiency in Generative Transformers

Machel Reid, Edison Marrese-Taylor, Yutaka Matsuo

Comments: EMNLP 2021 Findings

Subjects: Computation and Language (cs.CL); Machine Learning (cs.LG)

[25] arXiv:2101.00259 [pdf, other]

Title: Code Generation from Natural Language with Less Prior and More Monolingual Data

Sajad Norouzi, Keyi Tang, Yanshuai Cao

Comments: ACL 2021

Subjects: Computation and Language (cs.CL); Machine Learning (cs.LG)

[26] arXiv:2101.00288 [pdf, other]

Title: Polyjuice: Generating Counterfactuals for Explaining, Evaluating, and Improving Models

Tongshuang Wu, Marco Tulio Ribeiro, Jeffrey Heer, Daniel S. Weld

Comments: ACL 2021, main conference, long paper

Subjects: Computation and Language (cs.CL)

[27] arXiv:2101.00294 [pdf, other]

Title: Rider: Reader-Guided Passage Reranking for Open-Domain Question Answering

Yuning Mao, Pengcheng He, Xiaodong Liu, Yelong Shen, Jianfeng Gao, Jiawei Han, Weizhu Chen

Comments: Minor format updates. TLDR: Reranking retrieved passages by reader predictions can achieve 10~20 gains in top-1 retrieval accuracy and 1~4 gains in Exact Match (EM) without any training

Subjects: Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Information Retrieval (cs.IR)

[28] arXiv:2101.00297 [pdf, other]

Title: Analyzing Commonsense Emergence in Few-shot Knowledge Models

Jeff Da, Ronan Le Bras, Ximing Lu, Yejin Choi, Antoine Bosselut

Comments: AKBC 2021

Subjects: Computation and Language (cs.CL)

[29] arXiv:2101.00345 [pdf, other]

Title: Modeling Fine-Grained Entity Types with Box Embeddings

Yasumasa Onoe, Michael Boratko, Andrew McCallum, Greg Durrett

Comments: ACL 2021

Subjects: Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Machine Learning (cs.LG)

[30] arXiv:2101.00371 [pdf, other]

Title: On-the-Fly Attention Modulation for Neural Generation

Yue Dong, Chandra Bhagavatula, Ximing Lu, Jena D. Hwang, Antoine Bosselut, Jackie Chi Kit Cheung, Yejin Choi

Comments: 10 pages, 3 figures

Subjects: Computation and Language (cs.CL)

[31] arXiv:2101.00376 [pdf, other]

Title: RiddleSense: Reasoning about Riddle Questions Featuring Linguistic Creativity and Commonsense Knowledge

Bill Yuchen Lin, Ziyi Wu, Yichi Yang, Dong-Ho Lee, Xiang Ren

Comments: Accepted to ACL 2021 (Findings). Project page: this https URL

Subjects: Computation and Language (cs.CL); Artificial Intelligence (cs.AI)

[32] arXiv:2101.00379 [pdf, other]

Title: Investigating Memorization of Conspiracy Theories in Text Generation

Sharon Levy, Michael Saxon, William Yang Wang

Comments: ACL 2021 Findings

Subjects: Computation and Language (cs.CL); Computers and Society (cs.CY)

[33] arXiv:2101.00387 [pdf, other]

Title: What all do audio transformer models hear? Probing Acoustic Representations for Language Delivery and its Structure

Jui Shah, Yaman Kumar Singla, Changyou Chen, Rajiv Ratn Shah

Subjects: Computation and Language (cs.CL); Machine Learning (cs.LG); Sound (cs.SD); Audio and Speech Processing (eess.AS)

[34] arXiv:2101.00388 [pdf, other]

Title: A Robust and Domain-Adaptive Approach for Low-Resource Named Entity Recognition

Houjin Yu, Xian-Ling Mao, Zewen Chi, Wei Wei, Heyan Huang

Comments: Best Student Paper of 2020 IEEE International Conference on Knowledge Graph (ICKG)

Journal-ref: 2020 IEEE International Conference on Knowledge Graph (ICKG) (pp. 297-304)-

Subjects: Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Machine Learning (cs.LG)

[35] arXiv:2101.00389 [pdf, other]

Title: Multitask Learning for Class-Imbalanced Discourse Classification

Alexander Spangher, Jonathan May, Sz-rung Shiang, Lingjia Deng

Comments: 17 pages, 11 figures

Subjects: Computation and Language (cs.CL)

[36] arXiv:2101.00390 [pdf, other]

Title: VoxPopuli: A Large-Scale Multilingual Speech Corpus for Representation Learning, Semi-Supervised Learning and Interpretation

Changhan Wang, Morgane Rivière, Ann Lee, Anne Wu, Chaitanya Talnikar, Daniel Haziza, Mary Williamson, Juan Pino, Emmanuel Dupoux

Comments: Accepted to ACL 2021 (long paper)

Subjects: Computation and Language (cs.CL); Audio and Speech Processing (eess.AS)

[37] arXiv:2101.00391 [pdf, other]

Title: Which Linguist Invented the Lightbulb? Presupposition Verification for Question-Answering

Najoung Kim, Ellie Pavlick, Burcu Karagol Ayan, Deepak Ramachandran

Comments: ACL 2021 Camera-ready

Subjects: Computation and Language (cs.CL)

[38] arXiv:2101.00394 [pdf, other]

Title: End-to-end Semantic Role Labeling with Neural Transition-based Model

Hao Fei, Meishan Zhang, Bobo Li, Donghong Ji

Comments: Accepted at AAAI 2021

Subjects: Computation and Language (cs.CL)

[39] arXiv:2101.00396 [pdf, other]

Title: Lex-BERT: Enhancing BERT based NER with lexicons

Wei Zhu, Daniel Cheung

Comments: Will incorporate new ideas, and more experiments, and better descriptions

Subjects: Computation and Language (cs.CL)

[40] arXiv:2101.00403 [pdf, other]

Title: Superbizarre Is Not Superb: Derivational Morphology Improves BERT's Interpretation of Complex Words

Valentin Hofmann, Janet B. Pierrehumbert, Hinrich Schütze

Comments: ACL 2021

Subjects: Computation and Language (cs.CL)

[41] arXiv:2101.00406 [pdf, other]

Title: CDLM: Cross-Document Language Modeling

Avi Caciularu, Arman Cohan, Iz Beltagy, Matthew E. Peters, Arie Cattan, Ido Dagan

Comments: EMNLP 2021, findings

Subjects: Computation and Language (cs.CL)

[42] arXiv:2101.00408 [pdf, other]

Title: End-to-End Training of Neural Retrievers for Open-Domain Question Answering

Devendra Singh Sachan, Mostofa Patwary, Mohammad Shoeybi, Neel Kant, Wei Ping, William L Hamilton, Bryan Catanzaro

Comments: ACL 2021

Subjects: Computation and Language (cs.CL); Artificial Intelligence (cs.AI)

[43] arXiv:2101.00411 [pdf, other]

Title: Substructure Substitution: Structured Data Augmentation for NLP

Haoyue Shi, Karen Livescu, Kevin Gimpel

Subjects: Computation and Language (cs.CL)

[44] arXiv:2101.00416 [pdf, other]

Title: Improving Sequence-to-Sequence Pre-training via Sequence Span Rewriting

Wangchunshu Zhou, Tao Ge, Canwen Xu, Ke Xu, Furu Wei

Subjects: Computation and Language (cs.CL)

[45] arXiv:2101.00419 [pdf, other]

Title: KM-BART: Knowledge Enhanced Multimodal BART for Visual Commonsense Generation

Yiran Xing, Zai Shi, Zhao Meng, Gerhard Lakemeyer, Yunpu Ma, Roger Wattenhofer

Comments: ACL-IJCNLP 2021 main conference. The first three authors contribute equally to this work

Subjects: Computation and Language (cs.CL)

[46] arXiv:2101.00420 [pdf, other]

Title: Learning to Generate Task-Specific Adapters from Task Description

Qinyuan Ye, Xiang Ren

Comments: Accepted to ACL 2021. Camera-ready version. Code: this https URL

Subjects: Computation and Language (cs.CL); Machine Learning (cs.LG)

[47] arXiv:2101.00421 [pdf, other]

Title: The Highs and Lows of Simple Lexical Domain Adaptation Approaches for Neural Machine Translation

Nikolay Bogoychev, Pinzhen Chen

Comments: Accepted at Workshop on Insights from Negative Results in NLP 2021

Subjects: Computation and Language (cs.CL)

[48] arXiv:2101.00430 [pdf, other]

Title: Assessing Emoji Use in Modern Text Processing Tools

Abu Awal Md Shoeb, Gerard de Melo

Subjects: Computation and Language (cs.CL); Information Retrieval (cs.IR); Social and Information Networks (cs.SI)

[49] arXiv:2101.00433 [pdf, other]

Title: Modeling Disclosive Transparency in NLP Application Descriptions

Michael Saxon, Sharon Levy, Xinyi Wang, Alon Albalak, William Yang Wang

Comments: To appear at EMNLP 2021. 15 pages, 10 figures, 7 tables

Journal-ref: Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pp 2023-2037

Subjects: Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Human-Computer Interaction (cs.HC)

[50] arXiv:2101.00434 [pdf, other]

Title: Coreference Resolution without Span Representations

Yuval Kirstain, Ori Ram, Omer Levy

Comments: Accepted to ACL 2021

Subjects: Computation and Language (cs.CL)