Computation and Language

Authors and titles for May 2024

[1501] arXiv:2405.18572 (cross-list from cs.LG) [pdf, html, other]

Title: Low-rank finetuning for LLMs: A fairness perspective

Saswat Das, Marco Romanelli, Cuong Tran, Zarreen Reza, Bhavya Kailkhura, Ferdinando Fioretto

Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Computation and Language (cs.CL)

[1502] arXiv:2405.18620 (cross-list from cs.HC) [pdf, html, other]

Title: RealitySummary: Exploring On-Demand Mixed Reality Text Summarization and Question Answering using Large Language Models

Aditya Gunturu, Shivesh Jadon, Nandi Zhang, Morteza Faraji, Jarin Thundathil, Tafreed Ahmad, Wesley Willett, Ryo Suzuki

Subjects: Human-Computer Interaction (cs.HC); Artificial Intelligence (cs.AI); Computation and Language (cs.CL)

[1503] arXiv:2405.18628 (cross-list from cs.LG) [pdf, html, other]

Title: Hardware-Aware Parallel Prompt Decoding for Memory-Efficient Acceleration of LLM Inference

Hao Mark Chen, Wayne Luk, Ka Fai Cedric Yiu, Rui Li, Konstantin Mishchenko, Stylianos I. Venieris, Hongxiang Fan

Comments: The code for this implementation is available at this https URL

Subjects: Machine Learning (cs.LG); Computation and Language (cs.CL)

[1504] arXiv:2405.18634 (cross-list from cs.LG) [pdf, html, other]

Title: A Theoretical Understanding of Self-Correction through In-context Alignment

Yifei Wang, Yuyang Wu, Zeming Wei, Stefanie Jegelka, Yisen Wang

Comments: Accepted at NeurIPS 2024

Subjects: Machine Learning (cs.LG); Computation and Language (cs.CL); Machine Learning (stat.ML)

[1505] arXiv:2405.18639 (cross-list from q-bio.NC) [pdf, other]

Title: Improving Speech Decoding from ECoG with Self-Supervised Pretraining

Brian A. Yuan, Joseph G. Makin

Subjects: Neurons and Cognition (q-bio.NC); Computation and Language (cs.CL); Machine Learning (cs.LG); Sound (cs.SD); Audio and Speech Processing (eess.AS)

[1506] arXiv:2405.18642 (cross-list from cs.AI) [pdf, html, other]

Title: JADS: A Framework for Self-supervised Joint Aspect Discovery and Summarization

Xiaobo Guo, Jay Desai, Srinivasan H. Sengamedu

Comments: preprint

Subjects: Artificial Intelligence (cs.AI); Computation and Language (cs.CL)

[1507] arXiv:2405.18669 (cross-list from cs.LG) [pdf, html, other]

Title: Zipper: A Multi-Tower Decoder Architecture for Fusing Modalities

Vicky Zayats, Peter Chen, Melissa Ferrari, Dirk Padfield

Comments: Under review at NeurIPS

Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Audio and Speech Processing (eess.AS)

[1508] arXiv:2405.18672 (cross-list from cs.CV) [pdf, html, other]

Title: LLM-based Hierarchical Concept Decomposition for Interpretable Fine-Grained Image Classification

Renyi Qu, Mark Yatskar

Subjects: Computer Vision and Pattern Recognition (cs.CV); Computation and Language (cs.CL)

[1509] arXiv:2405.18688 (cross-list from cs.LG) [pdf, html, other]

Title: Efficient Preference-based Reinforcement Learning via Aligned Experience Estimation

Fengshuo Bai, Rui Zhao, Hongming Zhang, Sijia Cui, Ying Wen, Yaodong Yang, Bo Xu, Lei Han

Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Computation and Language (cs.CL)

[1510] arXiv:2405.18711 (cross-list from cs.AI) [pdf, html, other]

Title: Calibrating Reasoning in Language Models with Internal Consistency

Zhihui Xie, Jizhou Guo, Tong Yu, Shuai Li

Comments: NeurIPS 2024 camera ready

Subjects: Artificial Intelligence (cs.AI); Computation and Language (cs.CL)

[1511] arXiv:2405.18721 (cross-list from cs.CV) [pdf, html, other]

Title: Correctable Landmark Discovery via Large Models for Vision-Language Navigation

Bingqian Lin, Yunshuang Nie, Ziming Wei, Yi Zhu, Hang Xu, Shikui Ma, Jianzhuang Liu, Xiaodan Liang

Comments: Accepted by TPAMI 2024

Subjects: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Computation and Language (cs.CL)

[1512] arXiv:2405.18742 (cross-list from cs.AI) [pdf, html, other]

Title: Musical Phrase Segmentation via Grammatical Induction

Reed Perkins, Dan Ventura

Comments: Extended version of a paper appearing in the proceedings of IJCAI 2024 that includes additional material in an appendix. Please cite the IJCAI version

Journal-ref: Proceedings of the International Joint Conference on Artificial Intelligence, 2024

Subjects: Artificial Intelligence (cs.AI); Computation and Language (cs.CL)

[1513] arXiv:2405.18776 (cross-list from cs.CR) [pdf, html, other]

Title: LMO-DP: Optimizing the Randomization Mechanism for Differentially Private Fine-Tuning (Large) Language Models

Qin Yang, Meisam Mohammad, Han Wang, Ali Payani, Ashish Kundu, Kai Shu, Yan Yan, Yuan Hong

Comments: 18 pages, 15 figures

Subjects: Cryptography and Security (cs.CR); Computation and Language (cs.CL); Machine Learning (cs.LG)

[1514] arXiv:2405.18870 (cross-list from cs.AI) [pdf, html, other]

Title: LLMs achieve adult human performance on higher-order theory of mind tasks

Winnie Street, John Oliver Siy, Geoff Keeling, Adrien Baranes, Benjamin Barnett, Michael McKibben, Tatenda Kanyere, Alison Lentz, Blaise Aguera y Arcas, Robin I. M. Dunbar

Subjects: Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Human-Computer Interaction (cs.HC)

[1515] arXiv:2405.18874 (cross-list from cond-mat.dis-nn) [pdf, html, other]

Title: Are queries and keys always relevant? A case study on Transformer wave functions

Riccardo Rende, Luciano Loris Viteritti

Comments: 10 pages, 5 figures, 1 table

Journal-ref: Mach. Learn.: Sci. Technol. 6 010501 (2024)

Subjects: Disordered Systems and Neural Networks (cond-mat.dis-nn); Computation and Language (cs.CL); Computational Physics (physics.comp-ph)

[1516] arXiv:2405.18937 (cross-list from cs.CV) [pdf, html, other]

Title: Kestrel: Point Grounding Multimodal LLM for Part-Aware 3D Vision-Language Understanding

Junjie Fei, Mahmoud Ahmed, Jian Ding, Eslam Mohamed Bakr, Mohamed Elhoseiny

Subjects: Computer Vision and Pattern Recognition (cs.CV); Computation and Language (cs.CL)

[1517] arXiv:2405.18991 (cross-list from cs.CV) [pdf, html, other]

Title: EasyAnimate: A High-Performance Long Video Generation Method based on Transformer Architecture

Jiaqi Xu, Xinyi Zou, Kunzhe Huang, Yunkuo Chen, Bo Liu, MengLi Cheng, Xing Shi, Jun Huang

Comments: 8 pages, 6 figures

Subjects: Computer Vision and Pattern Recognition (cs.CV); Computation and Language (cs.CL); Multimedia (cs.MM)

[1518] arXiv:2405.19026 (cross-list from cs.LG) [pdf, html, other]

Title: DiveR-CT: Diversity-enhanced Red Teaming Large Language Model Assistants with Relaxing Constraints

Andrew Zhao, Quentin Xu, Matthieu Lin, Shenzhi Wang, Yong-jin Liu, Zilong Zheng, Gao Huang

Comments: Accepted by the 39th Annual AAAI Conference on Artificial Intelligence (AAAI-25)

Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Cryptography and Security (cs.CR)

[1519] arXiv:2405.19076 (cross-list from cs.CV) [pdf, other]

Title: Cephalo: Multi-Modal Vision-Language Models for Bio-Inspired Materials Analysis and Design

Markus J. Buehler

Subjects: Computer Vision and Pattern Recognition (cs.CV); Mesoscale and Nanoscale Physics (cond-mat.mes-hall); Materials Science (cond-mat.mtrl-sci); Computation and Language (cs.CL); Machine Learning (cs.LG)

[1520] arXiv:2405.19186 (cross-list from cs.CV) [pdf, html, other]

Title: MetaToken: Detecting Hallucination in Image Descriptions by Meta Classification

Laura Fieback, Jakob Spiegelberg, Hanno Gottschalk

Subjects: Computer Vision and Pattern Recognition (cs.CV); Computation and Language (cs.CL); Machine Learning (cs.LG)

[1521] arXiv:2405.19209 (cross-list from cs.CV) [pdf, html, other]

Title: VideoTree: Adaptive Tree-based Video Representation for LLM Reasoning on Long Videos

Ziyang Wang, Shoubin Yu, Elias Stengel-Eskin, Jaehong Yoon, Feng Cheng, Gedas Bertasius, Mohit Bansal

Comments: CVPR 2025; First three authors contributed equally; Project page: this https URL

Subjects: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Computation and Language (cs.CL)

[1522] arXiv:2405.19313 (cross-list from cs.AI) [pdf, html, other]

Title: Language Models Trained to do Arithmetic Predict Human Risky and Intertemporal Choice

Jian-Qiao Zhu, Haijiang Yan, Thomas L. Griffiths

Subjects: Artificial Intelligence (cs.AI); Computation and Language (cs.CL); General Economics (econ.GN)

[1523] arXiv:2405.19315 (cross-list from cs.CV) [pdf, html, other]

Title: Matryoshka Query Transformer for Large Vision-Language Models

Wenbo Hu, Zi-Yi Dou, Liunian Harold Li, Amita Kamath, Nanyun Peng, Kai-Wei Chang

Comments: Preprint. Our code and model are publicly available at this https URL

Subjects: Computer Vision and Pattern Recognition (cs.CV); Computation and Language (cs.CL); Machine Learning (cs.LG)

[1524] arXiv:2405.19316 (cross-list from cs.LG) [pdf, html, other]

Title: Robust Preference Optimization through Reward Model Distillation

Adam Fisch, Jacob Eisenstein, Vicky Zayats, Alekh Agarwal, Ahmad Beirami, Chirag Nagpal, Pete Shaw, Jonathan Berant

Subjects: Machine Learning (cs.LG); Computation and Language (cs.CL)

[1525] arXiv:2405.19334 (cross-list from cs.AI) [pdf, html, other]

Title: LLMs Meet Multimodal Generation and Editing: A Survey

Yingqing He, Zhaoyang Liu, Jingye Chen, Zeyue Tian, Hongyu Liu, Xiaowei Chi, Runtao Liu, Ruibin Yuan, Yazhou Xing, Wenhai Wang, Jifeng Dai, Yong Zhang, Wei Xue, Qifeng Liu, Yike Guo, Qifeng Chen

Comments: 52 Pages with 16 Figures, 12 Tables, and 545 References. GitHub Repository at: this https URL

Subjects: Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Computer Vision and Pattern Recognition (cs.CV); Multimedia (cs.MM); Sound (cs.SD)

[1526] arXiv:2405.19335 (cross-list from cs.CV) [pdf, html, other]

Title: X-VILA: Cross-Modality Alignment for Large Language Model

Hanrong Ye, De-An Huang, Yao Lu, Zhiding Yu, Wei Ping, Andrew Tao, Jan Kautz, Song Han, Dan Xu, Pavlo Molchanov, Hongxu Yin

Comments: Technical Report

Subjects: Computer Vision and Pattern Recognition (cs.CV); Computation and Language (cs.CL); Machine Learning (cs.LG)

[1527] arXiv:2405.19342 (cross-list from cs.SD) [pdf, html, other]

Title: Sonos Voice Control Bias Assessment Dataset: A Methodology for Demographic Bias Assessment in Voice Assistants

Chloé Sekkat, Fanny Leroy, Salima Mdhaffar, Blake Perry Smith, Yannick Estève, Joseph Dureau, Alice Coucke

Subjects: Sound (cs.SD); Computation and Language (cs.CL); Machine Learning (cs.LG); Audio and Speech Processing (eess.AS)

[1528] arXiv:2405.19343 (cross-list from cs.SD) [pdf, html, other]

Title: Luganda Speech Intent Recognition for IoT Applications

Andrew Katumba, Sudi Murindanyi, John Trevor Kasule, Elvis Mugume

Comments: Presented as a conference paper at ICLR 2024/AfricaNLP

Subjects: Sound (cs.SD); Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Audio and Speech Processing (eess.AS)

[1529] arXiv:2405.19534 (cross-list from cs.LG) [pdf, html, other]

Title: Preference Learning Algorithms Do Not Learn Preference Rankings

Angelica Chen, Sadhika Malladi, Lily H. Zhang, Xinyi Chen, Qiuyi Zhang, Rajesh Ranganath, Kyunghyun Cho

Comments: NeurIPS 2024 camera-ready

Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Computation and Language (cs.CL)

[1530] arXiv:2405.19544 (cross-list from cs.AI) [pdf, html, other]

Title: One-Shot Safety Alignment for Large Language Models via Optimal Dualization

Xinmeng Huang, Shuo Li, Edgar Dobriban, Osbert Bastani, Hamed Hassani, Dongsheng Ding

Comments: 32 pages, 6 figures, 8 tables

Subjects: Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Machine Learning (cs.LG); Optimization and Control (math.OC); Machine Learning (stat.ML)

[1531] arXiv:2405.19550 (cross-list from cs.LG) [pdf, html, other]

Title: Stress-Testing Capability Elicitation With Password-Locked Models

Ryan Greenblatt, Fabien Roger, Dmitrii Krasheninnikov, David Krueger

Subjects: Machine Learning (cs.LG); Computation and Language (cs.CL)

[1532] arXiv:2405.19561 (cross-list from cs.AI) [pdf, html, other]

Title: Quo Vadis ChatGPT? From Large Language Models to Large Knowledge Models

Venkat Venkatasubramanian, Arijit Chakraborty

Subjects: Artificial Intelligence (cs.AI); Computation and Language (cs.CL)

[1533] arXiv:2405.19562 (cross-list from cs.CY) [pdf, html, other]

Title: Selective Explanations

Lucas Monteiro Paes, Dennis Wei, Flavio P. Calmon

Subjects: Computers and Society (cs.CY); Computation and Language (cs.CL); Machine Learning (cs.LG)

[1534] arXiv:2405.19567 (cross-list from cs.AI) [pdf, html, other]

Title: Dr-LLaVA: Visual Instruction Tuning with Symbolic Clinical Grounding

Shenghuan Sun, Alexander Schubert, Gregory M. Goldgof, Zhiqing Sun, Thomas Hartvigsen, Atul J. Butte, Ahmed Alaa

Comments: Code available at: this https URL

Subjects: Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG)

[1535] arXiv:2405.19581 (cross-list from cs.SE) [pdf, html, other]

Title: Source Code Foundation Models are Transferable Binary Analysis Knowledge Bases

Zian Su, Xiangzhe Xu, Ziyang Huang, Kaiyuan Zhang, Xiangyu Zhang

Subjects: Software Engineering (cs.SE); Artificial Intelligence (cs.AI); Computation and Language (cs.CL)

[1536] arXiv:2405.19592 (cross-list from cs.LG) [pdf, html, other]

Title: Why Larger Language Models Do In-context Learning Differently?

Zhenmei Shi, Junyi Wei, Zhuoyan Xu, Yingyu Liang

Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Computation and Language (cs.CL)

[1537] arXiv:2405.19597 (cross-list from cs.LG) [pdf, html, other]

Title: SVFT: Parameter-Efficient Fine-Tuning with Singular Vectors

Vijay Lingam, Atula Tejaswi, Aditya Vavre, Aneesh Shetty, Gautham Krishna Gudur, Joydeep Ghosh, Alex Dimakis, Eunsol Choi, Aleksandar Bojchevski, Sujay Sanghavi

Comments: 17 pages, 5 figures, 14 tables

Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Computation and Language (cs.CL)

[1538] arXiv:2405.19616 (cross-list from cs.AI) [pdf, html, other]

Title: Easy Problems That LLMs Get Wrong

Sean Williams, James Huckle

Comments: AutogenAI Ltd. GitHub Repo: this https URL

Subjects: Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Machine Learning (cs.LG)

[1539] arXiv:2405.19653 (cross-list from cs.LG) [pdf, html, other]

Title: SysCaps: Language Interfaces for Simulation Surrogates of Complex Systems

Patrick Emami, Zhaonan Li, Saumya Sinha, Truc Nguyen

Comments: Accepted at ICLR 2025. 23 pages. Updated with final camera ready version

Subjects: Machine Learning (cs.LG); Computation and Language (cs.CL); Systems and Control (eess.SY)

[1540] arXiv:2405.19716 (cross-list from cs.CV) [pdf, html, other]

Title: Enhancing Large Vision Language Models with Self-Training on Image Comprehension

Yihe Deng, Pan Lu, Fan Yin, Ziniu Hu, Sheng Shen, Quanquan Gu, James Zou, Kai-Wei Chang, Wei Wang

Comments: 22 pages, 14 figures, 9 tables

Subjects: Computer Vision and Pattern Recognition (cs.CV); Computation and Language (cs.CL)

[1541] arXiv:2405.19732 (cross-list from cs.CV) [pdf, html, other]

Title: LLM as a Complementary Optimizer to Gradient Descent: A Case Study in Prompt Tuning

Zixian Guo, Ming Liu, Zhilong Ji, Jinfeng Bai, Yiwen Guo, Wangmeng Zuo

Subjects: Computer Vision and Pattern Recognition (cs.CV); Computation and Language (cs.CL); Machine Learning (cs.LG)

[1542] arXiv:2405.19782 (cross-list from cs.SE) [pdf, html, other]

Title: Dataflow-Guided Retrieval Augmentation for Repository-Level Code Completion

Wei Cheng, Yuhan Wu, Wei Hu

Comments: Accepted in the 62nd Annual Meeting of the Association for Computational Linguistics (ACL 2024)

Subjects: Software Engineering (cs.SE); Computation and Language (cs.CL)

[1543] arXiv:2405.19877 (cross-list from cs.AI) [pdf, html, other]

Title: KNOW: A Real-World Ontology for Knowledge Capture with Large Language Models

Arto Bendiken

Comments: 5 pages, 1 figure

Subjects: Artificial Intelligence (cs.AI); Computation and Language (cs.CL)

[1544] arXiv:2405.19883 (cross-list from cs.LG) [pdf, html, other]

Title: From Words to Actions: Unveiling the Theoretical Underpinnings of LLM-Driven Autonomous Systems

Jianliang He, Siyu Chen, Fengzhuo Zhang, Zhuoran Yang

Comments: 47 pages, accepted by ICML 2024

Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Computation and Language (cs.CL)

[1545] arXiv:2405.19893 (cross-list from cs.LG) [pdf, html, other]

Title: Similarity is Not All You Need: Endowing Retrieval Augmented Generation with Multi Layered Thoughts

Chunjing Gan, Dan Yang, Binbin Hu, Hanxiao Zhang, Siyuan Li, Ziqi Liu, Yue Shen, Lin Ju, Zhiqiang Zhang, Jinjie Gu, Lei Liang, Jun Zhou

Comments: 12 pages

Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Computation and Language (cs.CL)

[1546] arXiv:2405.19954 (cross-list from cs.CR) [pdf, html, other]

Title: GenKubeSec: LLM-Based Kubernetes Misconfiguration Detection, Localization, Reasoning, and Remediation

Ehud Malul, Yair Meidan, Dudu Mimran, Yuval Elovici, Asaf Shabtai

Subjects: Cryptography and Security (cs.CR); Computation and Language (cs.CL); Distributed, Parallel, and Cluster Computing (cs.DC); Machine Learning (cs.LG)

[1547] arXiv:2405.19988 (cross-list from cs.RO) [pdf, html, other]

Title: Video-Language Critic: Transferable Reward Functions for Language-Conditioned Robotics

Minttu Alakuijala, Reginald McLean, Isaac Woungang, Nariman Farsad, Samuel Kaski, Pekka Marttinen, Kai Yuan

Comments: 10 pages in the main text, 16 pages including references and supplementary materials. 4 figures and 3 tables in the main text, 1 table in supplementary materials

Subjects: Robotics (cs.RO); Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG)

[1548] arXiv:2405.20003 (cross-list from cs.LG) [pdf, html, other]

Title: Kernel Language Entropy: Fine-grained Uncertainty Quantification for LLMs from Semantic Similarities

Alexander Nikitin, Jannik Kossen, Yarin Gal, Pekka Marttinen

Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Computation and Language (cs.CL)

[1549] arXiv:2405.20015 (cross-list from cs.AI) [pdf, html, other]

Title: Efficient LLM-Jailbreaking by Introducing Visual Modality

Zhenxing Niu, Yuyao Sun, Haodong Ren, Haoxuan Ji, Quan Wang, Xiaoke Ma, Gang Hua, Rong Jin

Subjects: Artificial Intelligence (cs.AI); Computation and Language (cs.CL)

[1550] arXiv:2405.20018 (cross-list from cs.MA) [pdf, other]

Title: Safe Multi-agent Reinforcement Learning with Natural Language Constraints

Ziyan Wang, Meng Fang, Tristan Tomilin, Fei Fang, Yali Du

Comments: 23 pages, 6 figures

Subjects: Multiagent Systems (cs.MA); Computation and Language (cs.CL); Machine Learning (cs.LG)