Sound

Authors and titles for May 2024

[126] arXiv:2405.08295 (cross-list from cs.CL) [pdf, other]

Title: SpeechVerse: A Large-scale Generalizable Audio Language Model

Nilaksh Das, Saket Dingliwal, Srikanth Ronanki, Rohit Paturi, Zhaocheng Huang, Prashant Mathur, Jie Yuan, Dhanush Bekal, Xing Niu, Sai Muralidhar Jayanthi, Xilai Li, Karel Mundnich, Monica Sunkara, Sravan Bodapati, Sundararajan Srinivasan, Kyu J Han, Katrin Kirchhoff

Comments: Single Column, 13 page

Subjects: Computation and Language (cs.CL); Sound (cs.SD); Audio and Speech Processing (eess.AS)

[127] arXiv:2405.08317 (cross-list from cs.CL) [pdf, html, other]

Title: SpeechGuard: Exploring the Adversarial Robustness of Multimodal Large Language Models

Raghuveer Peri, Sai Muralidhar Jayanthi, Srikanth Ronanki, Anshu Bhatia, Karel Mundnich, Saket Dingliwal, Nilaksh Das, Zejiang Hou, Goeric Huybrechts, Srikanth Vishnubhotla, Daniel Garcia-Romero, Sundararajan Srinivasan, Kyu J Han, Katrin Kirchhoff

Comments: 9+6 pages, Submitted to ACL 2024

Subjects: Computation and Language (cs.CL); Sound (cs.SD); Audio and Speech Processing (eess.AS)

[128] arXiv:2405.08417 (cross-list from eess.AS) [pdf, html, other]

Title: Neural Speech Coding for Real-time Communications using Constant Bitrate Scalar Quantization

Andreas Brendel, Nicola Pia, Kishan Gupta, Lyonel Behringer, Guillaume Fuchs, Markus Multrus

Subjects: Audio and Speech Processing (eess.AS); Sound (cs.SD)

[129] arXiv:2405.08742 (cross-list from eess.AS) [pdf, other]

Title: A tunable binaural audio telepresence system capable of balancing immersive and enhanced modes

Yicheng Hsu, Mingsian R. Bai

Comments: 5 pages, 4 figures

Subjects: Audio and Speech Processing (eess.AS); Sound (cs.SD)

[130] arXiv:2405.09142 (cross-list from eess.AS) [pdf, html, other]

Title: Speaker Embeddings With Weakly Supervised Voice Activity Detection For Efficient Speaker Diarization

Jenthe Thienpondt, Kris Demuynck

Comments: Proceedings of Odyssey 2024: The Speaker and Language Recognition Workshop

Subjects: Audio and Speech Processing (eess.AS); Sound (cs.SD)

[131] arXiv:2405.09266 (cross-list from cs.CV) [pdf, html, other]

Title: Dance Any Beat: Blending Beats with Visuals in Dance Video Generation

Xuanchen Wang, Heng Wang, Dongnan Liu, Weidong Cai

Comments: WACV2025, 11 pages, 7 figures, demo page: this https URL

Subjects: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Multimedia (cs.MM); Sound (cs.SD); Audio and Speech Processing (eess.AS)

[132] arXiv:2405.09570 (cross-list from eess.SP) [pdf, html, other]

Title: FunnelNet: An End-to-End Deep Learning Framework to Monitor Digital Heart Murmur in Real-Time

Md Jobayer, Md. Mehedi Hasan Shawon, Md Rakibul Hasan, Shreya Ghosh, Tom Gedeon, Md Zakir Hossain

Comments: 8-page main paper and 4-page supplementary material

Subjects: Signal Processing (eess.SP); Machine Learning (cs.LG); Sound (cs.SD); Audio and Speech Processing (eess.AS)

[133] arXiv:2405.09589 (cross-list from cs.LG) [pdf, html, other]

Title: A Comprehensive Survey of Hallucination in Large Language, Image, Video and Audio Foundation Models

Pranab Sahoo, Prabhash Meharia, Akash Ghosh, Sriparna Saha, Vinija Jain, Aman Chadha

Comments: EMNLP 2024 Findings

Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Computer Vision and Pattern Recognition (cs.CV); Sound (cs.SD); Audio and Speech Processing (eess.AS)

[134] arXiv:2405.09768 (cross-list from eess.AS) [pdf, html, other]

Title: Evaluating Text-to-Speech Synthesis from a Large Discrete Token-based Speech Language Model

Siyang Wang, Éva Székely

Comments: 11 pages, 4 figures. Language Resources and Evaluation Conference (LREC) 2024. demo: this https URL

Subjects: Audio and Speech Processing (eess.AS); Sound (cs.SD)

[135] arXiv:2405.09814 (cross-list from cs.GR) [pdf, html, other]

Title: Semantic Gesticulator: Semantics-Aware Co-Speech Gesture Synthesis

Zeyi Zhang, Tenglong Ao, Yuyao Zhang, Qingzhe Gao, Chuan Lin, Baoquan Chen, Libin Liu

Comments: SIGGRAPH 2024 (Journal Track); Project page: this https URL

Subjects: Graphics (cs.GR); Computer Vision and Pattern Recognition (cs.CV); Sound (cs.SD); Audio and Speech Processing (eess.AS)

[136] arXiv:2405.09940 (cross-list from eess.AS) [pdf, html, other]

Title: Robust Singing Voice Transcription Serves Synthesis

Ruiqi Li, Yu Zhang, Yongqi Wang, Zhiqing Hong, Rongjie Huang, Zhou Zhao

Comments: ACL 2024

Subjects: Audio and Speech Processing (eess.AS); Sound (cs.SD)

[137] arXiv:2405.10018 (cross-list from eess.AS) [pdf, html, other]

Title: Data-Efficient Low-Complexity Acoustic Scene Classification in the DCASE 2024 Challenge

Florian Schmid, Paul Primus, Toni Heittola, Annamaria Mesaros, Irene Martín-Morató, Khaled Koutini, Gerhard Widmer

Comments: Task Description Page: this https URL

Subjects: Audio and Speech Processing (eess.AS); Sound (cs.SD)

[138] arXiv:2405.10022 (cross-list from eess.AS) [pdf, html, other]

Title: Monaural speech enhancement on drone via Adapter based transfer learning

Xingyu Chen, Hanwen Bi, Wei-Ting Lai, Fei Ma

Journal-ref: 2024 18th International Workshop on Acoustic Signal Enhancement (IWAENC), Aalborg, Denmark, 09-12 September 2024

Subjects: Audio and Speech Processing (eess.AS); Sound (cs.SD)

[139] arXiv:2405.10025 (cross-list from cs.CL) [pdf, html, other]

Title: Listen Again and Choose the Right Answer: A New Paradigm for Automatic Speech Recognition with Large Language Models

Yuchen Hu, Chen Chen, Chengwei Qin, Qiushi Zhu, Eng Siong Chng, Ruizhe Li

Comments: 14 pages, Accepted by ACL 2024

Subjects: Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Machine Learning (cs.LG); Sound (cs.SD); Audio and Speech Processing (eess.AS)

[140] arXiv:2405.10084 (cross-list from eess.AS) [pdf, html, other]

Title: Revisiting Deep Audio-Text Retrieval Through the Lens of Transportation

Manh Luong, Khai Nguyen, Nhat Ho, Reza Haf, Dinh Phung, Lizhen Qu

Subjects: Audio and Speech Processing (eess.AS); Artificial Intelligence (cs.AI); Sound (cs.SD)

[141] arXiv:2405.10272 (cross-list from cs.CV) [pdf, html, other]

Title: Faces that Speak: Jointly Synthesising Talking Face and Speech from Text

Youngjoon Jang, Ji-Hoon Kim, Junseok Ahn, Doyeop Kwak, Hong-Sun Yang, Yoon-Cheol Ju, Il-Hwan Kim, Byeong-Yeol Kim, Joon Son Chung

Comments: CVPR 2024

Subjects: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Sound (cs.SD); Audio and Speech Processing (eess.AS); Image and Video Processing (eess.IV)

[142] arXiv:2405.10502 (cross-list from cs.HC) [pdf, html, other]

Title: Enhancing DMI Interactions by Integrating Haptic Feedback for Intricate Vibrato Technique

Ziyue Piao, Christian Frisson, Bavo Van Kerrebroeck, Marcelo M.Wanderley

Subjects: Human-Computer Interaction (cs.HC); Sound (cs.SD); Audio and Speech Processing (eess.AS)

[143] arXiv:2405.11093 (cross-list from eess.AS) [pdf, html, other]

Title: AudioSetMix: Enhancing Audio-Language Datasets with LLM-Assisted Augmentations

David Xu

Comments: typos corrected

Subjects: Audio and Speech Processing (eess.AS); Computation and Language (cs.CL); Multimedia (cs.MM); Sound (cs.SD)

[144] arXiv:2405.11767 (cross-list from eess.AS) [pdf, html, other]

Title: Multi-speaker Text-to-speech Training with Speaker Anonymized Data

Wen-Chin Huang, Yi-Chiao Wu, Tomoki Toda

Comments: 5 pages. Submitted to Signal Processing Letters. Audio sample page: this https URL

Subjects: Audio and Speech Processing (eess.AS); Cryptography and Security (cs.CR); Sound (cs.SD)

[145] arXiv:2405.12221 (cross-list from cs.CV) [pdf, html, other]

Title: Images that Sound: Composing Images and Sounds on a Single Canvas

Ziyang Chen, Daniel Geng, Andrew Owens

Comments: Accepted to NeurIPS 2024. Project site: this https URL

Subjects: Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG); Multimedia (cs.MM); Sound (cs.SD); Audio and Speech Processing (eess.AS)

[146] arXiv:2405.12496 (cross-list from eess.AS) [pdf, html, other]

Title: A Survey of Integrating Wireless Technology into Active Noise Control

Xiaoyi Shen, Dongyuan Shi, Zhengding Luo, Junwei Ji, Woon-Seng Gan

Subjects: Audio and Speech Processing (eess.AS); Networking and Internet Architecture (cs.NI); Sound (cs.SD); Signal Processing (eess.SP)

[147] arXiv:2405.12609 (cross-list from eess.AS) [pdf, html, other]

Title: Mamba in Speech: Towards an Alternative to Self-Attention

Xiangyu Zhang, Qiquan Zhang, Hexin Liu, Tianyi Xiao, Xinyuan Qian, Beena Ahmed, Eliathamby Ambikairajah, Haizhou Li, Julien Epps

Subjects: Audio and Speech Processing (eess.AS); Sound (cs.SD)

[148] arXiv:2405.12847 (cross-list from cs.IR) [pdf, html, other]

Title: A Dataset and Baselines for Measuring and Predicting the Music Piece Memorability

Li-Yang Tseng, Tzu-Ling Lin, Hong-Han Shuai, Jen-Wei Huang, Wen-Whei Chang

Journal-ref: Proceedings of the 24th International Society for Music Information Retrieval Conference, 174-181. Milan, Italy, November 5-9, 2023

Subjects: Information Retrieval (cs.IR); Machine Learning (cs.LG); Multimedia (cs.MM); Sound (cs.SD); Audio and Speech Processing (eess.AS)

[149] arXiv:2405.12899 (cross-list from math.FA) [pdf, html, other]

Title: On a time-frequency blurring operator with applications in data augmentation

Simon Halvdansson

Comments: 22 pages, 4 figures

Subjects: Functional Analysis (math.FA); Sound (cs.SD); Audio and Speech Processing (eess.AS)

[150] arXiv:2405.12983 (cross-list from eess.AS) [pdf, html, other]

Title: Multilingual Audio-Visual Speech Recognition with Hybrid CTC/RNN-T Fast Conformer

Maxime Burchi, Krishna C. Puvvada, Jagadeesh Balam, Boris Ginsburg, Radu Timofte

Subjects: Audio and Speech Processing (eess.AS); Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV); Multimedia (cs.MM); Sound (cs.SD)

[151] arXiv:2405.13344 (cross-list from eess.AS) [pdf, html, other]

Title: Contextualized Automatic Speech Recognition with Dynamic Vocabulary

Yui Sudo, Yosuke Fukumoto, Muhammad Shakeel, Yifan Peng, Shinji Watanabe

Subjects: Audio and Speech Processing (eess.AS); Computation and Language (cs.CL); Sound (cs.SD)

[152] arXiv:2405.13379 (cross-list from cs.CL) [pdf, html, other]

Title: You don't understand me!: Comparing ASR results for L1 and L2 speakers of Swedish

Ronald Cumbal, Birger Moell, Jose Lopes, Olof Engwall

Subjects: Computation and Language (cs.CL); Sound (cs.SD); Audio and Speech Processing (eess.AS)

[153] arXiv:2405.13477 (cross-list from cs.HC) [pdf, html, other]

Title: A Near-Real-Time Processing Ego Speech Filtering Pipeline Designed for Speech Interruption During Human-Robot Interaction

Yue Li, Florian A. Kunneman, Koen V. Hindriks

Comments: 8 pages,16 figures, Under review by RoMan 2024 conference

Subjects: Human-Computer Interaction (cs.HC); Sound (cs.SD); Audio and Speech Processing (eess.AS)

[154] arXiv:2405.13514 (cross-list from eess.AS) [pdf, html, other]

Title: Joint Optimization of Streaming and Non-Streaming Automatic Speech Recognition with Multi-Decoder and Knowledge Distillation

Muhammad Shakeel, Yui Sudo, Yifan Peng, Shinji Watanabe

Comments: Accepted to IEEE ICASSP 2024 workshop Hands-free Speech Communication and Microphone Arrays (HSCMA 2024)

Subjects: Audio and Speech Processing (eess.AS); Computation and Language (cs.CL); Sound (cs.SD)

[155] arXiv:2405.13762 (cross-list from cs.CV) [pdf, html, other]

Title: A Versatile Diffusion Transformer with Mixture of Noise Levels for Audiovisual Generation

Gwanghyun Kim, Alonso Martinez, Yu-Chuan Su, Brendan Jou, José Lezama, Agrim Gupta, Lijun Yu, Lu Jiang, Aren Jansen, Jacob Walker, Krishna Somandepalli

Subjects: Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG); Multimedia (cs.MM); Sound (cs.SD); Audio and Speech Processing (eess.AS)

[156] arXiv:2405.14161 (cross-list from cs.CL) [pdf, html, other]

Title: Self-Taught Recognizer: Toward Unsupervised Adaptation for Speech Foundation Models

Yuchen Hu, Chen Chen, Chao-Han Huck Yang, Chengwei Qin, Pin-Yu Chen, Eng Siong Chng, Chao Zhang

Comments: 23 pages, Preprint

Subjects: Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Machine Learning (cs.LG); Sound (cs.SD); Audio and Speech Processing (eess.AS)

[157] arXiv:2405.14598 (cross-list from cs.CV) [pdf, html, other]

Title: Visual Echoes: A Simple Unified Transformer for Audio-Visual Generation

Shiqi Yang, Zhi Zhong, Mengjie Zhao, Shusuke Takahashi, Masato Ishii, Takashi Shibuya, Yuki Mitsufuji

Comments: 10 pages

Subjects: Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG); Multimedia (cs.MM); Sound (cs.SD); Audio and Speech Processing (eess.AS)

[158] arXiv:2405.15085 (cross-list from eess.SP) [pdf, html, other]

Title: Acoustical Features as Knee Health Biomarkers: A Critical Analysis

Christodoulos Kechris, Jerome Thevenot, Tomas Teijeiro, Vincent A. Stadelmann, Nicola A. Maffiuletti, David Atienza

Subjects: Signal Processing (eess.SP); Sound (cs.SD); Audio and Speech Processing (eess.AS)

[159] arXiv:2405.15216 (cross-list from cs.LG) [pdf, html, other]

Title: Denoising LM: Pushing the Limits of Error Correction Models for Speech Recognition

Zijin Gu, Tatiana Likhomanenko, He Bai, Erik McDermott, Ronan Collobert, Navdeep Jaitly

Comments: under review

Subjects: Machine Learning (cs.LG); Computation and Language (cs.CL); Sound (cs.SD); Audio and Speech Processing (eess.AS)

[160] arXiv:2405.15923 (cross-list from eess.SP) [pdf, other]

Title: Spiketrum: An FPGA-based Implementation of a Neuromorphic Cochlea

MHD Anas Alsakkal, Jayawan Wijekoon

Comments: To be published at "IEEE Transactions on Circuits and Systems"

Subjects: Signal Processing (eess.SP); Neural and Evolutionary Computing (cs.NE); Sound (cs.SD); Audio and Speech Processing (eess.AS)

[161] arXiv:2405.16136 (cross-list from cs.AI) [pdf, html, other]

Title: C3LLM: Conditional Multimodal Content Generation Using Large Language Models

Zixuan Wang, Qinkai Duan, Yu-Wing Tai, Chi-Keung Tang

Subjects: Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Machine Learning (cs.LG); Sound (cs.SD); Audio and Speech Processing (eess.AS)

[162] arXiv:2405.16677 (cross-list from eess.AS) [pdf, html, other]

Title: Crossmodal ASR Error Correction with Discrete Speech Units

Yuanchao Li, Pinzhen Chen, Peter Bell, Catherine Lai

Comments: Accepted to IEEE SLT 2024

Subjects: Audio and Speech Processing (eess.AS); Computation and Language (cs.CL); Sound (cs.SD)

[163] arXiv:2405.17100 (cross-list from cs.CR) [pdf, html, other]

Title: Sok: Comprehensive Security Overview, Challenges, and Future Directions of Voice-Controlled Systems

Haozhe Xu, Cong Wu, Yangyang Gu, Xingcan Shang, Jing Chen, Kun He, Ruiying Du

Subjects: Cryptography and Security (cs.CR); Sound (cs.SD); Audio and Speech Processing (eess.AS)

[164] arXiv:2405.17569 (cross-list from cs.LG) [pdf, html, other]

Title: Discriminant audio properties in deep learning based respiratory insufficiency detection in Brazilian Portuguese

Marcelo Matheus Gauy, Larissa Cristina Berti, Arnaldo Cândido Jr, Augusto Camargo Neto, Alfredo Goldman, Anna Sara Shafferman Levin, Marcus Martins, Beatriz Raposo de Medeiros, Marcelo Queiroz, Ester Cerdeira Sabino, Flaviane Romani Fernandes Svartman, Marcelo Finger

Comments: 5 pages, 2 figures, 1 table. Published in Artificial Intelligence in Medicine (AIME) 2023

Journal-ref: Artificial Intellingence in Medicine Proceedings 2023, page 271-275

Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Sound (cs.SD); Audio and Speech Processing (eess.AS)

[165] arXiv:2405.17809 (cross-list from cs.CL) [pdf, html, other]

Title: TransVIP: Speech to Speech Translation System with Voice and Isochrony Preservation

Chenyang Le, Yao Qian, Dongmei Wang, Long Zhou, Shujie Liu, Xiaofei Wang, Midia Yousefi, Yanmin Qian, Jinyu Li, Sheng Zhao, Michael Zeng

Comments: Neural Information Processing Systems, poster

Subjects: Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Sound (cs.SD); Audio and Speech Processing (eess.AS)

[166] arXiv:2405.17842 (cross-list from cs.CV) [pdf, html, other]

Title: MMDisCo: Multi-Modal Discriminator-Guided Cooperative Diffusion for Joint Audio and Video Generation

Akio Hayakawa, Masato Ishii, Takashi Shibuya, Yuki Mitsufuji

Comments: ICLR 2025

Subjects: Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG); Multimedia (cs.MM); Sound (cs.SD); Audio and Speech Processing (eess.AS)

[167] arXiv:2405.18639 (cross-list from q-bio.NC) [pdf, other]

Title: Improving Speech Decoding from ECoG with Self-Supervised Pretraining

Brian A. Yuan, Joseph G. Makin

Subjects: Neurons and Cognition (q-bio.NC); Computation and Language (cs.CL); Machine Learning (cs.LG); Sound (cs.SD); Audio and Speech Processing (eess.AS)

[168] arXiv:2405.19041 (cross-list from cs.CL) [pdf, html, other]

Title: BLSP-KD: Bootstrapping Language-Speech Pre-training via Knowledge Distillation

Chen Wang, Minpeng Liao, Zhongqiang Huang, Jiajun Zhang

Subjects: Computation and Language (cs.CL); Sound (cs.SD); Audio and Speech Processing (eess.AS)

[169] arXiv:2405.19334 (cross-list from cs.AI) [pdf, html, other]

Title: LLMs Meet Multimodal Generation and Editing: A Survey

Yingqing He, Zhaoyang Liu, Jingye Chen, Zeyue Tian, Hongyu Liu, Xiaowei Chi, Runtao Liu, Ruibin Yuan, Yazhou Xing, Wenhai Wang, Jifeng Dai, Yong Zhang, Wei Xue, Qifeng Liu, Yike Guo, Qifeng Chen

Comments: 52 Pages with 16 Figures, 12 Tables, and 545 References. GitHub Repository at: this https URL

Subjects: Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Computer Vision and Pattern Recognition (cs.CV); Multimedia (cs.MM); Sound (cs.SD)

[170] arXiv:2405.19426 (cross-list from cs.CL) [pdf, html, other]

Title: Deep Learning for Assessment of Oral Reading Fluency

Mithilesh Vaidya, Binaya Kumar Sahoo, Preeti Rao

Subjects: Computation and Language (cs.CL); Sound (cs.SD); Audio and Speech Processing (eess.AS)

[171] arXiv:2405.19497 (cross-list from eess.AS) [pdf, html, other]

Title: Gaussian Flow Bridges for Audio Domain Transfer with Unpaired Data

Eloi Moliner, Sebastian Braun, Hannes Gamper

Comments: Submitted to IWAENC 2024

Subjects: Audio and Speech Processing (eess.AS); Machine Learning (cs.LG); Sound (cs.SD)

[172] arXiv:2405.20064 (cross-list from eess.AS) [pdf, html, other]

Title: 1st Place Solution to Odyssey Emotion Recognition Challenge Task1: Tackling Class Imbalance Problem

Mingjie Chen, Hezhao Zhang, Yuanchao Li, Jiachen Luo, Wen Wu, Ziyang Ma, Peter Bell, Catherine Lai, Joshua Reiss, Lin Wang, Philip C. Woodland, Xie Chen, Huy Phan, Thomas Hain

Subjects: Audio and Speech Processing (eess.AS); Sound (cs.SD)

[173] arXiv:2405.20336 (cross-list from cs.CV) [pdf, html, other]

Title: RapVerse: Coherent Vocals and Whole-Body Motions Generations from Text

Jiaben Chen, Xin Yan, Yihang Chen, Siyuan Cen, Qinwei Ma, Haoyu Zhen, Kaizhi Qian, Lie Lu, Chuang Gan

Comments: Project website: this https URL

Subjects: Computer Vision and Pattern Recognition (cs.CV); Sound (cs.SD); Audio and Speech Processing (eess.AS)

[174] arXiv:2405.20402 (cross-list from eess.AS) [pdf, html, other]

Title: Cross-Talk Reduction

Zhong-Qiu Wang, Anurag Kumar, Shinji Watanabe

Comments: in International Joint Conference on Artificial Intelligence (IJCAI), 2024

Subjects: Audio and Speech Processing (eess.AS); Sound (cs.SD); Signal Processing (eess.SP)

[175] arXiv:2405.20410 (cross-list from cs.CL) [pdf, html, other]

Title: SeamlessExpressiveLM: Speech Language Model for Expressive Speech-to-Speech Translation with Chain-of-Thought

Hongyu Gong, Bandhav Veluri

Subjects: Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Sound (cs.SD); Audio and Speech Processing (eess.AS)