Multimedia

Authors and titles for February 2024

[51] arXiv:2402.08577 (cross-list from cs.CL) [pdf, other]

Title: Test-Time Backdoor Attacks on Multimodal Large Language Models

Dong Lu, Tianyu Pang, Chao Du, Qian Liu, Xianjun Yang, Min Lin

Subjects: Computation and Language (cs.CL); Cryptography and Security (cs.CR); Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG); Multimedia (cs.MM)

[52] arXiv:2402.08846 (cross-list from cs.CL) [pdf, html, other]

Title: An Embarrassingly Simple Approach for LLM with Strong ASR Capacity

Ziyang Ma, Guanrou Yang, Yifan Yang, Zhifu Gao, Jiaming Wang, Zhihao Du, Fan Yu, Qian Chen, Siqi Zheng, Shiliang Zhang, Xie Chen

Comments: Working in progress and will open-source soon

Subjects: Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Multimedia (cs.MM); Sound (cs.SD); Audio and Speech Processing (eess.AS)

[53] arXiv:2402.09318 (cross-list from cs.SD) [pdf, other]

Title: Leveraging Pre-Trained Autoencoders for Interpretable Prototype Learning of Music Audio

Pablo Alonso-Jiménez, Leonardo Pepino, Roser Batlle-Roca, Pablo Zinemanas, Dmitry Bogdanov, Xavier Serra, Martín Rocamora

Subjects: Sound (cs.SD); Artificial Intelligence (cs.AI); Multimedia (cs.MM); Audio and Speech Processing (eess.AS)

[54] arXiv:2402.09430 (cross-list from eess.SP) [pdf, html, other]

Title: WiMANS: A Benchmark Dataset for WiFi-based Multi-user Activity Sensing

Shuokang Huang, Kaihan Li, Di You, Yichong Chen, Arvin Lin, Siying Liu, Xiaohui Li, Julie A. McCann

Comments: We present WiMANS, to our knowledge, the first dataset for multi-user activity sensing based on WiFi

Subjects: Signal Processing (eess.SP); Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV); Multimedia (cs.MM)

[55] arXiv:2402.09871 (cross-list from cs.SD) [pdf, html, other]

Title: MuChin: A Chinese Colloquial Description Benchmark for Evaluating Language Models in the Field of Music

Zihao Wang, Shuyu Li, Tao Zhang, Qi Wang, Pengfei Yu, Jinyang Luo, Yan Liu, Ming Xi, Kejun Zhang

Comments: Accepted by International Joint Conference on Artificial Intelligence 2024 (IJCAI 2024)

Subjects: Sound (cs.SD); Artificial Intelligence (cs.AI); Multimedia (cs.MM); Audio and Speech Processing (eess.AS)

[56] arXiv:2402.09883 (cross-list from cs.CV) [pdf, other]

Title: Lester: rotoscope animation through video object segmentation and tracking

Ruben Tous

Subjects: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Graphics (cs.GR); Multimedia (cs.MM)

[57] arXiv:2402.10002 (cross-list from cs.CV) [pdf, html, other]

Title: MM-Point: Multi-View Information-Enhanced Multi-Modal Self-Supervised 3D Point Cloud Understanding

Hai-Tao Yu, Mofei Song

Comments: Accepted by AAAI 2024

Journal-ref: AAAI 2024

Subjects: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Multimedia (cs.MM)

[58] arXiv:2402.10294 (cross-list from cs.HC) [pdf, html, other]

Title: LAVE: LLM-Powered Agent Assistance and Language Augmentation for Video Editing

Bryan Wang, Yuliang Li, Zhaoyang Lv, Haijun Xia, Yan Xu, Raj Sodhi

Comments: Paper accepted to the ACM Conference on Intelligent User Interfaces (ACM IUI) 2024

Subjects: Human-Computer Interaction (cs.HC); Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Multimedia (cs.MM)

[59] arXiv:2402.11250 (cross-list from eess.IV) [pdf, other]

Title: Hierarchical Prior-based Super Resolution for Point Cloud Geometry Compression

Dingquan Li, Kede Ma, Jing Wang, Ge Li

Subjects: Image and Video Processing (eess.IV); Computer Vision and Pattern Recognition (cs.CV); Multimedia (cs.MM)

[60] arXiv:2402.11520 (cross-list from cs.CV) [pdf, html, other]

Title: Cross-Attention Fusion of Visual and Geometric Features for Large Vocabulary Arabic Lipreading

Samar Daou, Achraf Ben-Hamadou, Ahmed Rekik, Abdelaziz Kallel

Comments: submitted for review

Subjects: Computer Vision and Pattern Recognition (cs.CV); Multimedia (cs.MM)

[61] arXiv:2402.11812 (cross-list from cs.CV) [pdf, other]

Title: Interpretable Embedding for Ad-hoc Video Search

Jiaxin Wu, Chong-Wah Ngo

Comments: accepted in ACMMM 2020

Subjects: Computer Vision and Pattern Recognition (cs.CV); Multimedia (cs.MM)

[62] arXiv:2402.11954 (cross-list from cs.SD) [pdf, html, other]

Title: Multimodal Emotion Recognition from Raw Audio with Sinc-convolution

Xiaohui Zhang, Wenjie Fu, Mangui Liang

Subjects: Sound (cs.SD); Multimedia (cs.MM); Audio and Speech Processing (eess.AS)

[63] arXiv:2402.12121 (cross-list from cs.CL) [pdf, html, other]

Title: IRR: Image Review Ranking Framework for Evaluating Vision-Language Models

Kazuki Hayashi, Kazuma Onishi, Toma Suzuki, Yusuke Ide, Seiji Gobara, Shigeki Saito, Yusuke Sakai, Hidetaka Kamigaito, Katsuhiko Hayashi, Taro Watanabe

Comments: 18pages, Accepted at COLING25

Subjects: Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV); Multimedia (cs.MM)

[64] arXiv:2402.12412 (cross-list from cs.HC) [pdf, html, other]

Title: Dynamic and Super-Personalized Media Ecosystem Driven by Generative AI: Unpredictable Plays Never Repeating The Same

Sungjun Ahn, Hyun-Jeong Yim, Youngwan Lee, Sung-Ik Park

Comments: 13 pages, 7 figures

Subjects: Human-Computer Interaction (cs.HC); Artificial Intelligence (cs.AI); Multimedia (cs.MM); Signal Processing (eess.SP)

[65] arXiv:2402.12451 (cross-list from cs.CV) [pdf, html, other]

Title: The Revolution of Multimodal Large Language Models: A Survey

Davide Caffagni, Federico Cocchi, Luca Barsellotti, Nicholas Moratelli, Sara Sarto, Lorenzo Baraldi, Lorenzo Baraldi, Marcella Cornia, Rita Cucchiara

Comments: ACL 2024 (Findings)

Subjects: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Multimedia (cs.MM)

[66] arXiv:2402.13022 (cross-list from cs.CL) [pdf, html, other]

Title: SoMeLVLM: A Large Vision Language Model for Social Media Processing

Xinnong Zhang, Haoyu Kuang, Xinyi Mou, Hanjia Lyu, Kun Wu, Siming Chen, Jiebo Luo, Xuanjing Huang, Zhongyu Wei

Subjects: Computation and Language (cs.CL); Multimedia (cs.MM)

[67] arXiv:2402.14947 (cross-list from cs.HC) [pdf, html, other]

Title: An Avalanche of Images on Telegram Preceded Russia's Full-Scale Invasion of Ukraine

William Theisen, Michael Yankoski, Kristina Hook, Ernesto Verdeja, Walter Scheirer, Tim Weninger

Comments: 20 pages, 7 figures

Subjects: Human-Computer Interaction (cs.HC); Multimedia (cs.MM); Social and Information Networks (cs.SI)

[68] arXiv:2402.15096 (cross-list from cs.LG) [pdf, html, other]

Title: Multimodal Transformer With a Low-Computational-Cost Guarantee

Sungjin Park, Edward Choi

Comments: Accepted to ICASSP 2024 (5 pages)

Subjects: Machine Learning (cs.LG); Computer Vision and Pattern Recognition (cs.CV); Multimedia (cs.MM)

[69] arXiv:2402.15300 (cross-list from cs.CV) [pdf, html, other]

Title: Seeing is Believing: Mitigating Hallucination in Large Vision-Language Models via CLIP-Guided Decoding

Ailin Deng, Zhirui Chen, Bryan Hooi

Comments: Code URL: this https URL

Subjects: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Machine Learning (cs.LG); Multimedia (cs.MM)

[70] arXiv:2402.15444 (cross-list from cs.AI) [pdf, html, other]

Title: Unleashing the Power of Imbalanced Modality Information for Multi-modal Knowledge Graph Completion

Yichi Zhang, Zhuo Chen, Lei Liang, Huajun Chen, Wen Zhang

Comments: Accepted by LREC-COLING 2024

Subjects: Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Machine Learning (cs.LG); Multimedia (cs.MM)

[71] arXiv:2402.15695 (cross-list from cs.HC) [pdf, other]

Title: Applied User Research in Virtual Reality: Tools, Methods, and Challenges

Leonie Bensch, Andrea Casini, Aidan Cowley, Florian Dufresne, Enrico Guerra, Paul de Medeiros, Tommy Nilsson, Flavie Rometsch, Andreas Treuer, Anna Vock

Subjects: Human-Computer Interaction (cs.HC); Multimedia (cs.MM)

[72] arXiv:2402.15746 (cross-list from cs.CV) [pdf, html, other]

Title: Intelligent Director: An Automatic Framework for Dynamic Visual Composition using ChatGPT

Sixiao Zheng, Jingyang Huo, Yu Wang, Yanwei Fu

Comments: Project Page: this https URL

Subjects: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Multimedia (cs.MM)

[73] arXiv:2402.15923 (cross-list from cs.LG) [pdf, html, other]

Title: Predicting Outcomes in Video Games with Long Short Term Memory Networks

Kittimate Chulajata, Sean Wu, Fabien Scalzo, Eun Sang Cha

Comments: 7 pages, 2 Figures, 2 Tables. Kittimate Chulajata and Sean Wu are considered co-first authors

Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Multimedia (cs.MM)

[74] arXiv:2402.16110 (cross-list from cs.IR) [pdf, html, other]

Title: Disentangled Graph Variational Auto-Encoder for Multimodal Recommendation with Interpretability

Xin Zhou, Chunyan Miao

Comments: 12 pages, 7 figures

Subjects: Information Retrieval (cs.IR); Multimedia (cs.MM)

[75] arXiv:2402.16153 (cross-list from cs.SD) [pdf, html, other]

Title: ChatMusician: Understanding and Generating Music Intrinsically with LLM

Ruibin Yuan, Hanfeng Lin, Yi Wang, Zeyue Tian, Shangda Wu, Tianhao Shen, Ge Zhang, Yuhang Wu, Cong Liu, Ziya Zhou, Ziyang Ma, Liumeng Xue, Ziyu Wang, Qin Liu, Tianyu Zheng, Yizhi Li, Yinghao Ma, Yiming Liang, Xiaowei Chi, Ruibo Liu, Zili Wang, Pengfei Li, Jingcheng Wu, Chenghua Lin, Qifeng Liu, Tao Jiang, Wenhao Huang, Wenhu Chen, Emmanouil Benetos, Jie Fu, Gus Xia, Roger Dannenberg, Wei Xue, Shiyin Kang, Yike Guo

Comments: GitHub: this https URL

Subjects: Sound (cs.SD); Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Machine Learning (cs.LG); Multimedia (cs.MM); Audio and Speech Processing (eess.AS)

[76] arXiv:2402.16318 (cross-list from cs.CV) [pdf, html, other]

Title: Gradient-Guided Modality Decoupling for Missing-Modality Robustness

Hao Wang, Shengda Luo, Guosheng Hu, Jianguo Zhang

Comments: AAAI24

Subjects: Computer Vision and Pattern Recognition (cs.CV); Multimedia (cs.MM)

[77] arXiv:2402.16364 (cross-list from cs.CL) [pdf, html, other]

Title: Where Do We Go from Here? Multi-scale Allocentric Relational Inference from Natural Spatial Descriptions

Tzuf Paz-Argaman, Sayali Kulkarni, John Palowitch, Jason Baldridge, Reut Tsarfaty

Journal-ref: EACL 2024

Subjects: Computation and Language (cs.CL); Machine Learning (cs.LG); Multimedia (cs.MM)

[78] arXiv:2402.16366 (cross-list from cs.CV) [pdf, html, other]

Title: SPC-NeRF: Spatial Predictive Compression for Voxel Based Radiance Field

Zetian Song, Wenhong Duan, Yuhuai Zhang, Shiqi Wang, Siwei Ma, Wen Gao

Subjects: Computer Vision and Pattern Recognition (cs.CV); Multimedia (cs.MM)

[79] arXiv:2402.16665 (cross-list from cs.HC) [pdf, html, other]

Title: The Interaction Fidelity Model: A Taxonomy to Distinguish the Aspects of Fidelity in Virtual Reality

Michael Bonfert, Thomas Muender, Ryan P. McMahan, Frank Steinicke, Doug Bowman, Rainer Malaka, Tanja Döring

Comments: 34 pages incl. references and appendix

Journal-ref: International Journal of Human-Computer Interaction by Taylor and Francis, 2024, 1-33

Subjects: Human-Computer Interaction (cs.HC); Graphics (cs.GR); Multimedia (cs.MM)

[80] arXiv:2402.17723 (cross-list from cs.CV) [pdf, html, other]

Title: Seeing and Hearing: Open-domain Visual-Audio Generation with Diffusion Latent Aligners

Yazhou Xing, Yingqing He, Zeyue Tian, Xintao Wang, Qifeng Chen

Comments: Accepted to CVPR 2024. Project website: this https URL

Subjects: Computer Vision and Pattern Recognition (cs.CV); Multimedia (cs.MM); Sound (cs.SD); Audio and Speech Processing (eess.AS)

[81] arXiv:2402.18122 (cross-list from cs.CV) [pdf, html, other]

Title: G4G:A Generic Framework for High Fidelity Talking Face Generation with Fine-grained Intra-modal Alignment

Juan Zhang, Jiahao Chen, Cheng Wang, Zhiwang Yu, Tangquan Qi, Di Wu

Subjects: Computer Vision and Pattern Recognition (cs.CV); Multimedia (cs.MM)

[82] arXiv:2402.18208 (cross-list from cs.SI) [pdf, html, other]

Title: Shorts on the Rise: Assessing the Effects of YouTube Shorts on Long-Form Video Content

Prajit T. Rajendran, Kevin Creusy, Vivien Garnes

Subjects: Social and Information Networks (cs.SI); Multimedia (cs.MM)

[83] arXiv:2402.18761 (cross-list from eess.IV) [pdf, html, other]

Title: Exploration of Learned Lifting-Based Transform Structures for Fully Scalable and Accessible Wavelet-Like Image Compression

Xinyue Li, Aous Naman, David Taubman

Comments: This work has been submitted to the IEEE for possible publication

Subjects: Image and Video Processing (eess.IV); Computer Vision and Pattern Recognition (cs.CV); Multimedia (cs.MM)

[84] arXiv:2402.18844 (cross-list from cs.CV) [pdf, html, other]

Title: Deep learning for 3D human pose estimation and mesh recovery: A survey

Yang Liu, Changzhen Qiu, Zhiyong Zhang

Subjects: Computer Vision and Pattern Recognition (cs.CV); Multimedia (cs.MM)

[85] arXiv:2402.18927 (cross-list from cs.CV) [pdf, html, other]

Title: Edge Computing Enabled Real-Time Video Analysis via Adaptive Spatial-Temporal Semantic Filtering

Xiang Chen, Wenjie Zhu, Jiayuan Chen, Tong Zhang, Changyan Yi, Jun Cai

Subjects: Computer Vision and Pattern Recognition (cs.CV); Multimedia (cs.MM); Networking and Internet Architecture (cs.NI)

[86] arXiv:2402.19330 (cross-list from cs.CV) [pdf, html, other]

Title: A Novel Approach to Industrial Defect Generation through Blended Latent Diffusion Model with Online Adaptation

Hanxi Li, Zhengxun Zhang, Hao Chen, Lin Wu, Bo Li, Deyin Liu, Mingwen Wang

Comments: 13 pages,7 figures

Subjects: Computer Vision and Pattern Recognition (cs.CV); Multimedia (cs.MM)