Multimedia

Authors and titles for September 2024

[1] arXiv:2409.00022 [pdf, other]

Title: Detecting Misinformation in Multimedia Content through Cross-Modal Entity Consistency: A Dual Learning Approach

Zhe Fu, Kanlun Wang, Wangjiaxuan Xin, Lina Zhou, Shi Chen, Yaorong Ge, Daniel Janies, Dongsong Zhang

Comments: Accepted to PACIS 2024. 15 pages, 3 figures

Journal-ref: https://aisel.aisnet.org/pacis2024/track07_secprivacy/track07_secprivacy/2

Subjects: Multimedia (cs.MM); Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV)

[2] arXiv:2409.00597 [pdf, html, other]

Title: Multimodal Multi-turn Conversation Stance Detection: A Challenge Dataset and Effective Model

Fuqiang Niu, Zebang Cheng, Xianghua Fu, Xiaojiang Peng, Genan Dai, Yin Chen, Hu Huang, Bowen Zhang

Comments: ACM MM2024

Subjects: Multimedia (cs.MM); Computation and Language (cs.CL)

[3] arXiv:2409.01029 [pdf, html, other]

Title: Multi-Reference Generative Face Video Compression with Contrastive Learning

Goluck Konuko, Giuseppe Valenzise

Subjects: Multimedia (cs.MM)

[4] arXiv:2409.01710 [pdf, html, other]

Title: Privacy-Preserving Multimedia Mobile Cloud Computing Using Protective Perturbation

Zhongze Tang, Mengmei Ye, Yao Liu, Sheng Wei

Subjects: Multimedia (cs.MM)

[5] arXiv:2409.05297 [pdf, html, other]

Title: Adaptive Offloading and Enhancement for Low-Light Video Analytics on Mobile Devices

Yuanyi He, Peng Yang, Tian Qin, Jiawei Hou, Ning Zhang

Subjects: Multimedia (cs.MM)

[6] arXiv:2409.05772 [pdf, html, other]

Title: A CLIP-based siamese approach for meme classification

Javier Huertas-Tato, Christos Koutlis, Symeon Papadopoulos, David Camacho, Ioannis Kompatsiaris

Subjects: Multimedia (cs.MM)

[7] arXiv:2409.06051 [pdf, html, other]

Title: REVISION: A Roadmap on Adaptive Video Streaming Optimization

Farzad Tashtarian, Christian Timmerer

Comments: 10 pages, 5 figures

Subjects: Multimedia (cs.MM)

[8] arXiv:2409.06709 [pdf, html, other]

Title: Unveiling Visual Biases in Audio-Visual Localization Benchmarks

Liangyu Chen, Zihao Yue, Boshen Xu, Qin Jin

Comments: Accepted by ECCV24 AVGenL Workshop

Subjects: Multimedia (cs.MM); Artificial Intelligence (cs.AI); Sound (cs.SD); Audio and Speech Processing (eess.AS)

[9] arXiv:2409.07450 [pdf, html, other]

Title: VMAS: Video-to-Music Generation via Semantic Alignment in Web Music Videos

Yan-Bo Lin, Yu Tian, Linjie Yang, Gedas Bertasius, Heng Wang

Comments: Project Page: this https URL

Subjects: Multimedia (cs.MM); Computer Vision and Pattern Recognition (cs.CV); Sound (cs.SD); Audio and Speech Processing (eess.AS)

[10] arXiv:2409.07759 [pdf, other]

Title: SwinGS: Sliding Window Gaussian Splatting for Volumetric Video Streaming with Arbitrary Length

Bangya Liu, Suman Banerjee

Subjects: Multimedia (cs.MM); Computer Vision and Pattern Recognition (cs.CV)

[11] arXiv:2409.07901 [pdf, html, other]

Title: Bridging Discrete and Continuous: A Multimodal Strategy for Complex Emotion Detection

Jiehui Jia, Huan Zhang, Jinhua Liang

Subjects: Multimedia (cs.MM)

[12] arXiv:2409.08489 [pdf, html, other]

Title: Confidence Calibration for Audio Captioning Models

Rehana Mahfuz, Yinyi Guo, Erik Visser

Subjects: Multimedia (cs.MM); Sound (cs.SD); Audio and Speech Processing (eess.AS)

[13] arXiv:2409.08772 [pdf, html, other]

Title: On the Computation of BD-Rate over a Set of Videos for Fair Assessment of Performance of Learned Video Codecs

M.Akin Yilmaz, Onur Keleş, A.Murat Tekalp

Comments: Submitted to IEEE ICASSP 2025

Subjects: Multimedia (cs.MM); Computer Vision and Pattern Recognition (cs.CV); Image and Video Processing (eess.IV)

[14] arXiv:2409.09427 [pdf, html, other]

Title: Prototypical Prompting for Text-to-image Person Re-identification

Shuanglin Yan, Jun Liu, Neng Dong, Liyan Zhang, Jinhui Tang

Comments: Accepted by ACM MM2024

Subjects: Multimedia (cs.MM)

[15] arXiv:2409.09638 [pdf, html, other]

Title: Multi-view Hypergraph-based Contrastive Learning Model for Cold-Start Micro-video Recommendation

Sisuo Lyu, Xiuze Zhou, Xuming Hu

Subjects: Multimedia (cs.MM)

[16] arXiv:2409.10281 [pdf, html, other]

Title: DreamHead: Learning Spatial-Temporal Correspondence via Hierarchical Diffusion for Audio-driven Talking Head Synthesis

Fa-Ting Hong, Yunfei Liu, Yu Li, Changyin Zhou, Fei Yu, Dan Xu

Subjects: Multimedia (cs.MM); Artificial Intelligence (cs.AI); Sound (cs.SD); Audio and Speech Processing (eess.AS)

[17] arXiv:2409.10525 [pdf, html, other]

Title: "Is This It?": Towards Ecologically Valid Benchmarks for Situated Collaboration

Dan Bohus, Sean Andrist, Yuwei Bao, Eric Horvitz, Ann Paradiso

Subjects: Multimedia (cs.MM); Artificial Intelligence (cs.AI); Computation and Language (cs.CL)

[18] arXiv:2409.10958 [pdf, html, other]

Title: Towards Effective User Attribution for Latent Diffusion Models via Watermark-Informed Blending

Yongyang Pan, Xiaohong Liu, Siqi Luo, Yi Xin, Xiao Guo, Xiaoming Liu, Xiongkuo Min, Guangtao Zhai

Comments: 9 pages, 7 figures

Subjects: Multimedia (cs.MM); Cryptography and Security (cs.CR); Computer Vision and Pattern Recognition (cs.CV); Image and Video Processing (eess.IV)

[19] arXiv:2409.11286 [pdf, html, other]

Title: Enhancing Few-Shot Classification without Forgetting through Multi-Level Contrastive Constraints

Bingzhi Chen, Haoming Zhou, Yishu Liu, Biqing Zeng, Jiahui Pan, Guangming Lu

Subjects: Multimedia (cs.MM)

[20] arXiv:2409.11729 [pdf, html, other]

Title: DETECLAP: Enhancing Audio-Visual Representation Learning with Object Information

Shota Nakada, Taichi Nishimura, Hokuto Munakata, Masayoshi Kondo, Tatsuya Komatsu

Comments: under review

Subjects: Multimedia (cs.MM); Computer Vision and Pattern Recognition (cs.CV); Sound (cs.SD); Audio and Speech Processing (eess.AS)

[21] arXiv:2409.14829 [pdf, html, other]

Title: RoWSFormer: A Robust Watermarking Framework with Swin Transformer for Enhanced Geometric Attack Resilience

Weitong Chen, Yuheng Li

Subjects: Multimedia (cs.MM); Computer Vision and Pattern Recognition (cs.CV); Image and Video Processing (eess.IV)

[22] arXiv:2409.16297 [pdf, html, other]

Title: Analyzing Recursiveness in Multimodal Generative Artificial Intelligence: Stability or Divergence?

Javier Conde, Tobias Cheung, Gonzalo Martínez, Pedro Reviriego, Rik Sarkar

Subjects: Multimedia (cs.MM)

[23] arXiv:2409.16404 [pdf, html, other]

Title: FastTalker: Jointly Generating Speech and Conversational Gestures from Text

Zixin Guo, Jian Zhang

Comments: European Conference on Computer Vision Workshop

Subjects: Multimedia (cs.MM); Sound (cs.SD); Audio and Speech Processing (eess.AS)

[24] arXiv:2409.17104 [pdf, html, other]

Title: Language-oriented Semantic Communication for Image Transmission with Fine-Tuned Diffusion Model

Xinfeng Wei, Haonan Tong, Nuocheng Yang, Changchuan Yin

Comments: 6 pages, 9 figures, accepted by Wireless Communications and Signal Processing (WCSP) 2024

Subjects: Multimedia (cs.MM)

[25] arXiv:2409.17596 [pdf, html, other]

Title: Subjective and Objective Quality-of-Experience Evaluation Study for Live Video Streaming

Zehao Zhu, Wei Sun, Jun Jia, Wei Wu, Sibin Deng, Kai Li, Ying Chen, Xiongkuo Min, Jia Wang, Guangtao Zhai

Comments: 14 pages, 5 figures

Subjects: Multimedia (cs.MM); Artificial Intelligence (cs.AI); Image and Video Processing (eess.IV)

[26] arXiv:2409.17678 [pdf, html, other]

Title: Modeling the Popularity of Events on Web by Sparsity and Mutual-Excitation Guided Graph Neural Network

Jiaxin Deng, Linlin Jia, Junbiao Pang, Qingming Huang

Subjects: Multimedia (cs.MM)

[27] arXiv:2409.18971 [pdf, html, other]

Title: Early Joint Learning of Emotion Information Makes MultiModal Model Understand You Better

Mengying Ge, Mingyang Li, Dongkai Tang, Pengbo Li, Kuo Liu, Shuhao Deng, Songbai Pu, Long Liu, Yang Song, Tao Zhang

Subjects: Multimedia (cs.MM); Artificial Intelligence (cs.AI); Sound (cs.SD); Audio and Speech Processing (eess.AS)

[28] arXiv:2409.19132 [pdf, html, other]

Title: From Vision to Audio and Beyond: A Unified Model for Audio-Visual Representation and Generation

Kun Su, Xiulong Liu, Eli Shlizerman

Comments: Accepted by ICML 2024

Subjects: Multimedia (cs.MM); Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG); Sound (cs.SD); Audio and Speech Processing (eess.AS)

[29] arXiv:2409.19506 [pdf, html, other]

Title: IWN: Image Watermarking Based on Idempotency

Kaixin Deng

Subjects: Multimedia (cs.MM); Computer Vision and Pattern Recognition (cs.CV)

[30] arXiv:2409.19627 [pdf, html, other]

Title: IDEAW: Robust Neural Audio Watermarking with Invertible Dual-Embedding

Pengcheng Li, Xulong Zhang, Jing Xiao, Jianzong Wang

Comments: Accepted by the 2024 Conference on Empirical Methods in Natural Language Processing (EMNLP 2024)

Subjects: Multimedia (cs.MM); Cryptography and Security (cs.CR); Sound (cs.SD); Audio and Speech Processing (eess.AS)

[31] arXiv:2409.20142 [pdf, html, other]

Title: Signal Processing for Haptic Surface Modeling: a Review

Antonio Luigi Stefani, Niccolò Bisagno, Andrea Rosani, Nicola Conci, Francesco De Natale

Comments: 19 pages, 6 figures

Subjects: Multimedia (cs.MM)

[32] arXiv:2409.00486 (cross-list from cs.CV) [pdf, html, other]

Title: Multi-scale Multi-instance Visual Sound Localization and Segmentation

Shentong Mo, Haofan Wang

Subjects: Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG); Multimedia (cs.MM); Sound (cs.SD); Audio and Speech Processing (eess.AS)

[33] arXiv:2409.00552 (cross-list from eess.AS) [pdf, html, other]

Title: Digit Recognition using Multimodal Spiking Neural Networks

William Bjorndahl, Jack Easton, Austin Modoff, Eric C. Larson, Joseph Camp, Prasanna Rangarajan

Comments: 4 pages, 2 figures, submitted to 2025 IEEE International Conference on Acoustics, Speech, and Signal Processing

Subjects: Audio and Speech Processing (eess.AS); Computer Vision and Pattern Recognition (cs.CV); Multimedia (cs.MM); Sound (cs.SD)

[34] arXiv:2409.00562 (cross-list from eess.AS) [pdf, html, other]

Title: Comparative Analysis of Modality Fusion Approaches for Audio-Visual Person Identification and Verification

Aref Farhadipour, Masoumeh Chapariniya, Teodora Vukovic, Volker Dellwo

Comments: This paper was accepted at the ICNLSP2024 conference

Subjects: Audio and Speech Processing (eess.AS); Computer Vision and Pattern Recognition (cs.CV); Multimedia (cs.MM); Sound (cs.SD)

[35] arXiv:2409.00615 (cross-list from cs.HC) [pdf, html, other]

Title: MetaDigiHuman: Haptic Interfaces for Digital Humans in Metaverse

Senthil Kumar Jagatheesaperumal, Praveen Sathikumar, Harikrishnan Rajan

Comments: 5 pages, 2 figures

Subjects: Human-Computer Interaction (cs.HC); Multimedia (cs.MM)

[36] arXiv:2409.00971 (cross-list from cs.CV) [pdf, html, other]

Title: Interpretable Convolutional SyncNet

Sungjoon Park, Jaesub Yun, Donggeon Lee, Minsik Park

Comments: 8+5 pages

Subjects: Computer Vision and Pattern Recognition (cs.CV); Multimedia (cs.MM); Sound (cs.SD); Audio and Speech Processing (eess.AS)

[37] arXiv:2409.01352 (cross-list from cs.SD) [pdf, html, other]

Title: Spectron: Target Speaker Extraction using Conditional Transformer with Adversarial Refinement

Tathagata Bandyopadhyay

Subjects: Sound (cs.SD); Machine Learning (cs.LG); Multimedia (cs.MM); Audio and Speech Processing (eess.AS)

[38] arXiv:2409.01534 (cross-list from cs.CV) [pdf, html, other]

Title: Think Twice Before Recognizing: Large Multimodal Models for General Fine-grained Traffic Sign Recognition

Yaozong Gan, Guang Li, Ren Togo, Keisuke Maeda, Takahiro Ogawa, Miki Haseyama

Subjects: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Multimedia (cs.MM)

[39] arXiv:2409.01761 (cross-list from cs.CV) [pdf, html, other]

Title: PRoGS: Progressive Rendering of Gaussian Splats

Brent Zoomers, Maarten Wijnants, Ivan Molenaers, Joni Vanherck, Jeroen Put, Lode Jorissen, Nick Michiels

Subjects: Computer Vision and Pattern Recognition (cs.CV); Multimedia (cs.MM)

[40] arXiv:2409.02049 (cross-list from cs.CV) [pdf, html, other]

Title: Low-Resolution Face Recognition via Adaptable Instance-Relation Distillation

Ruixin Shi, Weijia Guo, Shiming Ge

Comments: Accepted by IJCNN 2024

Subjects: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Machine Learning (cs.LG); Multimedia (cs.MM)

[41] arXiv:2409.02101 (cross-list from cs.CV) [pdf, html, other]

Title: Towards Real-World Adverse Weather Image Restoration: Enhancing Clearness and Semantics with Vision-Language Models

Jiaqi Xu, Mengyang Wu, Xiaowei Hu, Chi-Wing Fu, Qi Dou, Pheng-Ann Heng

Comments: Accepted by ECCV 2024

Subjects: Computer Vision and Pattern Recognition (cs.CV); Multimedia (cs.MM)

[42] arXiv:2409.02108 (cross-list from cs.CV) [pdf, html, other]

Title: Unveiling Deep Shadows: A Survey on Image and Video Shadow Detection, Removal, and Generation in the Era of Deep Learning

Xiaowei Hu, Zhenghao Xing, Tianyu Wang, Chi-Wing Fu, Pheng-Ann Heng

Comments: Publicly available results, trained models, and evaluation metrics at this https URL

Subjects: Computer Vision and Pattern Recognition (cs.CV); Graphics (cs.GR); Multimedia (cs.MM)

[43] arXiv:2409.02266 (cross-list from cs.SD) [pdf, html, other]

Title: LSTMSE-Net: Long Short Term Speech Enhancement Network for Audio-visual Speech Enhancement

Arnav Jain, Jasmer Singh Sanjotra, Harshvardhan Choudhary, Krish Agrawal, Rupal Shah, Rohan Jha, M. Sajid, Amir Hussain, M. Tanveer

Journal-ref: INTERSPEECH 2024

Subjects: Sound (cs.SD); Machine Learning (cs.LG); Multimedia (cs.MM); Audio and Speech Processing (eess.AS)

[44] arXiv:2409.02376 (cross-list from cs.CV) [pdf, other]

Title: Coral Model Generation from Single Images for Virtual Reality Applications

Jie Fu (University of the Arts London, Creative Computing Institute, London, United Kingdom), Shun Fu (Bloks Technology Company, Shanghai, China), Mick Grierson (University of the Arts London, Creative Computing Institute, London, United Kingdom)

Comments: In Proceedings of Explainable AI for the Arts Workshop 2024 (XAIxArts 2024) arXiv:2406.14485

Subjects: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Graphics (cs.GR); Human-Computer Interaction (cs.HC); Multimedia (cs.MM)

[45] arXiv:2409.02453 (cross-list from eess.IV) [pdf, html, other]

Title: FrameCorr: Adaptive, Autoencoder-based Neural Compression for Video Reconstruction in Resource and Timing Constrained Network Settings

John Li, Shehab Sarar Ahmed, Deepak Nair

Subjects: Image and Video Processing (eess.IV); Computer Vision and Pattern Recognition (cs.CV); Emerging Technologies (cs.ET); Multimedia (cs.MM)

[46] arXiv:2409.02555 (cross-list from cs.CV) [pdf, html, other]

Title: Low-Resolution Object Recognition with Cross-Resolution Relational Contrastive Distillation

Kangkai Zhang, Shiming Ge, Ruixin Shi, Dan Zeng

Comments: This paper is accepted by IEEE Transactions on Circuits and Systems for Video Technology (TCSVT)

Subjects: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Machine Learning (cs.LG); Multimedia (cs.MM)

[47] arXiv:2409.02657 (cross-list from cs.CV) [pdf, html, other]

Title: PoseTalk: Text-and-Audio-based Pose Control and Motion Refinement for One-Shot Talking Head Generation

Jun Ling, Yiwen Wang, Han Xue, Rong Xie, Li Song

Comments: 7+5 pages, 15 figures

Subjects: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Multimedia (cs.MM)

[48] arXiv:2409.02828 (cross-list from cs.CV) [pdf, html, other]

Title: ExpLLM: Towards Chain of Thought for Facial Expression Recognition

Xing Lan, Jian Xue, Ji Qi, Dongmei Jiang, Ke Lu, Tat-Seng Chua

Comments: project page: this https URL

Subjects: Computer Vision and Pattern Recognition (cs.CV); Multimedia (cs.MM)

[49] arXiv:2409.02845 (cross-list from cs.SD) [pdf, html, other]

Title: Multi-Track MusicLDM: Towards Versatile Music Generation with Latent Diffusion Model

Tornike Karchkhadze, Mohammad Rasool Izadi, Ke Chen, Gerard Assayag, Shlomo Dubnov

Subjects: Sound (cs.SD); Multimedia (cs.MM); Audio and Speech Processing (eess.AS)

[50] arXiv:2409.02889 (cross-list from cs.CL) [pdf, html, other]

Title: LongLLaVA: Scaling Multi-modal LLMs to 1000 Images Efficiently via a Hybrid Architecture

Xidong Wang, Dingjie Song, Shunian Chen, Chen Zhang, Benyou Wang

Comments: 20 pages, 9 figures, 9 tables

Subjects: Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV); Multimedia (cs.MM)