Multimedia

Authors and titles for January 2024

[1] arXiv:2401.03195 [pdf, other]

Title: Efficient Bitrate Ladder Construction using Transfer Learning and Spatio-Temporal Features

Ali Falahati, Mohammad Karim Safavi, Ardavan Elahi, Farhad Pakdaman, Moncef Gabbouj

Comments: 7 pages, 9 figures, 7 tables, Copyright 2024 IEEE - Presented in IEEE MVIP 2024

Journal-ref: Proc. 2024 13th Iranian/3rd Int. Conf. Mach. Vis. Image Process. (MVIP) (2024) 1-7

Subjects: Multimedia (cs.MM); Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG)

[2] arXiv:2401.03476 [pdf, html, other]

Title: Freetalker: Controllable Speech and Text-Driven Gesture Generation Based on Diffusion Models for Enhanced Speaker Naturalness

Sicheng Yang, Zunnan Xu, Haiwei Xue, Yongkang Cheng, Shaoli Huang, Mingming Gong, Zhiyong Wu

Comments: 6 pages, 3 figures, ICASSP 2024

Subjects: Multimedia (cs.MM); Artificial Intelligence (cs.AI); Human-Computer Interaction (cs.HC); Sound (cs.SD); Audio and Speech Processing (eess.AS)

[3] arXiv:2401.04039 [pdf, other]

Title: Bj{\o}ntegaard Delta (BD): A Tutorial Overview of the Metric, Evolution, Challenges, and Recommendations

Nabajeet Barman, Maria G. Martini, Yuriy Reznik

Subjects: Multimedia (cs.MM); Information Theory (cs.IT); Image and Video Processing (eess.IV)

[4] arXiv:2401.04394 [pdf, html, other]

Title: SonicVisionLM: Playing Sound with Vision Language Models

Zhifeng Xie, Shengye Yu, Qile He, Mengtian Li

Comments: CVPR 2024

Subjects: Multimedia (cs.MM); Sound (cs.SD); Audio and Speech Processing (eess.AS)

[5] arXiv:2401.04405 [pdf, html, other]

Title: Optimal Transcoding Resolution Prediction for Efficient Per-Title Bitrate Ladder Estimation

Jinhai Yang, Mengxi Guo, Shijie Zhao, Junlin Li, Li Zhang

Comments: Accepted by the 2024 Data Compression Conference (DCC) for presentation as a poster. This is the full paper

Subjects: Multimedia (cs.MM); Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV); Image and Video Processing (eess.IV)

[6] arXiv:2401.04935 [pdf, html, other]

Title: Learning Audio Concepts from Counterfactual Natural Language

Ali Vosoughi, Luca Bondi, Ho-Hsiang Wu, Chenliang Xu

Comments: Accepted at ICASSP 2024

Subjects: Multimedia (cs.MM); Computation and Language (cs.CL); Sound (cs.SD); Audio and Speech Processing (eess.AS)

[7] arXiv:2401.05746 [pdf, html, other]

Title: Cross-Modality and Within-Modality Regularization for Audio-Visual DeepFake Detection

Heqing Zou, Meng Shen, Yuchen Hu, Chen Chen, Eng Siong Chng, Deepu Rajan

Comments: Accepted by ICASSP 2024

Subjects: Multimedia (cs.MM)

[8] arXiv:2401.05995 [pdf, html, other]

Title: A Multi-Embedding Convergence Network on Siamese Architecture for Fake Reviews

Sankarshan Dasgupta, James Buckley

Comments: 8 pages, 4 figures, 1 table

Subjects: Multimedia (cs.MM)

[9] arXiv:2401.06386 [pdf, html, other]

Title: Generative AI-enabled Mobile Tactical Multimedia Networks: Distribution, Generation, and Perception

Minrui Xu, Dusit Niyato, Jiawen Kang, Zehui Xiong, Song Guo, Yuguang Fang, Dong In Kim

Subjects: Multimedia (cs.MM)

[10] arXiv:2401.07084 [pdf, html, other]

Title: ScripTONES: Sentiment-Conditioned Music Generation for Movie Scripts

Vishruth Veerendranath, Vibha Masti, Utkarsh Gupta, Hrishit Chaudhuri, Gowri Srinivasa

Comments: Presented at NeurIPS 2023 - ML For Audio workshop. To appear in proceedings of AIML Systems 2023 - Generative AI

Subjects: Multimedia (cs.MM); Sound (cs.SD); Audio and Speech Processing (eess.AS)

[11] arXiv:2401.07411 [pdf, html, other]

Title: Startup Delay Aware Short Video Ordering: Problem, Model, and A Reinforcement Learning based Algorithm

Zhipeng Gao, Chunxi Li, Yongxiang Zhao, Baoxian Zhang

Subjects: Multimedia (cs.MM)

[12] arXiv:2401.08449 [pdf, other]

Title: CLIPRerank: An Extremely Simple Method for Improving Ad-hoc Video Search

Aozhu Chen, Fangming Zhou, Ziyuan Wang, Xirong Li

Comments: Accepted by ICASSP 2024

Subjects: Multimedia (cs.MM)

[13] arXiv:2401.09774 [pdf, html, other]

Title: On the Audio Hallucinations in Large Audio-Video Language Models

Taichi Nishimura, Shota Nakada, Masayoshi Kondo

Comments: 6 pages

Subjects: Multimedia (cs.MM); Computation and Language (cs.CL); Computer Vision and Pattern Recognition (cs.CV); Sound (cs.SD); Audio and Speech Processing (eess.AS)

[14] arXiv:2401.09854 [pdf, other]

Title: A Survey on Energy Consumption and Environmental Impact of Video Streaming

Samira Afzal, Narges Mehran, Zoha Azimi Ourimi, Farzad Tashtarian, Hadi Amirpour, Radu Prodan, Christian Timmerer

Subjects: Multimedia (cs.MM)

[15] arXiv:2401.11764 [pdf, html, other]

Title: Identity-Driven Multimedia Forgery Detection via Reference Assistance

Junhao Xu, Jingjing Chen, Xue Song, Feng Han, Haijun Shan, Yugang Jiang

Subjects: Multimedia (cs.MM)

[16] arXiv:2401.11818 [pdf, html, other]

Title: MInD: Improving Multimodal Sentiment Analysis via Multimodal Information Disentanglement

Weichen Dai, Xingyu Li, Zeyu Wang, Pengbo Hu, Ji Qi, Jianlin Peng, Yi Zhou

Subjects: Multimedia (cs.MM)

[17] arXiv:2401.13967 [pdf, html, other]

Title: Perceptual-oriented Learned Image Compression with Dynamic Kernel

Nianxiang Fu, Junxi Zhang, Huairui Wang, Zhenzhong Chen

Subjects: Multimedia (cs.MM)

[18] arXiv:2401.15343 [pdf, other]

Title: Optimal Quality and Efficiency in Adaptive Live Streaming with JND-Aware Low latency Encoding

Vignesh V Menon, Jingwen Zhu, Prajit T Rajendran, Samira Afzal, Klaus Schoeffmann, Patrick Le Callet, Christian Timmerer

Comments: 2024 Mile High Video (MHV)

Subjects: Multimedia (cs.MM)

[19] arXiv:2401.15346 [pdf, other]

Title: Energy-efficient Adaptive Video Streaming with Latency-Aware Dynamic Resolution Encoding

Vignesh V Menon, Amritha Premkumar, Prajit T Rajendran, Adam Wieckowski, Benjamin Bross, Christian Timmerer, Detlev Marpe

Comments: 2024 Mile High Video (MHV)

Subjects: Multimedia (cs.MM)

[20] arXiv:2401.16825 [pdf, html, other]

Title: Smart Fitting Room: A One-stop Framework for Matching-aware Virtual Try-on

Mingzhe Yu, Yunshan Ma, Lei Wu, Kai Cheng, Xue Li, Lei Meng, Tat-Seng Chua

Subjects: Multimedia (cs.MM)

[21] arXiv:2401.16830 [pdf, other]

Title: LATENTPATCH: A Non-Parametric Approach for Face Generation and Editing

Benjamin Samuth (UNICAEN, ENSICAEN), Julien Rabin (ENSICAEN, UNICAEN), David Tschumperlé (ENSICAEN, UNICAEN), Frédéric Jurie (ENSICAEN, UNICAEN)

Journal-ref: 2023 IEEE International Conference on Image Processing (ICIP), Oct 2023, Kuala Lumpur, Malaysia. pp.1790-1794

Subjects: Multimedia (cs.MM); Image and Video Processing (eess.IV); Signal Processing (eess.SP)

[22] arXiv:2401.17151 [pdf, other]

Title: An Open Software Suite for Event-Based Video

Andrew C. Freeman

Subjects: Multimedia (cs.MM); Computer Vision and Pattern Recognition (cs.CV)

[23] arXiv:2401.00393 (cross-list from cs.CV) [pdf, other]

Title: Generative Model-Driven Synthetic Training Image Generation: An Approach to Cognition in Rail Defect Detection

Rahatara Ferdousi, Chunsheng Yang, M. Anwar Hossain, Fedwa Laamarti, M. Shamim Hossain, Abdulmotaleb El Saddik

Comments: 26 pages, 13 figures, Springer Journal

Subjects: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Machine Learning (cs.LG); Multimedia (cs.MM); Image and Video Processing (eess.IV)

[24] arXiv:2401.00403 (cross-list from cs.LG) [pdf, html, other]

Title: Overcome Modal Bias in Multi-modal Federated Learning via Balanced Modality Selection

Yunfeng Fan, Wenchao Xu, Haozhao Wang, Fushuo Huo, Jinyu Chen, Song Guo

Comments: Accepted by ECCV24, 23 pages

Subjects: Machine Learning (cs.LG); Computer Vision and Pattern Recognition (cs.CV); Multimedia (cs.MM)

[25] arXiv:2401.00416 (cross-list from cs.CV) [pdf, html, other]

Title: SVFAP: Self-supervised Video Facial Affect Perceiver

Licai Sun, Zheng Lian, Kexin Wang, Yu He, Mingyu Xu, Haiyang Sun, Bin Liu, Jianhua Tao

Comments: Published in: IEEE Transactions on Affective Computing (Early Access). The code and models are available at this https URL

Journal-ref: IEEE Transactions on Affective Computing, 2024

Subjects: Computer Vision and Pattern Recognition (cs.CV); Human-Computer Interaction (cs.HC); Multimedia (cs.MM)

[26] arXiv:2401.00658 (cross-list from cs.IT) [pdf, html, other]

Title: Point Cloud in the Air

Yulin Shao, Chenghong Bian, Li Yang, Qianqian Yang, Zhaoyang Zhang, Deniz Gunduz

Subjects: Information Theory (cs.IT); Machine Learning (cs.LG); Multimedia (cs.MM); Signal Processing (eess.SP)

[27] arXiv:2401.00736 (cross-list from cs.CV) [pdf, html, other]

Title: Diffusion Models, Image Super-Resolution And Everything: A Survey

Brian B. Moser, Arundhati S. Shanbhag, Federico Raue, Stanislav Frolov, Sebastian Palacio, Andreas Dengel

Subjects: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Machine Learning (cs.LG); Multimedia (cs.MM)

[28] arXiv:2401.00763 (cross-list from cs.SE) [pdf, html, other]

Title: New Job, New Gender? Measuring the Social Bias in Image Generation Models

Wenxuan Wang, Haonan Bai, Jen-tse Huang, Yuxuan Wan, Youliang Yuan, Haoyi Qiu, Nanyun Peng, Michael R. Lyu

Comments: ACM MM 2024 Oral

Subjects: Software Engineering (cs.SE); Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Computer Vision and Pattern Recognition (cs.CV); Multimedia (cs.MM)

[29] arXiv:2401.00894 (cross-list from cs.LG) [pdf, html, other]

Title: Balanced Multi-modal Federated Learning via Cross-Modal Infiltration

Yunfeng Fan, Wenchao Xu, Haozhao Wang, Jiaqi Zhu, Song Guo

Comments: 10 pages, 5 figures 4 tables

Subjects: Machine Learning (cs.LG); Computer Vision and Pattern Recognition (cs.CV); Multimedia (cs.MM)

[30] arXiv:2401.01163 (cross-list from cs.CV) [pdf, html, other]

Title: NU-Class Net: A Novel Approach for Video Quality Enhancement

Parham Zilouchian Moghaddam, Mehdi Modarressi, Mohammad Amin Sadeghi

Subjects: Computer Vision and Pattern Recognition (cs.CV); Multimedia (cs.MM)

[31] arXiv:2401.01470 (cross-list from cs.CV) [pdf, other]

Title: TPC-ViT: Token Propagation Controller for Efficient Vision Transformer

Wentao Zhu

Comments: Accepted by the main conference of WACV 2024; well-formatted PDF is in this https URL ; supplementary is in this https URL

Subjects: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Machine Learning (cs.LG); Multimedia (cs.MM); Neural and Evolutionary Computing (cs.NE)

[32] arXiv:2401.01699 (cross-list from cs.CV) [pdf, html, other]

Title: WordArt Designer API: User-Driven Artistic Typography Synthesis with Large Language Models on ModelScope

Jun-Yan He, Zhi-Qi Cheng, Chenyang Li, Jingdong Sun, Wangmeng Xiang, Yusen Hu, Xianhui Lin, Xiaoyang Kang, Zengke Jin, Bin Luo, Yifeng Geng, Xuansong Xie, Jingren Zhou

Comments: Spotlight Paper at the Workshop on Machine Learning for Creativity and Design, 37th Conference on Neural Information Processing Systems (NeurIPS 2023). 5 pages, 5 figures

Subjects: Computer Vision and Pattern Recognition (cs.CV); Computation and Language (cs.CL); Multimedia (cs.MM)

[33] arXiv:2401.01759 (cross-list from cs.SI) [pdf, html, other]

Title: VGA: Vision and Graph Fused Attention Network for Rumor Detection

Lin Bai, Caiyan Jia, Ziying Song, Chaoqun Cui

Subjects: Social and Information Networks (cs.SI); Computation and Language (cs.CL); Computer Vision and Pattern Recognition (cs.CV); Multimedia (cs.MM)

[34] arXiv:2401.01955 (cross-list from cs.HC) [pdf, html, other]

Title: MULTI-CASE: A Transformer-based Ethics-aware Multimodal Investigative Intelligence Framework

Maximilian T. Fischer, Yannick Metz, Lucas Joos, Matthias Miller, Daniel A. Keim

Comments: 6 pages, 3 figures, 1 table

Subjects: Human-Computer Interaction (cs.HC); Multimedia (cs.MM)

[35] arXiv:2401.02309 (cross-list from cs.CV) [pdf, html, other]

Title: TR-DETR: Task-Reciprocal Transformer for Joint Moment Retrieval and Highlight Detection

Hao Sun, Mingyao Zhou, Wenjing Chen, Wei Xie

Comments: Accepted by AAAI-24

Subjects: Computer Vision and Pattern Recognition (cs.CV); Multimedia (cs.MM)

[36] arXiv:2401.02566 (cross-list from cs.SD) [pdf, other]

Title: Siamese Residual Neural Network for Musical Shape Evaluation in Piano Performance Assessment

Xiaoquan Li, Stephan Weiss, Yijun Yan, Yinhe Li, Jinchang Ren, John Soraghan, Ming Gong

Comments: this http URL, this http URL, this http URL, this http URL, this http URL, this http URL, this http URL,"Siamese residual neural network for musical shape evaluation in piano performance assessment" in Proc. of the 31st European Signal Processing Conference, Helsinki, Finland

Subjects: Sound (cs.SD); Machine Learning (cs.LG); Multimedia (cs.MM); Audio and Speech Processing (eess.AS)

[37] arXiv:2401.02614 (cross-list from cs.CV) [pdf, html, other]

Title: Scaling and Masking: A New Paradigm of Data Sampling for Image and Video Quality Assessment

Yongxu Liu, Yinghui Quan, Guoyao Xiao, Aobo Li, Jinjian Wu

Comments: Accepted by AAAI2024. Code has been released at this https URL

Subjects: Computer Vision and Pattern Recognition (cs.CV); Multimedia (cs.MM)

[38] arXiv:2401.02678 (cross-list from cs.SD) [pdf, html, other]

Title: MusicAOG: an Energy-Based Model for Learning and Sampling a Hierarchical Representation of Symbolic Music

Yikai Qian, Tianle Wang, Xinyi Tong, Xin Jin, Duo Xu, Bo Zheng, Tiezheng Ge, Feng Yu, Song-Chun Zhu

Subjects: Sound (cs.SD); Multimedia (cs.MM); Audio and Speech Processing (eess.AS)

[39] arXiv:2401.02838 (cross-list from cs.CV) [pdf, other]

Title: CrisisViT: A Robust Vision Transformer for Crisis Image Classification

Zijun Long, Richard McCreadie, Muhammad Imran

Journal-ref: Proceedings of the 20th International ISCRAM Conference 2023, pp. 309--319

Subjects: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Multimedia (cs.MM); Social and Information Networks (cs.SI)

[40] arXiv:2401.03105 (cross-list from cs.CV) [pdf, html, other]

Title: Incorporating Visual Experts to Resolve the Information Loss in Multimodal Large Language Models

Xin He, Longhui Wei, Lingxi Xie, Qi Tian

Subjects: Computer Vision and Pattern Recognition (cs.CV); Multimedia (cs.MM)

[41] arXiv:2401.03115 (cross-list from cs.CV) [pdf, html, other]

Title: Transferable Learned Image Compression-Resistant Adversarial Perturbations

Yang Sui, Zhuohang Li, Ding Ding, Xiang Pan, Xiaozhong Xu, Shan Liu, Zhenzhong Chen

Comments: Accepted by BMVC 2024

Subjects: Computer Vision and Pattern Recognition (cs.CV); Multimedia (cs.MM); Image and Video Processing (eess.IV)

[42] arXiv:2401.03201 (cross-list from cs.CV) [pdf, other]

Title: 3DMIT: 3D Multi-modal Instruction Tuning for Scene Understanding

Zeju Li, Chao Zhang, Xiaoyan Wang, Ruilong Ren, Yifan Xu, Ruifei Ma, Xiangde Liu

Comments: 9 pages, 5 figures

Subjects: Computer Vision and Pattern Recognition (cs.CV); Multimedia (cs.MM)

[43] arXiv:2401.03499 (cross-list from cs.CV) [pdf, html, other]

Title: Re:Draw -- Context Aware Translation as a Controllable Method for Artistic Production

Joao Liborio Cardoso, Francesco Banterle, Paolo Cignoni, Michael Wimmer

Subjects: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Graphics (cs.GR); Multimedia (cs.MM)

[44] arXiv:2401.03890 (cross-list from cs.CV) [pdf, html, other]

Title: A Survey on 3D Gaussian Splatting

Guikun Chen, Wenguan Wang

Comments: Ongoing project

Subjects: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Graphics (cs.GR); Multimedia (cs.MM)

[45] arXiv:2401.04023 (cross-list from cs.CV) [pdf, other]

Title: Efficient Multiscale Multimodal Bottleneck Transformer for Audio-Video Classification

Wentao Zhu

Comments: Accepted by WACV 2024; well-formatted PDF is in this https URL

Subjects: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Machine Learning (cs.LG); Multimedia (cs.MM); Sound (cs.SD); Audio and Speech Processing (eess.AS)

[46] arXiv:2401.04154 (cross-list from cs.CV) [pdf, other]

Title: Efficient Selective Audio Masked Multimodal Bottleneck Transformer for Audio-Video Classification

Wentao Zhu

Comments: Accepted by WACV 2024; well-formatted PDF is in this https URL. arXiv admin note: text overlap with arXiv:2401.04023

Subjects: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Machine Learning (cs.LG); Multimedia (cs.MM); Sound (cs.SD); Audio and Speech Processing (eess.AS)

[47] arXiv:2401.04210 (cross-list from cs.CV) [pdf, html, other]

Title: FunnyNet-W: Multimodal Learning of Funny Moments in Videos in the Wild

Zhi-Song Liu, Robin Courant, Vicky Kalogeiton

Comments: 22 pages, 14 figures

Subjects: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Multimedia (cs.MM); Sound (cs.SD); Audio and Speech Processing (eess.AS)

[48] arXiv:2401.05614 (cross-list from cs.SD) [pdf, other]

Title: Self-Attention and Hybrid Features for Replay and Deep-Fake Audio Detection

Lian Huang, Chi-Man Pun

Subjects: Sound (cs.SD); Multimedia (cs.MM); Audio and Speech Processing (eess.AS)

[49] arXiv:2401.05698 (cross-list from cs.CV) [pdf, html, other]

Title: HiCMAE: Hierarchical Contrastive Masked Autoencoder for Self-Supervised Audio-Visual Emotion Recognition

Licai Sun, Zheng Lian, Bin Liu, Jianhua Tao

Comments: Accepted by Information Fusion. The code is available at this https URL

Journal-ref: Information Fusion, 2024

Subjects: Computer Vision and Pattern Recognition (cs.CV); Human-Computer Interaction (cs.HC); Multimedia (cs.MM); Sound (cs.SD); Audio and Speech Processing (eess.AS)

[50] arXiv:2401.06390 (cross-list from cs.SD) [pdf, html, other]

Title: LCB-net: Long-Context Biasing for Audio-Visual Speech Recognition

Fan Yu, Haoxu Wang, Xian Shi, Shiliang Zhang

Comments: Accepted by ICASPP 2024

Subjects: Sound (cs.SD); Multimedia (cs.MM); Audio and Speech Processing (eess.AS)