Information Theory

Authors and titles for October 2024

[201] arXiv:2410.01374 (cross-list from math.OC) [pdf, html, other]

Title: Newton Meets Marchenko-Pastur: Massively Parallel Second-Order Optimization with Hessian Sketching and Debiasing

Elad Romanov, Fangzhao Zhang, Mert Pilanci

Subjects: Optimization and Control (math.OC); Information Theory (cs.IT); Signal Processing (eess.SP); Machine Learning (stat.ML)

[202] arXiv:2410.01444 (cross-list from cs.CL) [pdf, html, other]

Title: Geometric Signatures of Compositionality Across a Language Model's Lifetime

Jin Hwa Lee, Thomas Jiralerspong, Lei Yu, Yoshua Bengio, Emily Cheng

Comments: Under review at ARR

Subjects: Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Information Theory (cs.IT); Machine Learning (cs.LG)

[203] arXiv:2410.01673 (cross-list from quant-ph) [pdf, html, other]

Title: MaxSAT decoders for arbitrary CSS codes

Mohammadreza Noormandipour, Tobias Haug

Comments: 11 pages including appendix, 7 figures

Subjects: Quantum Physics (quant-ph); Information Theory (cs.IT)

[204] arXiv:2410.02450 (cross-list from cs.LG) [pdf, html, other]

Title: Personalized Federated Learning for Generative AI-Assisted Semantic Communications

Yubo Peng, Feibo Jiang, Li Dong, Kezhi Wang, Kun Yang

Subjects: Machine Learning (cs.LG); Distributed, Parallel, and Cluster Computing (cs.DC); Information Theory (cs.IT)

[205] arXiv:2410.02575 (cross-list from cs.CR) [pdf, html, other]

Title: Assessing the Viability of Synthetic Physical Copy Detection Patterns on Different Imaging Systems

Roman Chaban, Brian Pulfer, Slava Voloshynovskiy

Subjects: Cryptography and Security (cs.CR); Information Theory (cs.IT)

[206] arXiv:2410.02583 (cross-list from quant-ph) [pdf, html, other]

Title: Sample-Optimal Quantum State Tomography for Structured Quantum States in One Dimension

Zhen Qin, Casey Jameson, Alireza Goldar, Michael B. Wakin, Zhexuan Gong, Zhihui Zhu

Subjects: Quantum Physics (quant-ph); Information Theory (cs.IT); Signal Processing (eess.SP); Optimization and Control (math.OC)

[207] arXiv:2410.02686 (cross-list from quant-ph) [pdf, html, other]

Title: Optimal continuity bound for the von Neumann entropy under energy constraints

S.Becker, N.Datta, M.G.Jabbour, M.E.Shirokov

Comments: 15 pages, any comments are still welcome

Subjects: Quantum Physics (quant-ph); Information Theory (cs.IT); Mathematical Physics (math-ph)

[208] arXiv:2410.02733 (cross-list from cs.LG) [pdf, html, other]

Title: Data Similarity-Based One-Shot Clustering for Multi-Task Hierarchical Federated Learning

Abdulmoneam Ali, Ahmed Arafa

Comments: To appear in Asilomar 2024

Subjects: Machine Learning (cs.LG); Information Theory (cs.IT); Networking and Internet Architecture (cs.NI); Signal Processing (eess.SP)

[209] arXiv:2410.02775 (cross-list from eess.SP) [pdf, html, other]

Title: A Deep Learning Approach for User-Centric Clustering in Cell-Free Massive MIMO Systems

Giovanni Di Gennaro, Amedeo Buonanno, Gianmarco Romano, Stefano Buzzi, Francesco A.N Palmieri

Comments: Accepted to 25th IEEE International Workshop on Signal Processing Advances in Wireless Communications (SPAWC), 2024

Subjects: Signal Processing (eess.SP); Information Theory (cs.IT); Machine Learning (cs.LG)

[210] arXiv:2410.02796 (cross-list from eess.SP) [pdf, html, other]

Title: Toward Adaptive Tracking and Communication via an Airborne Maneuverable Bi-Static ISAC System

Mingliang Wei, Ruoguang Li, Li Wang, Lianming Xu, Zhu Han

Subjects: Signal Processing (eess.SP); Emerging Technologies (cs.ET); Information Theory (cs.IT); Networking and Internet Architecture (cs.NI)

[211] arXiv:2410.02833 (cross-list from stat.ML) [pdf, other]

Title: Asymmetry of the Relative Entropy in the Regularization of Empirical Risk Minimization

Francisco Daunas, Iñaki Esnaola, Samir M. Perlaza, H. Vincent Poor

Subjects: Machine Learning (stat.ML); Information Theory (cs.IT); Machine Learning (cs.LG)

[212] arXiv:2410.02890 (cross-list from cs.CR) [pdf, html, other]

Title: Theoretically Grounded Framework for LLM Watermarking: A Distribution-Adaptive Approach

Haiyun He, Yepeng Liu, Ziqiao Wang, Yongyi Mao, Yuheng Bu

Subjects: Cryptography and Security (cs.CR); Information Theory (cs.IT); Machine Learning (cs.LG)

[213] arXiv:2410.02983 (cross-list from cs.RO) [pdf, html, other]

Title: Information-Driven Search and Track of Novel Space Objects

Trevor N. Wolf, Brandon A. Jones

Comments: Submitted to the Journal of Astronautical Sciences

Subjects: Robotics (cs.RO); Information Theory (cs.IT); Systems and Control (eess.SY)

[214] arXiv:2410.03395 (cross-list from physics.bio-ph) [pdf, html, other]

Title: Receptors cluster in high-curvature membrane regions for optimal spatial gradient sensing

Albert Alonso, Robert G. Endres, Julius B. Kirkegaard

Journal-ref: Phys. Rev. Lett. 134, 158401 (2025)

Subjects: Biological Physics (physics.bio-ph); Soft Condensed Matter (cond-mat.soft); Information Theory (cs.IT); Cell Behavior (q-bio.CB)

[215] arXiv:2410.03459 (cross-list from cs.SD) [pdf, html, other]

Title: Generative Semantic Communication for Text-to-Speech Synthesis

Jiahao Zheng, Jinke Ren, Peng Xu, Zhihao Yuan, Jie Xu, Fangxin Wang, Gui Gui, Shuguang Cui

Comments: The paper has been accepted by IEEE Globecom Workshop

Subjects: Sound (cs.SD); Information Theory (cs.IT); Machine Learning (cs.LG); Audio and Speech Processing (eess.AS)

[216] arXiv:2410.03562 (cross-list from quant-ph) [pdf, html, other]

Title: Class of codes correcting absorptions and emissions

Arda Aydin, Alexander Barg

Subjects: Quantum Physics (quant-ph); Information Theory (cs.IT)

[217] arXiv:2410.03674 (cross-list from cs.NI) [pdf, other]

Title: Trends, Advancements and Challenges in Intelligent Optimization in Satellite Communication

Philippe Krajsic, Viola Suess, Zehong Cao, Ryszard Kowalczyk, Bogdan Franczyk

Comments: 10 pages, 2 figures, 3 tables

Subjects: Networking and Internet Architecture (cs.NI); Information Theory (cs.IT); Machine Learning (cs.LG); Signal Processing (eess.SP)

[218] arXiv:2410.03972 (cross-list from cs.LG) [pdf, html, other]

Title: Measuring and Controlling Solution Degeneracy across Task-Trained Recurrent Neural Networks

Ann Huang, Satpreet H. Singh, Kanaka Rajan

Subjects: Machine Learning (cs.LG); Information Theory (cs.IT); Neural and Evolutionary Computing (cs.NE); Neurons and Cognition (q-bio.NC)

[219] arXiv:2410.05078 (cross-list from cs.LG) [pdf, html, other]

Title: Compression via Pre-trained Transformers: A Study on Byte-Level Multimodal Data

David Heurtel-Depeiges, Anian Ruoss, Joel Veness, Tim Genewein

Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Information Theory (cs.IT)

[220] arXiv:2410.05117 (cross-list from cs.LG) [pdf, html, other]

Title: Assouad, Fano, and Le Cam with Interaction: A Unifying Lower Bound Framework and Characterization for Bandit Learnability

Fan Chen, Dylan J. Foster, Yanjun Han, Jian Qian, Alexander Rakhlin, Yunbei Xu

Subjects: Machine Learning (cs.LG); Information Theory (cs.IT); Statistics Theory (math.ST); Machine Learning (stat.ML)

[221] arXiv:2410.05493 (cross-list from cs.LG) [pdf, html, other]

Title: Transformers learn variable-order Markov chains in-context

Ruida Zhou, Chao Tian, Suhas Diggavi

Subjects: Machine Learning (cs.LG); Information Theory (cs.IT)

[222] arXiv:2410.05646 (cross-list from cs.LG) [pdf, html, other]

Title: Score-Based Variational Inference for Inverse Problems

Zhipeng Xue, Penghao Cai, Xiaojun Yuan, Xiqi Gao

Comments: 10 pages, 7 figures, conference

Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Information Theory (cs.IT)

[223] arXiv:2410.05734 (cross-list from cs.LG) [pdf, html, other]

Title: Diminishing Exploration: A Minimalist Approach to Piecewise Stationary Multi-Armed Bandits

Kuan-Ta Li, Ping-Chun Hsieh, Yu-Chih Huang

Subjects: Machine Learning (cs.LG); Information Theory (cs.IT)

[224] arXiv:2410.05844 (cross-list from eess.SY) [pdf, html, other]

Title: Spectrally Efficient LDPC Codes For IRIG-106 Waveforms via Random Puncturing

Andrew D. Cummins, David G. M. Mitchell, Erik Perrins

Comments: Accepted for inclusion in the 2024 International Telemetry Conference

Subjects: Systems and Control (eess.SY); Information Theory (cs.IT); Signal Processing (eess.SP)

[225] arXiv:2410.05911 (cross-list from cs.LG) [pdf, html, other]

Title: Accelerating Error Correction Code Transformers

Matan Levy, Yoni Choukroun, Lior Wolf

Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Information Theory (cs.IT)

[226] arXiv:2410.06339 (cross-list from cs.LG) [pdf, html, other]

Title: Filtered Randomized Smoothing: A New Defense for Robust Modulation Classification

Wenhan Zhang, Meiyu Zhong, Ravi Tandon, Marwan Krunz

Comments: IEEE Milcom 2024

Subjects: Machine Learning (cs.LG); Cryptography and Security (cs.CR); Information Theory (cs.IT); Networking and Internet Architecture (cs.NI); Signal Processing (eess.SP)

[227] arXiv:2410.06378 (cross-list from stat.ML) [pdf, html, other]

Title: Covering Numbers for Deep ReLU Networks with Applications to Function Approximation and Nonparametric Regression

Weigutian Ou, Helmut Bölcskei

Subjects: Machine Learning (stat.ML); Artificial Intelligence (cs.AI); Information Theory (cs.IT); Machine Learning (cs.LG)

[228] arXiv:2410.06843 (cross-list from eess.SP) [pdf, html, other]

Title: Point-to-Point MIMO Channel Estimation by Exploiting Array Geometry and Clustered Multipath Propagation

Özlem Tuğfe Demir, Emil Björnson

Comments: 6 pages, 2 figures, accepted to be presented at IEEE GLOBECOM 2024

Subjects: Signal Processing (eess.SP); Information Theory (cs.IT)

[229] arXiv:2410.06884 (cross-list from cs.LG) [pdf, html, other]

Title: Adaptive Refinement Protocols for Distributed Distribution Estimation under $\ell^p$-Losses

Deheng Yuan, Tao Guo, Zhongyi Huang

Subjects: Machine Learning (cs.LG); Information Theory (cs.IT); Statistics Theory (math.ST)

[230] arXiv:2410.06993 (cross-list from cs.LG) [pdf, other]

Title: Efficient Distribution Matching of Representations via Noise-Injected Deep InfoMax

Ivan Butakov, Alexander Semenenko, Alexander Tolmachev, Andrey Gladkov, Marina Munkhoeva, Alexey Frolov

Comments: 25 pages, 7 fugures

Subjects: Machine Learning (cs.LG); Information Theory (cs.IT); Machine Learning (stat.ML)

[231] arXiv:2410.07393 (cross-list from eess.SP) [pdf, html, other]

Title: How Much Power Must We Extract From a Receiver Antenna to Effect Communications?

Thomas L. Marzetta, Brian McMinn, Amritpal Singh, Thorkild B. Hansen

Comments: 10 pages

Subjects: Signal Processing (eess.SP); Information Theory (cs.IT)

[232] arXiv:2410.07548 (cross-list from stat.ML) [pdf, html, other]

Title: Hybrid Summary Statistics

T. Lucas Makinen, Ce Sui, Benjamin D. Wandelt, Natalia Porqueres, Alan Heavens

Comments: 7 pages, 4 figures. Accepted to ML4PS2024 at NeurIPS 2024

Subjects: Machine Learning (stat.ML); Cosmology and Nongalactic Astrophysics (astro-ph.CO); Information Theory (cs.IT); Machine Learning (cs.LG); Data Analysis, Statistics and Probability (physics.data-an)

[233] arXiv:2410.07595 (cross-list from quant-ph) [pdf, html, other]

Title: Geometric structure and transversal logic of quantum Reed-Muller codes

Alexander Barg, Nolan J. Coble, Dominik Hangleiter, Christopher Kang

Subjects: Quantum Physics (quant-ph); Information Theory (cs.IT); Combinatorics (math.CO)

[234] arXiv:2410.07603 (cross-list from cs.DB) [pdf, other]

Title: An Analysis of XML Compression Efficiency

Christopher James Augeri, Barry E. Mullins, Leemon C. Baird III, Dursun A. Bulutoglu, Rusty O. Baldwin

Comments: 1. test data at this https URL 2. one next step is testing newer compressors, e.g., Brotli, along with Zstandard, which leverages the asymmetric numeral system (ANS) 3. citations at this https URL

Journal-ref: Proceedings of the 2007 workshop on Experimental Computer Science (ExpCS) at ACM FCRC 2007

Subjects: Databases (cs.DB); Information Theory (cs.IT); Performance (cs.PF)

[235] arXiv:2410.07616 (cross-list from cs.LG) [pdf, other]

Title: The Plug-in Approach for Average-Reward and Discounted MDPs: Optimal Sample Complexity Analysis

Matthew Zurek, Yudong Chen

Comments: Accepted to 36th International Conference on Algorithmic Learning Theory (ALT 2025)

Subjects: Machine Learning (cs.LG); Information Theory (cs.IT); Optimization and Control (math.OC); Machine Learning (stat.ML)

[236] arXiv:2410.07638 (cross-list from cs.LG) [pdf, other]

Title: Almost Minimax Optimal Best Arm Identification in Piecewise Stationary Linear Bandits

Yunlong Hou, Vincent Y. F. Tan, Zixin Zhong

Comments: 69 pages. Accepted to NeurIPS 2024

Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Information Theory (cs.IT); Machine Learning (stat.ML)

[237] arXiv:2410.07650 (cross-list from math.CO) [pdf, html, other]

Title: Optimal additive quaternary codes of dimension $3.5$

Sascha Kurz

Comments: 14 pages, 2 table; extended details

Subjects: Combinatorics (math.CO); Information Theory (cs.IT)

[238] arXiv:2410.07651 (cross-list from stat.ML) [pdf, html, other]

Title: Theoretical limits of descending $\ell_0$ sparse-regression ML algorithms

Mihailo Stojnic

Subjects: Machine Learning (stat.ML); Information Theory (cs.IT); Machine Learning (cs.LG); Statistics Theory (math.ST)

[239] arXiv:2410.07687 (cross-list from cs.LG) [pdf, html, other]

Title: Learning to Compress: Local Rank and Information Compression in Deep Neural Networks

Niket Patel, Ravid Shwartz-Ziv

Comments: Accepted to Compression Workshop @ NeurIPS 2024

Subjects: Machine Learning (cs.LG); Information Theory (cs.IT)

[240] arXiv:2410.08113 (cross-list from cs.CL) [pdf, html, other]

Title: Robust AI-Generated Text Detection by Restricted Embeddings

Kristian Kuznetsov, Eduard Tulchinskii, Laida Kushnareva, German Magai, Serguei Barannikov, Sergey Nikolenko, Irina Piontkovskaya

Comments: Accepted to Findings of EMNLP 2024

Subjects: Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Information Theory (cs.IT)

[241] arXiv:2410.08140 (cross-list from quant-ph) [pdf, html, other]

Title: Optimality of meta-converse for channel simulation

Aadil Oufkir, Omar Fawzi, Mario Berta

Comments: 23+6 pages

Subjects: Quantum Physics (quant-ph); Information Theory (cs.IT)

[242] arXiv:2410.08160 (cross-list from quant-ph) [pdf, html, other]

Title: Optimal Strategies for Winning Certain Coset-Guessing Quantum Games

Michael Schleppy, Emina Soljanin, Nicolas Swanson

Comments: Submitted to JSAIT Special Issue on Quantum Error Correction and Fault Tolerance

Subjects: Quantum Physics (quant-ph); Information Theory (cs.IT)

[243] arXiv:2410.08222 (cross-list from eess.SP) [pdf, html, other]

Title: Variational Source-Channel Coding for Semantic Communication

Yulong Feng, Jing Xu, Liujun Hu, Guanghui Yu, Xiangyang Duan

Subjects: Signal Processing (eess.SP); Information Theory (cs.IT); Machine Learning (cs.LG)

[244] arXiv:2410.08360 (cross-list from cs.LG) [pdf, html, other]

Title: Minimax Hypothesis Testing for the Bradley-Terry-Luce Model

Anuran Makur, Japneet Singh

Comments: 54 pages, 6 figures

Subjects: Machine Learning (cs.LG); Information Theory (cs.IT); Statistics Theory (math.ST)

[245] arXiv:2410.08459 (cross-list from eess.SP) [pdf, html, other]

Title: Beamforming Design for Intelligent Reffecting Surface Aided Near-Field THz Communications

Chi Qiu, Qingqing Wu, Wen Chen, Meng Hua, Wanming Hao, Mengnan Jian, Fen Hou

Subjects: Signal Processing (eess.SP); Information Theory (cs.IT); Networking and Internet Architecture (cs.NI)

[246] arXiv:2410.08634 (cross-list from cs.LG) [pdf, html, other]

Title: GAI-Enabled Explainable Personalized Federated Semi-Supervised Learning

Yubo Peng, Feibo Jiang, Li Dong, Kezhi Wang, Kun Yang

Subjects: Machine Learning (cs.LG); Information Theory (cs.IT)

[247] arXiv:2410.08937 (cross-list from quant-ph) [pdf, html, other]

Title: Distributed Quantum Hypothesis Testing under Zero-rate Communication Constraints

Sreejith Sreekumar, Christoph Hirche, Hao-Chung Cheng, Mario Berta

Comments: 34+2 pages; Added single-letter characterization of Stein's exponent under zero-rate (fully) quantum communication when state under alternative is product

Subjects: Quantum Physics (quant-ph); Information Theory (cs.IT)

[248] arXiv:2410.09121 (cross-list from quant-ph) [pdf, html, other]

Title: Comparing Quantum Encoding Techniques

Nidhi Munikote

Subjects: Quantum Physics (quant-ph); Emerging Technologies (cs.ET); Information Theory (cs.IT); Machine Learning (cs.LG)

[249] arXiv:2410.09557 (cross-list from physics.med-ph) [pdf, html, other]

Title: Optimizing Quantitative Photoacoustic Imaging Systems: The Bayesian Cramér-Rao Bound Approach

Evan Scope Crafts, Mark A. Anastasio, Umberto Villa

Subjects: Medical Physics (physics.med-ph); Information Theory (cs.IT); Numerical Analysis (math.NA)

[250] arXiv:2410.09701 (cross-list from stat.ML) [pdf, html, other]

Title: Transformers as Game Players: Provable In-context Game-playing Capabilities of Pre-trained Models

Chengshuai Shi, Kun Yang, Jing Yang, Cong Shen

Comments: Accepted to NeurIPS 2024

Subjects: Machine Learning (stat.ML); Computer Science and Game Theory (cs.GT); Information Theory (cs.IT); Machine Learning (cs.LG); Multiagent Systems (cs.MA)