Information Theory
- [1] arXiv:2405.13255 [pdf, ps, html, other]
-
Title: Low-Complexity PSCL Decoding of Polar CodesComments: 11 pages, 19 figuresSubjects: Information Theory (cs.IT)
Successive cancellation list (SCL) decoding enables polar codes and their generalizations to deliver satisfactory performance in finite-length scenarios but it comes with high latency and complexity. To reduce latency, a partitioned SCL (PSCL) decoding algorithm, implemented over a PSCL decoding tree, can be utilized. In this work, we aim to lower down the complexity of the PSCL decoding, resulting in an efficient decoding algorithm with low latency and complexity for polar-like codes. To achieve this, we define two metrics at each level of the PSCL decoding tree. One is for evaluating the reliability of a path and the other is for estimating the probability of the correct path being included in a list of paths. Then, we propose a double-threshold strategy in the PSCL decoding process where unreliable valid paths are pruned based on the first metric, and then a list of surviving paths is selected based on the second metric. Simulation results demonstrate that when polar/CRC-polar/PAC codes are decoded using the proposed low-complexity PSCL decoder, both the sorting complexity and the computational complexity are reduced and significantly decrease as the signal-to-noise ratio (SNR) increases.
- [2] arXiv:2405.13312 [pdf, ps, html, other]
-
Title: Iterative Detection and Decoding Schemes with LLR Refinements in Cell-Free Massive MIMO NetworksComments: 6 pages, 2 figuresSubjects: Information Theory (cs.IT); Signal Processing (eess.SP)
In this paper, we propose low-complexity local detectors and log-likelihood ratio (LLR) refinement techniques for a coded cell-free massive multiple input multiple output (CF- mMIMO) systems, where an iterative detection and decoding (IDD) scheme is applied using parallel interference cancellation (PIC) and access point (AP) selection. In particular, we propose three LLR processing schemes based on the individual processing of the LLRs of each AP, LLR censoring, and a linear combination of LLRs by assuming statistical independence. We derive new closed-form expressions for the local soft minimum mean square error (MMSE)-PIC detector and receive matched filter (RMF). We also examine the system performance as the number of iterations increases. Simulations assess the performance of the proposed techniques against existing approaches.
- [3] arXiv:2405.13320 [pdf, ps, html, other]
-
Title: Self-dual 2-quasi Negacyclic Codes over Finite FieldsSubjects: Information Theory (cs.IT)
In this paper, we investigate the existence and asymptotic property of self-dual $2$-quasi negacyclic codes of length $2n$ over a finite field of cardinality $q$. When $n$ is odd, we show that the $q$-ary self-dual $2$-quasi negacyclic codes exist if and only if $q\,{\not\equiv}-\!1~({\rm mod}~4)$. When $n$ is even, we prove that the $q$-ary self-dual $2$-quasi negacyclic codes always exist. By using the technique introduced in this paper, we prove that $q$-ary self-dual $2$-quasi negacyclic codes are asymptotically good.
- [4] arXiv:2405.13368 [pdf, ps, html, other]
-
Title: Static Deep Q-learning for Green Downlink C-RANSubjects: Information Theory (cs.IT)
Power saving is a main pillar in the operation of wireless communication systems. In this paper, we investigate cloud radio access network (C-RAN) capability to reduce power consumption based on the user equipment (UE) requirement. Aiming to save the long-term C-RAN energy consumption, an optimization problem is formulated to manage the downlink power without degrading the UE requirement by designing the power offset parameter. Considering stochastic traffic arrivals at UEs, we first formulate the problem as a Markov decision process (MDP) and then set up a dual objective optimization problem in terms of the downlink throughput and power. To solve this optimization problem, we develop a novel static deep Q-learning (SDQL) algorithm to maximize the downlink throughput and minimize the downlink power. In our proposed algorithm, we design multi-Q-tables to simultaneously optimize power reductions of activated RRHs by assigning one Q-table for each UE. To maximize the accumulative reward in terms of the downlink throughput loss and power reduction, our proposed algorithm performs power reductions of activated RRHs through continuous environmental interactions. Simulation results1 show that our proposed algorithm enjoys a superior average power reduction compared to the activation and sleep schemes, and enjoys a low computational complexity.
- [5] arXiv:2405.13413 [pdf, ps, html, other]
-
Title: Boosted Neural Decoders: Achieving Extreme Reliability of LDPC Codes for 6G NetworksComments: 12 pages, 11 figuresSubjects: Information Theory (cs.IT); Machine Learning (cs.LG); Signal Processing (eess.SP)
Ensuring extremely high reliability is essential for channel coding in 6G networks. The next-generation of ultra-reliable and low-latency communications (xURLLC) scenario within 6G networks requires a frame error rate (FER) below 10-9. However, low-density parity-check (LDPC) codes, the standard in 5G new radio (NR), encounter a challenge known as the error floor phenomenon, which hinders to achieve such low rates. To tackle this problem, we introduce an innovative solution: boosted neural min-sum (NMS) decoder. This decoder operates identically to conventional NMS decoders, but is trained by novel training methods including: i) boosting learning with uncorrected vectors, ii) block-wise training schedule to address the vanishing gradient issue, iii) dynamic weight sharing to minimize the number of trainable weights, iv) transfer learning to reduce the required sample count, and v) data augmentation to expedite the sampling process. Leveraging these training strategies, the boosted NMS decoder achieves the state-of-the art performance in reducing the error floor as well as superior waterfall performance. Remarkably, we fulfill the 6G xURLLC requirement for 5G LDPC codes without the severe error floor. Additionally, the boosted NMS decoder, once its weights are trained, can perform decoding without additional modules, making it highly practical for immediate application.
- [6] arXiv:2405.13483 [pdf, ps, html, other]
-
Title: Distributed Indirect Source Coding with Decoder Side InformationSubjects: Information Theory (cs.IT)
This paper studies a variant of the rate-distortion problem motivated by task-oriented semantic communication and distributed learning problems, where $M$ correlated sources are independently encoded for a central decoder. The decoder has access to a correlated side information in addition to the messages received from the encoders, and aims to recover a latent random variable correlated with the sources observed by the encoders within a given distortion constraint rather than recovering the sources themselves. We provide bounds on the rate-distortion region for this scenario in general, and characterize the rate-distortion function exactly when the sources are conditionally independent given the side information.
- [7] arXiv:2405.13678 [pdf, ps, html, other]
-
Title: Integrated Sensing and Communication Exploiting Prior Information: How Many Sensing Beams are Needed?Comments: This is the longer version of a paper to appear in IEEE International Symposium on Information Theory (ISIT), 2024Subjects: Information Theory (cs.IT); Signal Processing (eess.SP)
This paper studies an integrated sensing and communication (ISAC) system where a multi-antenna base station (BS) aims to communicate with a single-antenna user in the downlink and sense the unknown and random angle parameter of a target via exploiting its prior distribution information. We consider a general transmit beamforming structure where the BS sends one communication beam and potentially one or multiple dedicated sensing beam(s). Firstly, motivated by the periodic feature of the angle parameter, we derive the periodic posterior Cramér-Rao bound (PCRB) for quantifying a lower bound of the mean-cyclic error (MCE), which is more accurate than the conventional PCRB for bounding the mean-squared error (MSE). Then, note that more sensing beams enable higher flexibility in enhancing the sensing performance, while also generating extra interference to the communication user. To resolve this trade-off, we formulate the transmit beamforming optimization problem to minimize the periodic PCRB subject to a communication rate requirement for the user. Despite the non-convexity of this problem, we derive the optimal solution by leveraging the semi-definite relaxation (SDR) technique and Lagrange duality theory. Moreover, we analytically prove that at most one dedicated sensing beam is needed. Numerical results validate our analysis and the advantage of having a dedicated sensing beam.
- [8] arXiv:2405.13946 [pdf, ps, html, other]
-
Title: Coded Computing Meets Quantum Circuit Simulation: Coded Parallel Tensor Network Contraction AlgorithmComments: Accepted to ISIT2024Subjects: Information Theory (cs.IT)
Parallel tensor network contraction algorithms have emerged as the pivotal benchmarks for assessing the classical limits of computation, exemplified by Google's demonstration of quantum supremacy through random circuit sampling. However, the massive parallelization of the algorithm makes it vulnerable to computer node failures. In this work, we apply coded computing to a practical parallel tensor network contraction algorithm. To the best of our knowledge, this is the first attempt to code tensor network contractions. Inspired by matrix multiplication codes, we provide two coding schemes: 2-node code for practicality in quantum simulation and hyperedge code for generality. Our 2-node code successfully achieves significant gain for $f$-resilient number compared to naive replication, proportional to both the number of node failures and the dimension product of sliced indices. Our hyperedge code can cover tensor networks out of the scope of quantum, with degraded gain in the exchange of its generality.
- [9] arXiv:2405.14029 [pdf, ps, html, other]
-
Title: Analog Beamforming Enabled Multicasting: Finite-Alphabet Inputs and Statistical CSIComments: 5 pagesSubjects: Information Theory (cs.IT); Signal Processing (eess.SP)
The average multicast rate (AMR) is analyzed in a multicast channel utilizing analog beamforming with finite-alphabet inputs, considering statistical channel state information (CSI). New expressions for the AMR are derived for non-cooperative and cooperative multicasting scenarios. Asymptotic analyses are conducted in the high signal-to-noise ratio regime to derive the array gain and diversity order. It is proved that the analog beamformer influences the AMR through its array gain, leading to the proposal of efficient beamforming algorithms aimed at maximizing the array gain to enhance the AMR.
- [10] arXiv:2405.14046 [pdf, ps, html, other]
-
Title: Deep Reinforcement Learning Based Resource Allocation for MIMO Bistatic Backscatter NetworksComments: Submitted to an IEEE Transactions JournalSubjects: Information Theory (cs.IT); Signal Processing (eess.SP)
Bistatic backscatter communication promises ubiquitous, massive connectivity by utilizing passive tags to connect with a reader by reflecting carrier emitter (CE) signals for future Internet-of-Things (IoT) networks. This study focuses on the joint design of the transmit/received beamformers at the CE/reader and the reflection coefficient of the tag. A throughput maximization problem is thus formulated, subject to satisfying the tag requirements. We develop a joint design through a series of trial-and-error interactions within the environment, driven by a predefined reward system in a continuous state and action context. We propose two deep reinforcement learning (DRL) algorithms to address the underlying optimization problem, namely deep deterministic policy gradient (DDPG) and soft actor-critic (SAC). Simulation results indicate that the proposed algorithm can learn from the environment and incrementally enhance its behavior, achieving performance that is on par with two leading benchmarks. Further, we also compared the performance of the proposed method with deep Q-network (DQN), double deep Q-network (DDQN), and dueling DQN (DuelDQN). For a system with twelve antennas, SAC leads with a 26.76% gain over DQN, followed by alternative optimization (AO) and DDPG at 23.02% and 19.16%. DDQN and DuelDQN show smaller improvements of 10.40% and 14.36%, respectively, against DQN.
- [11] arXiv:2405.14123 [pdf, ps, html, other]
-
Title: Equations for the overlaps of a SICComments: 19 PagesSubjects: Information Theory (cs.IT)
We give a holomorphic quartic polynomial in the overlap variables whose zeros on the torus are precisely the Weyl-Heisenberg SICs (symmetric informationally complete positive operator valued measures). By way of comparison, all the other known systems of equations that determine a Weyl-Heisenberg SIC involve variables and their complex conjugates. We also give a related interesting result about the powers of the projective Fourier transform of the group G = Z d x Z d .
- [12] arXiv:2405.14570 [pdf, ps, html, other]
-
Title: A general method for the development of constrained codesSubjects: Information Theory (cs.IT)
Nowadays there are several classes of constrained codes intended for different applications. The following two large classes can be distinguished. The first class contains codes with local constraints; for example, the source data must be encoded by binary sequences containing no sub-words 00 and 111. The second class contains codes with global constraints; for example, the code-words must be binary sequences of certain even length with half zeros and half this http URL is important to note that often the necessary codes must fulfill some requirements of both classes.
In this paper we propose a general polynomial complexity method for constructing codes for both classes, as well as for combinations thereof. The proposed method uses the enumerative Cover's code, but the main difference between known applications of this code is that the known algorithms require the use of combinatorial formulae when applied, whereas the proposed method calculates all parameters on-the-fly using a polynomial complexity algorithm.
New submissions for Friday, 24 May 2024 (showing 12 of 12 entries )
- [13] arXiv:2405.13027 (cross-list from cs.HC) [pdf, ps, html, other]
-
Title: Cognitive Effort Measures Driven by Fixation Induced Retinal Flow in Visual Scanning Behavior during Virtual DrivingRunlin Zhang (1), Qing Xu (1), Simon Parkinson (2), Klaus Schoeffmann (3), Yu Chen (4) ((1) College of Intelligence and Computing, Tianjin University (2) School of Computing and Engineering, University of Huddersfield (3) Institute of Information Technology, Alpen-Adria Universitat Klagenfurt (4) School of Foreign Languages, Southeast University)Subjects: Human-Computer Interaction (cs.HC); Information Theory (cs.IT)
In this paper, we consider the problem of visual scanning mechanism underpinning sensorimotor tasks, such as walking and driving, in dynamic environments. We exploit eye tracking data for offering two new cognitive effort measures in visual scanning behavior of virtual driving. By utilizing the retinal flow induced by fixation, two novel measures of cognitive effort are proposed through the importance of grids in the viewing plane and the concept of information quantity, respectively. Psychophysical studies are conducted to reveal the effectiveness of the two proposed measures. Both these two cognitive effort measures have shown their significant correlation with pupil size change. Our results suggest that the quantitative exploitation of eye tracking data provides an effective approach for the evaluation of sensorimotor activities.
- [14] arXiv:2405.13549 (cross-list from eess.SP) [pdf, ps, html, other]
-
Title: Multi-Objective Optimization-Based Waveform Design for Multi-User and Multi-Target MIMO-ISAC SystemsComments: 13 pages, submitted to IEEE TWCSubjects: Signal Processing (eess.SP); Information Theory (cs.IT)
Integrated sensing and communication (ISAC) opens up new service possibilities for sixth-generation (6G) systems, where both communication and sensing (C&S) functionalities co-exist by sharing the same hardware platform and radio resource. In this paper, we investigate the waveform design problem in a downlink multi-user and multi-target ISAC system under different C&S performance preferences. The multi-user interference (MUI) may critically degrade the communication performance. To eliminate the MUI, we employ the constructive interference mechanism into the ISAC system, which saves the power budget for communication. However, due to the conflict between C&S metrics, it is intractable for the ISAC system to achieve the optimal performance of C&S objective simultaneously. Therefore, it is important to strike a tradeoff between C&S objectives. By virtue of the multi-objective optimization theory, we propose a weighted Tchebycheff-based transformation method to re-frame the C&S trade-off problem as a Pareto-optimal problem, thus effectively tackling the constraints in ISAC systems. Finally, simulation results reveal the trade-off relation between C&S performances, which provides insights for the flexible waveform design under different C&S performance preferences in MIMO-ISAC systems.
- [15] arXiv:2405.13798 (cross-list from cs.CL) [pdf, ps, html, other]
-
Title: Slaves to the Law of Large Numbers: An Asymptotic Equipartition Property for Perplexity in Generative Language ModelsSubjects: Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Information Theory (cs.IT)
We propose a new asymptotic equipartition property for the perplexity of a large piece of text generated by a language model and present theoretical arguments for this property. Perplexity, defined as a inverse likelihood function, is widely used as a performance metric for training language models. Our main result states that the logarithmic perplexity of any large text produced by a language model must asymptotically converge to the average entropy of its token distributions. This means that language models are constrained to only produce outputs from a ``typical set", which we show, is a vanishingly small subset of all possible grammatically correct outputs. We present preliminary experimental results from an open-source language model to support our theoretical claims. This work has possible practical applications for understanding and improving ``AI detection" tools and theoretical implications for the uniqueness, predictability and creative potential of generative models.
- [16] arXiv:2405.13912 (cross-list from math.ST) [pdf, ps, html, other]
-
Title: Matrix Denoising with Doubly Heteroscedastic Noise: Fundamental Limits and Optimal Spectral MethodsSubjects: Statistics Theory (math.ST); Information Theory (cs.IT); Machine Learning (cs.LG); Probability (math.PR); Machine Learning (stat.ML)
We study the matrix denoising problem of estimating the singular vectors of a rank-$1$ signal corrupted by noise with both column and row correlations. Existing works are either unable to pinpoint the exact asymptotic estimation error or, when they do so, the resulting approaches (e.g., based on whitening or singular value shrinkage) remain vastly suboptimal. On top of this, most of the literature has focused on the special case of estimating the left singular vector of the signal when the noise only possesses row correlation (one-sided heteroscedasticity). In contrast, our work establishes the information-theoretic and algorithmic limits of matrix denoising with doubly heteroscedastic noise. We characterize the exact asymptotic minimum mean square error, and design a novel spectral estimator with rigorous optimality guarantees: under a technical condition, it attains positive correlation with the signals whenever information-theoretically possible and, for one-sided heteroscedasticity, it also achieves the Bayes-optimal error. Numerical experiments demonstrate the significant advantage of our theoretically principled method with the state of the art. The proofs draw connections with statistical physics and approximate message passing, departing drastically from standard random matrix theory techniques.
- [17] arXiv:2405.13939 (cross-list from quant-ph) [pdf, ps, html, other]
-
Title: Principal eigenstate classical shadowsComments: 38 pagesSubjects: Quantum Physics (quant-ph); Information Theory (cs.IT); Machine Learning (cs.LG)
Given many copies of an unknown quantum state $\rho$, we consider the task of learning a classical description of its principal eigenstate. Namely, assuming that $\rho$ has an eigenstate $|\phi\rangle$ with (unknown) eigenvalue $\lambda > 1/2$, the goal is to learn a (classical shadows style) classical description of $|\phi\rangle$ which can later be used to estimate expectation values $\langle \phi |O| \phi \rangle$ for any $O$ in some class of observables. We consider the sample-complexity setting in which generating a copy of $\rho$ is expensive, but joint measurements on many copies of the state are possible. We present a protocol for this task scaling with the principal eigenvalue $\lambda$ and show that it is optimal within a space of natural approaches, e.g., applying quantum state purification followed by a single-copy classical shadows scheme. Furthermore, when $\lambda$ is sufficiently close to $1$, the performance of our algorithm is optimal--matching the sample complexity for pure state classical shadows.
- [18] arXiv:2405.14228 (cross-list from math.CO) [pdf, ps, html, other]
-
Title: $t$-Balanced Codes with the Kendall-$\tau$ MetricSubjects: Combinatorics (math.CO); Information Theory (cs.IT)
We investigate the maximum cardinality and the mathematical structure of error-correcting codes endowed with the Kendall-$\tau$ metric. We establish an averaging bound for the cardinality of a code with prescribed minimum distance, discuss its sharpness, and characterize codes attaining it. This leads to introducing the family of $t$-balanced codes in the Kendall-$\tau$ metric. The results are based on novel arguments that shed new light on the structure of the Kendall-$\tau$ metric space.
- [19] arXiv:2405.14482 (cross-list from math.ST) [pdf, ps, html, other]
-
Title: Quantifying Multivariate Graph Dependencies: Theory and Estimation for Multiplex GraphsSubjects: Statistics Theory (math.ST); Information Theory (cs.IT); Combinatorics (math.CO); Probability (math.PR)
Multiplex graphs, characterised by their layered structure, exhibit informative interdependencies within layers that are crucial for understanding complex network dynamics. Quantifying the interaction and shared information among these layers is challenging due to the non-Euclidean structure of graphs. Our paper introduces a comprehensive theory of multivariate information measures for multiplex graphs. We introduce graphon mutual information for pairs of graphs and expand this to graphon interaction information for three or more graphs, including their conditional variants. We then define graphon total correlation and graphon dual total correlation, along with their conditional forms, and introduce graphon $O-$information. We discuss and quantify the concepts of synergy and redundancy in graphs for the first time, introduce consistent nonparametric estimators for these multivariate graphon information--theoretic measures, and provide their convergence rates. We also conduct a simulation study to illustrate our theoretical findings and demonstrate the relationship between the introduced measures, multiplex graph structure, and higher--order interdependecies. Real-world applications further show the utility of our estimators in revealing shared information and dependence structures in real-world multiplex graphs. This work not only answers fundamental questions about information sharing across multiple graphs but also sets the stage for advanced pattern analysis in complex networks.
- [20] arXiv:2405.14724 (cross-list from eess.SP) [pdf, ps, html, other]
-
Title: Learning-Based Intermittent CSI Estimation with Adaptive Intervals in Integrated Sensing and Communication SystemsSubjects: Signal Processing (eess.SP); Information Theory (cs.IT)
Due to the distinct objectives and multipath utilization mechanisms between the communication module and radar module, the system design of integrated sensing and communication (ISAC) necessitates two types of channel state information (CSI), i.e., communication CSI representing the whole channel gain and phase shifts, and radar CSI exclusively focused on target mobility and position information. However, current ISAC systems apply an identical mechanism to estimate both types of CSI at the same predetermined estimation interval, leading to significant overhead and compromised performances. Therefore, this paper proposes an intermittent communication and radar CSI estimation scheme with adaptive intervals for individual users/targets, where both types of CSI can be predicted using channel temporal correlations for cost reduction or re-estimated via training signal transmission for improved estimation accuracy. Specifically, we jointly optimize the binary CSI re-estimation/prediction decisions and transmit beamforming matrices for individual users/targets to maximize communication transmission rates and minimize radar tracking errors and costs in a multiple-input single-output (MISO) ISAC system. Unfortunately, this problem has causality issues because it requires comparing system performances under re-estimated CSI and predicted CSI during the optimization. Additionally, the binary decision makes the joint design a mixed integer nonlinear programming (MINLP) problem, resulting in high complexity when using conventional optimization algorithms. Therefore, we propose a deep reinforcement online learning (DROL) framework that first implements an online deep neural network (DNN) to learn the binary CSI updating decisions from the experiences. Given the learned decisions, we propose an efficient algorithm to solve the remaining beamforming design problem efficiently.
Cross submissions for Friday, 24 May 2024 (showing 8 of 8 entries )
- [21] arXiv:2006.02643 (replaced) [pdf, ps, html, other]
-
Title: Universal Graph Compression: Stochastic Block ModelsSubjects: Information Theory (cs.IT); Databases (cs.DB); Statistics Theory (math.ST)
Motivated by the prevalent data science applications of processing large-scale graph data such as social networks and biological networks, this paper investigates lossless compression of data in the form of a labeled graph. Particularly, we consider a widely used random graph model, stochastic block model (SBM), which captures the clustering effects in social networks. An information-theoretic universal compression framework is applied, in which one aims to design a single compressor that achieves the asymptotically optimal compression rate, for every SBM distribution, without knowing the parameters of the SBM. Such a graph compressor is proposed in this paper, which universally achieves the optimal compression rate with polynomial time complexity for a wide class of SBMs. Existing universal compression techniques are developed mostly for stationary ergodic one-dimensional sequences. However, the adjacency matrix of SBM has complex two-dimensional correlations. The challenge is alleviated through a carefully designed transform that converts two-dimensional correlated data into almost i.i.d. submatrices. The sequence of submatrices is then compressed by a Krichevsky--Trofimov compressor, whose length analysis is generalized to identically distributed but arbitrarily correlated sequences. In four benchmark graph datasets, the compressed files from competing algorithms take 2.4 to 27 times the space needed by the proposed scheme.
- [22] arXiv:2307.14882 (replaced) [pdf, ps, html, other]
-
Title: Knot Theory and Error-Correcting CodesSubjects: Information Theory (cs.IT); Algebraic Topology (math.AT); General Topology (math.GN)
This paper builds a novel bridge between algebraic coding theory and mathematical knot theory, with applications in both directions. We give methods to construct error-correcting codes starting from the colorings of a knot, describing through a series of results how the properties of the knot translate into code parameters. We show that knots can be used to obtain error-correcting codes with prescribed parameters and an efficient decoding algorithm.
- [23] arXiv:2310.00263 (replaced) [pdf, ps, html, other]
-
Title: RIS-Aided Cell-Free Massive MIMO Systems for 6G: Fundamentals, System Design, and ApplicationsComments: Proceedings of the IEEE, Accept, 2024Subjects: Information Theory (cs.IT); Signal Processing (eess.SP)
An introduction of intelligent interconnectivity for people and things has posed higher demands and more challenges for sixth-generation (6G) networks, such as high spectral efficiency and energy efficiency, ultra-low latency, and ultra-high reliability. Cell-free (CF) massive multiple-input multiple-output (mMIMO) and reconfigurable intelligent surface (RIS), also called intelligent reflecting surface (IRS), are two promising technologies for coping with these unprecedented demands. Given their distinct capabilities, integrating the two technologies to further enhance wireless network performances has received great research and development attention. In this paper, we provide a comprehensive survey of research on RIS-aided CF mMIMO wireless communication systems. We first introduce system models focusing on system architecture and application scenarios, channel models, and communication protocols. Subsequently, we summarize the relevant studies on system operation and resource allocation, providing in-depth analyses and discussions. Following this, we present practical challenges faced by RIS-aided CF mMIMO systems, particularly those introduced by RIS, such as hardware impairments and electromagnetic interference. We summarize corresponding analyses and solutions to further facilitate the implementation of RIS-aided CF mMIMO systems. Furthermore, we explore an interplay between RIS-aided CF mMIMO and other emerging 6G technologies, such as next-generation multiple-access (NGMA), simultaneous wireless information and power transfer (SWIPT), and millimeter wave (mmWave). Finally, we outline several research directions for future RIS-aided CF mMIMO systems.
- [24] arXiv:2310.20504 (replaced) [pdf, ps, html, other]
-
Title: SumComp: Coding for Digital Over-the-Air Computation via the Ring of IntegersSubjects: Information Theory (cs.IT)
Communication and computation are traditionally treated as separate entities, allowing for individual optimizations. However, many applications focus on local information's functionality rather than the information itself. For such cases, harnessing interference for computation in a multiple access channel through digital over-the-air computation can notably increase the computation, as established by the ChannelComp method. However, the coding scheme originally proposed in ChannelComp may suffer from high computational complexity because it is general and is not optimized for specific modulation categories. Therefore, this study considers a specific category of digital modulations for over-the-air computations, QAM and PAM, for which we introduce a novel coding scheme called SumComp. Furthermore, we derive an MSE analysis for SumComp coding in the computation of the arithmetic mean function and establish an upper bound on the MAE for a set of nomographic functions. Simulation results affirm the superior performance of SumComp coding compared to traditional analog over-the-air computation and the original coding in ChannelComp approaches regarding both MSE and MAE over a noisy multiple access channel. Specifically, SumComp coding shows approximately $10$ dB improvements for computing arithmetic and geometric mean on the normalized MSE for low noise scenarios.
- [25] arXiv:2402.02853 (replaced) [pdf, ps, html, other]
-
Title: Repeated-Root Cyclic Codes with Optimal Parameters or Best Parameters KnownComments: 27 pagesSubjects: Information Theory (cs.IT)
Cyclic codes are the most studied subclass of linear codes and widely used in data storage and communication systems. Many cyclic codes have optimal parameters or the best parameters known. They are divided into simple-root cyclic codes and repeated-root cyclic codes. Although there are a huge number of references on cyclic codes, few of them are on repeated-root cyclic codes. Hence, repeated-root cyclic codes are rarely studied. There are a few families of distance-optimal repeated-root binary and $p$-ary cyclic codes for odd prime $p$ in the literature. However, it is open whether there exists an infinite family of distance-optimal repeated-root cyclic codes over $\bF_q$ for each even $q \geq 4$.
In this paper, three infinite families of distance-optimal repeated-root cyclic codes with minimum distance 3 or 4 are constructed; two other infinite families of repeated-root cyclic codes with minimum distance 3 or 4 are developed; seven infinite families of repeated-root cyclic codes with minimum distance 6 or 8 or 10 are presented; and two infinite families of repeated-root binary cyclic codes with parameters $[2n, k, d \geq (n-1)/\log_2 n]$, where $n=2^m-1$ and $k \geq n$, are constructed. In addition, 27 repeated-root cyclic codes of length up to $254$ over $\bF_q$ for $q \in \{2, 4, 8\}$ with optimal parameters or best parameters known are obtained in this paper. The results of this paper show that repeated-root cyclic codes could be very attractive and are worth of further investigation. - [26] arXiv:2403.10089 (replaced) [pdf, ps, html, other]
-
Title: Approximation and bounding techniques for the Fisher-Rao distances between parametric statistical modelsComments: 47 pagesSubjects: Information Theory (cs.IT); Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG)
The Fisher-Rao distance between two probability distributions of a statistical model is defined as the Riemannian geodesic distance induced by the Fisher information metric. In order to calculate the Fisher-Rao distance in closed-form, we need (1) to elicit a formula for the Fisher-Rao geodesics, and (2) to integrate the Fisher length element along those geodesics. We consider several numerically robust approximation and bounding techniques for the Fisher-Rao distances: First, we report generic upper bounds on Fisher-Rao distances based on closed-form 1D Fisher-Rao distances of submodels. Second, we describe several generic approximation schemes depending on whether the Fisher-Rao geodesics or pregeodesics are available in closed-form or not. In particular, we obtain a generic method to guarantee an arbitrarily small additive error on the approximation provided that Fisher-Rao pregeodesics and tight lower and upper bounds are available. Third, we consider the case of Fisher metrics being Hessian metrics, and report generic tight upper bounds on the Fisher-Rao distances using techniques of information geometry. Uniparametric and biparametric statistical models always have Fisher Hessian metrics, and in general a simple test allows to check whether the Fisher information matrix yields a Hessian metric or not. Fourth, we consider elliptical distribution families and show how to apply the above techniques to these models. We also propose two new distances based either on the Fisher-Rao lengths of curves serving as proxies of Fisher-Rao geodesics, or based on the Birkhoff/Hilbert projective cone distance. Last, we consider an alternative group-theoretic approach for statistical transformation models based on the notion of maximal invariant which yields insights on the structures of the Fisher-Rao distance formula which may be used fruitfully in applications.
- [27] arXiv:2210.10524 (replaced) [pdf, ps, html, other]
-
Title: Over-the-Air Computation for 6G: Foundations, Technologies, and ApplicationsComments: This work has been accepted by IEEE Internet of Things JournalSubjects: Signal Processing (eess.SP); Information Theory (cs.IT)
The rapid advancement of artificial intelligence technologies has given rise to diversified intelligent services, which place unprecedented demands on massive connectivity and gigantic data aggregation. However, the scarce radio resources and stringent latency requirement make it challenging to meet these demands. To tackle these challenges, over-the-air computation (AirComp) emerges as a potential technology. Specifically, AirComp seamlessly integrates the communication and computation procedures through the superposition property of multiple-access channels, which yields a revolutionary multiple-access paradigm shift from "compute-after-communicate" to "compute-when-communicate". By this means, AirComp enables spectral-efficient and low-latency wireless data aggregation by allowing multiple devices to occupy the same channel for transmission. In this paper, we aim to present the recent advancement of AirComp in terms of foundations, technologies, and applications. The mathematical form and communication design are introduced as the foundations of AirComp, and the critical issues of AirComp over different network architectures are then discussed along with the review of existing literature. The technologies employed for the analysis and optimization on AirComp are reviewed from the information theory and signal processing perspectives. Moreover, we present the existing studies that tackle the practical implementation issues in AirComp systems, and elaborate the applications of AirComp in Internet of Things and edge intelligent networks. Finally, potential research directions are highlighted to motivate the future development of AirComp.
- [28] arXiv:2301.02469 (replaced) [pdf, ps, html, other]
-
Title: Cox Point Processes for Multi Altitude LEO Satellite NetworksComments: accepted to IEEE Trans. Veh. TechnolSubjects: Signal Processing (eess.SP); Information Theory (cs.IT); Probability (math.PR)
To model existing or future low Earth orbit (LEO) satellite networks leveraging multiple constellations, we propose a simple analytical approach to represent the clustering of satellites on orbits. More precisely, we develop a variable-altitude Poisson orbit process that effectively captures the geometric fact that satellites are always positioned on orbits, and these orbits may vary in altitude. Conditionally on the orbit process, satellites situated on these orbits are modeled as linear Poisson point processes, thereby forming a Cox point process. For this model, we derive useful statistics, including the distribution of the distance from the typical user to its nearest visible satellite, the outage probability, the Laplace functional of the proposed Cox satellite point process, and the Laplace transform of the interference power from the Cox-distributed satellites under general fading. The derived statistics enable the evaluation of the performance of such LEO satellite communication systems as functions of network parameters.
- [29] arXiv:2304.07278 (replaced) [pdf, ps, html, other]
-
Title: Minimax-Optimal Reward-Agnostic Exploration in Reinforcement LearningComments: accepted for presentation in COLT 2024Subjects: Machine Learning (cs.LG); Information Theory (cs.IT); Systems and Control (eess.SY); Statistics Theory (math.ST); Machine Learning (stat.ML)
This paper studies reward-agnostic exploration in reinforcement learning (RL) -- a scenario where the learner is unware of the reward functions during the exploration stage -- and designs an algorithm that improves over the state of the art. More precisely, consider a finite-horizon inhomogeneous Markov decision process with $S$ states, $A$ actions, and horizon length $H$, and suppose that there are no more than a polynomial number of given reward functions of interest. By collecting an order of \begin{align*}
\frac{SAH^3}{\varepsilon^2} \text{ sample episodes (up to log factor)} \end{align*} without guidance of the reward information, our algorithm is able to find $\varepsilon$-optimal policies for all these reward functions, provided that $\varepsilon$ is sufficiently small. This forms the first reward-agnostic exploration scheme in this context that achieves provable minimax optimality. Furthermore, once the sample size exceeds $\frac{S^2AH^3}{\varepsilon^2}$ episodes (up to log factor), our algorithm is able to yield $\varepsilon$ accuracy for arbitrarily many reward functions (even when they are adversarially designed), a task commonly dubbed as ``reward-free exploration.'' The novelty of our algorithm design draws on insights from offline RL: the exploration scheme attempts to maximize a critical reward-agnostic quantity that dictates the performance of offline RL, while the policy learning paradigm leverages ideas from sample-optimal offline RL paradigms. - [30] arXiv:2307.04191 (replaced) [pdf, ps, html, other]
-
Title: On the sample complexity of parameter estimation in logistic regression with normal designSubjects: Statistics Theory (math.ST); Information Theory (cs.IT); Machine Learning (cs.LG); Machine Learning (stat.ML)
The logistic regression model is one of the most popular data generation model in noisy binary classification problems. In this work, we study the sample complexity of estimating the parameters of the logistic regression model up to a given $\ell_2$ error, in terms of the dimension and the inverse temperature, with standard normal covariates. The inverse temperature controls the signal-to-noise ratio of the data generation process. While both generalization bounds and asymptotic performance of the maximum-likelihood estimator for logistic regression are well-studied, the non-asymptotic sample complexity that shows the dependence on error and the inverse temperature for parameter estimation is absent from previous analyses. We show that the sample complexity curve has two change-points in terms of the inverse temperature, clearly separating the low, moderate, and high temperature regimes.
- [31] arXiv:2309.14011 (replaced) [pdf, ps, html, other]
-
Title: A Truly Concurrent Semantics for Reversible CCSSubjects: Logic in Computer Science (cs.LO); Information Theory (cs.IT)
Reversible CCS (RCCS) is a well-established, formal model for reversible communicating systems, which has been built on top of the classical Calculus of Communicating Systems (CCS). In its original formulation, each CCS process is equipped with a memory that records its performed actions, which is then used to reverse computations. More recently, abstract models for RCCS have been proposed in the literature, basically, by directly associating RCCS processes with (reversible versions of) event structures. In this paper we propose a different abstract model: starting from one of the well-known encoding of CCS into Petri nets we apply a recently proposed approach to incorporate causally-consistent reversibility to Petri nets, obtaining as result the (reversible) net counterpart of every RCCS term.
- [32] arXiv:2311.06840 (replaced) [pdf, ps, html, other]
-
Title: Omitted Labels in Causality: A Study of ParadoxesSubjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Information Theory (cs.IT); Social and Information Networks (cs.SI); Methodology (stat.ME)
We explore what we call ``omitted label contexts,'' in which training data is limited to a subset of the possible labels. This setting is common among specialized human experts or specific focused studies. We lean on well-studied paradoxes (Simpson's and Condorcet) to illustrate the more general difficulties of causal inference in omitted label contexts. Contrary to the fundamental principles on which much of causal inference is built, we show that ``correct'' adjustments sometimes require non-exchangeable treatment and control groups. These pitfalls lead us to the study networks of conclusions drawn from different contexts and the structures the form, proving an interesting connection between these networks and social choice theory.
- [33] arXiv:2404.08483 (replaced) [pdf, ps, html, other]
-
Title: Semantic Communication for Cooperative Multi-Task Processing over Wireless NetworksComments: This work has been submitted to the IEEE for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessibleSubjects: Signal Processing (eess.SP); Information Theory (cs.IT); Machine Learning (cs.LG)
In this paper, we have expanded the current status of semantic communication limited to processing one task to a more general system that can handle multiple tasks concurrently. In pursuit of this, we first introduced our definition of the "semantic source", enabling the interpretation of multiple semantics based on a single observation. A semantic encoder design is then introduced, featuring the division of the encoder into a common unit and multiple specific units enabling cooperative multi-task processing. Simulation results demonstrate the effectiveness of the proposed semantic source and the system design. Our approach employs information maximization (infomax) and end-to-end design principles.
- [34] arXiv:2405.00105 (replaced) [pdf, ps, html, other]
-
Title: Quantum Doeblin coefficients: A simple upper bound on contraction coefficientsComments: 15 pages, 6 figures. Short version accepted at ISIT 2024, v2: 18 pages, 8 figures. Improved bounds addedSubjects: Quantum Physics (quant-ph); Information Theory (cs.IT)
Contraction coefficients give a quantitative strengthening of the data processing inequality. As such, they have many natural applications whenever closer analysis of information processing is required. However, it is often challenging to calculate these coefficients. As a remedy we discuss a quantum generalization of Doeblin coefficients. These give an efficiently computable upper bound on many contraction coefficients. We prove several properties and discuss generalizations and applications. In particular, we give additional stronger bounds. One especially for PPT channels and one for general channels based on a constraint relaxation. Additionally, we introduce reverse Doeblin coefficients that bound certain expansion coefficients.
- [35] arXiv:2405.12807 (replaced) [pdf, ps, html, other]
-
Title: FAdam: Adam is a natural gradient optimizer using diagonal empirical Fisher informationComments: 19 pages, 1 figures, 5 tablesSubjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Information Theory (cs.IT)
This paper establishes a mathematical foundation for the Adam optimizer, elucidating its connection to natural gradient descent through Riemannian and information geometry. We rigorously analyze the diagonal empirical Fisher information matrix (FIM) in Adam, clarifying all detailed approximations and advocating for the use of log probability functions as loss, which should be based on discrete distributions, due to the limitations of empirical FIM. Our analysis uncovers flaws in the original Adam algorithm, leading to proposed corrections such as enhanced momentum calculations, adjusted bias corrections, adaptive epsilon, and gradient clipping. We refine the weight decay term based on our theoretical framework. Our modified algorithm, Fisher Adam (FAdam), demonstrates superior performance across diverse domains including LLM, ASR, and VQ-VAE, achieving state-of-the-art results in ASR.