Information Theory
See recent articles
- [1] arXiv:2406.17070 [pdf, html, other]
-
Title: Collective Bit Flipping-Based Decoding of Quantum LDPC CodesComments: 13 pages, 12 figuresSubjects: Information Theory (cs.IT); Quantum Physics (quant-ph)
Quantum low-density parity-check (QLDPC) codes have been proven to achieve higher minimum distances at higher code rates than surface codes. However, this family of codes imposes stringent latency requirements and poor performance under iterative decoding, especially when the variable degree is low. In this work, we improve both the error correction performance and decoding latency of variable degree-3 (dv-3) QLDPC codes under iterative decoding. Firstly, we perform a detailed analysis of the structure of a well-known family of QLDPC codes, i.e., hypergraph product-based codes. Then, we propose a decoding approach that stems from the knowledge of harmful configurations apparent in these codes. Our decoding scheme is based on applying a modified version of bit flipping (BF) decoding, namely two-bit bit flipping (TBF) decoding, which adds more degrees of freedom to BF decoding. The granularity offered by TBF decoding helps us design sets of decoders that operate in parallel and can collectively decode error patterns appearing in harmful configurations of the code, thus addressing both the latency and performance requirements. Finally, simulation results demonstrate that the proposed decoding scheme surpasses other iterative decoding approaches for various dv-3 QLDPC codes.
- [2] arXiv:2406.17144 [pdf, other]
-
Title: An information-geometric approach for network decomposition using the q-state Potts modelSubjects: Information Theory (cs.IT); Social and Information Networks (cs.SI); Mathematical Physics (math-ph); Adaptation and Self-Organizing Systems (nlin.AO); Applications (stat.AP)
Complex networks are critical in many scientific, technological, and societal contexts due to their ability to represent and analyze intricate systems with interdependent components. Often, after labeling the nodes of a network with a community detection algorithm, its modular organization emerges, allowing a better understanding of the underlying structure by uncovering hidden relationships. In this paper, we introduce a novel information-geometric framework for the filtering and decomposition of networks whose nodes have been labeled. Our approach considers the labeled network as the outcome of a Markov random field modeled by a q-state Potts model. According to information geometry, the first and second order Fisher information matrices are related to the metric and curvature tensor of the parametric space of a statistical model. By computing an approximation to the local shape operator, the proposed methodology is able to identify low and high information nodes, allowing the decomposition of the labeled network in two complementary subgraphs. Hence, we call this method as the LO-HI decomposition. Experimental results with several kinds of networks show that the high information subgraph is often related to edges and boundaries, while the low information subgraph is a smoother version of the network, in the sense that the modular structure is improved.
- [3] arXiv:2406.17196 [pdf, html, other]
-
Title: Coded Kalman Filtering over MIMO Gaussian Channels with FeedbackComments: Accepted for presentation at the 2024 IEEE International Symposium on Information TheorySubjects: Information Theory (cs.IT); Systems and Control (eess.SY)
We consider the problem of remotely stabilizing a linear dynamical system. In this setting, a sensor co-located with the system communicates the system's state to a controller over a noisy communication channel with feedback. The objective of the controller (decoder) is to use the channel outputs to estimate the vector state with finite zero-delay mean squared error (MSE) at the infinite horizon. It has been shown in [1] that for a vector Gauss-Markov source and either a single-input multiple-output (SIMO) or a multiple-input single-output (MISO) channel, linear codes require the minimum capacity to achieve finite MSE. This paper considers the more general problem of linear zero-delay joint-source channel coding (JSCC) of a vector-valued source over a multiple-input multiple-output (MIMO) Gaussian channel with feedback. We study sufficient and necessary conditions for linear codes to achieve finite MSE. For sufficiency, we introduce a coding scheme where each unstable source mode is allocated to a single channel for estimation. Our proof for the necessity of this scheme relies on a matrix-algebraic conjecture that we prove to be true if either the source or channel is scalar. We show that linear codes achieve finite MSE for a scalar source over a MIMO channel if and only if the best scalar sub-channel can achieve finite MSE. Finally, we provide a new counter-example demonstrating that linear codes are generally sub-optimal for coding over MIMO channels.
- [4] arXiv:2406.17223 [pdf, html, other]
-
Title: On Zero-Error Capacity of Graphs with One EdgeSubjects: Information Theory (cs.IT)
In this paper, we study the zero-error capacity of channels with memory, which are represented by graphs. We provide a method to construct code for any graph with one edge, thereby determining a lower bound on its zero-error capacity. Moreover, this code can achieve zero-error capacity when the symbols in a vertex with degree one are the same. We further apply our method to the one-edge graphs representing the binary channels with two memories. There are 28 possible graphs, which can be organized into 11 categories based on their symmetries. The code constructed by our method is proved to achieve the zero-error capacity for all these graphs except for the two graphs in Case 11.
- [5] arXiv:2406.17339 [pdf, other]
-
Title: Optimizing Configuration Selection in Reconfigurable-Antenna MIMO Systems: Physics-Inspired Heuristic SolversComments: arXiv admin note: text overlap with arXiv:2403.12571Journal-ref: IEEE Transactions on Communications, 2004Subjects: Information Theory (cs.IT); Signal Processing (eess.SP)
Reconfigurable antenna multiple-input multiple-output (MIMO) is a foundational technology for the continuing evolution of cellular systems, including upcoming 6G communication systems. In this paper, we address the problem of flexible/reconfigurable antenna configuration selection for point-to-point MIMO antenna systems by using physics-inspired heuristics. Firstly, we optimize the antenna configuration to maximize the signal-to-noise ratio (SNR) at the receiver by leveraging two basic heuristic solvers, i.e., coherent Ising machines (CIMs), that mimic quantum mechanical dynamics, and quantum annealing (QA), where a real-world QA architecture is considered (D-Wave). A mathematical framework that converts the configuration selection problem into CIM- and QA- compatible unconstrained quadratic formulations is investigated. Numerical and experimental results show that the proposed designs outperform classical counterparts and achieve near-optimal performance (similar to exhaustive search with exponential complexity) while ensuring polynomial complexity. Moreover, we study the optimal antenna configuration that maximizes the end-to-end Shannon capacity. A simulated annealing (SA) heuristic which achieves near-optimal performance through appropriate parameterization is adopted. A modified version of the basic SA that exploits parallel tempering to avoid local maxima is also studied, which provides additional performance gains. Extended numerical studies show that the SA solutions outperform conventional heuristics (which are also developed for comparison purposes), while the employment of the SNR-based solutions is highly sub-optimal.
- [6] arXiv:2406.17393 [pdf, other]
-
Title: Timely and Painless Breakups: Off-the-grid Blind Message Recovery and Users' DemixingSubjects: Information Theory (cs.IT)
In the near future, the Internet of Things will interconnect billions of devices, forming a vast network where users sporadically transmit short messages through multi-path wireless channels. These channels are characterized by the superposition of a small number of scaled and delayed copies of Dirac spikes. At the receiver, the observed signal is a sum of these convolved signals, and the task is to find the amplitudes, continuous-indexed delays, and transmitted messages from a single signal. This task is inherently ill-posed without additional assumptions on the channel or messages. In this work, we assume the channel exhibits sparsity in the delay domain and that i.i.d. random linear encoding is applied to the messages at the devices. Leveraging these assumptions, we propose a semidefinite programming optimization capable of simultaneously recovering both messages and the delay parameters of the channels from only a single received signal. Our theoretical analysis establishes that the required number of samples at the receiver scales proportionally to the sum-product of sparsity and message length of all users, aligning with the degrees of freedom in the proposed convex optimization framework. Numerical experiments confirm the efficacy of the proposed method in accurately estimating closely-spaced delay parameters and recovering messages.
- [7] arXiv:2406.17476 [pdf, html, other]
-
Title: Pre-Decoder Processing Functions for a DMC with Mismatched DecodingSubjects: Information Theory (cs.IT)
This paper analyzes the effect of adding a pre-decoder processing function to a receiver that contains a fixed mismatched decoder at the output of a discrete memoryless channel. We study properties of the symbolwise pre-processing function and show that it is a simple yet very powerful tool which enables to obtain reliable transmission at a positive rate for almost every metric. We present lower and upper bounds on the capacity of a channel with mismatched decoding and symbolwise(scalar-to-scalar) pre-processing, and show that the optimal pre-processing function for random coding is deterministic. We also characterize achievable error exponents. Finally, we prove that a separation principle holds for vectorwise(vector-to-vector) pre-processing functions and further, that deterministic functions maximize the reliably transmitted rate in this case.
- [8] arXiv:2406.17669 [pdf, other]
-
Title: Capacity-Achieving Gray CodesComments: 10 pages, 3 figuresSubjects: Information Theory (cs.IT); Data Structures and Algorithms (cs.DS)
To ensure differential privacy, one can reveal an integer fuzzily in two ways: (a) add some Laplace noise to the integer, or (b) encode the integer as a binary string and add iid BSC noise. The former is simple and natural while the latter is flexible and affordable, especially when one wants to reveal a sparse vector of integers. In this paper, we propose an implementation of (b) that achieves the capacity of the BSC with positive error exponents. Our implementation adds error-correcting functionality to Gray codes by mimicking how software updates back up the files that are getting updated ("coded Gray code"). In contrast, the old implementation of (b) interpolates between codewords of a black-box error-correcting code ("Grayed code").
- [9] arXiv:2406.17689 [pdf, other]
-
Title: Robust Gray Codes Approaching the Optimal RateSubjects: Information Theory (cs.IT); Data Structures and Algorithms (cs.DS)
Robust Gray codes were introduced by (Lolck and Pagh, SODA 2024). Informally, a robust Gray code is a (binary) Gray code $\mathcal{G}$ so that, given a noisy version of the encoding $\mathcal{G}(j)$ of an integer $j$, one can recover $\hat{j}$ that is close to $j$ (with high probability over the noise). Such codes have found applications in differential privacy.
In this work, we present near-optimal constructions of robust Gray codes. In more detail, we construct a Gray code $\mathcal{G}$ of rate $1 - H_2(p) - \varepsilon$ that is efficiently encodable, and that is robust in the following sense. Supposed that $\mathcal{G}(j)$ is passed through the binary symmetric channel $\text{BSC}_p$ with cross-over probability $p$, to obtain $x$. We present an efficient decoding algorithm that, given $x$, returns an estimate $\hat{j}$ so that $|j - \hat{j}|$ is small with high probability.
New submissions for Wednesday, 26 June 2024 (showing 9 of 9 entries )
- [10] arXiv:2406.16878 (cross-list from eess.SP) [pdf, other]
-
Title: Benchmarking Semantic Communications for Image Transmission Over MIMO Interference ChannelsSubjects: Signal Processing (eess.SP); Artificial Intelligence (cs.AI); Information Theory (cs.IT)
Semantic communications offer promising prospects for enhancing data transmission efficiency. However, existing schemes have predominantly concentrated on point-to-point transmissions. In this paper, we aim to investigate the validity of this claim in interference scenarios compared to baseline approaches. Specifically, our focus is on general multiple-input multiple-output (MIMO) interference channels, where we propose an interference-robust semantic communication (IRSC) scheme. This scheme involves the development of transceivers based on neural networks (NNs), which integrate channel state information (CSI) either solely at the receiver or at both transmitter and receiver ends. Moreover, we establish a composite loss function for training IRSC transceivers, along with a dynamic mechanism for updating the weights of various components in the loss function to enhance system fairness among users. Experimental results demonstrate that the proposed IRSC scheme effectively learns to mitigate interference and outperforms baseline approaches, particularly in low signal-to-noise (SNR) regimes.
- [11] arXiv:2406.17133 (cross-list from quant-ph) [pdf, html, other]
-
Title: The Infinite-Dimensional Quantum Entropy: the Unified Entropy CaseComments: 12 pages, 2 figuresSubjects: Quantum Physics (quant-ph); Information Theory (cs.IT)
By a use of the Fredholm determinant theory, the unified quantum entropy notion has been extended to a case of infinite-dimensional systems. Some of the known (in the finite-dimensional case) basic properties of the introduced unified entropies have been extended to the case study. Certain numerical approaches for computing the proposed finite and infinite-dimensional entropies are being outlined as well.
- [12] arXiv:2406.17470 (cross-list from cs.LG) [pdf, html, other]
-
Title: Dynamic Scheduling for Vehicle-to-Vehicle Communications Enhanced Federated LearningComments: Submitted to IEEE for possible publicationSubjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Distributed, Parallel, and Cluster Computing (cs.DC); Information Theory (cs.IT)
Leveraging the computing and sensing capabilities of vehicles, vehicular federated learning (VFL) has been applied to edge training for connected vehicles. The dynamic and interconnected nature of vehicular networks presents unique opportunities to harness direct vehicle-to-vehicle (V2V) communications, enhancing VFL training efficiency. In this paper, we formulate a stochastic optimization problem to optimize the VFL training performance, considering the energy constraints and mobility of vehicles, and propose a V2V-enhanced dynamic scheduling (VEDS) algorithm to solve it. The model aggregation requirements of VFL and the limited transmission time due to mobility result in a stepwise objective function, which presents challenges in solving the problem. We thus propose a derivative-based drift-plus-penalty method to convert the long-term stochastic optimization problem to an online mixed integer nonlinear programming (MINLP) problem, and provide a theoretical analysis to bound the performance gap between the online solution and the offline optimal solution. Further analysis of the scheduling priority reduces the original problem into a set of convex optimization problems, which are efficiently solved using the interior-point method. Experimental results demonstrate that compared with the state-of-the-art benchmarks, the proposed algorithm enhances the image classification accuracy on the CIFAR-10 dataset by 3.18% and reduces the average displacement errors on the Argoverse trajectory prediction dataset by 10.21%.
- [13] arXiv:2406.17610 (cross-list from quant-ph) [pdf, html, other]
-
Title: YAQQ: Yet Another Quantum Quantizer -- Design Space Exploration of Quantum Gate Sets using Novelty SearchAritra Sarkar, Akash Kundu, Matthew Steinberg, Sibasish Mishra, Sebastiaan Fauquenot, Tamal Acharya, Jarosław A. Miszczak, Sebastian FeldSubjects: Quantum Physics (quant-ph); Information Theory (cs.IT); Symbolic Computation (cs.SC)
In the standard circuit model of quantum computation, the number and quality of the quantum gates composing the circuit influence the runtime and fidelity of the computation. The fidelity of the decomposition of quantum algorithms, represented as unitary matrices, to bounded depth quantum circuits depends strongly on the set of gates available for the decomposition routine. To investigate this dependence, we explore the design space of discrete quantum gate sets and present a software tool for comparative analysis of quantum processing units and control protocols based on their native gates. The evaluation is conditioned on a set of unitary transformations representing target use cases on the quantum processors. The cost function considers three key factors: (i) the statistical distribution of the decomposed circuits' depth, (ii) the statistical distribution of process fidelities for the approximate decomposition, and (iii) the relative novelty of a gate set compared to other gate sets in terms of the aforementioned properties. The developed software, YAQQ (Yet Another Quantum Quantizer), enables the discovery of an optimized set of quantum gates through this tunable joint cost function. To identify these gate sets, we use the novelty search algorithm, circuit decomposition techniques, and stochastic optimization to implement YAQQ within the Qiskit quantum simulator environment. YAQQ exploits reachability tradeoffs conceptually derived from quantum algorithmic information theory. Our results demonstrate the pragmatic application of identifying gate sets that are advantageous to popularly used quantum gate sets in representing quantum algorithms. Consequently, we demonstrate pragmatic use cases of YAQQ in comparing transversal logical gate sets in quantum error correction codes, designing optimal quantum instruction sets, and compiling to specific quantum processors.
Cross submissions for Wednesday, 26 June 2024 (showing 4 of 4 entries )
- [14] arXiv:2402.05583 (replaced) [pdf, html, other]
-
Title: On the Spectral Efficiency of Indoor Wireless Networks with a Rotary Uniform Linear ArrayComments: 6 pages, 7 figures. Manuscript submitted to the IEEE Wireless Communications and Networking Conference (WCNC), Milan, Italy, 2025Subjects: Information Theory (cs.IT); Signal Processing (eess.SP)
Contemporary wireless communication systems rely on Multi-User Multiple-Input Multiple-Output (MU-MIMO) techniques. In such systems, each Access Point (AP) is equipped with multiple antenna elements and serves multiple devices simultaneously. Notably, traditional systems utilize fixed antennas, i.e., antennas without any movement capabilities, while the idea of movable antennas has recently gained traction among the research community. By moving in a confined region, movable antennas are able to exploit the wireless channel variation in the continuous domain. This additional degree of freedom may enhance the quality of the wireless links, and consequently the communication performance. However, movable antennas for MU-MIMO proposed in the literature are complex, bulky, expensive and present a high power consumption. In this paper, we propose an alternative to such systems that has lower complexity and lower cost. More specifically, we propose the incorporation of rotation capabilities to APs equipped with Uniform Linear Arrays (ULAs) of antennas. We consider the uplink of an indoor scenario where the AP serves multiple devices simultaneously. The optimal rotation of the ULA is computed based on estimates of the positions of the active devices and aiming at maximizing the per-user mean achievable Spectral Efficiency (SE). Adopting a spatially correlated Rician channel model, our numerical results show that the rotation capabilities of the AP can bring substantial improvements in the SE in scenarios where the line-of-sight component of the channel vectors is strong. Moreover, our proposed system is robust against imperfect positioning estimates.
- [15] arXiv:2403.10002 (replaced) [pdf, html, other]
-
Title: Fast Group Scheduling for Downlink Large-Scale Multi-Group Multicast BeamformingComments: 13 pages, 8 figuresSubjects: Information Theory (cs.IT); Signal Processing (eess.SP)
Next-generation wireless networks need to handle massive user access effectively. This paper addresses the problem of joint group scheduling and multicast beamforming for downlink transmission with many active user groups. Aiming to maximize the minimum user throughput, we propose a three-phase approach to tackle this difficult joint optimization problem efficiently. In Phase 1, we utilize the optimal multicast beamforming structure obtained recently to find the group-channel directions for all groups. We propose two low-complexity group scheduling algorithms in Phase 2, which determine the subset of groups in each time slot sequentially and the total number of time slots required for all groups. The first algorithm measures the level of spatial separation among groups and selects the dissimilar groups that maximize the minimum user rate into the same time slot. In contrast, the second algorithm first identifies the spatially correlated groups via a learning-based clustering method based on the group-channel directions, and then separates spatially similar groups into different time slots. Finally, the multicast beamformers for the scheduled groups are obtained in each time slot by a computationally efficient method. Simulation results show that our proposed scheduling methods can effectively capture the level of spatial separation among groups to improve the minimum user throughput over the conventional approach that serves all groups in a single time slot or one group per time slot, and can be executed with low computational complexity.
- [16] arXiv:2406.06280 (replaced) [pdf, html, other]
-
Title: Optimal sensing policy with interference-model uncertaintyComments: Submitted to IEEE communications lettersSubjects: Information Theory (cs.IT); Signal Processing (eess.SP)
Assume that an interferer behaves according to a parametric model but one does not know the value of the model parameters. Sensing enables to improve the model knowledge and therefore perform a better link adaptation. However, we consider a half-duplex scenario where, at each time slot, the communication system should decide between sensing and communication. We thus propose to investigate the optimal policy to maximize the expected sum rate given a finite-time communication. % the following question therefore arises: At a given time slot, should one sense or communicate? We first show that this problem can be modelled in the Markov decision process (MDP) framework. We then demonstrate that the optimal open-loop and closed-loop policies can be found significantly faster than the standard backward-induction algorithm.
- [17] arXiv:2406.15945 (replaced) [pdf, html, other]
-
Title: Full-Space Wireless Sensing Enabled by Multi-Sector Intelligent SurfacesComments: 13 pages, 9 figuresSubjects: Signal Processing (eess.SP); Information Theory (cs.IT)
The multi-sector intelligent surface (IS), benefiting from a smarter wave manipulation capability, has been shown to enhance channel gain and offer full-space coverage in communications. However, the benefits of multi-sector IS in wireless sensing remain unexplored. This paper introduces the application of multi-sector IS for wireless sensing/localization. Specifically, we propose a new self-sensing system, where an active source controller uses the multi-sector IS geometry to reflect/scatter the emitted signals towards the entire space, thereby achieving full-space coverage for wireless sensing. Additionally, dedicated sensors are installed aligned with the IS elements at each sector, which collect echo signals from the target and cooperate to sense the target angle. In this context, we develop a maximum likelihood estimator of the target angle for the proposed multi-sector IS self-sensing system, along with the corresponding theoretical limits defined by the Cramér-Rao Bound. The analysis reveals that the advantages of the multi-sector IS self-sensing system stem from two aspects: enhancing the probing power on targets (thereby improving power efficiency) and increasing the rate of target angle (thereby enhancing the transceiver's sensitivity to target angles). Finally, our analysis and simulations confirm that the multi-sector IS self-sensing system, particularly the 4-sector architecture, achieves full-space sensing capability beyond the single-sector IS configuration. Furthermore, similarly to communications, employing directive antenna patterns on each sector's IS elements and sensors significantly enhances sensing capabilities. This enhancement originates from both aspects of improved power efficiency and target angle sensitivity, with the former also being observed in communications while the latter being unique in sensing.