Information Theory
- [1] arXiv:2405.15935 [pdf, ps, other]
-
Title: Optimizing Secrecy Codes Using Gradient DescentComments: 25 pages, 23 figures. Submitted to Transactions on Information TheorySubjects: Information Theory (cs.IT)
Recent theoretical developments in coset coding theory have provided continuous-valued functions which give the equivocation and maximum likelihood (ML) decoding probability of coset secrecy codes. In this work, we develop a method for incorporating these functions, along with a complex set of constraints, into a gradient descent optimization algorithm. This algorithm employs a movement cost function and trigonometric update step to ensure that the continuous-valued code definition vector ultimately reaches a value which yields a realizable coset code. This algorithm is used to produce coset codes with blocklength up to a few thousand. These codes were compared against published codes, including both short-blocklength and capacity-achieving constructions. For most code sizes, codes generated using gradient descent outperformed all others, especially capacity-achieving constructions, which performed significantly worse than randomly-generated codes at short blocklength.
- [2] arXiv:2405.15969 [pdf, ps, html, other]
-
Title: Massive Digital Over-the-Air Computation for Communication-Efficient Federated Edge LearningComments: To be published in the IEEE Journal on Selected Areas in CommunicationsSubjects: Information Theory (cs.IT); Signal Processing (eess.SP)
Over-the-air computation (AirComp) is a promising technology converging communication and computation over wireless networks, which can be particularly effective in model training, inference, and more emerging edge intelligence applications. AirComp relies on uncoded transmission of individual signals, which are added naturally over the multiple access channel thanks to the superposition property of the wireless medium. Despite significantly improved communication efficiency, how to accommodate AirComp in the existing and future digital communication networks, that are based on discrete modulation schemes, remains a challenge. This paper proposes a massive digital AirComp (MD-AirComp) scheme, that leverages an unsourced massive access protocol, to enhance compatibility with both current and next-generation wireless networks. MD-AirComp utilizes vector quantization to reduce the uplink communication overhead, and employs shared quantization and modulation codebooks. At the receiver, we propose a near-optimal approximate message passing-based algorithm to compute the model aggregation results from the superposed sequences, which relies on estimating the number of devices transmitting each code sequence, rather than trying to decode the messages of individual transmitters. We apply MD-AirComp to the federated edge learning (FEEL), and show that it significantly accelerates FEEL convergence compared to state-of-the-art while using the same amount of communication resources. To support further research and ensure reproducibility, we have made our code available at this https URL.
- [3] arXiv:2405.16047 [pdf, ps, html, other]
-
Title: Unified Timing Analysis for Closed-Loop Goal-Oriented Wireless CommunicationComments: submitted to IEEE Trans. Wireless CommunSubjects: Information Theory (cs.IT); Signal Processing (eess.SP)
Goal-oriented communication has become one of the focal concepts in sixth-generation communication systems owing to its potential to provide intelligent, immersive, and real-time mobile services. The emerging paradigms of goal-oriented communication constitute closed loops integrating communication, computation, and sensing. However, challenges arise for closed-loop timing analysis due to multiple random factors that affect the communication/computation latency, as well as the heterogeneity of feedback mechanisms across multi-modal sensing data. To tackle these problems, we aim to provide a unified timing analysis framework for closed-loop goal-oriented communication (CGC) systems over fading channels. The proposed framework is unified as it considers computation, compression, and communication latency in the loop with different configurations. To capture the heterogeneity across multi-modal feedback, we categorize the sensory data into the periodic-feedback and event-triggered, respectively. We formulate timing constraints based on average and tail performance, covering timeliness, jitter, and reliability of CGC systems. A method based on saddlepoint approximation is proposed to obtain the distribution of closed-loop latency. The results show that the modified saddlepoint approximation is capable of accurately characterizing the latency distribution of the loop with analytically tractable expressions. This sets the basis for low-complexity co-design of communication and computation.
- [4] arXiv:2405.16048 [pdf, ps, html, other]
-
Title: A New Construction of Optimal Symmetrical ZCCSComments: This paper has been accepted in 'IEEE International Symposium on Information Theory (ISIT 2024)'Subjects: Information Theory (cs.IT)
We propose new constructions for a two-dimensional ($2$D) perfect array, complete complementary code (CCC), and multiple CCCs as an optimal symmetrical $Z$-complementary code set (ZCCS). We propose a method to generate a two-dimensional perfect array and CCC. By utilising mutually orthogonal sequences, we developed a method to extend the length of a CCC without affecting the set or code size. Additionally, this concept is extended to include the development of multiple CCCs, and the correlation characteristics of these multiple CCCs are identical with the characteristics of optimal symmetrical ZCCS.
- [5] arXiv:2405.16060 [pdf, ps, html, other]
-
Title: Delay-Effective Task Offloading Technology in Internet of Vehicles: From the Perspective of the Vehicle PlatooningSubjects: Information Theory (cs.IT)
The task offloading technology plays a crucial vital role in the Internet of Vehicle (IoV) with the demands of delay minimum, by jointly optimizing the heterogeneous computing resources supported by the vehicles, roadside units (RSUs), and macro base stations (MBSs). In previous works, on the one hand, they ignored the wireless interference among the exchange and sharing of the task data. On the other hand, the available resources supported by the vehicles that have similar driving behaviors, which can form a vehicle platooning (VEH-PLA) and effectively integrate the resources of individual vehicle, has not been addressed. In addition, as a novel resource management paradigm, the VEH-PLA should consider the task categorization, since vehicles in VEH-PLA may have the same task offloading requests, which also has not attracted enough attention. In this paper, considering the wireless interference, mobility, VEH-PLA, and task categorization, we propose four kinds of task offloading models for the purpose of the processing delay minimum. Furthermore, by utilizing centralized training and decentralized execution (CTDE) based on multi-agent deep reinforcement learning (MADRL), we present a task offloading decision-making method to find the global optimal offloading decision, resulting in a significant enhancement in the load balancing of resources and processing delay. Finally, the simulations demonstrate that the proposed method significantly outperforms traditional task offloading methods in terms of the processing delay minimum while keeping the resource load balancing.
- [6] arXiv:2405.16062 [pdf, ps, html, other]
-
Title: Movable Antenna Empowered Physical Layer Security Without Eve's CSI: Joint Optimization of Beamforming and Antenna PositionsSubjects: Information Theory (cs.IT); Signal Processing (eess.SP)
Physical layer security (PLS) technology based on the fixed-position antenna (FPA) has {attracted widespread attention}. Due to the fixed feature of the antennas, current FPA-based PLS schemes cannot fully utilize the spatial degree of freedom, and thus a weaken secure gain in the desired/undesired direction may exist. Different from the concept of FPA, mobile antenna (MA) is a novel technology that {reconfigures} the wireless channels and enhances the corresponding capacity through the flexible movement of antennas on a minor scale. MA-empowered PLS enjoys huge potential and deserves further investigation. In this paper, we, for the first time, investigate the secrecy performance of MA-enabled PLS system where a MA-based Alice transmits the confidential information to multiple single-antenna Bobs, in the presence of the single-antenna eavesdropper (Eve) {in the absence} of perfect channel state information (CSI). For the purpose of the secrecy rate maximization of the worst Bob, we jointly design the transmit beamforming and antenna positions at the Alice, subject to the minimum moving distance of the antenna, uncertainty CSI of Eve, and maximum transmit power. Furthermore, the projected gradient ascent (PGA), alternating optimization (AO), and simulated annealing (SA) {are} adopted to solve the non-convex characteristics of the problem of the secrecy rate maximization. Simulation results demonstrate the effectiveness and correctness of the proposed method. In particular, MA-enabled PLS scheme can significantly enhance the secrecy rate compared to the conventional FPA-based ones for different settings of key system parameters.
- [7] arXiv:2405.16078 [pdf, ps, html, other]
-
Title: An Multi-resources Integration Empowered Task Offloading in Internet of Vehicles: From the Perspective of Wireless InterferenceSubjects: Information Theory (cs.IT)
The task offloading technology plays a vital role in the Internet of Vehicles (IoV), by satisfying the diversified demands of the vehicles, such as the energy consumption and processing latency of the computing task. Different from the previous works, on the one hand, they ignored the wireless interference of communications among vehicle-to-vehicle (V2V), as well as between vehicles and roadside units (RSU); on the other hand, the available resources of parked vehicles on the roadside and other moving vehicles on the road are also ignored. In this paper, first of all, we adopt a truncated Gaussian distribution for modeling the vehicle moving speed, instead of the simplistic average speed models in prior studies. Then, with the consideration of wireless interference and effective communication duration existing in V2V and RSUs, we establish an analytical framework of the task offloading, characterized by the energy consumption and processing delay, by integrating computing resources of parked/moving vehicles and RSUs. Furthermore, inspired by the method of multi-agent deterministic policy gradient (MADDPG), we address a joint optimization of the energy consumption and processing delay of the computing task, while ensuring the load balancing of the resources. Finally, the simulations demonstrate the effectiveness and correctness of the proposed MADDPG. In particular, compared with the current popular methods of the task offloading, the MADDPG shows the best performance, in terms of convergence speed, energy consumption and processing delay.
- [8] arXiv:2405.16174 [pdf, ps, html, other]
-
Title: Dynamic Scattering Arrays for Simultaneous Electromagnetic Processing and Radiation in Holographic MIMO SystemsComments: Submittted to IEEE Trans. on Wireless CommunicationsSubjects: Information Theory (cs.IT); Signal Processing (eess.SP)
To meet the stringent requirements of next-generation wireless networks, multiple-input multiple-output (MIMO) technology is expected to become massive and pervasive. Unfortunately, this could pose scalability issues in terms of complexity, power consumption, cost, and processing latency. Therefore, novel technologies and design approaches, such as the recently introduced holographic MIMO paradigm, must be investigated to make future networks sustainable. In this context, we propose the concept of a dynamic scattering array (DSA) as a versatile 3D structure capable of performing joint wave-based computing and radiation by moving the processing from the digital domain to the electromagnetic (EM) domain. We provide a general analytical framework for modeling DSAs, introduce specific design algorithms, and apply them to various use cases. The examples presented in the numerical results demonstrate the potential of DSAs to further reduce complexity and the number of radiofrequency (RF) chains in holographic MIMO systems while achieving enhanced EM wave processing and radiation flexibility for tasks such as beamforming and single- and multi-user MIMO.
- [9] arXiv:2405.16360 [pdf, ps, html, other]
-
Title: How Many Matrices Should I Prepare To Polarize Channels Optimally Fast?Comments: 6 pages, 2 figures, ISIT 2023Subjects: Information Theory (cs.IT)
Polar codes that approach capacity at a near-optimal speed, namely with scaling exponents close to $2$, have been shown possible for $q$-ary erasure channels (Pfister and Urbanke), the BEC (Fazeli, Hassani, Mondelli, and Vardy), all BMS channels (Guruswami, Riazanov, and Ye), and all DMCs (Wang and Duursma). There is, nevertheless, a subtlety separating the last two papers from the first two, namely the usage of multiple dynamic kernels in the polarization process, which leads to increased complexity and fewer opportunities to hardware-accelerate. This paper clarifies this subtlety, providing a trade-off between the number of kernels in the construction and the scaling exponent. We show that the number of kernels can be bounded by $O(\ell^{3/\mu-1})$ where $\mu$ is the targeted scaling exponent and $\ell$ is the kernel size. In particular, if one settles for scaling exponent approaching $3$, a single kernel suffices, and to approach the optimal scaling exponent of $2$, about $O(\sqrt{\ell})$ kernels suffice.
- [10] arXiv:2405.16370 [pdf, ps, html, other]
-
Title: Quickly-Decodable Group Testing with Fewer Tests: Price-Scarlett and Cheraghchi-Nakos's Nonadaptive Splitting with Explicit ScalarsComments: 6 pages, 3 figures, ISIT 2023Subjects: Information Theory (cs.IT)
We modify Cheraghchi-Nakos [CN20] and Price-Scarlett's [PS20] fast binary splitting approach to nonadaptive group testing. We show that, to identify a uniformly random subset of $k$ infected persons among a population of $n$, it takes only $\ln(2 - 4\varepsilon) ^{-2} k \ln n$ tests and decoding complexity $O(\varepsilon^{-2} k \ln n)$, for any small $\varepsilon > 0$, with vanishing error probability. In works prior to ours, only two types of group testing schemes exist. Those that use $\ln(2)^{-2} k \ln n$ or fewer tests require linear-in-$n$ complexity, sometimes even polynomial in $n$; those that enjoy sub-$n$ complexity employ $O(k \ln n)$ tests, where the big-$O$ scalar is implicit, presumably greater than $\ln(2)^{-2}$. We almost achieve the best of both worlds, namely, the almost-$\ln(2)^{-2}$ scalar and the sub-$n$ decoding complexity. How much further one can reduce the scalar $\ln(2)^{-2}$ remains an open problem.
- [11] arXiv:2405.16373 [pdf, ps, html, other]
-
Title: Successive Cancellation Sampling Decoder: An Attempt to Analyze List Decoding TheoreticallyComments: 6 pages, 4 figures, ISIT 2024Subjects: Information Theory (cs.IT)
Successive cancellation list (SCL) decoders of polar codes excel in practical performance but pose challenges for theoretical analysis. Existing works either limit their scope to erasure channels or address general channels without taking advantage of soft information. In this paper, we propose the "successive cancellation sampling" (SCS) decoder. SCS hires iid "agents" to sample codewords using posterior probabilities. This makes it fully parallel and amenable for some theoretical analysis. As an example, when comparing SCS with $a$ agents to any list decoder with list size $\ell$, we can prove that the error probability of the former is at most $\ell/ae$ more than that of the latter. In this paper, we also describe how to adjust the "temperature" of agents. Warmer agents are less likely to sample the same codewords and hence can further reduce error probability.
- [12] arXiv:2405.16374 [pdf, ps, html, other]
-
Title: Isolate and then Identify: Rethinking Adaptive Group TestingComments: 6 pages, 1 figure, ISIT 2024Subjects: Information Theory (cs.IT)
Group testing (GT) is the art of identifying binary signals and the marketplace for exchanging new ideas for related fields such as unique-element counting, compressed sensing, traitor tracing, and geno-typing. A GT scheme can be nonadaptive or adaptive; the latter is preferred when latency is ess of an issue. To construct adaptive GT schemes, a popular strategy is to spend the majority of tests in the first few rounds to gain as much information as possible, and uses later rounds to refine details. In this paper, we propose a transparent strategy called "isolate and then identify" (I@I). In the first few rounds, I@I divides the population into teams until every team contains at most one sick person. Then, in the last round, I@I identifies the sick person in each team. Performance-wise, I@I is the first GT scheme that achieves the optimal coefficient $1/$capacity$(Z)$ for the $k \log_2 (n/k)$ term in the number of tests when $Z$ is a generic channel corrupting the test outcomes. I@I follows a modular methodology whereby the isolating part and the identification part can be optimized separately.
- [13] arXiv:2405.16463 [pdf, ps, html, other]
-
Title: InfoMat: A Tool for the Analysis and Visualization Sequential Information TransferComments: Accepted at ISIT2024Subjects: Information Theory (cs.IT)
Despite the popularity of information measures in analysis of probabilistic systems, proper tools for their visualization are not common. This work develops a simple matrix representation of information transfer in sequential systems, termed information matrix (InfoMat). The simplicity of the InfoMat provides a new visual perspective on existing decomposition formulas of mutual information, and enables us to prove new relations between sequential information theoretic measures. We study various estimation schemes of the InfoMat, facilitating the visualization of information transfer in sequential datasets. By drawing a connection between visual patterns in the InfoMat and various dependence structures, we observe how information transfer evolves in the dataset. We then leverage this tool to visualize the effect of capacity-achieving coding schemes on the underlying exchange of information. We believe the InfoMat is applicable to any time-series task for a better understanding of the data at hand.
- [14] arXiv:2405.16753 [pdf, ps, html, other]
-
Title: Multi-answer Constrained Optimal Querying: Maximum Information Gain CodingComments: 17 pages, 11 figuresSubjects: Information Theory (cs.IT)
As the rapidly developments of artificial intelligence and machine learning, behavior tree design in multiagent system or AI game become more important. The behavior tree design problem is highly related to the source coding in information theory. "Twenty Questions" problem is a typical example for the behavior tree design, usually used to explain the source coding application in information theory and can be solved by Huffman coding. In some realistic scenarios, there are some constraints on the asked questions. However, for general question set, finding the minimum expected querying length is an open problem, belongs to NP-hard. Recently, a new coding scheme has been proposed to provide a near optimal solution for binary cases with some constraints, named greedy binary separation coding (GBSC). In this work, we shall generalize it to D-ary cases and propose maximum information gain coding (MIGC) approach to solve the multi-answer decision constrained querying problem. The optimality of the proposed MIGC is discussed in theory. Later on, we also apply MIGC to discuss three practical scenarios and showcase that MIGC has better performance than GBSC and Shannon Coding in terms of bits persymbol.
- [15] arXiv:2405.16784 [pdf, ps, html, other]
-
Title: The second-order zero differential uniformity of the swapped inverse functions over finite fieldsSubjects: Information Theory (cs.IT); Cryptography and Security (cs.CR)
The Feistel Boomerang Connectivity Table (FBCT) was proposed as the feistel counterpart of the Boomerang Connectivity Table. The entries of the FBCT are actually related to the second-order zero differential spectrum. Recently, several results on the second-order zero differential uniformity of some functions were introduced. However, almost all of them were focused on power functions, and there are only few results on non-power functions. In this paper, we investigate the second-order zero differential uniformity of the swapped inverse functions, which are functions obtained from swapping two points in the inverse function. We also present the second-order zero differential spectrum of the swapped inverse functions for certain cases. In particular, this paper is the first result to characterize classes of non-power functions with the second-order zero differential uniformity equal to 4, in even characteristic.
- [16] arXiv:2405.16791 [pdf, ps, html, other]
-
Title: Joint Node Selection and Resource Allocation Optimization for Cooperative Sensing with a Shared Wireless BackhaulComments: 13 pages, 10 figuresSubjects: Information Theory (cs.IT); Signal Processing (eess.SP)
In this paper, we consider a cooperative sensing framework in the context of future multi-functional network with both communication and sensing ability, where one base station (BS) serves as a sensing transmitter and several nearby BSs serve as sensing receivers. Each receiver receives the sensing signal reflected by the target and communicates with the fusion center (FC) through a wireless multiple access channel (MAC) for cooperative target localization. To improve the localization performance, we present a hybrid information-signal domain cooperative sensing (HISDCS) design, where each sensing receiver transmits both the estimated time delay/effective reflecting coefficient and the received sensing signal sampled around the estimated time delay to the FC. Then, we propose to minimize the number of channel uses by utilizing an efficient Karhunen-Loéve transformation (KLT) encoding scheme for signal quantization and proper node selection, under the Cramér-Rao lower bound (CRLB) constraint and the capacity limits of MAC. A novel matrix-inequality constrained successive convex approximation (MCSCA) algorithm is proposed to optimize the wireless backhaul resource allocation, together with a greedy strategy for node selection. Despite the high non-convexness of the considered problem, we prove that the proposed MCSCA algorithm is able to converge to the set of Karush-Kuhn-Tucker (KKT) solutions of a relaxed problem obtained by relaxing the discrete variables. Besides, a low-complexity quantization bit reallocation algorithm is designed, which does not perform explicit node selection, and is able to harvest most of the performance gain brought by HISDCS. Finally, numerical simulations are presented to show that the proposed HISDCS design is able to significantly outperform the baseline schemes.
- [17] arXiv:2405.16893 [pdf, ps, html, other]
-
Title: Cross Far- and Near-Field Channel Measurement and Modeling in Extremely Large-scale Antenna Array (ELAA) SystemsComments: 14 pages, 33 figuresSubjects: Information Theory (cs.IT); Signal Processing (eess.SP)
Technologies like ultra-massive multiple-input-multiple-output (UM-MIMO) and reconfigurable intelligent surfaces (RISs) are of special interest to meet the key performance indicators of future wireless systems including ubiquitous connectivity and lightning-fast data rates. One of their common features, the extremely large-scale antenna array (ELAA) systems with hundreds or thousands of antennas, give rise to near-field (NF) propagation and bring new challenges to channel modeling and characterization. In this paper, a cross-field channel model for ELAA systems is proposed, which improves the statistical model in 3GPP TR 38.901 by refining the propagation path with its first and last bounces and differentiating the characterization of parameters like path loss, delay, and angles in near- and far-fields. A comprehensive analysis of cross-field boundaries and closed-form expressions of corresponding NF or FF parameters are provided. Furthermore, cross-field experiments carried out in a typical indoor scenario at 300 GHz verify the variation of MPC parameters across the antenna array, and demonstrate the distinction of channels between different antenna elements. Finally, detailed generation procedures of the cross-field channel model are provided, based on which simulations and analysis on NF probabilities and channel coefficients are conducted for $4\times4$, $8\times8$, $16\times16$, and $9\times21$ uniform planar arrays at different frequency bands.
- [18] arXiv:2405.16965 [pdf, ps, html, other]
-
Title: Timeliness of Status Update System: The Effect of Parallel Transmission Using Heterogeneous Updating DevicesSubjects: Information Theory (cs.IT); Signal Processing (eess.SP)
Timely status updating is the premise of emerging interaction-based applications in the Internet of Things (IoT). Using redundant devices to update the status of interest is a promising method to improve the timeliness of information. However, parallel status updating leads to out-of-order arrivals at the monitor, significantly challenging timeliness analysis. This work studies the Age of Information (AoI) of a multi-queue status update system where multiple devices monitor the same physical process. Specifically, two systems are considered: the Basic System, which only has type-1 devices that are ad hoc devices located close to the source, and the Hybrid System, which contains additional type-2 devices that are infrastructure-based devices located in fixed points compared to the Basic System. Using the Stochastic Hybrid Systems (SHS) framework, a mathematical model that combines discrete and continuous dynamics, we derive the expressions of the average AoI of the considered two systems in closed form. Numerical results verify the accuracy of the analysis. It is shown that when the number and parameters of the type-1 devices/type-2 devices are fixed, the logarithm of average AoI will linearly decrease with the logarithm of the total arrival rate of type-2 devices or that of the number of type-1 devices under specific condition. It has also been demonstrated that the proposed systems can significantly outperform the FCFS M/M/N status update system.
- [19] arXiv:2405.17114 [pdf, ps, html, other]
-
Title: Holographic MIMO Systems, Their Channel Estimation and PerformanceComments: This article has been accepted for publication in IEEE VTMSubjects: Information Theory (cs.IT); Signal Processing (eess.SP)
Holographic multiple-input multiple-output (MIMO) systems constitute a promising technology in support of next-generation wireless communications, thus paving the way for a smart programmable radio environment. However, despite its significant potential, further fundamental issues remain to be addressed, such as the acquisition of accurate channel information. Indeed, the conventional angular-domain channel representation is no longer adequate for characterizing the sparsity inherent in holographic MIMO channels. To fill this knowledge gap, in this article, we conceive a decomposition and reconstruction (DeRe)-based framework for facilitating the estimation of sparse channels in holographic MIMOs. In particular, the channel parameters involved in the steering vector, namely the azimuth and elevation angles plus the distance (AED), are decomposed for independently constructing their own covariance matrices. Then, the acquisition of each parameter can be formulated as a compressive sensing (CS) problem by harnessing the covariance matrix associated with each individual parameter. We demonstrate that our solution exhibits an improved performance and imposes a reduced pilot overhead, despite its reduced complexity. Finally, promising open research topics are highlighted to bridge the gap between the theory and the practical employment of holographic MIMO schemes.
- [20] arXiv:2405.17150 [pdf, ps, html, other]
-
Title: Deep Learning-based Joint Channel Prediction and Multibeam Precoding for LEO Satellite Internet of ThingsComments: IEEE Transactions on Wireless Communications, 2024Subjects: Information Theory (cs.IT); Signal Processing (eess.SP)
Low earth orbit (LEO) satellite internet of things (IoT) is a promising way achieving global Internet of Everything, and thus has been widely recognized as an important component of sixth-generation (6G) wireless networks. Yet, due to high-speed movement of the LEO satellite, it is challenging to acquire timely channel state information (CSI) and design effective multibeam precoding for various IoT applications. To this end, this paper provides a deep learning (DL)-based joint channel prediction and multibeam precoding scheme under adverse environments, e.g., high Doppler shift, long propagation delay, and low satellite payload. {Specifically, this paper first designs a DL-based channel prediction scheme by using convolutional neural networks (CNN) and long short term memory (LSTM), which predicts the CSI of current time slot according to that of previous time slots. With the predicted CSI, this paper designs a DL-based robust multibeam precoding scheme by using a channel augmentation method based on variational auto-encoder (VAE).} Finally, extensive simulation results confirm the effectiveness and robustness of the proposed scheme in LEO satellite IoT.
- [21] arXiv:2405.17329 [pdf, ps, html, other]
-
Title: Joint MIMO Transceiver and Reflector Design for Reconfigurable Intelligent Surface-Assisted CommunicationComments: 14 pages, 12 figuresSubjects: Information Theory (cs.IT); Signal Processing (eess.SP)
In this paper, we consider a reconfigurable intelligent surface (RIS)-assisted multiple-input multiple-output communication system with multiple antennas at both the base station (BS) and the user. We plan to maximize the achievable rate through jointly optimizing the transmit precoding matrix, the receive combining matrix, and the RIS reflection matrix under the constraints of the transmit power at the BS and the unit-modulus reflection at the RIS. Regarding the non-trivial problem form, we initially reformulate it into an considerable problem to make it tractable by utilizing the relationship between the achievable rate and the weighted minimum mean squared error. Next, the transmit precoding matrix, the receive combining matrix, and the RIS reflection matrix are alternately optimized. In particular, the optimal transmit precoding matrix and receive combining matrix are obtained in closed forms. Furthermore, a pair of computationally efficient methods are proposed for the RIS reflection matrix, namely the semi-definite relaxation (SDR) method and the successive closed form (SCF) method. We theoretically prove that both methods are ensured to converge, and the SCF-based algorithm is able to converges to a Karush-Kuhn-Tucker point of the problem.
New submissions for Tuesday, 28 May 2024 (showing 21 of 21 entries )
- [22] arXiv:2405.15057 (cross-list from quant-ph) [pdf, ps, html, other]
-
Title: Characterization of Nearly Self-Orthogonal Quasi-Twisted Codes and Related Quantum CodesComments: 18 pages, 8 tables; see also this http URL This work has been submitted to the IEEE for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessibleSubjects: Quantum Physics (quant-ph); Information Theory (cs.IT)
Quasi-twisted codes are used here as the classical ingredients in the so-called Construction X for quantum error-control codes. The construction utilizes nearly self-orthogonal codes to design quantum stabilizer codes. We expand the choices of the inner product to also cover the symplectic and trace-symplectic inner products, in addition to the original Hermitian one. A refined lower bound on the minimum distance of the resulting quantum codes is established and illustrated. We report numerous record breaking quantum codes from our randomized search for inclusion in the updated online database.
- [23] arXiv:2405.15789 (cross-list from cs.AI) [pdf, ps, html, other]
-
Title: Semantic Objective Functions: A distribution-aware method for adding logical constraints in deep learningComments: 12 pages,4 figuresSubjects: Artificial Intelligence (cs.AI); Information Theory (cs.IT); Machine Learning (cs.LG); Logic in Computer Science (cs.LO)
Issues of safety, explainability, and efficiency are of increasing concern in learning systems deployed with hard and soft constraints. Symbolic Constrained Learning and Knowledge Distillation techniques have shown promising results in this area, by embedding and extracting knowledge, as well as providing logical constraints during neural network training. Although many frameworks exist to date, through an integration of logic and information geometry, we provide a construction and theoretical framework for these tasks that generalize many approaches. We propose a loss-based method that embeds knowledge-enforces logical constraints-into a machine learning model that outputs probability distributions. This is done by constructing a distribution from the external knowledge/logic formula and constructing a loss function as a linear combination of the original loss function with the Fisher-Rao distance or Kullback-Leibler divergence to the constraint distribution. This construction includes logical constraints in the form of propositional formulas (Boolean variables), formulas of a first-order language with finite variables over a model with compact domain (categorical and continuous variables), and in general, likely applicable to any statistical model that was pretrained with semantic information. We evaluate our method on a variety of learning tasks, including classification tasks with logic constraints, transferring knowledge from logic formulas, and knowledge distillation from general distributions.
- [24] arXiv:2405.16033 (cross-list from cs.DB) [pdf, ps, other]
-
Title: Wrangling Data Issues to be Wrangled: Literature Review, Taxonomy, and Industry Case StudySubjects: Databases (cs.DB); Information Theory (cs.IT)
Data quality is vital for user experience in products reliant on data. As solutions for data quality problems, researchers have developed various taxonomies for different types of issues. However, although some of the existing taxonomies are near-comprehensive, the over-complexity has limited their actionability in data issue solution development. Hence, recent researchers issued new sets of data issue categories that are more concise for better usability. Although more concise, modern data issue labeling's over-catering to the solution systems may sometimes cause the taxonomy to be not mutually exclusive. Consequently, different categories sometimes overlap in determining the issue types, or the same categories share different definitions across research. This hinders solution development and confounds issue detection. Therefore, based on observations from a literature review and feedback from our industry partner, we propose a comprehensive taxonomy of data quality issues from two distinct dimensions: the attribute dimension represents the intrinsic characteristics and the outcome dimension that indicates the manifestation of the issues. With the categories redefined, we labeled the reported data issues in our industry partner's data warehouse. The labeled issues provide us with a general idea of the distributions of each type of problem and which types of issues require the most effort and care to deal with. Our work aims to address a widely generalizable taxonomy rule in modern data quality issue engineering and helps practitioners and researchers understand their data issues and estimate the efforts required for issue fixing.
- [25] arXiv:2405.16227 (cross-list from cs.CR) [pdf, ps, other]
-
Title: SIGNLINE: Digital signature scheme based on linear equations cryptosystemSubjects: Cryptography and Security (cs.CR); Information Theory (cs.IT); Group Theory (math.GR)
The paper explores a novel cryptosystem for digital signatures based on linear equa-tions for logarithmic signatures. A logarithmic signature serves as a fundamental cryptographic primitive, characterized by properties such as nonlinearity, non-commutability, unidirectionality, and key-dependent factorability. The proposed cryptosystem ensures the secrecy of logarithmic signatures through its foundation in linear equations. Quantum security is achieved by eliminating any possible mapping between the input and output of the logarithmic signature, thereby rendering Grover's quantum attack ineffective. The public key sizes for the NIST security levels of 128, 192, and 256 bits are 1, 1.5, and 2 KB, respectively. The algorithm demonstrates scalability concerning computational costs, memory usage, and hardware limitations without compromising security. Its primary operation involves bitwise XOR over logarithmic arrays of 8, 16, 32, and 64 bits.
- [26] arXiv:2405.16881 (cross-list from cs.CC) [pdf, ps, html, other]
-
Title: Half-duplex communication complexity with adversary can be less than the classical communication complexitySubjects: Computational Complexity (cs.CC); Information Theory (cs.IT)
Half-duplex communication complexity with adversary was defined in [Hoover, K., Impagliazzo, R., Mihajlin, I., Smal, A. V. Half-Duplex Communication Complexity, ISAAC 2018.] Half-duplex communication protocols generalize classical protocols defined by Andrew Yao in [Yao, A. C.-C. Some Complexity Questions Related to Distributive Computing (Preliminary Report), STOC 1979]. It has been unknown so far whether the communication complexities defined by these models are different or not. In the present paper we answer this question: we exhibit a function whose half-duplex communication complexity with adversary is strictly less than its classical communication complexity.
Cross submissions for Tuesday, 28 May 2024 (showing 5 of 5 entries )
- [27] arXiv:1810.11826 (replaced) [pdf, ps, html, other]
-
Title: $m$-adic residue codes over $\mathbb{F}_q[v]/(v^s-v)$ and their application to quantum codesJournal-ref: K\"{u}r\"{u}z, F., Sar\i , M., \& K\"{o}ro\u{g}lu, M.E. (2022). $m$-adic residue codes over $\mathbb{F}_{q}[v]/(v^{s}-v)$ and their application to quantum codes, Quantum Information \& Computation, 22(5-6), 427-439Subjects: Information Theory (cs.IT)
Due to their rich algebraic structure, cyclic codes have a great deal of significance amongst linear codes. Duadic codes are the generalization of the quadratic residue codes, a special case of cyclic codes. The $m$-adic residue codes are the generalization of the duadic codes. The aim of this paper is to study the structure of the $m$-adic residue codes over the quotient ring $\frac{{{\mathbb{F}_q}\left[ v \right]}}{{\left\langle {{v^s} - v} \right\rangle }}$. We determine the idempotent generators of the $m$-adic residue codes over $\frac{{{\mathbb{F}_q}\left[ v \right]}}{{\left\langle {{v^s} - v} \right\rangle }}$. We obtain some parameters of optimal $m$-adic residue codes over $\frac{{{\mathbb{F}_q}\left[ v \right]}}{{\left\langle {{v^s} - v} \right\rangle }}$ with respect to Griesmer bound for rings. Furthermore, we derive a condition for $m$-adic residue codes over $\frac{{{\mathbb{F}_q}\left[ v \right]}}{{\left\langle {{v^s} - v} \right\rangle }}$ to contain their dual. By making use of a preserving-orthogonality Gray map, we construct a family of quantum error correcting codes from the Gray images of dual-containing $m$-adic residue codes over $\frac{{{\mathbb{F}_q}\left[ v \right]}}{{\left\langle {{v^s} - v} \right\rangle }}$ and give some examples to illustrate our findings.
- [28] arXiv:2208.06177 (replaced) [pdf, ps, html, other]
-
Title: Status Update Control and Analysis under Two-Way DelaySubjects: Information Theory (cs.IT)
We study status updating under two-way delay in a system consisting of a sampler, a sink, and a controller residing at the sink. The controller controls the sampling process by sending request packets to the sampler. Upon receiving a request, the sampler generates a sample and transmits the status update packet to the sink. Transmissions of both request and status update packets encounter random delays. We develop optimal control policies to minimize the average age of information (AoI) using the tools of Markov decision processes in two scenarios. We begin with the system having at most one active request, i.e., a generated request for which the sink has not yet received a status update packet. Then, as the main distinctive feature of this paper, we initiate pipelining-type status updating by studying a system having at most two active requests. Furthermore, we conduct AoI analysis by deriving the average AoI expressions for the Zero-Wait-1, Zero-Wait-2, and Wait-1 policies. According to the Zero-Wait-1 policy, whenever a status update packet is delivered to the sink, a new request packet is inserted into the system. The Zero-Wait-2 policy operates similarly, except that the system holds two active requests. According to the Wait-1 policy, whenever a status update packet is delivered to the sink, a new request is sent after a waiting time which is a function of the current AoI. Numerical results illustrate the performance of each status updating policy under different system parameter values.
- [29] arXiv:2303.14086 (replaced) [pdf, ps, html, other]
-
Title: Finite Field Multiple AccessComments: 38 pages, 11 figuresSubjects: Information Theory (cs.IT)
In the past several decades, various techniques have been developed and used for multiple-access (MA) communications. With the new applications for 6G, it is desirable to find new resources, physical or virtual, to confront the fast development of MA communication systems. For binary source transmission, this paper proposes an element-pair (EP) coding scheme for supporting massive users with short packet traffic, which solves the finite blocklength (FBL) of multiuser reliability transmission problem. Each user is assigned a unique EP, and the collection of EPs assigned to the users possesses the unique sum-pattern mapping (USPM) structural property. We present methods for constructing symbol-wise EP codes with USPM structural property based on the prime field and extension field of prime field, respectively. Based on the orthogonal EP code constructed over GF($2^m$), we propose finite-field MA (FFMA) systems over a Gaussian multiple-access channel (GMAC), including both the sparse-form and diagonal-from structures. The proposed FFMA is then applied to network layer and forms network FFMA systems for pure digital networks, in which an EP is viewed as a virtual resource block (VRB). Simulation results show that the error performance of the proposed FFMA over a GMAC can approach the error performance as that of the single-user transmission.
- [30] arXiv:2312.06504 (replaced) [pdf, ps, html, other]
-
Title: An infinite class of quantum codes derived from duadic constacyclic codesReza Dastbasteh, Josu Etxezarreta Martinez, Andrew Nemec, Antonio deMarti iOlius, Pedro Crespo BofillComments: 31 pages, 2 tablesSubjects: Information Theory (cs.IT)
We present a family of quantum stabilizer codes using the structure of duadic constacyclic codes over $\mathbb{F}_4$. Within this family, quantum codes can possess varying dimensions, and their minimum distances are lower bounded by a square root bound. For each fixed dimension, this allows us to construct an infinite sequence of binary quantum codes with a growing minimum distance. Additionally, we prove that this family of quantum codes includes an infinite subclass of degenerate codes. We also introduce a technique for extending splittings of duadic constacyclic codes, providing new insights into the minimum distance and minimum odd-like weight of specific duadic constacyclic codes. Finally, we provide numerical examples of some quantum codes with short lengths within this family.
- [31] arXiv:2401.07100 (replaced) [pdf, ps, html, other]
-
Title: Meta-Learning for Resource Allocation in Uplink Multi STAR-RIS-aided NOMA SystemSubjects: Information Theory (cs.IT); Signal Processing (eess.SP)
Simultaneously transmitting and reflecting reconfigurable intelligent surface (STAR-RIS) is a novel technology which enables the full-space coverage. In this letter, a multi STAR-RIS-aided system using non-orthogonal multiple access in an uplink transmission is considered, where the multi-order reflections among multiple STAR-RISs assist the transmission from the single-antenna users to the multi-antenna base station. Specifically, the total sum rate maximization problem is solved by jointly optimizing the active beamforming, power allocation, transmission and reflection beamforming at the STAR-RIS, and user-STAR-RIS assignment. To solve the non-convex optimization problem, a novel deep reinforcement learning algorithm is proposed which integrates meta-learning and deep deterministic policy gradient (DDPG), denoted by Meta-DDPG. Numerical results demonstrate that our proposed Meta-DDPG algorithm outperforms the conventional DDPG algorithm with $53\%$ improvement, while multi-order reflections among multi STAR-RISs yields to $14.1\%$ enhancement in the total data rate.
- [32] arXiv:2401.10688 (replaced) [pdf, ps, html, other]
-
Title: Unraveling codes: fast, robust, beyond-bound error correction for DRAMComments: Changes vs first arxiv version: wordsmithing, typo corrections and citation fixesSubjects: Information Theory (cs.IT); Hardware Architecture (cs.AR)
Generalized Reed-Solomon (RS) codes are a common choice for efficient, reliable error correction in memory and communications systems. These codes add $2t$ extra parity symbols to a block of memory, and can efficiently and reliably correct up to $t$ symbol errors in that block. Decoding is possible beyond this bound, but it is imperfectly reliable and often computationally expensive. Beyond-bound decoding is an important problem to solve for error-correcting Dynamic Random Access Memory (DRAM). These memories are often designed so that each access touches two extra memory devices, so that a failure in any one device can be corrected. But system architectures increasingly require DRAM to store metadata in addition to user data. When the metadata replaces parity data, a single-device failure is then beyond-bound. An error-correction system can either protect each access with a single RS code, or divide it into several segments protected with a shorter code, usually in an Interleaved Reed-Solomon (IRS) configuration. The full-block RS approach is more reliable, both at correcting errors and at preventing silent data corruption (SDC). The IRS option is faster, and is especially efficient at beyond-bound correction of single- or double-device failures. Here we describe a new family of "unraveling" Reed-Solomon codes that bridges the gap between these options. Our codes are full-block generalized RS codes, but they can also be decoded using an IRS decoder. As a result, they combine the speed and beyond-bound correction capabilities of interleaved codes with the robustness of full-block codes, including the ability of the latter to reliably correct failures across multiple devices. We show that unraveling codes are an especially good fit for high-reliability DRAM error correction.
- [33] arXiv:2401.11390 (replaced) [pdf, ps, html, other]
-
Title: A Transformation of Repairing Reed-Solomon Codes from Rack-Aware Storage Model to Homogeneous Storage ModelComments: 11 pages, 2 figures, 3 tablesSubjects: Information Theory (cs.IT)
In this paper, we address the node repair problem of Reed-Solomon (RS) coded distributed storage systems. Specifically, to overcome the challenges of multiple-node failures of RS codes under the rack-aware storage model, we employ good polynomials to guide the placement of the conventional RS codes into racks and then propose a novel repair framework for the resultant rack-aware RS codes, which can transform its repair to that under the homogeneous storage model. As applications of our repair framework, firstly we present the repair scheme of multiple-node failures for some existing constructions, which only have non-trivial solutions for repairing a single-node failure before. Secondly, we deduce several new constructions of rack-aware RS codes supporting the repair of multiple-node failures within a single rack and across multiple racks respectively.
- [34] arXiv:2401.15666 (replaced) [pdf, ps, html, other]
-
Title: Error-Correcting Codes for Combinatorial Composite DNASubjects: Information Theory (cs.IT)
Data storage in DNA is developing as a possible solution for archival digital data. Recently, to further increase the potential capacity of DNA-based data storage systems, the combinatorial composite DNA synthesis method was suggested. This approach extends the DNA alphabet by harnessing short DNA fragment reagents, known as shortmers. The shortmers are building blocks of the alphabet symbols, consisting of a fixed number of shortmers. Thus, when information is read, it is possible that one of the shortmers that forms part of the composition of a symbol is missing and therefore the symbol cannot be determined. In this paper, we model this type of error as a type of asymmetric error and propose code constructions that can correct such errors in this setup. We also provide a lower bound on the redundancy of such error-correcting codes and give an explicit encoder and decoder pair for our construction. Our suggested error model is also supported by an analysis of data from actual experiments that produced DNA according to the combinatorial scheme. Lastly, we also provide a statistical evaluation of the probability of observing such error events, as a function of read depth.
- [35] arXiv:2403.12268 (replaced) [pdf, ps, html, other]
-
Title: Near-Field Channel Modeling for Electromagnetic Information TheoryComments: In this paper, we propose the near-field channel model for EIT based on electromagnetic scattering theory. Then, we derive the analytical expression of the correlation function of the fields and analyze the characteristics of it. Finally, we design a channel estimation scheme for near-field scenarioSubjects: Information Theory (cs.IT); Signal Processing (eess.SP)
Electromagnetic information theory (EIT) is one of the emerging topics for 6G communication due to its potential to reveal the performance limit of wireless communication systems. For EIT, the research foundation is reasonable and accurate channel modeling. Existing channel modeling works for EIT in non-line-of-sight (NLoS) scenario focus on far-field modeling, which can not accurately capture the characteristics of the channel in near-field. In this paper, we propose the near-field channel model for EIT based on electromagnetic scattering theory. We model the channel by using non-stationary Gaussian random fields and derive the analytical expression of the correlation function of the fields. Furthermore, we analyze the characteristics of the proposed channel model, e.g., channel degrees of freedom (DoF). Finally, we design a channel estimation scheme for near-field scenario by integrating the electromagnetic prior information of the proposed model. Numerical analysis verifies the correctness of the proposed scheme and shows that it can outperform existing schemes like least square (LS) and orthogonal matching pursuit (OMP).
- [36] arXiv:2403.20228 (replaced) [pdf, ps, html, other]
-
Title: Cooperative Sensing and Communication for ISAC Networks: Performance Analysis and OptimizationComments: 7 pages, 5 figures, this paper has been submitted to IEEE for possible publicationSubjects: Information Theory (cs.IT); Signal Processing (eess.SP)
In this work, we study integrated sensing and communication (ISAC) networks intending to effectively balance sensing and communication (S&C) performance at the network level. Through the simultaneous utilization of multi-point (CoMP) coordinated joint transmission and distributed multiple-input multiple-output (MIMO) radar techniques, we propose a cooperative networked ISAC scheme to enhance both S&C services. Then, the tool of stochastic geometry is exploited to capture the S&C performance, which allows us to illuminate key cooperative dependencies in the ISAC network. Remarkably, the derived expression of the Cramer-Rao lower bound (CRLB) of the localization accuracy unveils a significant finding: Deploying $N$ ISAC transceivers yields an enhanced sensing performance across the entire network, in accordance with the $\ln^2N$ scaling law. Simulation results demonstrate that compared to the time-sharing scheme, the proposed cooperative ISAC scheme can effectively improve the average data rate and reduce the CRLB.
- [37] arXiv:2404.14514 (replaced) [pdf, ps, html, other]
-
Title: Cooperative ISAC Networks: Performance Analysis, Scaling Laws and OptimizationComments: 13 pages, 10 figures, this work has been submitted to IEEE for possible publication. arXiv admin note: text overlap with arXiv:2403.20228Subjects: Information Theory (cs.IT); Signal Processing (eess.SP)
Integrated sensing and communication (ISAC) networks are investigated with the objective of effectively balancing the sensing and communication (S&C) performance at the network level. Through the simultaneous utilization of multi-point (CoMP) coordinated joint transmission and distributed multiple-input multiple-output (MIMO) radar techniques, we propose an innovative networked ISAC scheme, where multiple transceivers are employed for collaboratively enhancing the S&C services. Then, the potent tool of stochastic geometry is exploited for characterizing the S&C performance, which allows us to illuminate the key cooperative dependencies in the ISAC network and optimize salient network-level parameters. Remarkably, the Cramer-Rao lower bound (CRLB) expression of the localization accuracy derived unveils a significant finding: Deploying N ISAC transceivers yields an enhanced average cooperative sensing performance across the entire network, in accordance with the ln^2N scaling law. Crucially, this scaling law is less pronounced in comparison to the performance enhancement of N^2 achieved when the transceivers are equidistant from the target, which is primarily due to the substantial path loss from the distant base stations (BSs) and leads to reduced contributions to sensing performance gain. Moreover, we derive a tight expression of the communication rate, and present a low-complexity algorithm to determine the optimal cooperative cluster size. Based on our expression derived for the S&C performance, we formulate the optimization problem of maximizing the network performance in terms of two joint S&C metrics. To this end, we jointly optimize the cooperative BS cluster sizes and the transmit power to strike a flexible tradeoff between the S&C performance.
- [38] arXiv:2405.07666 (replaced) [pdf, ps, html, other]
-
Title: New Solutions to Delsarte's Dual Linear ProgramsSubjects: Information Theory (cs.IT); Discrete Mathematics (cs.DM)
Understanding the maximum size of a code with a given minimum distance is a major question in computer science and discrete mathematics. The most fruitful approach for finding asymptotic bounds on such codes is by using Delsarte's theory of association schemes. With this approach, Delsarte constructs a linear program such that its maximum value is an upper bound on the maximum size of a code with a given minimum distance. Bounding this value can be done by finding solutions to the corresponding dual linear program. Delsarte's theory is very general and goes way beyond binary codes. In this work, we provide universal bounds in the framework of association schemes that generalize the Elias-Bassalygo bound, which can be applied to any association scheme constructed from a distance function. These bounds are obtained by constructing new solutions to Delsarte's dual linear program. We instantiate these results and we recover known bounds for $q$-ary codes and for constant-weight binary codes. Our other contribution is to recover, for essentially any $Q$-polynomial scheme, MRRW-type solutions to Delsarte's dual linear program which are inspired by the Laplacian approach of Friedman and Tillich instead of using the Christoffel-Darboux formulas. We show in particular how the second linear programming bound can be interpreted in this framework.
- [39] arXiv:2405.08584 (replaced) [pdf, ps, html, other]
-
Title: When Do Low-Rate Concatenated Codes Approach The Gilbert-Varshamov Bound?Subjects: Information Theory (cs.IT); Computational Complexity (cs.CC)
The Gilbert--Varshamov (GV) bound is a classical existential result in coding theory. It implies that a random linear binary code of rate $\epsilon^2$ has relative distance at least $\frac{1}{2} - O(\epsilon)$ with high probability. However, it is a major challenge to construct explicit codes with similar parameters.
One hope to derandomize the Gilbert--Varshamov construction is with code concatenation: We begin with a (hopefully explicit) outer code ${C}_\mathrm{out}$ over a large alphabet, and concatenate that with a small binary random linear code ${C}_\mathrm{in}$. It is known that when we use independent small codes for each coordinate, then the result lies on the GV bound with high probability, but this still uses a lot of randomness. In this paper, we consider the question of whether code concatenation with a single random linear inner code ${C}_\mathrm{in}$ can lie on the GV bound; and if so what conditions on ${C}_\mathrm{out}$ are sufficient for this.
We show that first, there do exist linear outer codes ${C}_\mathrm{out}$ that are "good" for concatenation in this sense (in fact, most linear codes codes are good). We also provide two sufficient conditions for ${C}_\mathrm{out}$, so that if ${C}_\mathrm{out}$ satisfies these, ${C}_\mathrm{out}\circ {C}_\mathrm{in}$ will likely lie on the GV bound. We hope that these conditions may inspire future work towards constructing explicit codes ${C}_\mathrm{out}$. - [40] arXiv:2405.10007 (replaced) [pdf, ps, html, other]
-
Title: Sampling Theorem and interpolation formula for non-vanishing signalsComments: arXiv admin note: substantial text overlap with arXiv:2405.05566Subjects: Information Theory (cs.IT)
The paper establishes an analog Whittaker-Shannon-Kotelnikov sampling theorem with fast decreasing coefficient, as well as a new modification of the corresponding interpolation formula applicable for general type non-vanishing bounded continuous signals.
- [41] arXiv:2405.10496 (replaced) [pdf, ps, html, other]
-
Title: Electromagnetic Information Theory for Holographic MIMO CommunicationsLi Wei, Tierui Gong, Chongwen Huang, Zhaoyang Zhang, Wei E. I. Sha, Zhi Ning Chen, Linglong Dai, Merouane Debbah, Chau YuenSubjects: Information Theory (cs.IT); Signal Processing (eess.SP)
Holographic multiple-input multiple-output (HMIMO) utilizes a compact antenna array to form a nearly continuous aperture, thereby enhancing higher capacity and more flexible configurations compared with conventional MIMO systems, making it attractive in current scientific research. Key questions naturally arise regarding the potential of HMIMO to surpass Shannon's theoretical limits and how far its capabilities can be extended. However, the traditional Shannon information theory falls short in addressing these inquiries because it only focuses on the information itself while neglecting the underlying carrier, electromagnetic (EM) waves, and environmental interactions. To fill up the gap between the theoretical analysis and the practical application for HMIMO systems, we introduce electromagnetic information theory (EIT) in this paper. This paper begins by laying the foundation for HMIMO-oriented EIT, encompassing EM wave equations and communication regions. In the context of HMIMO systems, the resultant physical limitations are presented, involving Chu's limit, Harrington's limit, Hannan's limit, and the evaluation of coupling effects. Field sampling and HMIMO-assisted oversampling are also discussed to guide the optimal HMIMO design within the EIT framework. To comprehensively depict the EM-compliant propagation process, we present the approximate and exact channel modeling approaches in near-/far-field zones. Furthermore, we discuss both traditional Shannon's information theory, employing the probabilistic method, and Kolmogorov information theory, utilizing the functional analysis, for HMIMO-oriented EIT systems.
- [42] arXiv:2308.13728 (replaced) [pdf, ps, html, other]
-
Title: Indicator functions, v-numbers and Gorenstein rings in the theory of projective Reed-Muller-type codesManuel González-Sarabia, Humberto Muñoz-George, Jorge A. Ordaz, Eduardo Sáenz-de-Cabezón, Rafael H. VillarrealComments: 33 pagesSubjects: Commutative Algebra (math.AC); Information Theory (cs.IT); Algebraic Geometry (math.AG)
For projective Reed--Muller-type codes we give a global duality criterion in terms of the v-number and the Hilbert function of a vanishing ideal. As an application, we provide a global duality theorem for projective Reed--Muller-type codes over Gorenstein vanishing ideals, generalizing the known case where the vanishing ideal is a complete intersection. We classify self dual Reed-Muller-type codes over Gorenstein ideals using the regularity and a parity check matrix. For projective evaluation codes, we give a duality theorem inspired by that of affine evaluation codes. We show how to compute the regularity index of the $r$-th generalized Hamming weight function in terms of the standard indicator functions of the set of evaluation points.
- [43] arXiv:2312.03867 (replaced) [pdf, ps, html, other]
-
Title: Multi-Group Fairness Evaluation via Conditional Value-at-Risk TestingComments: Accepted for publication in the IEEE Journal on Selected Areas in Information Theory (JSAIT)Subjects: Machine Learning (cs.LG); Computers and Society (cs.CY); Information Theory (cs.IT); Machine Learning (stat.ML)
Machine learning (ML) models used in prediction and classification tasks may display performance disparities across population groups determined by sensitive attributes (e.g., race, sex, age). We consider the problem of evaluating the performance of a fixed ML model across population groups defined by multiple sensitive attributes (e.g., race and sex and age). Here, the sample complexity for estimating the worst-case performance gap across groups (e.g., the largest difference in error rates) increases exponentially with the number of group-denoting sensitive attributes. To address this issue, we propose an approach to test for performance disparities based on Conditional Value-at-Risk (CVaR). By allowing a small probabilistic slack on the groups over which a model has approximately equal performance, we show that the sample complexity required for discovering performance violations is reduced exponentially to be at most upper bounded by the square root of the number of groups. As a byproduct of our analysis, when the groups are weighted by a specific prior distribution, we show that Rényi entropy of order 2/3 of the prior distribution captures the sample complexity of the proposed CVaR test algorithm. Finally, we also show that there exists a non-i.i.d. data collection strategy that results in a sample complexity independent of the number of groups.
- [44] arXiv:2402.02001 (replaced) [pdf, ps, html, other]
-
Title: PANDA: Query Evaluation in Submodular WidthSubjects: Databases (cs.DB); Information Theory (cs.IT)
In recent years, several information-theoretic upper bounds have been introduced on the output size and evaluation cost of database join queries. These bounds vary in their power depending on both the type of statistics on input relations and the query plans that they support. This motivated the search for algorithms that can compute the output of a join query in times that are bounded by the corresponding information-theoretic bounds. In this paper, we describe $\panda$, an algorithm that takes a Shannon-inequality that underlies the bound, and translates each proof step into an algorithmic step corresponding to some database operation. \panda computes answers to a conjunctive query in time given by the the submodular width plus the output size of the query. The version in this paper represents a significant simplification of the original version in \cite{DBLP:conf/pods/Khamis0S17}.
- [45] arXiv:2403.02187 (replaced) [pdf, ps, other]
-
Title: Mutual Information Estimation via Normalizing FlowsComments: 20 pages, 6 figuresSubjects: Machine Learning (cs.LG); Information Theory (cs.IT); Machine Learning (stat.ML)
We propose a novel approach to the problem of mutual information (MI) estimation via introducing a family of estimators based on normalizing flows. The estimator maps original data to the target distribution, for which MI is easier to estimate. We additionally explore the target distributions with known closed-form expressions for MI. Theoretical guarantees are provided to demonstrate that our approach yields MI estimates for the original data. Experiments with high-dimensional data are conducted to highlight the practical advantages of the proposed method.
- [46] arXiv:2404.16101 (replaced) [pdf, ps, other]
-
Title: Multivariate FidelitiesComments: v2: 100 pages, 1 figure; includes additional content on multivariate geometric fidelities and secrecy-based multivariate fidelitiesSubjects: Quantum Physics (quant-ph); Information Theory (cs.IT); Mathematical Physics (math-ph); Operator Algebras (math.OA)
The main contribution of our paper is to introduce a number of multivariate quantum fidelities and show that they satisfy several desirable properties that are natural extensions of those of the Uhlmann and Holevo fidelities. We propose three variants that reduce to the average pairwise fidelity for commuting states: average pairwise $z$-fidelities, the multivariate semi-definite programming (SDP) fidelity, and a multivariate fidelity inspired by an existing secrecy measure. The second one is obtained by extending the SDP formulation of the Uhlmann fidelity to more than two states. All three of these variants satisfy the following properties: (i) reduction to multivariate classical fidelities for commuting states, (ii) the data-processing inequality, (iii) invariance under permutations of the states, (iv) its values are in the interval $[0,1]$; they are faithful, that is, their values are equal to one if and only if all the states are equal, and they satisfy orthogonality, that is their values are equal to zero if and only if the states are mutually orthogonal to each other, (v) direct-sum property, (vi) joint concavity, and (vii) uniform continuity bounds under certain conditions. Furthermore, we establish inequalities relating these different variants, indeed clarifying that all these definitions coincide with the average pairwise fidelity for commuting states. Lastly, we introduce another multivariate fidelity called multivariate log-Euclidean fidelity, which is a quantum generalization of the Matusita multivariate fidelity. We also show that it satisfies most of the desirable properties listed above, it is a function of a multivariate log-Euclidean divergence, and has an operational interpretation in terms of quantum hypothesis testing with an arbitrarily varying null hypothesis.
- [47] arXiv:2405.12807 (replaced) [pdf, ps, html, other]
-
Title: FAdam: Adam is a natural gradient optimizer using diagonal empirical Fisher informationComments: 19 pages, 1 figures, 5 tablesSubjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Information Theory (cs.IT)
This paper establishes a mathematical foundation for the Adam optimizer, elucidating its connection to natural gradient descent through Riemannian and information geometry. We rigorously analyze the diagonal empirical Fisher information matrix (FIM) in Adam, clarifying all detailed approximations and advocating for the use of log probability functions as loss, which should be based on discrete distributions, due to the limitations of empirical FIM. Our analysis uncovers flaws in the original Adam algorithm, leading to proposed corrections such as enhanced momentum calculations, adjusted bias corrections, adaptive epsilon, and gradient clipping. We refine the weight decay term based on our theoretical framework. Our modified algorithm, Fisher Adam (FAdam), demonstrates superior performance across diverse domains including LLM, ASR, and VQ-VAE, achieving state-of-the-art results in ASR.
- [48] arXiv:2405.15413 (replaced) [pdf, ps, html, other]
-
Title: MambaVC: Learned Visual Compression with Selective State SpacesShiyu Qin, Jinpeng Wang, Yimin Zhou, Bin Chen, Tianci Luo, Baoyi An, Tao Dai, Shutao Xia, Yaowei WangComments: 17pages,15 figuresSubjects: Image and Video Processing (eess.IV); Computer Vision and Pattern Recognition (cs.CV); Information Theory (cs.IT)
Learned visual compression is an important and active task in multimedia. Existing approaches have explored various CNN- and Transformer-based designs to model content distribution and eliminate redundancy, where balancing efficacy (i.e., rate-distortion trade-off) and efficiency remains a challenge. Recently, state-space models (SSMs) have shown promise due to their long-range modeling capacity and efficiency. Inspired by this, we take the first step to explore SSMs for visual compression. We introduce MambaVC, a simple, strong and efficient compression network based on SSM. MambaVC develops a visual state space (VSS) block with a 2D selective scanning (2DSS) module as the nonlinear activation function after each downsampling, which helps to capture informative global contexts and enhances compression. On compression benchmark datasets, MambaVC achieves superior rate-distortion performance with lower computational and memory overheads. Specifically, it outperforms CNN and Transformer variants by 9.3% and 15.6% on Kodak, respectively, while reducing computation by 42% and 24%, and saving 12% and 71% of memory. MambaVC shows even greater improvements with high-resolution images, highlighting its potential and scalability in real-world applications. We also provide a comprehensive comparison of different network designs, underscoring MambaVC's advantages.