Disordered Systems and Neural Networks
- [1] arXiv:2405.13240 [pdf, ps, html, other]
-
Title: Quantum Hopfield Model with Dilute MemoriesSubjects: Disordered Systems and Neural Networks (cond-mat.dis-nn)
We discuss adiabatic spectra and dynamics of the quantum, i.e. transverse field, Hopfield model with dilute memories (the number of stored patterns $p < log_2 N$, where $N$ is the number of qubits). At some critical transverse field the model undergoes the quantum phase transition from the ordered to the paramagnetic state. The corresponding critical exponents are calculated and used to determine efficiency of quantum annealing protocols. We also discuss implications of these results for the quantum annealing of generic spin glass models.
- [2] arXiv:2405.13277 [pdf, ps, html, other]
-
Title: Topology Obstructing Anderson Localization of LightComments: 4 pages, supplemental materialSubjects: Disordered Systems and Neural Networks (cond-mat.dis-nn); Mesoscale and Nanoscale Physics (cond-mat.mes-hall); Optics (physics.optics)
Light propagation in random media is notorious for its resilience to (Anderson) wave localization. We here argue that in materials with slow helicity relaxation previously unnoticed principles of topology reinforce delocalization. We show that the effective electromagnetic action is governed by principles, identical to those protecting delocalization at topological insulator surface states. The length scales over which these exert a protective influence depend on the conservation of helicity, leading to the prediction that light scattering in media with, e.g., strong magnetic and electric dipolar resonances evades localization over large scales.
- [3] arXiv:2405.13508 [pdf, ps, html, other]
-
Title: Furutsu-Novikov--like cross-correlation--response relations for systems driven by shot noiseComments: 12 pages, 9 figuresSubjects: Disordered Systems and Neural Networks (cond-mat.dis-nn); Biological Physics (physics.bio-ph)
We consider a dynamic system that is driven by an intensity-modulated Poisson process with intensity $\Lambda(t)=\lambda(t)+\varepsilon\nu(t)$. We derive an exact relation between the input-output cross-correlation in the spontaneous state ($\varepsilon=0$) and the linear response to the modulation ($\varepsilon>0$). This can be regarded as a variant of the Furutsu-Novikov theorem for the case of shot noise. As we show, the relation is still valid in the presence of additional independent noise. Furthermore, we derive an extension to Cox-process input, i.e. to colored shot noise. We discuss applications to particle detection and to neuroscience. Using the new relation, we obtain a fluctuation-response-relation for a leaky integrate-and-fire neuron. We also show how the new relation can be used in a remote control problem in a recurrent neural network. The relations are numerically tested for both stationary and non-stationary dynamics. Lastly, extensions to marked Poisson processes and to higher-order statistics are presented.
- [4] arXiv:2405.14215 [pdf, ps, html, other]
-
Title: Exact finite-size scaling spectra in the quenched and annealed Sherrington-Kirkpatrick spin glassComments: 17 pages, 8 figures, Submission to SciPostSubjects: Disordered Systems and Neural Networks (cond-mat.dis-nn); Statistical Mechanics (cond-mat.stat-mech)
Fine resolution of the discrete eigenvalues at the spectral edge of an $N\times N$ random matrix is required in many applications. Starting from a finite-size scaling ansatz for the Stieltjes transform of the maximum likelihood spectrum, we show that the scaling function satisfies a first-order ODE of the Riccati type. Further transformation yields a linear second-order ODE for the characteristic function, whose nodes yield the exact locations of leading eigenvalues. With this new technique, we examine in detail the spectral crossover of the annealed Sherrington-Kirkpatrick (SK) spin glass model where a gap develops below a certain critical temperature. Our analysis yields analytic predictions for the finite-size scaling of the spin condensation phenomenon in both quenched and annealed SK models, which are then compared and validated in Monte Carlo simulations of the annealed model. More generally, rescaling of the spectral axis, adjusted to the distance of neighboring eigenvalues, offers a powerful strategy to deal with singularities that arise in the infinite size limit.
- [5] arXiv:2405.14289 [pdf, ps, html, other]
-
Title: Generating-functional analysis of random Lotka-Volterra systems: A step-by-step guideComments: 81 pages, 8 figuresSubjects: Disordered Systems and Neural Networks (cond-mat.dis-nn); Statistical Mechanics (cond-mat.stat-mech); Populations and Evolution (q-bio.PE)
This paper provides what is hopefully a self-contained set of notes describing the detailed steps of a generating-functional analysis of systems of generalised Lotka-Volterra equations with random interaction coefficients. Nothing in these notes is original, instead the generating-functional method (also known as the Martin-Siggia-Rose-DeDominic-Janssen formalism) and the resulting dynamic mean field theories have been used for the study of disordered systems and spin glasses for decades. But it is hard to find unifying sources which would allow a beginner to learn step-by-step how these methods can be used. My aim is to provide such a source. Most of the calculations are specific to generalised Lotka-Volterra systems, but much can be transferred to disordered systems in more general.
- [6] arXiv:2405.14376 [pdf, ps, html, other]
-
Title: Quantum Chaos in Random Ising NetworksAndrás Grabarits, Kasturi Ranjan Swain, Mahsa Seyed Heydari, Pranav Chandarana, Fernando J. Gómez-Ruiz, Adolfo del CampoSubjects: Disordered Systems and Neural Networks (cond-mat.dis-nn); Statistical Mechanics (cond-mat.stat-mech); Chaotic Dynamics (nlin.CD)
We report a systematic investigation of universal quantum chaotic signatures in the transverse field Ising model on an Erdős-Rényi network. This is achieved by studying local spectral measures such as the level spacing and the level velocity statistics. A spectral form factor analysis is also performed as a global measure, probing energy level correlations at arbitrary spectral distances. Our findings show that these measures capture the breakdown of chaotic behavior upon varying the connectivity and strength of the transverse field in various regimes. We demonstrate that the level spacing statistics and the spectral form factor signal this breakdown for sparsely and densely connected networks. The velocity statistics capture the surviving chaotic signatures in the sparse limit. However, these integrable-like regimes extend over a vanishingly small segment in the full range of connectivity.
- [7] arXiv:2405.14531 [pdf, ps, html, other]
-
Title: Low-resolution descriptions of model neural activity reveal hidden features and underlying system propertiesSubjects: Disordered Systems and Neural Networks (cond-mat.dis-nn); Computational Physics (physics.comp-ph)
The analysis of complex systems such as neural networks is made particularly difficult by the overwhelming number of their interacting components. In the absence of prior knowledge, identifying a small but informative subset of network nodes on which the analysis should focus is a rather difficult task. In this work, we address this problem in the context of a Hopfield model, for which we seek representations given in terms of a subgroup of its neurons with the aim of identifying those that can reveal the largest amount of information about the system. Using a methodology based on concepts from information theory, we reveal that such optimised low-resolution representations are not only informative per se, but rather they also serve as probes of the neural network properties; these results show a tight and potentially fruitful relation between the level of detail at which the network is inspected and the type and amount of information that can be extracted from it.
New submissions for Friday, 24 May 2024 (showing 7 of 7 entries )
- [8] arXiv:2405.13098 (cross-list from physics.comp-ph) [pdf, ps, html, other]
-
Title: Glassy dynamics in deep neural networks: A structural comparisonSubjects: Computational Physics (physics.comp-ph); Disordered Systems and Neural Networks (cond-mat.dis-nn); Statistical Mechanics (cond-mat.stat-mech)
Deep Neural Networks (DNNs) share important similarities with structural glasses. Both have many degrees of freedom, and their dynamics are governed by a high-dimensional, non-convex landscape representing either the loss or energy, respectively. Furthermore, both experience gradient descent dynamics subject to noise. In this work we investigate, by performing quantitative measurements on realistic networks trained on the MNIST and CIFAR-10 datasets, the extent to which this qualitative similarity gives rise to glass-like dynamics in neural networks. We demonstrate the existence of a Topology Trivialisation Transition as well as the previously studied under-to-overparameterised transition analogous to jamming. By training DNNs with overdamped Langevin dynamics in the resulting disordered phases, we do not observe diverging relaxation times at non-zero temperature, nor do we observe any caging effects, in contrast to glass phenomenology. However, the weight overlap function follows a power law in time, with an exponent of approximately -0.5, in agreement with the Mode-Coupling Theory of structural glasses. In addition, the DNN dynamics obey a form of time-temperature superposition. Finally, dynamic heterogeneity and ageing are observed at low temperatures. These results highlight important and surprising points of both difference and agreement between the behaviour of DNNs and structural glasses.
- [9] arXiv:2405.13282 (cross-list from quant-ph) [pdf, ps, html, other]
-
Title: Quantum criticality of generalized Aubry-Andr\'{e} models with exact mobility edges using fidelity susceptibilityComments: 9 pages, 5 figuresJournal-ref: Phys. Rev. E 109, 054123 (2024)Subjects: Quantum Physics (quant-ph); Disordered Systems and Neural Networks (cond-mat.dis-nn)
In this study, we explore the quantum critical phenomena in generalized Aubry-André models, with a particular focus on the scaling behavior at various filling states. Our approach involves using quantum fidelity susceptibility to precisely identify the mobility edges in these systems. Through a finite-size scaling analysis of the fidelity susceptibility, we are able to determine both the correlation-length critical exponent and the dynamical critical exponent at the critical point of the generalized Aubry-André model. Based on the Diophantine equation conjecture, we can determines the number of subsequences of the Fibonacci sequence and the corresponding scaling functions for a specific filling fraction, as well as the universality class. Our findings demonstrate the effectiveness of employing the generalized fidelity susceptibility for the analysis of unconventional quantum criticality and the associated universal information of quasiperiodic systems in cutting-edge quantum simulation experiments.
- [10] arXiv:2405.13690 (cross-list from math.ST) [pdf, ps, html, other]
-
Title: The effect of regularization in high dimensional Cox regressionSubjects: Statistics Theory (math.ST); Disordered Systems and Neural Networks (cond-mat.dis-nn)
We investigate analytically the behaviour of the penalized maximum partial likelihood estimator (PMPLE). Our results are derived for a generic separable regularization, but we focus on the elastic net. This penalization is routinely adopted for survival analysis in the high dimensional regime, where the Maximum Partial Likelihood estimator (no regularization) might not even exist. Previous theoretical results require that the number $s$ of non-zero association coefficients is $O(n^{\alpha})$, with $\alpha \in (0,1)$ and $n$ the sample size. Here we accurately characterize the behaviour of the PMPLE when $s$ is proportional to $n$ via the solution of a system of six non-linear equations that can be easily obtained by fixed point iteration. These equations are derived by means of the replica method and under the assumption that the covariates $\mathbf{X}\in \mathbb{R}^p$ follow a multivariate Gaussian law with covariance $\mathbf{I}_p/p$.
The solution of the previous equations allows us to investigate the dependency of various metrics of interest and hence their dependency on the ratio $\zeta = p/n$, the fraction of true active components $\nu = s/p$, and the regularization strength. We validate our results by extensive numerical simulations. - [11] arXiv:2405.14216 (cross-list from cond-mat.supr-con) [pdf, ps, html, other]
-
Title: Emergence of spatial patterns and synchronization in superconducting time crystalsComments: 30 pages, 23 figuresSubjects: Superconductivity (cond-mat.supr-con); Disordered Systems and Neural Networks (cond-mat.dis-nn); Strongly Correlated Electrons (cond-mat.str-el); Quantum Physics (quant-ph)
We identify a time crystal phase characterized by a frequency half of the driving frequency in disordered superconductors by employing the time dependent Bogoliubov-de Gennes formalism at zero temperature with a periodically driven coupling constant. After a period of exponential increase of spatial inhomogeneities and exponential suppression of the order parameter amplitude, the time crystal develops islands of different sizes. Each of these islands is a time crystal with the same frequency albeit with a phase shift $\pi$ with respect to the homogeneous time crystal. After its emergence, the island gradually becomes smaller, though the phase shift persists, until it is abruptly synchronized at a time that it depends on its initial size. We find a critical disorder strength, still deep in the metallic phase, at which the time crystal phase terminates. For even stronger disorder, the order parameter oscillates with the driving frequency in regions where localization effects are not important.
- [12] arXiv:2405.14689 (cross-list from cs.LG) [pdf, ps, html, other]
-
Title: Cascade of phase transitions in the training of Energy-based modelsComments: 19 pages, 6 figuresSubjects: Machine Learning (cs.LG); Disordered Systems and Neural Networks (cond-mat.dis-nn); Statistical Mechanics (cond-mat.stat-mech)
In this paper, we investigate the feature encoding process in a prototypical energy-based generative model, the Restricted Boltzmann Machine (RBM). We start with an analytical investigation using simplified architectures and data structures, and end with numerical analysis of real trainings on real datasets. Our study tracks the evolution of the model's weight matrix through its singular value decomposition, revealing a series of phase transitions associated to a progressive learning of the principal modes of the empirical probability distribution. The model first learns the center of mass of the modes and then progressively resolve all modes through a cascade of phase transitions. We first describe this process analytically in a controlled setup that allows us to study analytically the training dynamics. We then validate our theoretical results by training the Bernoulli-Bernoulli RBM on real data sets. By using data sets of increasing dimension, we show that learning indeed leads to sharp phase transitions in the high-dimensional limit. Moreover, we propose and test a mean-field finite-size scaling hypothesis. This shows that the first phase transition is in the same universality class of the one we studied analytically, and which is reminiscent of the mean-field paramagnetic-to-ferromagnetic phase transition.
- [13] arXiv:2405.14830 (cross-list from hep-lat) [pdf, ps, html, other]
-
Title: Deep learning lattice gauge theoriesSubjects: High Energy Physics - Lattice (hep-lat); Disordered Systems and Neural Networks (cond-mat.dis-nn); Strongly Correlated Electrons (cond-mat.str-el); Machine Learning (cs.LG); High Energy Physics - Theory (hep-th)
Monte Carlo methods have led to profound insights into the strong-coupling behaviour of lattice gauge theories and produced remarkable results such as first-principles computations of hadron masses. Despite tremendous progress over the last four decades, fundamental challenges such as the sign problem and the inability to simulate real-time dynamics remain. Neural network quantum states have emerged as an alternative method that seeks to overcome these challenges. In this work, we use gauge-invariant neural network quantum states to accurately compute the ground state of $\mathbb{Z}_N$ lattice gauge theories in $2+1$ dimensions. Using transfer learning, we study the distinct topological phases and the confinement phase transition of these theories. For $\mathbb{Z}_2$, we identify a continuous transition and compute critical exponents, finding excellent agreement with existing numerics for the expected Ising universality class. In the $\mathbb{Z}_3$ case, we observe a weakly first-order transition and identify the critical coupling. Our findings suggest that neural network quantum states are a promising method for precise studies of lattice gauge theory.
Cross submissions for Friday, 24 May 2024 (showing 6 of 6 entries )
- [14] arXiv:2106.05330 (replaced) [pdf, ps, html, other]
-
Title: Complete Realization of Energy Landscape and Non-equilibrium Trapping Dynamics in Spin Glass and Optimization ProblemSubjects: Disordered Systems and Neural Networks (cond-mat.dis-nn); Statistical Mechanics (cond-mat.stat-mech); Computational Physics (physics.comp-ph)
Energy landscapes are high-dimensional surfaces representing the dependence of system energy on variable configurations, which determine crucially the system's emergent behavior but are difficult to be analyzed due to their high-dimensional nature. In this article, we introduce an approach to reveal the complete energy landscapes of small spin glasses and Boolean satisfiability problems, which also unravels their non-equilibrium dynamics at an arbitrary temperature for an arbitrarily long time. In contrary to our common belief, our results show that it can be less likely to identify the ground states when temperature decreases, due to trapping in individual local minima, which ceases at different time, leading to multiple abrupt jumps with time in the ground-state probability. Simulations agree well with theoretical predictions on these remarkable phenomena. Finally, for large systems, we introduce a variant approach to extract partially the energy landscapes and observe both analytically and in simulations similar phenomena. This work introduces new methodology to unravel the non-equilibrium dynamics of glassy systems, and provides us with a clear, complete and new physical picture on their long-time behaviors inaccessible by modern numerics.
- [15] arXiv:2309.03813 (replaced) [pdf, ps, html, other]
-
Title: Anderson mobility edge as a percolation transitionComments: 6 pages, 6 figuresSubjects: Disordered Systems and Neural Networks (cond-mat.dis-nn)
The location of the mobility edge is a long standing problem in Anderson localization. In this paper, we show that the effective confining potential introduced in the localization landscape (LL) theory predicts the onset of delocalization in 3D tight-binding models, in a large part of the energy-disorder diagram. Near the edge of the spectrum, the eigenstates are confined inside the basins of the LL-based potential. The delocalization transition corresponds to the progressive merging of these basins resulting in the percolation of this classically-allowed region throughout the system. This approach, shown to be valid both in the cases of uniform and binary disorders despite their very different phase diagrams, allows us to reinterpret the Anderson transition in the tight-binding model: the mobility edge appears to be composed of two parts, one being understood as a percolation transition.
- [16] arXiv:2310.14904 (replaced) [pdf, ps, html, other]
-
Title: Dissipation induced extended-localized transitionJournal-ref: Phys. Rev. Lett. 132, 216301 (2024)Subjects: Disordered Systems and Neural Networks (cond-mat.dis-nn); Quantum Gases (cond-mat.quant-gas); Quantum Physics (quant-ph)
Mobility edge (ME), representing the critical energy that distinguishes between extended and localized states, is a key concept in understanding the transition between extended (metallic) and localized (insulating) states in disordered and quasiperiodic systems. Here we explore the impact of dissipation on a quasiperiodic system featuring MEs by calculating steady-state density matrix and analyzing quench dynamics with sudden introduction of dissipation, and demonstrate that dissipation can lead the system into specific states predominantly characterized by either extended or localized states, irrespective of the initial state. Our results establish the use of dissipation as a new avenue for inducing transitions between extended and localized states, and for manipulating dynamic behaviors of particles.
- [17] arXiv:2312.00873 (replaced) [pdf, ps, html, other]
-
Title: Critical quantum dynamics of observables at eigenstate transitionsComments: Data shown in the manuscript is available as ancillary filesSubjects: Statistical Mechanics (cond-mat.stat-mech); Disordered Systems and Neural Networks (cond-mat.dis-nn); Quantum Gases (cond-mat.quant-gas); Quantum Physics (quant-ph)
It is an outstanding goal to unveil the key features of quantum dynamics at eigenstate transitions. Focusing on quadratic fermionic Hamiltonians that exhibit localization transitions, we identify physical observables that exhibit scale-invariant critical dynamics at the transition when quenched from the initially localized charge density-wave states. The identification is based on two ingredients: (a) A relationship between the time evolution of observables in a many-body state and the transition probabilities of single-particle states, and (b) scale invariance of transition probabilities, which generalizes the corresponding recent result for survival probabilities [Phys. Rev. Lett. 131, 060404 (2023) and Phys. Rev. Res. 5, 043301 (2023)]. These properties suggest that the scale-invariant critical dynamics in the quantum-quench dynamics is also exhibited by the observables, which share the common eigenbasis with the Hamiltonian before the quench. Focusing on experimentally relevant observables such as site occupations and the particle imbalance, we numerically demonstrate their critical behavior at the eigenstate transitions in the three-dimensional Anderson model and the one-dimensional Aubry-André model model.
- [18] arXiv:2405.03797 (replaced) [pdf, ps, html, other]
-
Title: Tensor Network Computations That Capture Strict Variationality, Volume Law Behavior, and the Efficient Representation of Neural Network StatesComments: 4+9 pages, 13 figuresSubjects: Quantum Physics (quant-ph); Disordered Systems and Neural Networks (cond-mat.dis-nn); Strongly Correlated Electrons (cond-mat.str-el); Computational Physics (physics.comp-ph)
We introduce a change of perspective on tensor network states that is defined by the computational graph of the contraction of an amplitude. The resulting class of states, which we refer to as tensor network functions, inherit the conceptual advantages of tensor network states while removing computational restrictions arising from the need to converge approximate contractions. We use tensor network functions to compute strict variational estimates of the energy on loopy graphs, analyze their expressive power for ground-states, show that we can capture aspects of volume law time evolution, and provide a mapping of general feed-forward neural nets onto efficient tensor network functions. Our work expands the realm of computable tensor networks to ones where accurate contraction methods are not available, and opens up new avenues to use tensor networks.
- [19] arXiv:2405.12818 (replaced) [pdf, ps, html, other]
-
Title: Non-equilibrium dynamic hyperuniform statesSubjects: Soft Condensed Matter (cond-mat.soft); Disordered Systems and Neural Networks (cond-mat.dis-nn); Statistical Mechanics (cond-mat.stat-mech)
Disordered hyperuniform structures are an exotic state of matter having suppressed density fluctuations at large length-scale similar to perfect crystals and quasicrystals but without any long range orientational order. In the past decade, an increasing number of non-equilibrium systems were found to have dynamic hyperuniform states, which have emerged as a new research direction coupling both non-equilibrium physics and hyperuniformity. Here we review the recent progress in understanding dynamic hyperuniform states found in various non-equilibrium systems, including the critical hyperuniformity in absorbing phase transitions, non-equilibrium hyperuniform fluids and the hyperuniform structures in phase separating systems via spinodal decomposition.