Electrical Engineering and Systems Science
See recent articles
- [1] arXiv:2408.11828 [pdf, html, other]
-
Title: Online Electric Vehicle Charging Detection Based on Memory-based Transformer using Smart Meter DataSubjects: Signal Processing (eess.SP); Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
The growing popularity of Electric Vehicles (EVs) poses unique challenges for grid operators and infrastructure, which requires effectively managing these vehicles' integration into the grid. Identification of EVs charging is essential to electricity Distribution Network Operators (DNOs) for better planning and managing the distribution grid. One critical aspect is the ability to accurately identify the presence of EV charging in the grid. EV charging identification using smart meter readings obtained from behind-the-meter devices is a challenging task that enables effective managing the integration of EVs into the existing power grid. Different from the existing supervised models that require addressing the imbalance problem caused by EVs and non-EVs data, we propose a novel unsupervised memory-based transformer (M-TR) that can run in real-time (online) to detect EVs charging from a streaming smart meter. It dynamically leverages coarse-scale historical information using an M-TR encoder from an extended global temporal window, in conjunction with an M-TR decoder that concentrates on a limited time frame, local window, aiming to capture the fine-scale characteristics of the smart meter data. The M-TR is based on an anomaly detection technique that does not require any prior knowledge about EVs charging profiles, nor it does only require real power consumption data of non-EV users. In addition, the proposed model leverages the power of transfer learning. The M-TR is compared with different state-of-the-art methods and performs better than other unsupervised learning models. The model can run with an excellent execution time of 1.2 sec. for 1-minute smart recordings.
- [2] arXiv:2408.11842 [pdf, html, other]
-
Title: Non-Causal to Causal SSL-Supported Transfer Learning: Towards a High-Performance Low-Latency Speech VocodeComments: Accepted at IWAENC 2024Subjects: Audio and Speech Processing (eess.AS)
Recently, BigVGAN has emerged as high-performance speech vocoder. Its sequence-to-sequence-based synthesis, however, prohibits usage in low-latency conversational applications. Our work addresses this shortcoming in three steps. First, we introduce low latency into BigVGAN via implementing causal convolutions, yielding decreased performance. Second, to regain performance, we propose a teacher-student transfer learning scheme to distill the high-delay non-causal BigVGAN into our low-latency causal vocoder. Third, taking advantage of a self-supervised learning (SSL) model, in our case wav2vec 2.0, we align its encoder speech representations extracted from our low-latency causal vocoder to the ground truth ones. In speaker-independent settings, both proposed training schemes notably elevate the performance of our low-latency vocoder, closing up to the original high-delay BigVGAN. At only 21% higher complexity, our best small causal vocoder achieves 3.96 PESQ and 1.25 MCD, excelling even the original small non-causal BigVGAN (3.64 PESQ) by 0.32 PESQ and 0.1 MCD points, respectively.
- [3] arXiv:2408.11873 [pdf, html, other]
-
Title: Parameter-Efficient Transfer Learning under Federated Learning for Automatic Speech RecognitionSubjects: Audio and Speech Processing (eess.AS); Cryptography and Security (cs.CR); Machine Learning (cs.LG)
This work explores the challenge of enhancing Automatic Speech Recognition (ASR) model performance across various user-specific domains while preserving user data privacy. We employ federated learning and parameter-efficient domain adaptation methods to solve the (1) massive data requirement of ASR models from user-specific scenarios and (2) the substantial communication cost between servers and clients during federated learning. We demonstrate that when equipped with proper adapters, ASR models under federated tuning can achieve similar performance compared with centralized tuning ones, thus providing a potential direction for future privacy-preserved ASR services. Besides, we investigate the efficiency of different adapters and adapter incorporation strategies under the federated learning setting.
- [4] arXiv:2408.11934 [pdf, html, other]
-
Title: Decoupling Power Quality Issues in Grid-Microgrid Network Using Microgrid Building BlocksComments: This paper is accepted for publication in IEEE IECON 2024, Chicago, IL. The complete copyright version will be available on IEEE Xplore when the conference proceedings are publishedSubjects: Systems and Control (eess.SY)
Microgrids are evolving as promising options to enhance reliability of the connected transmission and distribution systems. Traditional design and deployment of microgrids require significant engineering analysis. Microgrid Building Blocks (MBB), consisting of modular blocks that integrate seamlessly to form effective microgrids, is an enabling concept for faster and broader adoption of microgrids. Back-to-Back converter placed at the point of common coupling of microgrid is an integral part of the MBB. This paper presents applications of MBB to decouple power quality issues in grid-microgrid network serving power quality sensitive loads such as data centers, new grid-edge technologies such as vehicle-to-grid generation, and serving electric vehicle charging loads during evacuation before disaster events. Simulation results show that MBB effectively decouples the power quality issues across networks and helps maintain good power quality in the power quality sensitive network based on the operational scenario.
- [5] arXiv:2408.11956 [pdf, html, other]
-
Title: The Whole Is Bigger Than the Sum of Its Parts: Modeling Individual Annotators to Capture Emotional VariabilityComments: Accepted to Interspeech 2024 ConferenceSubjects: Audio and Speech Processing (eess.AS); Machine Learning (cs.LG)
Emotion expression and perception are nuanced, complex, and highly subjective processes. When multiple annotators label emotional data, the resulting labels contain high variability. Most speech emotion recognition tasks address this by averaging annotator labels as ground truth. However, this process omits the nuance of emotion and inter-annotator variability, which are important signals to capture. Previous work has attempted to learn distributions to capture emotion variability, but these methods also lose information about the individual annotators. We address these limitations by learning to predict individual annotators and by introducing a novel method to create distributions from continuous model outputs that permit the learning of emotion distributions during model training. We show that this combined approach can result in emotion distributions that are more accurate than those seen in prior work, in both within- and cross-corpus settings.
- [6] arXiv:2408.11959 [pdf, html, other]
-
Title: On the design of stabilizing FIR controllersSubjects: Systems and Control (eess.SY); Cryptography and Security (cs.CR); Optimization and Control (math.OC)
Recently, it has been observed that finite impulse response controllers are an excellent basis for encrypted control, where privacy-preserving controller evaluations via special cryptosystems are the main focus. Beneficial properties of FIR filters are also well-known from digital signal processing, which makes them preferable over infinite impulse response filters in many applications. Their appeal extends to feedback control, offering design flexibility grounded solely on output measurements. However, designing FIR controllers is challenging, which motivates this work. To address the design challenge, we initially show that FIR controller designs for linear systems can equivalently be stated as static or dynamic output feedback problems. After focusing on the existence of stabilizing FIR controllers for a given plant, we tailor two common design approaches for output feedback to the case of FIR controllers. Unfortunately, it will turn out that the FIR characteristics add further restrictions to the LMI-based approaches. Hence, we finally turn to designs building on non-convex optimization, which provide satisfactory results for a selection of benchmark systems.
- [7] arXiv:2408.11965 [pdf, html, other]
-
Title: CT-AGRG: Automated Abnormality-Guided Report Generation from 3D Chest CT VolumesComments: 15 pages, 9 figures, submitted to ISBI 2025Subjects: Image and Video Processing (eess.IV); Computer Vision and Pattern Recognition (cs.CV)
The rapid increase of computed tomography (CT) scans and their time-consuming manual analysis have created an urgent need for robust automated analysis techniques in clinical settings. These aim to assist radiologists and help them managing their growing workload. Existing methods typically generate entire reports directly from 3D CT images, without explicitly focusing on observed abnormalities. This unguided approach often results in repetitive content or incomplete reports, failing to prioritize anomaly-specific descriptions. We propose a new anomaly-guided report generation model, which first predicts abnormalities and then generates targeted descriptions for each. Evaluation on a public dataset demonstrates significant improvements in report quality and clinical relevance. We extend our work by conducting an ablation study to demonstrate its effectiveness.
- [8] arXiv:2408.11982 [pdf, html, other]
-
Title: AIM 2024 Challenge on Compressed Video Quality Assessment: Methods and ResultsMaksim Smirnov, Aleksandr Gushchin, Anastasia Antsiferova, Dmitry Vatolin, Radu Timofte, Ziheng Jia, Zicheng Zhang, Wei Sun, Jiaying Qian, Yuqin Cao, Yinan Sun, Yuxin Zhu, Xiongkuo Min, Guangtao Zhai, Kanjar De, Qing Luo, Ao-Xiang Zhang, Peng Zhang, Haibo Lei, Linyan Jiang, Yaqing Li, Wenhui Meng, Xiaoheng Tan, Haiqiang Wang, Xiaozhong Xu, Shan Liu, Zhenzhong Chen, Zhengxue Cheng, Jiahao Xiao, Jun Xu, Chenlong He, Qi Zheng, Ruoxi Zhu, Min Li, Yibo Fan, Zhengzhong TuSubjects: Image and Video Processing (eess.IV); Computer Vision and Pattern Recognition (cs.CV); Multimedia (cs.MM)
Video quality assessment (VQA) is a crucial task in the development of video compression standards, as it directly impacts the viewer experience. This paper presents the results of the Compressed Video Quality Assessment challenge, held in conjunction with the Advances in Image Manipulation (AIM) workshop at ECCV 2024. The challenge aimed to evaluate the performance of VQA methods on a diverse dataset of 459 videos, encoded with 14 codecs of various compression standards (AVC/H.264, HEVC/H.265, AV1, and VVC/H.266) and containing a comprehensive collection of compression artifacts. To measure the methods performance, we employed traditional correlation coefficients between their predictions and subjective scores, which were collected via large-scale crowdsourced pairwise human comparisons. For training purposes, participants were provided with the Compressed Video Quality Assessment Dataset (CVQAD), a previously developed dataset of 1022 videos. Up to 30 participating teams registered for the challenge, while we report the results of 6 teams, which submitted valid final solutions and code for reproducing the results. Moreover, we calculated and present the performance of state-of-the-art VQA methods on the developed dataset, providing a comprehensive benchmark for future research. The dataset, results, and online leaderboard are publicly available at this https URL.
- [9] arXiv:2408.11992 [pdf, html, other]
-
Title: MBSS-T1: Model-Based Self-Supervised Motion Correction for Robust Cardiac T1 MappingSubjects: Image and Video Processing (eess.IV); Computer Vision and Pattern Recognition (cs.CV)
T1 mapping is a valuable quantitative MRI technique for diagnosing diffuse myocardial diseases. Traditional methods, relying on breath-hold sequences and echo triggering, face challenges with patient compliance and arrhythmias, limiting their effectiveness. Image registration can enable motion-robust T1 mapping, but inherent intensity differences between time points pose a challenge. We introduce MBSS-T1, a self-supervised model for motion correction in cardiac T1 mapping, constrained by physical and anatomical principles. The physical constraints ensure expected signal decay behavior, while the anatomical constraints maintain realistic deformations. The unique combination of these constraints ensures accurate T1 mapping along the longitudinal relaxation axis. MBSS-T1 outperformed baseline deep-learning-based image registration approaches in a 5-fold experiment on a public dataset of 210 patients (STONE sequence) and an internal dataset of 19 patients (MOLLI sequence). MBSS-T1 excelled in model fitting quality (R2: 0.974 vs. 0.941, 0.946), anatomical alignment (Dice score: 0.921 vs. 0.984, 0.988), and expert visual quality assessment for the presence of visible motion artifacts (4.33 vs. 3.34, 3.62). MBSS-T1 has the potential to enable motion-robust T1 mapping for a broader range of patients, overcoming challenges such as arrhythmias, and suboptimal compliance, and allowing for free-breathing T1 mapping without requiring large training datasets.
- [10] arXiv:2408.12013 [pdf, html, other]
-
Title: Detection of Under-represented Samples Using Dynamic Batch Training for Brain Tumor Segmentation from MR ImagesSubjects: Image and Video Processing (eess.IV); Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG)
Brain tumors in magnetic resonance imaging (MR) are difficult, time-consuming, and prone to human error. These challenges can be resolved by developing automatic brain tumor segmentation methods from MR images. Various deep-learning models based on the U-Net have been proposed for the task. These deep-learning models are trained on a dataset of tumor images and then used for segmenting the masks. Mini-batch training is a widely used method in deep learning for training. However, one of the significant challenges associated with this approach is that if the training dataset has under-represented samples or samples with complex latent representations, the model may not generalize well to these samples. The issue leads to skewed learning of the data, where the model learns to fit towards the majority representations while underestimating the under-represented samples. The proposed dynamic batch training method addresses the challenges posed by under-represented data points, data points with complex latent representation, and imbalances within the class, where some samples may be harder to learn than others. Poor performance of such samples can be identified only after the completion of the training, leading to the wastage of computational resources. Also, training easy samples after each epoch is an inefficient utilization of computation resources. To overcome these challenges, the proposed method identifies hard samples and trains such samples for more iterations compared to easier samples on the BraTS2020 dataset. Additionally, the samples trained multiple times are identified and it provides a way to identify hard samples in the BraTS2020 dataset. The comparison of the proposed training approach with U-Net and other models in the literature highlights the capabilities of the proposed training approach.
- [11] arXiv:2408.12014 [pdf, html, other]
-
Title: An Econometric Analysis of Large Flexible Cryptocurrency-mining Consumers in Electricity MarketsComments: 10 pages, 10 figures, accepted for publication in Hawaii International Conference on System Sciences-58Subjects: Systems and Control (eess.SY); Econometrics (econ.EM)
In recent years, power grids have seen a surge in large cryptocurrency mining firms, with individual consumption levels reaching 700MW. This study examines the behavior of these firms in Texas, focusing on how their consumption is influenced by cryptocurrency conversion rates, electricity prices, local weather, and other factors. We transform the skewed electricity consumption data of these firms, perform correlation analysis, and apply a seasonal autoregressive moving average model for analysis. Our findings reveal that, surprisingly, short-term mining electricity consumption is not correlated with cryptocurrency conversion rates. Instead, the primary influencers are the temperature and electricity prices. These firms also respond to avoid transmission and distribution network (T\&D) charges -- famously known as four Coincident peak (4CP) charges -- during summer times. As the scale of these firms is likely to surge in future years, the developed electricity consumption model can be used to generate public, synthetic datasets to understand the overall impact on power grid. The developed model could also lead to better pricing mechanisms to effectively use the flexibility of these resources towards improving power grid reliability.
- [12] arXiv:2408.12059 [pdf, other]
-
Title: MAC protocol classification in the ISM band using machine learning methodsSubjects: Signal Processing (eess.SP); Machine Learning (cs.LG); Networking and Internet Architecture (cs.NI)
With the emergence of new technologies and a growing number of wireless networks, we face the problem of radio spectrum shortages. As a result, identifying the wireless channel spectrum to exploit the channel's idle state while also boosting network security is a pivotal issue. Detecting and classifying protocols in the MAC sublayer enables Cognitive Radio users to improve spectrum utilization and minimize potential interference. In this paper, we classify the Wi-Fi and Bluetooth protocols, which are the most widely used MAC sublayer protocols in the ISM radio band. With the advent of various wireless technologies, especially in the 2.4 GHz frequency band, the ISM frequency spectrum has become crowded and high-traffic, which faces a lack of spectrum resources and user interference. Therefore, identifying and classifying protocols is an effective and useful method. Leveraging machine learning and deep learning techniques, known for their advanced classification capabilities, we apply Support Vector Machine and K-Nearest Neighbors algorithms, which are machine learning algorithms, to classify protocols into three classes: Wi-Fi, Wi-Fi Beacon, and Bluetooth. To capture the signals, we use the USRP N210 Software Defined Radio device and sample the real data in the indoor environment in different conditions of the presence and absence of transmitters and receivers for these two protocols. By assembling this dataset and studying the time and frequency features of the protocols, we extract the frame width and the silence gap between the two frames as time features and the PAPR of each frame as a power feature. By comparing the output of the protocols classification in different conditions and also adding Gaussian noise, it was found that the samples in the nonlinear SVM method with RBF and KNN functions have the best performance, with 97.83% and 98.12% classification accuracy, respectively.
- [13] arXiv:2408.12067 [pdf, html, other]
-
Title: Distributed Noncoherent Joint Transmission Based on Multi-Agent Reinforcement Learning for Dense Small Cell MISO SystemsSubjects: Signal Processing (eess.SP); Artificial Intelligence (cs.AI); Networking and Internet Architecture (cs.NI)
We consider a dense small cell (DSC) network where multi-antenna small cell base stations (SBSs) transmit data to single-antenna users over a shared frequency band. To enhance capacity, a state-of-the-art technique known as noncoherent joint transmission (JT) is applied, enabling users to receive data from multiple coordinated SBSs. However, the sum rate maximization problem with noncoherent JT is inherently nonconvex and NP-hard. While existing optimization-based noncoherent JT algorithms can provide near-optimal performance, they require global channel state information (CSI) and multiple iterations, which makes them difficult to be implemeted in DSC this http URL overcome these challenges, we first prove that the optimal beamforming structure is the same for both the power minimization problem and the sum rate maximization problem, and then mathematically derive the optimal beamforming structure for both problems by solving the power minimization problem.The optimal beamforming structure can effectively reduces the variable this http URL exploiting the optimal beamforming structure, we propose a deep deterministic policy gradient-based distributed noncoherent JT scheme to maximize the system sum this http URL the proposed scheme, each SBS utilizes global information for training and uses local CSI to determine beamforming vectors. Simulation results demonstrate that the proposed scheme achieves comparable performance with considerably lower computational complexity and information overhead compared to centralized iterative optimization-based techniques, making it more attractive for practical deployment.
- [14] arXiv:2408.12077 [pdf, html, other]
-
Title: Through-the-Wall Radar Human Activity Micro-Doppler Signature Representation Method Based on Joint Boulic-Sinusoidal Pendulum ModelComments: 17 pages, 14 figures, 7 tables, in IEEE Transactions on Microwave Theory and Techniques, 2024Subjects: Signal Processing (eess.SP); Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG)
With the help of micro-Doppler signature, ultra-wideband (UWB) through-the-wall radar (TWR) enables the reconstruction of range and velocity information of limb nodes to accurately identify indoor human activities. However, existing methods are usually trained and validated directly using range-time maps (RTM) and Doppler-time maps (DTM), which have high feature redundancy and poor generalization ability. In order to solve this problem, this paper proposes a human activity micro-Doppler signature representation method based on joint Boulic-sinusoidal pendulum motion model. In detail, this paper presents a simplified joint Boulic-sinusoidal pendulum human motion model by taking head, torso, both hands and feet into consideration improved from Boulic-Thalmann kinematic model. The paper also calculates the minimum number of key points needed to describe the Doppler and micro-Doppler information sufficiently. Both numerical simulations and experiments are conducted to verify the effectiveness. The results demonstrate that the proposed number of key points of micro-Doppler signature can precisely represent the indoor human limb node motion characteristics, and substantially improve the generalization capability of the existing methods for different testers.
- [15] arXiv:2408.12080 [pdf, html, other]
-
Title: Exploring the Feasibility of Automated Data Standardization using Large Language Models for Seamless PositioningComments: Accepted at IPIN 2024. To be published in IEEE XploreSubjects: Signal Processing (eess.SP); Artificial Intelligence (cs.AI); Networking and Internet Architecture (cs.NI)
We propose a feasibility study for real-time automated data standardization leveraging Large Language Models (LLMs) to enhance seamless positioning systems in IoT environments. By integrating and standardizing heterogeneous sensor data from smartphones, IoT devices, and dedicated systems such as Ultra-Wideband (UWB), our study ensures data compatibility and improves positioning accuracy using the Extended Kalman Filter (EKF). The core components include the Intelligent Data Standardization Module (IDSM), which employs a fine-tuned LLM to convert varied sensor data into a standardized format, and the Transformation Rule Generation Module (TRGM), which automates the creation of transformation rules and scripts for ongoing data standardization. Evaluated in real-time environments, our study demonstrates adaptability and scalability, enhancing operational efficiency and accuracy in seamless navigation. This study underscores the potential of advanced LLMs in overcoming sensor data integration complexities, paving the way for more scalable and precise IoT navigation solutions.
- [16] arXiv:2408.12085 [pdf, html, other]
-
Title: Controllability and Observability of Temporal HypergraphsComments: 6 pages, 3 figuresSubjects: Systems and Control (eess.SY); Social and Information Networks (cs.SI); Dynamical Systems (math.DS)
Numerous complex systems, such as those arisen in ecological networks, genomic contact networks, and social networks, exhibit higher-order and time-varying characteristics, which can be effectively modeled using temporal hypergraphs. However, analyzing and controlling temporal hypergraphs poses significant challenges due to their inherent time-varying and nonlinear nature, while most existing methods predominantly target static hypergraphs. In this article, we generalize the notions of controllability and observability to temporal hypergraphs by leveraging tensor and nonlinear systems theory. Specifically, we establish tensor-based rank conditions to determine the weak controllability and observability of temporal hypergraphs. The proposed framework is further demonstrated with synthetic and real-world examples.
- [17] arXiv:2408.12126 [pdf, other]
-
Title: Robust Input Shaping Vibration Control via Extended Kalman Filter-Incorporated Residual Neural NetworkSubjects: Systems and Control (eess.SY)
With the rapid development of industry, the vibration control of flexible structures and underactuated systems has been increasingly gaining attention. Input shaping technology enables stable performance for high-speed motion in industrial motion systems. However, existing input shapers generally suffer from the ineffective control performance due to the neglect of observation errors. To address this critical issue, this paper proposes an Extended Kalman Filter-incorporated Residual Neural Network-based input Shaping (ERS) model for vibration control. Its main ideas are two-fold: a) adopting an extended Kalman filter to address a vertical flexible beam's model errors; and b) adopting a residual neural network to cascade with the extended Kalman filter for eliminating the remaining observation errors. Detailed experiments on a real dataset collected from a vertical flexible beam demonstrate that the proposed ERS model has achieved significant vibration control performance over several state-of-the-art models.
- [18] arXiv:2408.12145 [pdf, html, other]
-
Title: Spectrum Sharing Between Low Earth Orbit Satellite and Terrestrial Networks: A Stochastic Geometry Perspective AnalysisSubjects: Signal Processing (eess.SP)
Low Earth orbit (LEO) satellite networks with mega constellations have the potential to provide 5G and beyond services ubiquitously. However, these networks may introduce mutual interference to both satellite and terrestrial networks, particularly when sharing spectrum resources. In this paper, we present a system-level performance analysis to address these interference issues using the tool of stochastic geometry. We model the spatial distributions of satellites, satellite users, terrestrial base stations (BSs), and terrestrial users using independent Poisson point processes on the surfaces of concentric spheres. Under these spatial models, we derive analytical expressions for the ergodic spectral efficiency of uplink (UL) and downlink (DL) satellite networks when they share spectrum with both UL and DL terrestrial networks. These derived ergodic expressions capture comprehensive network parameters, including the densities of satellite and terrestrial networks, the path-loss exponent, and fading. From our analysis, we determine the conditions under which spectrum sharing with UL terrestrial networks is advantageous for both UL and DL satellite networks. Our key finding is that the optimal spectrum sharing configuration among the four possible configurations depends on the density ratio between terrestrial BSs and users, providing a design guideline for spectrum management. Simulation results confirm the accuracy of our derived expressions.
- [19] arXiv:2408.12149 [pdf, html, other]
-
Title: New Collision Free Balanced Frequency Hopping Sequence Sets: Construction and AnalysisSubjects: Signal Processing (eess.SP)
Collision-free Frequency Hopping Sequences (FHS) are crucial for maintaining the throughput of Frequency Hopping Multiple Access (FHMA) communication systems. When multiple FHS deployed in the same geographical area hop into the same frequency spot or location, it can result in multiple access interference (MAI), leading to performance degradation of the co-habiting FHMA systems. Orthogonal FHS (O-FHS) could be used to avoid such collisions even while maintaining the pseudo-random hopping property. We propose a new Collision-Free Balanced Frequency Hopping Sequence (CFB-FHS) set based on $GF(p)$. The proposed algorithm generates a significantly longer O-FHS and ensures full-band hopping and uniform usage of all the available frequency spots, even while preserving the pseudo-random nature of the sequences. The Hamming correlation and these related properties of the proposed sequence set are analyzed and compared with the existing sequence sets. The length of this CFB-FHS can be chosen based on the application requirements, and once selected, the length remains fixed irrespective of the family size of the sequence set.
- [20] arXiv:2408.12150 [pdf, html, other]
-
Title: DeepHQ: Learned Hierarchical Quantizer for Progressive Deep Image CodingSubjects: Image and Video Processing (eess.IV); Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
Unlike fixed- or variable-rate image coding, progressive image coding (PIC) aims to compress various qualities of images into a single bitstream, increasing the versatility of bitstream utilization and providing high compression efficiency compared to simulcast compression. Research on neural network (NN)-based PIC is in its early stages, mainly focusing on applying varying quantization step sizes to the transformed latent representations in a hierarchical manner. These approaches are designed to compress only the progressively added information as the quality improves, considering that a wider quantization interval for lower-quality compression includes multiple narrower sub-intervals for higher-quality compression. However, the existing methods are based on handcrafted quantization hierarchies, resulting in sub-optimal compression efficiency. In this paper, we propose an NN-based progressive coding method that firstly utilizes learned quantization step sizes via learning for each quantization layer. We also incorporate selective compression with which only the essential representation components are compressed for each quantization layer. We demonstrate that our method achieves significantly higher coding efficiency than the existing approaches with decreased decoding time and reduced model size.
- [21] arXiv:2408.12193 [pdf, html, other]
-
Title: Empowering Wireless Network Applications with Deep Learning-based Radio Propagation ModelsComments: 7 pages, 3 Figures, 1 TableSubjects: Signal Processing (eess.SP); Machine Learning (cs.LG); Networking and Internet Architecture (cs.NI)
The efficient deployment and operation of any wireless communication ecosystem rely on knowledge of the received signal quality over the target coverage area. This knowledge is typically acquired through radio propagation solvers, which however suffer from intrinsic and well-known performance limitations. This article provides a primer on how integrating deep learning and conventional propagation modeling techniques can enhance multiple vital facets of wireless network operation, and yield benefits in terms of efficiency and reliability. By highlighting the pivotal role that the deep learning-based radio propagation models will assume in next-generation wireless networks, we aspire to propel further research in this direction and foster their adoption in additional applications.
- [22] arXiv:2408.12205 [pdf, html, other]
-
Title: Reconfigurable Intelligent Surfaces as Spatial FiltersComments: 13 pages, 9 figuresSubjects: Signal Processing (eess.SP)
The design of Reconfigurable Intelligent Surfaces (RISs) is typically based on treating the RIS as an infinitely large surface that steers incident plane waves toward the desired direction. In practical implementations, however, the RIS has finite size and the incident wave is a beam of finite $k$-content, rather than a plane wave of $\delta$-like $k$-content. To understand the implications of the finite extent of both the RIS and the incident beam, here we treat the RIS as a spatial filter, the transfer function of which is determined by both the prescribed RIS operation and the shape of the RIS boundary. Following this approach, we study how the RIS transforms the incident $k$-content and we demonstrate how, by engineering the RIS shape, size, and response, it is possible to shape beams with nontrivial $k$-content to suppress unwanted interference, while concentrating the reflected power to desired directions. We also demonstrate how our framework, when applied in the context of near-field communications, provides the necessary insights into how the wavefront of the beam is tailored to enable focusing, propagation with invariant profile, and bending, beyond conventional beamforming.
- [23] arXiv:2408.12239 [pdf, html, other]
-
Title: Fast Burst-Sparsity Learning Approach for Massive MIMO-OTFS Channel EstimationComments: 7 pages, 3 figuresSubjects: Signal Processing (eess.SP)
Accurate channel estimation in orthogonal time frequency space (OTFS) systems with massive multiple-input multiple-output (MIMO) configurations is challenging due to high-dimensional sparse representation (SR). Existing methods often face performance degradation and/or high computational complexity. To address these issues and exploit intricate channel sparsity structure, this letter first leverages a novel hybrid burst-sparsity prior to capture the burst/common sparse structure in the angle/delay domain, and then utilizes an independent variational Bayesian inference (VBI) factorization technique to efficiently solve the high-dimensional SR problem. Additionally, an angle/Doppler refinement approach is incorporated into the proposed method to automatically mitigate off-grid mismatches.
- [24] arXiv:2408.12252 [pdf, html, other]
-
Title: Synesthesia of Machines (SoM)-Enhanced Wideband Multi-User CSI LearningComments: 6 pages, 4 figures, 1 tableSubjects: Signal Processing (eess.SP)
Light detection and ranging (LiDAR) has been utilized for optimizing wireless communications due to its ability to detect the environment. This paper explores the use of LiDAR in channel estimation for wideband multi-user multiple-input-multiple-output orthogonal frequency division multiplexing systems and introduces a LiDAR-enhanced Channel State Information (CSI) learning network (LE-CLN). By utilizing user positioning information, LE-CLN first calculates user-localized over-complete angular measurements. It then investigates the correlation between LiDAR and CSI, transforming raw LiDAR data into a low-complexity format embedded with signal propagation characteristics. LE-CLN also adapts the use of LiDAR based on channel conditions through attention mechanisms. Thanks to the unique wireless features offered by LiDAR, LE-CLN achieves higher estimation accuracy and spectrum efficiency compared to benchmarks, particularly in latency-sensitive applications where pilot transmissions are expected to be reduced.
- [25] arXiv:2408.12266 [pdf, html, other]
-
Title: Accounts of using the Tustin-Net architecture on a rotary inverted pendulumSubjects: Systems and Control (eess.SY); Machine Learning (cs.LG)
In this report we investigate the use of the Tustin neural network architecture (Tustin-Net) for the identification of a physical rotary inverse pendulum. This physics-based architecture is of particular interest as it builds on the known relationship between velocities and positions. We here aim at discussing the advantages, limitations and performance of Tustin-Nets compared to first-principles grey-box models on a real physical apparatus, showing how, with a standard training procedure, the former can hardly achieve the same accuracy as the latter. To address this limitation, we present a training strategy based on transfer learning that yields Tustin-Nets that are competitive with the first-principles model, without requiring extensive knowledge of the setup as the latter.
- [26] arXiv:2408.12275 [pdf, html, other]
-
Title: Whole Slide Image Classification of Salivary Gland TumoursComments: 5 pages, 2 figures, 28th UK Conference on Medical Image Understanding and Analysis - clinical abstractSubjects: Image and Video Processing (eess.IV); Computer Vision and Pattern Recognition (cs.CV)
This work shows promising results using multiple instance learning on salivary gland tumours in classifying cancers on whole slide images. Utilising CTransPath as a patch-level feature extractor and CLAM as a feature aggregator, an F1 score of over 0.88 and AUROC of 0.92 are obtained for detecting cancer in whole slide images.
- [27] arXiv:2408.12277 [pdf, html, other]
-
Title: Modularized data-driven approximation of the Koopman operator and generatorComments: 31 pages, 11 figuresSubjects: Systems and Control (eess.SY); Dynamical Systems (math.DS)
Extended Dynamic Mode Decomposition (EDMD) is a widely-used data-driven approach to learn an approximation of the Koopman operator. Consequently, it provides a powerful tool for data-driven analysis, prediction, and control of nonlinear dynamical (control) systems. In this work, we propose a novel modularized EDMD scheme tailored to interconnected systems. To this end, we utilize the structure of the Koopman generator that allows to learn the dynamics of subsystems individually and thus alleviates the curse of dimensionality by considering observable functions on smaller state spaces. Moreover, our approach canonically enables transfer learning if a system encompasses multiple copies of a model as well as efficient adaption to topology changes without retraining. We provide finite-data bounds on the estimation error using tools from graph theory. The efficacy of the method is illustrated by means of various numerical examples.
- [28] arXiv:2408.12294 [pdf, html, other]
-
Title: Star-shaped Tilted Hexarotor Maneuverability: Analysis of the Role of the Tilt Cant AnglesComments: accepted for presentation at the 2024 IEEE 20th International Conference on Automation Science and Engineering (CASE2024)Subjects: Systems and Control (eess.SY); Robotics (cs.RO)
Star-shaped Tilted Hexarotors are rapidly emerging for applications highly demanding in terms of robustness and maneuverability. To ensure improvement in such features, a careful selection of the tilt angles is mandatory. In this work, we present a rigorous analysis of how the force subspace varies with the tilt cant angles, namely the tilt angles along the vehicle arms, taking into account gravity compensation and torque decoupling to abide by the hovering condition. Novel metrics are introduced to assess the performance of existing tilted platforms, as well as to provide some guidelines for the selection of the tilt cant angle in the design phase.
- [29] arXiv:2408.12323 [pdf, html, other]
-
Title: EUIS-Net: A Convolutional Neural Network for Efficient Ultrasound Image SegmentationSubjects: Image and Video Processing (eess.IV); Computer Vision and Pattern Recognition (cs.CV)
Segmenting ultrasound images is critical for various medical applications, but it offers significant challenges due to ultrasound images' inherent noise and unpredictability. To address these challenges, we proposed EUIS-Net, a CNN network designed to segment ultrasound images efficiently and precisely. The proposed EUIS-Net utilises four encoder-decoder blocks, resulting in a notable decrease in computational complexity while achieving excellent performance. The proposed EUIS-Net integrates both channel and spatial attention mechanisms into the bottleneck to improve feature representation and collect significant contextual information. In addition, EUIS-Net incorporates a region-aware attention module in skip connections, which enhances the ability to concentrate on the region of the injury. To enable thorough information exchange across various network blocks, skip connection aggregation is employed from the network's lowermost to the uppermost block. Comprehensive evaluations are conducted on two publicly available ultrasound image segmentation datasets. The proposed EUIS-Net achieved mean IoU and dice scores of 78. 12\%, 85. 42\% and 84. 73\%, 89. 01\% in the BUSI and DDTI datasets, respectively. The findings of our study showcase the substantial capabilities of EUIS-Net for immediate use in clinical settings and its versatility in various ultrasound imaging tasks.
- [30] arXiv:2408.12354 [pdf, html, other]
-
Title: LCM-SVC: Latent Diffusion Model Based Singing Voice Conversion with Inference Acceleration via Latent Consistency DistillationComments: Accepted to ISCSLP 2024. arXiv admin note: text overlap with arXiv:2406.05325Subjects: Audio and Speech Processing (eess.AS); Sound (cs.SD)
Any-to-any singing voice conversion (SVC) aims to transfer a target singer's timbre to other songs using a short voice sample. However many diffusion model based any-to-any SVC methods, which have achieved impressive results, usually suffered from low efficiency caused by a mass of inference steps. In this paper, we propose LCM-SVC, a latent consistency distillation (LCD) based latent diffusion model (LDM) to accelerate inference speed. We achieved one-step or few-step inference while maintaining the high performance by distilling a pre-trained LDM based SVC model, which had the advantages of timbre decoupling and sound quality. Experimental results show that our proposed method can significantly reduce the inference time and largely preserve the sound quality and timbre similarity comparing with other state-of-the-art SVC models. Audio samples are available at this https URL.
- [31] arXiv:2408.12402 [pdf, html, other]
-
Title: A Stable Polygamy Approach to Spectrum Access with Channel ReuseSubjects: Systems and Control (eess.SY)
We introduce a new and broader formulation of the stable marriage problem (SMP), called the stable polygamy problem (SPP), where multiple individuals from a larger group $L$ of $|L|$ individuals can be matched with a single individual from a smaller group $S$ of $|S|$ individuals. Each individual $\ell \in L$ possesses a social constraint set $C_{\ell}$ that contains a subset of $L$ with whom they cannot coexist harmoniously. We define a generalized concept of stability based on the preference and constraints of the individuals. We explore two common settings: common utility, where the utility of a match is the same for individuals from both sets, and preference ranking, where each individual has a preference ranking for every other individual in the opposite set. Our analysis is conducted using a novel graph-theoretical framework.
The classic SMP has been investigated in recent years for spectrum access to match cells or users to channels, where only one-to-one matching is allowed. By contrast, the new SPP formulation allows us to solve more general models with channel reuse, where multiple users may access the same channel simultaneously. Interestingly, we show that classic algorithms, such as propose and reject (P&R), and Hungarian method are no longer efficient in the polygamy setting.
We develop efficient algorithms to solve the SPP in polynomial time, tailored for implementations in spectrum access with channel reuse. We analytically show that our algorithm always solves the SPP with common utility. While the SPP with preference ranking cannot be solved by any algorithm in all cases, we prove that our algorithm effectively solves it in specific graph structures representing strong and weak interference regimes. Simulation results demonstrate the efficiency of our algorithms across various spectrum access scenarios. - [32] arXiv:2408.12425 [pdf, html, other]
-
Title: Dynamic Gated Recurrent Neural Network for Compute-efficient Speech EnhancementComments: Accepted to Interspeech 2024Subjects: Audio and Speech Processing (eess.AS); Machine Learning (cs.LG); Sound (cs.SD)
This paper introduces a new Dynamic Gated Recurrent Neural Network (DG-RNN) for compute-efficient speech enhancement models running on resource-constrained hardware platforms. It leverages the slow evolution characteristic of RNN hidden states over steps, and updates only a selected set of neurons at each step by adding a newly proposed select gate to the RNN model. This select gate allows the computation cost of the conventional RNN to be reduced during network inference. As a realization of the DG-RNN, we further propose the Dynamic Gated Recurrent Unit (D-GRU) which does not require additional parameters. Test results obtained from several state-of-the-art compute-efficient RNN-based speech enhancement architectures using the DNS challenge dataset, show that the D-GRU based model variants maintain similar speech intelligibility and quality metrics comparable to the baseline GRU based models even with an average 50% reduction in GRU computes.
- [33] arXiv:2408.12431 [pdf, html, other]
-
Title: The Hybrid Hospital: Balancing On-Site and Remote HospitalizationSubjects: Systems and Control (eess.SY)
Hybrid hospitals offer on-site and remote hospitalization through telemedicine. These new healthcare models require novel operational policies to balance costs, efficiency, and patient well-being. Our study addresses two first-order questions: (i) how to direct patient admission and call-in based on individual characteristics and proximity and (ii) how to determine the optimal allocation of medical resources between these two hospitalization options and among different patient types.
We develop a model that uses Brownian Motion to capture the patient's health evolution during remote/on-site hospitalization and during travel. Under cost-minimizing call-in policies, we find that remote hospitalization can be cost-effective for moderately distant patients, as the optimal call-in threshold is non-monotonic in the patient's travel time. Subject to scarce resources, the optimal solution structure becomes equivalent to a simultaneous, identically sized increase of remote and on-site costs under abundant resources. When limited resources must be divided among multiple patient types, the optimal thresholds shift in non-obvious ways as resource availability changes. Finally, we develop a practical and efficient policy that allows for swapping an on-site patient with a remote patient when the latter is called-in and sufficient resources are not available to treat both on-site.
Contrary to the widely held view that telemedicine can mitigate rural and non-rural healthcare disparities, our research suggests that on-site care may actually be more cost-effective than remote hospitalization for patients in distant locations, due to (potentially overlooked) risks during patient travel. This finding may be of particular concern in light of the growing number of ``hospital deserts'' amid recent rural hospital closures, as these communities may in fact not be well-served through at-home care. - [34] arXiv:2408.12457 [pdf, other]
-
Title: Data-driven MPC with terminal conditions in the Koopman frameworkComments: Accepted for presentation at the 63rd IEEE Conference on Decision and Control (CDC2024)Subjects: Systems and Control (eess.SY); Optimization and Control (math.OC)
We investigate nonlinear model predictive control (MPC) with terminal conditions in the Koopman framework using extended dynamic mode decomposition (EDMD) to generate a data-based surrogate model for prediction and optimization. We rigorously show recursive feasibility and prove practical asymptotic stability w.r.t. the approximation accuracy. To this end, finite-data error bounds are employed. The construction of the terminal conditions is based on recently derived proportional error bounds to ensure the required Lyapunov decrease. Finally, we illustrate the effectiveness of the proposed data-driven predictive controller including the design procedure to construct the terminal region and controller.
- [35] arXiv:2408.12510 [pdf, html, other]
-
Title: A Note on an Upper-Bound for the Sum of a Class K and an Extended Class K FunctionComments: 4 pagesSubjects: Systems and Control (eess.SY)
In this short note, we derive an upper-bound for the sum of two comparison functions, namely for the sum of a class K and an extended class K function. To the best of our knowledge, the relations derived in this note have not been previously derived in the literature.
- [36] arXiv:2408.12534 [pdf, html, other]
-
Title: Automatic Organ and Pan-cancer Segmentation in Abdomen CT: the FLARE 2023 ChallengeJun Ma, Yao Zhang, Song Gu, Cheng Ge, Ershuai Wang, Qin Zhou, Ziyan Huang, Pengju Lyu, Jian He, Bo WangComments: MICCAI 2024 FLARE Challenge SummarySubjects: Image and Video Processing (eess.IV); Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV)
Organ and cancer segmentation in abdomen Computed Tomography (CT) scans is the prerequisite for precise cancer diagnosis and treatment. Most existing benchmarks and algorithms are tailored to specific cancer types, limiting their ability to provide comprehensive cancer analysis. This work presents the first international competition on abdominal organ and pan-cancer segmentation by providing a large-scale and diverse dataset, including 4650 CT scans with various cancer types from over 40 medical centers. The winning team established a new state-of-the-art with a deep learning-based cascaded framework, achieving average Dice Similarity Coefficient scores of 92.3% for organs and 64.9% for lesions on the hidden multi-national testing set. The dataset and code of top teams are publicly available, offering a benchmark platform to drive further innovations this https URL.
- [37] arXiv:2408.12535 [pdf, html, other]
-
Title: Impact of the Inflation Reduction Act and Carbon Capture on Transportation Electrification for a Net-Zero Western U.S. GridComments: This is a preprint. It's complete copyright version will be available on the publisher's website after publicationSubjects: Systems and Control (eess.SY)
The electrification of transportation is critical to mitigate Greenhouse Gas (GHG) emissions. The United States (U.S.) government's Inflation Reduction Act (IRA) of 2022 introduces policies to promote the electrification of transportation. In addition to electrifying transportation, clean energy technologies such as Carbon Capture and Storage (CCS) may play a major role in achieving a net-zero energy system. Utilizing scenarios simulated by the U.S. version of the Global Change Analysis Model (GCAM-USA), we analyze the individual and compound contributions of the IRA and CCS to reach a clean U.S. grid by 2035 and net-zero GHG emissions by 2050. We analyze the contributions based on three metrics: i) transportation electrification rate, ii) transportation fuel mix, and iii) spatio-temporal charging loads. Our findings indicate that the IRA significantly accelerates transportation electrification in the near-term (until 2035). In contrast, CCS technologies, by enabling the continued use of internal combustion vehicles while still advancing torward net-zero, potentially suppresses the rate of transportation electrification in the long-term. This study underscores how policy and technology innovation can interact and sensitivity studies with different combination are essential to characterize the potential contributions of each to the transportation electrification.
New submissions for Friday, 23 August 2024 (showing 37 of 37 entries )
- [38] arXiv:2408.01463 (cross-list from nlin.PS) [pdf, html, other]
-
Title: Shock waves in nonlinear transmission linesComments: pdfLaTeX, 8 pages, 4 figures. The paper was edited substantiallySubjects: Pattern Formation and Solitons (nlin.PS); Systems and Control (eess.SY); Applied Physics (physics.app-ph)
We consider interaction between the small amplitude travelling waves ("sound") and the shock waves in the transmission line containing both nonlinear capacitors and nonlinear inductors. We calculate for the "sound" wave the coefficient of reflection from (the coefficient of transmission through) the shock wave. These coefficients are expressed in terms of the wave speeds and the wave impedances. When only the capacitors or only the inductors are nonlinear, the coefficients are expressed in terms of the wave speeds only. We explicitly include into consideration of the shocks the dissipation, introducing ohmic resistors shunting the inductors and also in series with the capacitors. This allows us to describe the shocks as physical objects of finite width and study their profiles. In some particular cases the profiles were obtained in terms of elementary functions.
- [39] arXiv:2408.11829 (cross-list from cs.CV) [pdf, other]
-
Title: FAKER: Full-body Anonymization with Human Keypoint Extraction for Real-time Video DeidentificationSubjects: Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG); Image and Video Processing (eess.IV)
In the contemporary digital era, protection of personal information has become a paramount issue. The exponential growth of the media industry has heightened concerns regarding the anonymization of individuals captured in video footage. Traditional methods, such as blurring or pixelation, are commonly employed, while recent advancements have introduced generative adversarial networks (GAN) to redraw faces in videos. In this study, we propose a novel approach that employs a significantly smaller model to achieve real-time full-body anonymization of individuals in videos. Unlike conventional techniques that often fail to effectively remove personal identification information such as skin color, clothing, accessories, and body shape while our method successfully eradicates all such details. Furthermore, by leveraging pose estimation algorithms, our approach accurately represents information regarding individuals' positions, movements, and postures. This algorithm can be seamlessly integrated into CCTV or IP camera systems installed in various industrial settings, functioning in real-time and thus facilitating the widespread adoption of full-body anonymization technology.
- [40] arXiv:2408.11837 (cross-list from cs.LG) [pdf, html, other]
-
Title: MicroXercise: A Micro-Level Comparative and Explainable System for Remote Physical TherapyComments: Accepted by IEEE/ACM CHASE 2024Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Human-Computer Interaction (cs.HC); Signal Processing (eess.SP)
Recent global estimates suggest that as many as 2.41 billion individuals have health conditions that would benefit from rehabilitation services. Home-based Physical Therapy (PT) faces significant challenges in providing interactive feedback and meaningful observation for therapists and patients. To fill this gap, we present MicroXercise, which integrates micro-motion analysis with wearable sensors, providing therapists and patients with a comprehensive feedback interface, including video, text, and scores. Crucially, it employs multi-dimensional Dynamic Time Warping (DTW) and attribution-based explainable methods to analyze the existing deep learning neural networks in monitoring exercises, focusing on a high granularity of exercise. This synergistic approach is pivotal, providing output matching the input size to precisely highlight critical subtleties and movements in PT, thus transforming complex AI analysis into clear, actionable feedback. By highlighting these micro-motions in different metrics, such as stability and range of motion, MicroXercise significantly enhances the understanding and relevance of feedback for end-users. Comparative performance metrics underscore its effectiveness over traditional methods, such as a 39% and 42% improvement in Feature Mutual Information (FMI) and Continuity. MicroXercise is a step ahead in home-based physical therapy, providing a technologically advanced and intuitively helpful solution to enhance patient care and outcomes.
- [41] arXiv:2408.11849 (cross-list from cs.CL) [pdf, html, other]
-
Title: Style-Talker: Finetuning Audio Language Model and Style-Based Text-to-Speech Model for Fast Spoken Dialogue GenerationComments: CoLM 2024Subjects: Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Audio and Speech Processing (eess.AS)
The rapid advancement of large language models (LLMs) has significantly propelled the development of text-based chatbots, demonstrating their capability to engage in coherent and contextually relevant dialogues. However, extending these advancements to enable end-to-end speech-to-speech conversation bots remains a formidable challenge, primarily due to the extensive dataset and computational resources required. The conventional approach of cascading automatic speech recognition (ASR), LLM, and text-to-speech (TTS) models in a pipeline, while effective, suffers from unnatural prosody because it lacks direct interactions between the input audio and its transcribed text and the output audio. These systems are also limited by their inherent latency from the ASR process for real-time applications. This paper introduces Style-Talker, an innovative framework that fine-tunes an audio LLM alongside a style-based TTS model for fast spoken dialog generation. Style-Talker takes user input audio and uses transcribed chat history and speech styles to generate both the speaking style and text for the response. Subsequently, the TTS model synthesizes the speech, which is then played back to the user. While the response speech is being played, the input speech undergoes ASR processing to extract the transcription and speaking style, serving as the context for the ensuing dialogue turn. This novel pipeline accelerates the traditional cascade ASR-LLM-TTS systems while integrating rich paralinguistic information from input speech. Our experimental results show that Style-Talker significantly outperforms the conventional cascade and speech-to-speech baselines in terms of both dialogue naturalness and coherence while being more than 50% faster.
- [42] arXiv:2408.11858 (cross-list from cs.CL) [pdf, html, other]
-
Title: Convexity-based Pruning of Speech Representation ModelsSubjects: Computation and Language (cs.CL); Sound (cs.SD); Audio and Speech Processing (eess.AS)
Speech representation models based on the transformer architecture and trained by self-supervised learning have shown great promise for solving tasks such as speech and speaker recognition, keyword spotting, emotion detection, and more. Typically, it is found that larger models lead to better performance. However, the significant computational effort involved in such large transformer systems is a challenge for embedded and real-world applications. Recent work has shown that there is significant redundancy in the transformer models for NLP and massive layer pruning is feasible (Sajjad et al., 2023). Here, we investigate layer pruning in audio models. We base the pruning decision on a convexity criterion. Convexity of classification regions has recently been proposed as an indicator of subsequent fine-tuning performance in a range of application domains, including NLP and audio. In empirical investigations, we find a massive reduction in the computational effort with no loss of performance or even improvements in certain cases.
- [43] arXiv:2408.11882 (cross-list from q-bio.NC) [pdf, other]
-
Title: Prosody of speech production in latent post-stroke aphasiaComments: Interspeech 2024Subjects: Neurons and Cognition (q-bio.NC); Sound (cs.SD); Audio and Speech Processing (eess.AS); Quantitative Methods (q-bio.QM)
This study explores prosodic production in latent aphasia, a mild form of aphasia associated with left-hemisphere brain damage (e.g. stroke). Unlike prior research on moderate to severe aphasia, we investigated latent aphasia, which can seem to have very similar speech production with neurotypical speech. We analysed the f0, intensity and duration of utterance-initial and utterance-final words of ten speakers with latent aphasia and ten matching controls. Regression models were fitted to improve our understanding of this understudied type of very mild aphasia. The results highlighted varying degrees of differences in all three prosodic measures between groups. We also investigated the diagnostic classification of latent aphasia versus neurotypical control using random forest, aiming to build a fast and reliable tool to assist with the identification of latent aphasia. The random forest analysis also reinforced the significance of prosodic features in distinguishing latent aphasia.
- [44] arXiv:2408.11885 (cross-list from physics.med-ph) [pdf, html, other]
-
Title: HDN:Hybrid Deep-learning and Non-line-of-sight Reconstruction Framework for Photoacoustic Brain ImagingComments: 8 pages, 8figuresSubjects: Medical Physics (physics.med-ph); Image and Video Processing (eess.IV); Optics (physics.optics)
Photoacoustic imaging (PAI) combines the high contrast of optical imaging with the deep penetration depth of ultrasonic imaging, showing great potential in cerebrovascular disease detection. However, the ultrasonic wave suffers strong attenuation and multi-scattering when it passes through the skull tissue, resulting in the distortion of the collected photoacoustic (PA) signal. In this paper, inspired by the principles of deep learning and non-line-of-sight (NLOS) imaging, we propose an image reconstruction framework named HDN (Hybrid Deep-learning and Non-line-of-sight), which consists of the signal extraction part and difference utilization part. The signal extraction part is used to correct the distorted signal and reconstruct an initial image. The difference utilization part is used to make further use of the signal difference between the distorted signal and corrected signal, reconstructing the residual image between the initial image and the target image. The test results on a PA digital brain simulation dataset show that compared with the traditional delay-and-sum (DAS) method and deep-learning-based method, HDN achieved superior performance in both signal correction and image reconstruction. Specifically for the SSIM index, the HDN reached 0.606 in imaging results, compared to 0.154 for the DAS method and 0.307 for the deep-learning-based method.
- [45] arXiv:2408.11915 (cross-list from cs.SD) [pdf, html, other]
-
Title: Video-Foley: Two-Stage Video-To-Sound Generation via Temporal Event Condition For Foley SoundSubjects: Sound (cs.SD); Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG); Multimedia (cs.MM); Audio and Speech Processing (eess.AS)
Foley sound synthesis is crucial for multimedia production, enhancing user experience by synchronizing audio and video both temporally and semantically. Recent studies on automating this labor-intensive process through video-to-sound generation face significant challenges. Systems lacking explicit temporal features suffer from poor controllability and alignment, while timestamp-based models require costly and subjective human annotation. We propose Video-Foley, a video-to-sound system using Root Mean Square (RMS) as a temporal event condition with semantic timbre prompts (audio or text). RMS, a frame-level intensity envelope feature closely related to audio semantics, ensures high controllability and synchronization. The annotation-free self-supervised learning framework consists of two stages, Video2RMS and RMS2Sound, incorporating novel ideas including RMS discretization and RMS-ControlNet with a pretrained text-to-audio model. Our extensive evaluation shows that Video-Foley achieves state-of-the-art performance in audio-visual alignment and controllability for sound timing, intensity, timbre, and nuance. Code, model weights, and demonstrations are available on the accompanying website. (this https URL)
- [46] arXiv:2408.11920 (cross-list from cs.IT) [pdf, html, other]
-
Title: Modular Hypernetworks for Scalable and Adaptive Deep MIMO ReceiversSubjects: Information Theory (cs.IT); Signal Processing (eess.SP)
Deep neural networks (DNNs) were shown to facilitate the operation of uplink multiple-input multiple-output (MIMO) receivers, with emerging architectures augmenting modules of classic receiver processing. Current designs consider static DNNs, whose architecture is fixed and weights are pre-trained. This induces a notable challenge, as the resulting MIMO receiver is suitable for a given configuration, i.e., channel distribution and number of users, while in practice these parameters change frequently with network variations and users leaving and joining the network. In this work, we tackle this core challenge of DNN-aided MIMO receivers. We build upon the concept of hypernetworks, augmenting the receiver with a pre-trained deep model whose purpose is to update the weights of the DNN-aided receiver upon instantaneous channel variations. We design our hypernetwork to augment modular deep receivers, leveraging their modularity to have the hypernetwork adapt not only the weights, but also the architecture. Our modular hypernetwork leads to a DNN-aided receiver whose architecture and resulting complexity adapts to the number of users, in addition to channel variations, without retraining. Our numerical studies demonstrate superior error-rate performance of modular hypernetworks in time-varying channels compared to static pre-trained receivers, while providing rapid adaptivity and scalability to network variations.
- [47] arXiv:2408.11937 (cross-list from math.OC) [pdf, other]
-
Title: Distributed alternating gradient descent for convex semi-infinite programs over a networkComments: 16 pages, 1 figureSubjects: Optimization and Control (math.OC); Systems and Control (eess.SY)
This paper presents a first-order distributed algorithm for solving a convex semi-infinite program (SIP) over a time-varying network. In this setting, the objective function associated with the optimization problem is a summation of a set of functions, each held by one node in a network. The semi-infinite constraint, on the other hand, is known to all agents. The nodes collectively aim to solve the problem using local data about the objective and limited communication capabilities depending on the network topology. Our algorithm is built on three key ingredients: consensus step, gradient descent in the local objective, and local gradient descent iterations in the constraint at a node when the estimate violates the semi-infinite constraint. The algorithm is constructed, and its parameters are prescribed in such a way that the iterates held by each agent provably converge to an optimizer. That is, as the algorithm progresses, the estimates achieve consensus, and the constraint violation and the error in the optimal value are bounded above by vanishing terms. A simulation example illustrates our results.
- [48] arXiv:2408.12021 (cross-list from cs.CR) [pdf, html, other]
-
Title: R-STELLAR: A Resilient Synthesizable Signature Attenuation SCA Protection on AES-256 with built-in Attack-on-Countermeasure DetectionComments: Extended from CICC. Now under revision at Journal of Solid-State CircuitsSubjects: Cryptography and Security (cs.CR); Signal Processing (eess.SP)
Side channel attacks (SCAs) remain a significant threat to the security of cryptographic systems in modern embedded devices. Even mathematically secure cryptographic algorithms, when implemented in hardware, inadvertently leak information through physical side channel signatures such as power consumption, electromagnetic (EM) radiation, light emissions, and acoustic emanations. Exploiting these side channels significantly reduces the search space of the attacker. In recent years, physical countermeasures have significantly increased the minimum traces to disclosure (MTD) to 1 billion. Among them, signature attenuation is the first method to achieve this mark. Signature attenuation often relies on analog techniques, and digital signature attenuation reduces MTD to 20 million, requiring additional methods for high resilience. We focus on improving the digital signature attenuation by an order of magnitude (MTD 200M). Additionally, we explore possible attacks against signature attenuation countermeasure. We introduce a Voltage drop Linear region Biasing (VLB) attack technique that reduces the MTD to over 2000 times less than the previous threshold. This is the first known attack against a physical side-channel attack (SCA) countermeasure. We have implemented an attack detector with a response time of 0.8 milliseconds to detect such attacks, limiting SCA leakage window to sub-ms, which is insufficient for a successful attack.
- [49] arXiv:2408.12048 (cross-list from cs.CV) [pdf, html, other]
-
Title: ISETHDR: A Physics-based Synthetic Radiance Dataset for High Dynamic Range Driving ScenesSubjects: Computer Vision and Pattern Recognition (cs.CV); Image and Video Processing (eess.IV)
This paper describes a physics-based end-to-end software simulation for image systems. We use the software to explore sensors designed to enhance performance in high dynamic range (HDR) environments, such as driving through daytime tunnels and under nighttime conditions. We synthesize physically realistic HDR spectral radiance images and use them as the input to digital twins that model the optics and sensors of different systems. This paper makes three main contributions: (a) We create a labeled (instance segmentation and depth), synthetic radiance dataset of HDR driving scenes. (b) We describe the development and validation of the end-to-end simulation framework. (c) We present a comparative analysis of two single-shot sensors designed for HDR. We open-source both the dataset and the software.
- [50] arXiv:2408.12069 (cross-list from cs.IT) [pdf, html, other]
-
Title: Rotatable Block-Controlled RIS: Bridging the Performance Gap to Element-Controlled SystemsComments: This work has been submitted to the IEEE for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessibleSubjects: Information Theory (cs.IT); Signal Processing (eess.SP)
The passive reconfigurable intelligent surface (RIS) requires numerous elements to achieve adequate array gain, which linearly increases power consumption (PC) with the number of reflection phases. To address this, this letter introduces a rotatable block-controlled RIS (BC-RIS) that preserves spectral efficiency (SE) while reducing power costs. Unlike the element-controlled RIS (EC-RIS), which necessitates independent phase control for each element, the BC-RIS uses a single phase control circuit for each block, substantially lowering power requirements. In the maximum ratio transmission, by customizing specular reflection channels through the rotation of blocks and coherently superimposing signals with optimized reflection phase of blocks, the BC-RIS achieves the same averaged SE as the EC-RIS. To counteract the added power demands from rotation, influenced by block size, we have developed a segmentation scheme to minimize overall PC. Furthermore, constraints for rotation power-related parameters have been established to enhance the energy efficiency of the BC-RIS compared to the EC-RIS. Numerical results confirm that this approach significantly improves energy efficiency while maintaining performance.
- [51] arXiv:2408.12102 (cross-list from cs.LG) [pdf, html, other]
-
Title: Integrating Audio, Visual, and Semantic Information for Enhanced Multimodal Speaker DiarizationSubjects: Machine Learning (cs.LG); Computer Vision and Pattern Recognition (cs.CV); Sound (cs.SD); Audio and Speech Processing (eess.AS)
Speaker diarization, the process of segmenting an audio stream or transcribed speech content into homogenous partitions based on speaker identity, plays a crucial role in the interpretation and analysis of human speech. Most existing speaker diarization systems rely exclusively on unimodal acoustic information, making the task particularly challenging due to the innate ambiguities of audio signals. Recent studies have made tremendous efforts towards audio-visual or audio-semantic modeling to enhance performance. However, even the incorporation of up to two modalities often falls short in addressing the complexities of spontaneous and unstructured conversations. To exploit more meaningful dialogue patterns, we propose a novel multimodal approach that jointly utilizes audio, visual, and semantic cues to enhance speaker diarization. Our method elegantly formulates the multimodal modeling as a constrained optimization problem. First, we build insights into the visual connections among active speakers and the semantic interactions within spoken content, thereby establishing abundant pairwise constraints. Then we introduce a joint pairwise constraint propagation algorithm to cluster speakers based on these visual and semantic constraints. This integration effectively leverages the complementary strengths of different modalities, refining the affinity estimation between individual speaker embeddings. Extensive experiments conducted on multiple multimodal datasets demonstrate that our approach consistently outperforms state-of-the-art speaker diarization methods.
- [52] arXiv:2408.12124 (cross-list from cs.LG) [pdf, html, other]
-
Title: Recording Brain Activity While Listening to Music Using Wearable EEG Devices Combined with Bidirectional Long Short-Term Memory NetworksComments: 15 pagesSubjects: Machine Learning (cs.LG); Human-Computer Interaction (cs.HC); Signal Processing (eess.SP)
Electroencephalography (EEG) signals are crucial for investigating brain function and cognitive processes. This study aims to address the challenges of efficiently recording and analyzing high-dimensional EEG signals while listening to music to recognize emotional states. We propose a method combining Bidirectional Long Short-Term Memory (Bi-LSTM) networks with attention mechanisms for EEG signal processing. Using wearable EEG devices, we collected brain activity data from participants listening to music. The data was preprocessed, segmented, and Differential Entropy (DE) features were extracted. We then constructed and trained a Bi-LSTM model to enhance key feature extraction and improve emotion recognition accuracy. Experiments were conducted on the SEED and DEAP datasets. The Bi-LSTM-AttGW model achieved 98.28% accuracy on the SEED dataset and 92.46% on the DEAP dataset in multi-class emotion recognition tasks, significantly outperforming traditional models such as SVM and EEG-Net. This study demonstrates the effectiveness of combining Bi-LSTM with attention mechanisms, providing robust technical support for applications in brain-computer interfaces (BCI) and affective computing. Future work will focus on improving device design, incorporating multimodal data, and further enhancing emotion recognition accuracy, aiming to achieve practical applications in real-world scenarios.
- [53] arXiv:2408.12129 (cross-list from cs.LG) [pdf, other]
-
Title: Deep Analysis of Time Series Data for Smart Grid Startup Strategies: A Transformer-LSTM-PSO Model ApproachComments: 46 pagesSubjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Systems and Control (eess.SY)
Grid startup, an integral component of the power system, holds strategic importance for ensuring the reliability and efficiency of the electrical grid. However, current methodologies for in-depth analysis and precise prediction of grid startup scenarios are inadequate. To address these challenges, we propose a novel method based on the Transformer-LSTM-PSO model. This model uniquely combines the Transformer's self-attention mechanism, LSTM's temporal modeling capabilities, and the parameter tuning features of the particle swarm optimization algorithm. It is designed to more effectively capture the complex temporal relationships in grid startup schemes. Our experiments demonstrate significant improvements, with our model achieving lower RMSE and MAE values across multiple datasets compared to existing benchmarks, particularly in the NYISO Electric Market dataset where the RMSE was reduced by approximately 15% and the MAE by 20% compared to conventional models. Our main contribution is the development of a Transformer-LSTM-PSO model that significantly enhances the accuracy and efficiency of smart grid startup predictions. The application of the Transformer-LSTM-PSO model represents a significant advancement in smart grid predictive analytics, concurrently fostering the development of more reliable and intelligent grid management systems.
- [54] arXiv:2408.12134 (cross-list from cs.IT) [pdf, html, other]
-
Title: Machine Learning-based Channel Prediction in Wideband Massive MIMO Systems with Small Overhead for Online TrainingComments: 16 pages, 16 figures, 4 tablesSubjects: Information Theory (cs.IT); Signal Processing (eess.SP)
Channel prediction compensates for outdated channel state information in multiple-input multiple-output (MIMO) systems. Machine learning (ML) techniques have recently been implemented to design channel predictors by leveraging the temporal correlation of wireless channels. However, most ML-based channel prediction techniques have only considered offline training when generating channel predictors, which can result in poor performance when encountering channel environments different from the ones they were trained on. To ensure prediction performance in varying channel conditions, we propose an online re-training framework that trains the channel predictor from scratch to effectively capture and respond to changes in the wireless environment. The training time includes data collection time and neural network training time, and should be minimized for practical channel predictors. To reduce the training time, especially data collection time, we propose a novel ML-based channel prediction technique called aggregated learning (AL) approach for wideband massive MIMO systems. In the proposed AL approach, the training data can be split and aggregated either in an array domain or frequency domain, which are the channel domains of MIMO-OFDM systems. This processing can significantly reduce the time for data collection. Our numerical results show that the AL approach even improves channel prediction performance in various scenarios with small training time overhead.
- [55] arXiv:2408.12162 (cross-list from cs.IT) [pdf, html, other]
-
Title: Empowering Over-the-Air Personalized Federated Learning via RISComments: Accepted by SCIENCE CHINA Information SciencesSubjects: Information Theory (cs.IT); Signal Processing (eess.SP)
Over-the-air computation (AirComp) integrates analog communication with task-oriented computation, serving as a key enabling technique for communication-efficient federated learning (FL) over wireless networks. However, AirComp-enabled FL (AirFL) with a single global consensus model fails to address the data heterogeneity in real-life FL scenarios with non-independent and identically distributed local datasets. In this paper, we introduce reconfigurable intelligent surface (RIS) technology to enable efficient personalized AirFL, mitigating the data heterogeneity issue. First, we achieve statistical interference elimination across different clusters in the personalized AirFL framework via RIS phase shift configuration. Then, we propose two personalized aggregation schemes involving power control and denoising factor design from the perspectives of first- and second-order moments, respectively, to enhance the FL convergence. Numerical results validate the superior performance of our proposed schemes over existing baselines.
- [56] arXiv:2408.12170 (cross-list from cs.HC) [pdf, html, other]
-
Title: VoiceX: A Text-To-Speech Framework for Custom VoicesSilvan Mertes, Daksitha Withanage Don, Otto Grothe, Johanna Kuch, Ruben Schlagowski, Elisabeth AndréSubjects: Human-Computer Interaction (cs.HC); Sound (cs.SD); Audio and Speech Processing (eess.AS)
Modern TTS systems are capable of creating highly realistic and natural-sounding speech. Despite these developments, the process of customizing TTS voices remains a complex task, mostly requiring the expertise of specialists within the field. One reason for this is the utilization of deep learning models, which are characterized by their expansive, non-interpretable parameter spaces, restricting the feasibility of manual customization. In this paper, we present a novel human-in-the-loop paradigm based on an evolutionary algorithm for directly interacting with the parameter space of a neural TTS model. We integrated our approach into a user-friendly graphical user interface that allows users to efficiently create original voices. Those voices can then be used with the backbone TTS model, for which we provide a Python API. Further, we present the results of a user study exploring the capabilities of VoiceX. We show that VoiceX is an appropriate tool for creating individual, custom voices.
- [57] arXiv:2408.12255 (cross-list from cs.IT) [pdf, html, other]
-
Title: Fast Iterative ELAA-MIMO Detection Exploiting Static Channel ComponentsComments: This work has been accepted by the IEEE Information Theory Workshop (ITW) 2024. Copyright may be transferred without notice, after which this version may no longer be accessibleSubjects: Information Theory (cs.IT); Signal Processing (eess.SP)
Extremely large aperture array (ELAA) is a promising multiple-input multiple-output (MIMO) technique for next generation mobile networks. In this paper, we propose two novel approaches to accelerate the convergence of current iterative MIMO detectors in ELAA channels. Our approaches exploit the static components of the ELAA channel, which include line of sight (LoS) paths and deterministic non-LoS (NLoS) components due to channel hardening effects. This paper proposes novel convergence acceleration techniques for fast iterative ELAA-MIMO detection by leveraging the static channel component, including the LoS paths and deterministic NLoS components that arise due to channel hardening. Specifically, these static channel components are utilized in two ways: as preconditioning matrices for general iterative algorithms, and as initialization for quasi-Newton (QN) methods. Simulation results show that the proposed approaches converge significantly faster compared to current iterative MIMO detectors, especially under strong LoS conditions with high Rician K-factor. Furthermore, QN methods with the proposed initialization matrix consistently achieve the best convergence performance while maintaining low complexity.
- [58] arXiv:2408.12279 (cross-list from cs.SD) [pdf, html, other]
-
Title: Developing vocal system impaired patient-aimed voice quality assessment approach using ASR representation-included multiple featuresShaoxiang Dang, Tetsuya Matsumoto, Yoshinori Takeuchi, Takashi Tsuboi, Yasuhiro Tanaka, Daisuke Nakatsubo, Satoshi Maesawa, Ryuta Saito, Masahisa Katsuno, Hiroaki KudoComments: Accepted by Interspeech 2024Subjects: Sound (cs.SD); Artificial Intelligence (cs.AI); Audio and Speech Processing (eess.AS)
The potential of deep learning in clinical speech processing is immense, yet the hurdles of limited and imbalanced clinical data samples loom large. This article addresses these challenges by showcasing the utilization of automatic speech recognition and self-supervised learning representations, pre-trained on extensive datasets of normal speech. This innovative approach aims to estimate voice quality of patients with impaired vocal systems. Experiments involve checks on PVQD dataset, covering various causes of vocal system damage in English, and a Japanese dataset focusing on patients with Parkinson's disease before and after undergoing subthalamic nucleus deep brain stimulation (STN-DBS) surgery. The results on PVQD reveal a notable correlation (>0.8 on PCC) and an extraordinary accuracy (<0.5 on MSE) in predicting Grade, Breathy, and Asthenic indicators. Meanwhile, progress has been achieved in predicting the voice quality of patients in the context of STN-DBS.
- [59] arXiv:2408.12329 (cross-list from cs.IT) [pdf, html, other]
-
Title: Asynchronous Cell-Free Massive MIMO-OFDM: Mixed Coherent and Non-Coherent TransmissionsComments: This work is submitted to IEEE for possible publicationSubjects: Information Theory (cs.IT); Signal Processing (eess.SP)
In this letter, we analyze the performance of mixed coherent and non-coherent transmissions approach, which can improve the performance of cell-free multiple-input multiple-output orthogonal frequency division multiplexing (CF mMIMO-OFDM) systems under asynchronous reception. To this end, we first obtain the achievable downlink sum-rate for the mixed coherent and non-coherent transmissions, and then provide a closed-form expression for the case with the maximum ratio precoding. Subsequently, an efficient clustering algorithm is proposed to group access points into different clusters with the same quantized phase shift in each cluster. Numerical results demonstrate that the mixed coherent and non-coherent transmissions can effectively improve the sum-rate of CF mMIMO-OFDM systems under asynchronous reception.
- [60] arXiv:2408.12337 (cross-list from cs.CL) [pdf, html, other]
-
Title: Fine-tuning Smaller Language Models for Question Answering over Financial DocumentsSubjects: Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Machine Learning (cs.LG); Systems and Control (eess.SY)
Recent research has shown that smaller language models can acquire substantial reasoning abilities when fine-tuned with reasoning exemplars crafted by a significantly larger teacher model. We explore this paradigm for the financial domain, focusing on the challenge of answering questions that require multi-hop numerical reasoning over financial texts. We assess the performance of several smaller models that have been fine-tuned to generate programs that encode the required financial reasoning and calculations. Our findings demonstrate that these fine-tuned smaller models approach the performance of the teacher model.
To provide a granular analysis of model performance, we propose an approach to investigate the specific student model capabilities that are enhanced by fine-tuning. Our empirical analysis indicates that fine-tuning refines the student models ability to express and apply the required financial concepts along with adapting the entity extraction for the specific data format. In addition, we hypothesize and demonstrate that comparable financial reasoning capability can be induced using relatively smaller datasets. - [61] arXiv:2408.12360 (cross-list from cs.RO) [pdf, other]
-
Title: Recursive Distributed Collaborative Aided Inertial NavigationComments: PhD thesisSubjects: Robotics (cs.RO); Systems and Control (eess.SY)
In this dissertation, we investigate the issue of robust localization in swarms of heterogeneous mobile agents with multiple and time-varying sensing modalities. Our focus is the development of filter-based and decoupled estimators under the assumption that agents possess communication and processing capabilities. Based on the findings from Distributed Collaborative State Estimation and modular sensor fusion, we propose a novel Kalman filter decoupling paradigm, which is termed Isolated Kalman Filtering (IKF). This paradigm is formally discussed and the treatment of delayed measurement is studied. The impact of approximation made was investigated on different observation graphs and the filter credibility was evaluated on a linear system in a Monte Carlo simulation. Finally, we propose a multi-agent modular sensor fusion approach based on the IKF paradigm, in order to cooperatively estimate the global state of a multi-agent system in a distributed way and fuse information provided by different on-board sensors in a computationally efficient way. As a consequence, this approach can be performed distributed among agents, while (i) communication between agents is only required at the moment of inter-agent joint observations, (ii) one agent acts as interim master to process state corrections isolated, (iii) agents can be added and removed from the swarm, (iv) each agent's full state can vary during mission (each local sensor suite can be truly modular), and (v) delayed and multi-rate sensor updates are supported. Extensive evaluation on realistic simulated and real-world data sets show that the proposed Isolated Kalman Filtering (IKF) paradigm, is applicable for both, truly modular single agent estimation and distributed collaborative multi-agent estimation problems.
- [62] arXiv:2408.12481 (cross-list from cs.SD) [pdf, html, other]
-
Title: Self-Learning for Personalized Keyword Spotting on Ultra-Low-Power Audio SensorsSubjects: Sound (cs.SD); Machine Learning (cs.LG); Audio and Speech Processing (eess.AS)
This paper proposes a self-learning framework to incrementally train (fine-tune) a personalized Keyword Spotting (KWS) model after the deployment on ultra-low power smart audio sensors. We address the fundamental problem of the absence of labeled training data by assigning pseudo-labels to the new recorded audio frames based on a similarity score with respect to few user recordings. By experimenting with multiple KWS models with a number of parameters up to 0.5M on two public datasets, we show an accuracy improvement of up to +19.2% and +16.0% vs. the initial models pretrained on a large set of generic keywords. The labeling task is demonstrated on a sensor system composed of a low-power microphone and an energy-efficient Microcontroller (MCU). By efficiently exploiting the heterogeneous processing engines of the MCU, the always-on labeling task runs in real-time with an average power cost of up to 8.2 mW. On the same platform, we estimate an energy cost for on-device training 10x lower than the labeling energy if sampling a new utterance every 5 s or 16.4 s with a DS-CNN-S or a DS-CNN-M model. Our empirical result paves the way to self-adaptive personalized KWS sensors at the extreme edge.
- [63] arXiv:2408.12500 (cross-list from cs.HC) [pdf, html, other]
-
Title: WhisperMask: A Noise Suppressive Mask-Type Microphone for Whisper SpeechComments: 14 pages, 14 figuresJournal-ref: Proceedings of the Augmented Humans International Conference 2024Subjects: Human-Computer Interaction (cs.HC); Sound (cs.SD); Audio and Speech Processing (eess.AS)
Whispering is a common privacy-preserving technique in voice-based interactions, but its effectiveness is limited in noisy environments. In conventional hardware- and software-based noise reduction approaches, isolating whispered speech from ambient noise and other speech sounds remains a challenge. We thus propose WhisperMask, a mask-type microphone featuring a large diaphragm with low sensitivity, making the wearer's voice significantly louder than the background noise. We evaluated WhisperMask using three key metrics: signal-to-noise ratio, quality of recorded voices, and speech recognition rate. Across all metrics, WhisperMask consistently outperformed traditional noise-suppressing microphones and software-based solutions. Notably, WhisperMask showed a 30% higher recognition accuracy for whispered speech recorded in an environment with 80 dB background noise compared with the pin microphone and earbuds. Furthermore, while a denoiser decreased the whispered speech recognition rate of these two microphones by approximately 20% at 30-60 dB noise, WhisperMask maintained a high performance even without denoising, surpassing the other microphones' performances by a significant margin.WhisperMask's design renders the wearer's voice as the dominant input and effectively suppresses background noise without relying on signal processing. This device allows for reliable voice interactions, such as phone calls and voice commands, in a wide range of noisy real-world scenarios while preserving user privacy.
- [64] arXiv:2408.12549 (cross-list from cs.SD) [pdf, html, other]
-
Title: Modeling Time-Variant Responses of Optical Compressors with Selective State Space ModelsSubjects: Sound (cs.SD); Artificial Intelligence (cs.AI); Audio and Speech Processing (eess.AS)
This paper presents a method for modeling optical dynamic range compressors using deep neural networks with Selective State Space models. The proposed approach surpasses previous methods based on recurrent layers by employing a Selective State Space block to encode the input audio. It features a refined technique integrating Feature-wise Linear Modulation and Gated Linear Units to adjust the network dynamically, conditioning the compression's attack and release phases according to external parameters. The proposed architecture is well-suited for low-latency and real-time applications, crucial in live audio processing. The method has been validated on the analog optical compressors TubeTech CL 1B and Teletronix LA-2A, which possess distinct characteristics. Evaluation is performed using quantitative metrics and subjective listening tests, comparing the proposed method with other state-of-the-art models. Results show that our black-box modeling methods outperform all others, achieving accurate emulation of the compression process for both seen and unseen settings during training. We further show a correlation between this accuracy and the sampling density of the control parameters in the dataset and identify settings with fast attack and slow release as the most challenging to emulate.
- [65] arXiv:2408.12573 (cross-list from math.OC) [pdf, html, other]
-
Title: Population Control of Giardia lambliaComments: 7 pages, 5 figures, 1 TableSubjects: Optimization and Control (math.OC); Systems and Control (eess.SY)
Giardia lamblia is a flagellate intestinal protozoan with global distribution causing the disease known as giardiasis. This parasite is responsable for 35.1% of outbreaks of diarrhea caused by contaminated water which and mainly affects children in whom it can cause physical and cognitive impairment. In this paper, we consider a model of population dynamics to represent the behavior of Giardia lamblia in vitro, taking into account its mutation characteristic that guarantees to the protozoan resistance to the drug metronidazole. Different from what is found in the literature, it is pursued as the control objective the extermination of the protozoan considering that the parameters of the model are uncertain and only the partial measurement of the state vector is possible. On these assumptions, a control law is designed and the stability of the closed-loop system is rigorously proved. Simulation and experimental results illustrate the benefits of the proposed population control method of Giardia lamblia.
- [66] arXiv:2408.12589 (cross-list from cs.IT) [pdf, html, other]
-
Title: Age and Value of Information Optimization for Systems with Multi-Class UpdatesComments: Appeared in IEEE ICC 2024Subjects: Information Theory (cs.IT); Networking and Internet Architecture (cs.NI); Signal Processing (eess.SP)
Received samples of a stochastic process are processed by a server for delivery as updates to a monitor. Each sample belongs to a class that specifies a distribution for its processing time and a function that describes how the value of the processed update decays with age at the monitor. The class of a sample is identified when the processed update is delivered. The server implements a form of M/G/1/1 blocking queue; samples arriving at a busy server are discarded and samples arriving at an idle server are subject to an admission policy that depends on the age and class of the prior delivered update. For the delivered updates, we characterize the average age of information (AoI) and average value of information (VoI). We derive the optimal stationary policy that minimizes the convex combination of the AoI and (negative) VoI. It is shown that the policy has a threshold structure, in which a new sample is allowed to arrive to the server only if the previous update's age and value difference surpasses a certain threshold that depends on the specifics of the value function and system statistics.
Cross submissions for Friday, 23 August 2024 (showing 29 of 29 entries )
- [67] arXiv:2305.18205 (replaced) [pdf, other]
-
Title: Pulse shape discrimination based on the Tempotron: a powerful classifier on GPUComments: 12 pages, 9 figuresJournal-ref: IEEE Transactions on Nuclear Science (2024)Subjects: Signal Processing (eess.SP); Machine Learning (cs.LG); Nuclear Experiment (nucl-ex)
This study utilized the Tempotron, a robust classifier based on a third-generation neural network model, for pulse shape discrimination. By eliminating the need for manual feature extraction, the Tempotron model can process pulse signals directly, generating discrimination results based on prior knowledge. The study performed experiments using GPU acceleration, resulting in over 500 times faster compared to the CPU-based model, and investigated the impact of noise augmentation on the Tempotron performance. Experimental results substantiated that Tempotron serves as a formidable classifier, adept at accomplishing high discrimination accuracy on both AmBe and time-of-flight PuBe datasets. Furthermore, analyzing the neural activity of Tempotron during training shed light on its learning characteristics and aided in selecting its hyperparameters. Moreover, the study addressed the constraints and potential avenues for future development in utilizing the Tempotron for pulse shape discrimination. The dataset used in this study and the GPU-based Tempotron are publicly available on GitHub at this https URL.
- [68] arXiv:2307.05352 (replaced) [pdf, other]
-
Title: Leveraging Variational Autoencoders for Parameterized MMSE EstimationComments: Accepted for publication in the IEEE Transactions on Signal ProcessingSubjects: Signal Processing (eess.SP); Information Theory (cs.IT); Machine Learning (stat.ML)
In this manuscript, we propose to use a variational autoencoder-based framework for parameterizing a conditional linear minimum mean squared error estimator. The variational autoencoder models the underlying unknown data distribution as conditionally Gaussian, yielding the conditional first and second moments of the estimand, given a noisy observation. The derived estimator is shown to approximate the minimum mean squared error estimator by utilizing the variational autoencoder as a generative prior for the estimation problem. We propose three estimator variants that differ in their access to ground-truth data during the training and estimation phases. The proposed estimator variant trained solely on noisy observations is particularly noteworthy as it does not require access to ground-truth data during training or estimation. We conduct a rigorous analysis by bounding the difference between the proposed and the minimum mean squared error estimator, connecting the training objective and the resulting estimation performance. Furthermore, the resulting bound reveals that the proposed estimator entails a bias-variance tradeoff, which is well-known in the estimation literature. As an example application, we portray channel estimation, allowing for a structured covariance matrix parameterization and low-complexity implementation. Nevertheless, the proposed framework is not limited to channel estimation but can be applied to a broad class of estimation problems. Extensive numerical simulations first validate the theoretical analysis of the proposed variational autoencoder-based estimators and then demonstrate excellent estimation performance compared to related classical and machine learning-based state-of-the-art estimators.
- [69] arXiv:2312.07137 (replaced) [pdf, html, other]
-
Title: The AIRI plug-and-play algorithm for image reconstruction in radio-interferometry: variations and robustnessSubjects: Image and Video Processing (eess.IV); Instrumentation and Methods for Astrophysics (astro-ph.IM)
Plug-and-Play (PnP) algorithms are appealing alternatives to proximal algorithms when solving inverse imaging problems. By learning a Deep Neural Network (DNN) denoiser behaving as a proximal operator, one waives the computational complexity of optimisation algorithms induced by sophisticated image priors, and the sub-optimality of handcrafted priors compared to DNNs. Such features are highly desirable in radio-interferometric (RI) imaging, where precision and scalability of the image reconstruction process are key. In previous work, we introduced AIRI, PnP counterpart to the unconstrained variant of the SARA optimisation algorithm, relying on a forward-backward algorithmic backbone. Here, we introduce variations of AIRI towards a more general and robust PnP paradigm in RI imaging. Firstly, we show that the AIRI denoisers can be used without any alteration to instantiate a PnP counterpart to the constrained SARA optimisation algorithm itself, relying on a primal-dual forward-backward algorithmic backbone, thus extending the remit of the AIRI paradigm. Secondly, we show that AIRI algorithms are robust to strong variations in the nature of the training dataset, with denoisers trained on medical images yielding similar reconstruction quality to those trained on astronomical images. Thirdly, we develop a functionality to quantify the model uncertainty introduced by the randomness in the training process. We validate the image reconstruction and uncertainty quantification functionality of AIRI algorithms against the SARA family and CLEAN, both in simulation and on real data of the ESO 137-006 galaxy acquired with the MeerKAT telescope. AIRI code is available in the BASPLib code library on GitHub.
- [70] arXiv:2312.16422 (replaced) [pdf, html, other]
-
Title: Selective-Memory Meta-Learning with Environment Representations for Sound Event Localization and DetectionComments: 14 pages, 11 figures, accepted by IEEE/ACM Transactions on Audio, Speech, and Language Processing (TASLP)Subjects: Audio and Speech Processing (eess.AS); Sound (cs.SD)
Environment shifts and conflicts present significant challenges for learning-based sound event localization and detection (SELD) methods. SELD systems, when trained in particular acoustic settings, often show restricted generalization capabilities for diverse acoustic environments. Furthermore, obtaining annotated samples for spatial sound events is notably costly. Deploying a SELD system in a new environment requires extensive time for re-training and fine-tuning. To overcome these challenges, we propose environment-adaptive Meta-SELD, designed for efficient adaptation to new environments using minimal data. Our method specifically utilizes computationally synthesized spatial data and employs Model-Agnostic Meta-Learning (MAML) on a pre-trained, environment-independent model. The method then utilizes fast adaptation to unseen real-world environments using limited samples from the respective environments. Inspired by the Learning-to-Forget approach, we introduce the concept of selective memory as a strategy for resolving conflicts across environments. This approach involves selectively memorizing target-environment-relevant information and adapting to the new environments through the selective attenuation of model parameters. In addition, we introduce environment representations to characterize different acoustic settings, enhancing the adaptability of our attenuation approach to various environments. We evaluate our proposed method on the development set of the Sony-TAu Realistic Spatial Soundscapes 2023 (STARSS23) dataset and computationally synthesized scenes. Experimental results demonstrate the superior performance of the proposed method compared to conventional supervised learning methods, particularly in localization.
- [71] arXiv:2401.00346 (replaced) [pdf, html, other]
-
Title: Internet of Harvester Nano Things: A Future ProspectsSubjects: Signal Processing (eess.SP)
The advancements in nanotechnology, material science, and electrical engineering have shrunk the sizes of electronic devices down to the micro/nanoscale. This brings the opportunity of developing the Internet of Nano Things (IoNT), an extension of the Internet of Things (IoT). With nanodevices, numerous new possibilities emerge in the biomedical, military fields, and industrial products. However, a continuous energy supply is mandatory for these devices to work. At the micro/nanoscale, batteries cannot supply this demand due to size limitations and the limited energy contained in the batteries. Internet of Harvester Nano Things (IoHNT), a concept of Energy Harvesting (EH) integrated with wireless power transmission (WPT) techniques, converts the existing different energy sources into electrical energy and transmits to IoNT nodes. As IoHNTs are not directly attached to IoNTs, it gives flexibility in size. However, we define the size of IoHNTs as up to 10 cm. In this review, we comprehensively investigate the available energy sources and EH principles to wirelessly power IoNTs. We discuss the IoHNT principles, material selections, and state-of-the-art applications of each energy source for different sectoral applications. The different technologies of WPT and how communication is influenced by the incorporation of IoHNTs to power IoNTs are discussed with the future research directions. IoHNTs represent a shift in the nanodevice power supply, leading us towards a future where wireless technology is widespread. Hence, it will motivate researchers to envision and contribute to advancing the following power revolution in IoNT, providing unmatched simplicity and efficiency.
- [72] arXiv:2401.04976 (replaced) [pdf, html, other]
-
Title: Full-frequency dynamic convolution: a physical frequency-dependent convolution for sound event detectionComments: Accepted by ICPR2024Subjects: Audio and Speech Processing (eess.AS); Sound (cs.SD)
Recently, 2D convolution has been found unqualified in sound event detection (SED). It enforces translation equivariance on sound events along frequency axis, which is not a shift-invariant dimension. To address this issue, dynamic convolution is used to model the frequency dependency of sound events. In this paper, we proposed the first full-dynamic method named full-frequency dynamic convolution (FFDConv). FFDConv generates frequency kernels for every frequency band, which is designed directly in the structure for frequency-dependent modeling. It physically furnished 2D convolution with the capability of frequency-dependent modeling. FFDConv outperforms not only the baseline by 6.6% in DESED real validation dataset in terms of PSDS1, but outperforms the other full-dynamic methods. In addition, by visualizing features of sound events, we observed that FFDConv could effectively extract coherent features in specific frequency bands, consistent with the vocal continuity of sound events. This proves that FFDConv has great frequency-dependent perception ability.
- [73] arXiv:2403.14179 (replaced) [pdf, html, other]
-
Title: AdaProj: Adaptively Scaled Angular Margin Subspace Projections for Anomalous Sound Detection with Auxiliary Classification TasksComments: Accepted for presentation at DCASE 2024Subjects: Audio and Speech Processing (eess.AS); Sound (cs.SD)
The state-of-the-art approach for semi-supervised anomalous sound detection is to first learn an embedding space by using auxiliary classification tasks based on meta information or self-supervised learning and then estimate the distribution of normal data. In this work, AdaProj a novel loss function for training the embedding model is presented. In contrast to commonly used angular margin losses, which project data of each class as close as possible to their corresponding class centers, AdaProj learns to project data onto class-specific subspaces while still ensuring an angular margin between classes. By doing so, the resulting distributions of the embeddings belonging to normal data are not required to be as restrictive as other loss functions allowing a more detailed view on the data. In experiments conducted on the DCASE2022 and DCASE2023 anomalous sound detection datasets, it is shown that using AdaProj to learn an embedding space significantly outperforms other commonly used loss functions.
- [74] arXiv:2403.16306 (replaced) [pdf, html, other]
-
Title: Control-Coherent Koopman Modeling: A Physical Modeling ApproachComments: Accepted at the Conference on Decision and Control (CDC 2024)Subjects: Systems and Control (eess.SY)
The modeling of nonlinear dynamics based on Koopman operator theory, which is originally applicable only to autonomous systems with no control, is extended to non-autonomous control system without approximation to input matrix B. Prevailing methods using a least square estimate of the B matrix may result in an erroneous input matrix, misinforming the controller about the structure of the input matrix in a lifted space. Here, a new method for constructing a Koopman model that comprises the exact input matrix B is presented. A set of state variables are introduced so that the control inputs are linearly involved in the dynamics of actuators. With these variables, a lifted linear model with the exact control matrix, called a Control-Coherent Koopman Model, is constructed by superposing control input terms, which are linear in local actuator dynamics, to the Koopman operator of the associated autonomous nonlinear system. The proposed method is applied to multi degree-of-freedom robotic arms and multi-cable manipulation systems. Model Predictive Control is applied to the former. It is demonstrated that the prevailing Dynamic Mode Decomposition with Control (DMDc) using an approximate control matrix B does not provide a satisfactory result, while the Control-Coherent Koopman Model performs well with the correct B matrix.
- [75] arXiv:2404.10071 (replaced) [pdf, html, other]
-
Title: Dynamic Complex-Frequency Control of Grid-Forming ConvertersComments: 6 Pages, 7 FiguresSubjects: Systems and Control (eess.SY)
Complex droop control, alternatively known as dispatchable virtual oscillator control (dVOC), stands out for its unique capabilities in synchronization and voltage stabilization among existing control strategies for grid-forming converters. Complex droop control leverages the novel concept of ``complex frequency'', thereby establishing a coupled connection between active and reactive power inputs and frequency and rate-of-change-of voltage outputs. However, its reliance on static droop gains limits its ability to exhibit crucial dynamic response behaviors required in future power systems. To address this limitation, this paper introduces dynamic complex-frequency control, upgrading static droop gains with dynamic transfer functions to enhance the richness and flexibility in dynamic responses for frequency and voltage control. Unlike existing approaches, the complex-frequency control framework treats frequency and voltage dynamics collectively, ensuring small-signal stability for frequency synchronization and voltage stabilization simultaneously. The control framework is validated through detailed numerical case studies on the IEEE nine-bus system, also showcasing its applicability in multi-converter setups.
- [76] arXiv:2404.10506 (replaced) [pdf, html, other]
-
Title: Restoring Connectivity in Vascular Segmentation using a Learned Post-Processing ModelJournal-ref: The First Workshop on Topology and Graph-Informed Imaging Informatics (TGI3), MICCAI 2024 WorkshopSubjects: Image and Video Processing (eess.IV)
Accurate segmentation of vascular networks is essential for computer-aided tools designed to address cardiovascular diseases. Despite more than thirty years of research, it remains a challenge to obtain vascular segmentation results that preserve the connectivity of the underlying vascular network. Yet connectivity is one of the key feature of these tools. In this work, we propose a post-processing algorithm aiming to reconnect vascular structures that have been disconnected by a segmentation algorithm. Connectivity being a complex property to model explicity, we propose to learn this geometric feature either through synthetic data or annotations of the application of interest. The resulting post-processing model can be used on the output of any supervised or unsupervised vascular segmentation algorithm. We show that this post-processing effectively restores the connectivity of vascular networks both in 2D and 3D images, leading to improved overall segmentation results.
- [77] arXiv:2404.14767 (replaced) [pdf, html, other]
-
Title: Remaining Discharge Energy Prediction for Lithium-Ion Batteries Over Broad Current Ranges: A Machine Learning ApproachComments: 15 pages, 13 figures, 4 tablesJournal-ref: Applied Energy 376 (2024) 124086Subjects: Systems and Control (eess.SY)
Lithium-ion batteries have found their way into myriad sectors of industry to drive electrification, decarbonization, and sustainability. A crucial aspect in ensuring their safe and optimal performance is monitoring their energy levels. In this paper, we present the first study on predicting the remaining energy of a battery cell undergoing discharge over wide current ranges from low to high C-rates. The complexity of the challenge arises from the cell's C-rate-dependent energy availability as well as its intricate electro-thermal dynamics especially at high C-rates. To address this, we introduce a new definition of remaining discharge energy and then undertake a systematic effort in harnessing the power of machine learning to enable its prediction. Our effort includes two parts in cascade. First, we develop an accurate dynamic model based on integration of physics with machine learning to capture a battery's voltage and temperature behaviors. Second, based on the model, we propose a machine learning approach to predict the remaining discharge energy under arbitrary C-rates and pre-specified cut-off limits in voltage and temperature. The experimental validation shows that the proposed approach can predict the remaining discharge energy with a relative error of less than 3% when the current varies between 0~8 C for an NCA cell and 0~15 C for an LFP cell. The approach, by design, is amenable to training and computation.
- [78] arXiv:2405.02487 (replaced) [pdf, html, other]
-
Title: Distributed Online Feedback Optimization for Real-time Distribution System Voltage RegulationSen Zhan, Nikolaos G. Paterakis, Wouter van den Akker, Anne van der Molen, Johan Morren, J. G. SlootwegSubjects: Systems and Control (eess.SY)
We investigate the real-time distribution system voltage regulation problem employing online feedback optimization (OFO) and short-range communication between physical neighbours. OFO does not need an accurate grid topology nor estimated consumption of non-controllable loads, affords fast calculations, and demonstrates robustness to uncertainties due to its feedback-based nature. These characteristics render it particularly suitable for real-time distribution system applications. However, many OFO controllers require centralized communication, which lack robustness to single-point failures. This paper proposes a distributed OFO design based on a nested feedback optimization strategy and analyzes its convergence. Numerical results reveal that the proposed design achieves satisfactory voltage regulation and outperforms other distributed approaches.
- [79] arXiv:2406.02918 (replaced) [pdf, html, other]
-
Title: U-KAN Makes Strong Backbone for Medical Image Segmentation and GenerationSubjects: Image and Video Processing (eess.IV); Computer Vision and Pattern Recognition (cs.CV)
U-Net has become a cornerstone in various visual applications such as image segmentation and diffusion probability models. While numerous innovative designs and improvements have been introduced by incorporating transformers or MLPs, the networks are still limited to linearly modeling patterns as well as the deficient interpretability. To address these challenges, our intuition is inspired by the impressive results of the Kolmogorov-Arnold Networks (KANs) in terms of accuracy and interpretability, which reshape the neural network learning via the stack of non-linear learnable activation functions derived from the Kolmogorov-Anold representation theorem. Specifically, in this paper, we explore the untapped potential of KANs in improving backbones for vision tasks. We investigate, modify and re-design the established U-Net pipeline by integrating the dedicated KAN layers on the tokenized intermediate representation, termed U-KAN. Rigorous medical image segmentation benchmarks verify the superiority of U-KAN by higher accuracy even with less computation cost. We further delved into the potential of U-KAN as an alternative U-Net noise predictor in diffusion models, demonstrating its applicability in generating task-oriented model architectures. These endeavours unveil valuable insights and sheds light on the prospect that with U-KAN, you can make strong backbone for medical image segmentation and generation. Project page:\url{this https URL}.
- [80] arXiv:2406.17976 (replaced) [pdf, html, other]
-
Title: The Role of Electric Grid Research in Addressing Climate ChangeComments: 17 pages, 2 figuresJournal-ref: Nat. Clim. Chang. (2024)Subjects: Systems and Control (eess.SY)
Addressing the urgency of climate change necessitates a coordinated and inclusive effort from all relevant stakeholders. Critical to this effort is the modeling, analysis, control, and integration of technological innovations within the electric energy system, which plays a crucial role in scaling up climate change solutions. This perspective article presents a set of research challenges and opportunities in the area of electric power systems that would be crucial in accelerating Gigaton-level decarbonization. Furthermore, it highlights institutional challenges associated with developing market mechanisms and regulatory architectures, ensuring that incentives are aligned for stakeholders to effectively implement the technological solutions on a large scale.
- [81] arXiv:2407.03656 (replaced) [pdf, html, other]
-
Title: WildDESED: An LLM-Powered Dataset for Wild Domestic Environment Sound Event Detection SystemComments: DCASE WS 2024Subjects: Audio and Speech Processing (eess.AS); Sound (cs.SD)
This work aims to advance sound event detection (SED) research by presenting a new large language model (LLM)-powered dataset namely wild domestic environment sound event detection (WildDESED). It is crafted as an extension to the original DESED dataset to reflect diverse acoustic variability and complex noises in home settings. We leveraged LLMs to generate eight different domestic scenarios based on target sound categories of the DESED dataset. Then we enriched the scenarios with a carefully tailored mixture of noises selected from AudioSet and ensured no overlap with target sound. We consider widely popular convolutional neural recurrent network to study WildDESED dataset, which depicts its challenging nature. We then apply curriculum learning by gradually increasing noise complexity to enhance the model's generalization capabilities across various noise levels. Our results with this approach show improvements within the noisy environment, validating the effectiveness on the WildDESED dataset promoting noise-robust SED advancements.
- [82] arXiv:2407.09645 (replaced) [pdf, html, other]
-
Title: Hamilton-Jacobi Reachability in Reinforcement Learning: A SurveyComments: Accepted in IEEE Open Journal of Control Systems (OJ-CSYS)Subjects: Systems and Control (eess.SY); Machine Learning (cs.LG); Robotics (cs.RO)
Recent literature has proposed approaches that learn control policies with high performance while maintaining safety guarantees. Synthesizing Hamilton-Jacobi (HJ) reachable sets has become an effective tool for verifying safety and supervising the training of reinforcement learning-based control policies for complex, high-dimensional systems. Previously, HJ reachability was restricted to verifying low-dimensional dynamical systems primarily because the computational complexity of the dynamic programming approach it relied on grows exponentially with the number of system states. In recent years, a litany of proposed methods addresses this limitation by computing the reachability value function simultaneously with learning control policies to scale HJ reachability analysis while still maintaining a reliable estimate of the true reachable set. These HJ reachability approximations are used to improve the safety, and even reward performance, of learned control policies and can solve challenging tasks such as those with dynamic obstacles and/or with lidar-based or vision-based observations. In this survey paper, we review the recent developments in the field of HJ reachability estimation in reinforcement learning that would provide a foundational basis for further research into reliability in high-dimensional systems.
- [83] arXiv:2408.03647 (replaced) [pdf, html, other]
-
Title: Real-time Event Recognition of Long-distance Distributed Vibration Sensing with Knowledge Distillation and Hardware AccelerationComments: 9 pages, 10 figuresSubjects: Systems and Control (eess.SY); Signal Processing (eess.SP)
Fiber-optic sensing, especially distributed optical fiber vibration (DVS) sensing, is gaining importance in internet of things (IoT) applications, such as industrial safety monitoring and intrusion detection. Despite their wide application, existing post-processing methods that rely on deep learning models for event recognition in DVS systems face challenges with real-time processing of large sample data volumes, particularly in long-distance applications. To address this issue, we propose to use a four-layer convolutional neural network (CNN) with ResNet as the teacher model for knowledge distillation. This results in a significant improvement in accuracy, from 83.41% to 95.39%, on data from previously untrained environments. Additionally, we propose a novel hardware design based on field-programmable gate arrays (FPGA) to further accelerate model inference. This design replaces multiplication with binary shift operations and quantizes model weights, enabling high parallelism and low latency. Our implementation achieves an inference time of 0.083 ms for a spatial-temporal sample covering a 12.5 m fiber length and 0.256 s time frame. This performance enables real-time signal processing over approximately 38.55 km of fiber, about $2.14\times$ the capability of an Nvidia GTX 4090 GPU. The proposed method greatly enhances the efficiency of vibration pattern recognition, promoting the use of DVS as a smart IoT system. The data and code are available at this https URL.
- [84] arXiv:2408.05892 (replaced) [pdf, html, other]
-
Title: Polyp SAM 2: Advancing Zero shot Polyp Segmentation in Colorectal Cancer DetectionSubjects: Image and Video Processing (eess.IV); Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG)
Polyp segmentation plays a crucial role in the early detection and diagnosis of colorectal cancer. However, obtaining accurate segmentations often requires labor-intensive annotations and specialized models. Recently, Meta AI Research released a general Segment Anything Model 2 (SAM 2), which has demonstrated promising performance in several segmentation tasks. In this work, we evaluate the performance of SAM 2 in segmenting polyps under various prompted settings. We hope this report will provide insights to advance the field of polyp segmentation and promote more interesting work in the future. This project is publicly available at this https URL sajjad-sh33/Polyp-SAM-2.
- [85] arXiv:2408.06425 (replaced) [pdf, html, other]
-
Title: Bayesian Learning in a Nonlinear Multiscale State-Space ModelComments: Included additional figuresSubjects: Signal Processing (eess.SP); Machine Learning (cs.LG); Machine Learning (stat.ML)
The ubiquity of multiscale interactions in complex systems is well-recognized, with development and heredity serving as a prime example of how processes at different temporal scales influence one another. This work introduces a novel multiscale state-space model to explore the dynamic interplay between systems interacting across different time scales, with feedback between each scale. We propose a Bayesian learning framework to estimate unknown states by learning the unknown process noise covariances within this multiscale model. We develop a Particle Gibbs with Ancestor Sampling (PGAS) algorithm for inference and demonstrate through simulations the efficacy of our approach.
- [86] arXiv:2408.09218 (replaced) [pdf, other]
-
Title: FQGA-single: Towards Fewer Training Epochs and Fewer Model Parameters for Image-to-Image Translation TasksSubjects: Image and Video Processing (eess.IV); Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG)
CycleGAN was trained on SynthRAD Grand Challenge Dataset using the single-epoch modification (SEM) method proposed in this paper which is referred to as (CycleGAN-single) compared to the usual method of training CycleGAN on around 200 epochs (CycleGAN-multi). Model performance were evaluated qualitatively and quantitatively with quantitative performance metrics like PSNR, SSIM, MAE and MSE. The consideration of both quantitative and qualitative performance when evaluating a model is unique to certain image-to-image translation tasks like medical imaging of patient data as detailed in this paper. Also, this paper shows that good quantitative performance does not always imply good qualitative performance and the converse is also not always True (i.e. good qualitative performance does not always imply good quantitative performance). This paper also proposes a lightweight model called FQGA (Fast Paired Image-to-Image Translation Quarter-Generator Adversary) which has 1/4 the number of parameters compared to CycleGAN (when comparing their Generator Models). FQGA outperforms CycleGAN qualitatively and quantitatively even only after training on 20 epochs. Finally, using SEM method on FQGA allowed it to again outperform CycleGAN both quantitatively and qualitatively. These performance gains even with fewer model parameters and fewer epochs (which will result in time and computational savings) may also be applicable to other image-to-image translation tasks in Machine Learning apart from the Medical image-translation task discussed in this paper between Cone Beam Computed Tomography (CBCT) and Computed Tomography (CT) images.
- [87] arXiv:2408.11329 (replaced) [pdf, html, other]
-
Title: Full-Duplex ISAC-Enabled D2D Underlaid Cellular Networks: Joint Transceiver Beamforming and Power AllocationComments: This work has been submitted to IEEE Transactions on Wireless Communications on 7 June,2024Subjects: Signal Processing (eess.SP)
Integrating device-to-device (D2D) communication into cellular networks can significantly reduce the transmission burden on base stations (BSs). Besides, integrated sensing and communication (ISAC) is envisioned as a key feature in future wireless networks. In this work, we consider a full-duplex ISAC- based D2D underlaid system, and propose a joint beamforming and power allocation scheme to improve the performance of the coexisting ISAC and D2D networks. To enhance spectral efficiency, a sum rate maximization problem is formulated for the full-duplex ISAC-based D2D underlaid system, which is non-convex. To solve the non-convex optimization problem, we propose a successive convex approximation (SCA)-based iterative algorithm and prove its convergence. Numerical results are provided to validate the effectiveness of the proposed scheme with the iterative algorithm, demonstrating that the proposed scheme outperforms state-of-the-art ones in both communication and sensing performance.
- [88] arXiv:2110.02710 (replaced) [pdf, html, other]
-
Title: Contextual Tuning of Model Predictive Control for Autonomous RacingLukas P. Fröhlich, Christian Küttel, Elena Arcari, Lukas Hewing, Melanie N. Zeilinger, Andrea CarronSubjects: Robotics (cs.RO); Systems and Control (eess.SY)
Learning-based model predictive control has been widely applied in autonomous racing to improve the closed-loop behaviour of vehicles in a data-driven manner. When environmental conditions change, e.g., due to rain, often only the predictive model is adapted, but the controller parameters are kept constant. However, this can lead to suboptimal behaviour. In this paper, we address the problem of data-efficient controller tuning, adapting both the model and objective simultaneously. The key novelty of the proposed approach is that we leverage a learned dynamics model to encode the environmental condition as a so-called context. This insight allows us to employ contextual Bayesian optimization to efficiently transfer knowledge across different environmental conditions. Consequently, we require fewer data to find the optimal controller configuration for each context. The proposed framework is extensively evaluated with more than 3'000 laps driven on an experimental platform with 1:28 scale RC race cars. The results show that our approach successfully optimizes the lap time across different contexts requiring fewer data compared to other approaches based on standard Bayesian optimization.
- [89] arXiv:2112.12979 (replaced) [pdf, html, other]
-
Title: Integrating Physics-Based Modeling with Machine Learning for Lithium-Ion BatteriesComments: 15 pages, 10 figures, 2 tables. arXiv admin note: text overlap with arXiv:2103.11580Journal-ref: Applied Energy 329 (2023) 120289Subjects: Computational Engineering, Finance, and Science (cs.CE); Machine Learning (cs.LG); Systems and Control (eess.SY)
Mathematical modeling of lithium-ion batteries (LiBs) is a primary challenge in advanced battery management. This paper proposes two new frameworks to integrate physics-based models with machine learning to achieve high-precision modeling for LiBs. The frameworks are characterized by informing the machine learning model of the state information of the physical model, enabling a deep integration between physics and machine learning. Based on the frameworks, a series of hybrid models are constructed, through combining an electrochemical model and an equivalent circuit model, respectively, with a feedforward neural network. The hybrid models are relatively parsimonious in structure and can provide considerable voltage predictive accuracy under a broad range of C-rates, as shown by extensive simulations and experiments. The study further expands to conduct aging-aware hybrid modeling, leading to the design of a hybrid model conscious of the state-of-health to make prediction. The experiments show that the model has high voltage predictive accuracy throughout a LiB's cycle life.
- [90] arXiv:2209.04042 (replaced) [pdf, other]
-
Title: Assessing Lower Limb Strength using Internet-of-Things Enabled ChairComments: 12 PagesSubjects: Machine Learning (cs.LG); Signal Processing (eess.SP)
This project describes the application of the technologies of Machine Learning and Internet-of-Things to assess the lower limb strength of individuals undergoing rehabilitation or therapy. Specifically, it seeks to measure and assess the progress of individuals by sensors attached to chairs and processing the data through Google GPU Tensorflow CoLab. Pressure sensors are attached to various locations on a chair, including but not limited to the seating area, backrest, hand rests, and legs. Sensor data from the individual performing both sit-to-stand transition and stand-to-sit transition provides a time series dataset regarding the pressure distribution and vibratory motion on the chair. The dataset and timing information can then be fed into a machine learning model to estimate the relative strength and weakness during various phases of the movement.
- [91] arXiv:2210.02899 (replaced) [pdf, html, other]
-
Title: Self-supervised Learning for Clustering of Wireless Spectrum ActivityJournal-ref: Computer Communications, Volume 212, 1 December 2023, Pages 353-365Subjects: Networking and Internet Architecture (cs.NI); Artificial Intelligence (cs.AI); Machine Learning (cs.LG); Signal Processing (eess.SP)
In recent years, much work has been done on processing of wireless spectrum data involving machine learning techniques in domain-related problems for cognitive radio networks, such as anomaly detection, modulation classification, technology classification and device fingerprinting. Most of the solutions are based on labeled data, created in a controlled manner and processed with supervised learning approaches. However, spectrum data measured in real-world environment is highly nondeterministic, making its labeling a laborious and expensive process, requiring domain expertise, thus being one of the main drawbacks of using supervised learning approaches in this domain. In this paper, we investigate the use of self-supervised learning (SSL) for exploring spectrum activities in a real-world unlabeled data. In particular, we compare the performance of two SSL models, one based on a reference DeepCluster architecture and one adapted for spectrum activity identification and clustering, and a baseline model based on K-means clustering algorithm. We show that SSL models achieve superior performance regarding the quality of extracted features and clustering performance. With SSL models we achieve reduction of the feature vectors size by two orders of magnitude, while improving the performance by a factor of 2 to 2.5 across the evaluation metrics, supported by visual assessment. Additionally we show that adaptation of the reference SSL architecture to the domain data provides reduction of model complexity by one order of magnitude, while preserving or even improving the clustering performance.
- [92] arXiv:2404.06493 (replaced) [pdf, html, other]
-
Title: Flying with Photons: Rendering Novel Views of Propagating LightComments: ECCV 2024, Project page: this https URLSubjects: Computer Vision and Pattern Recognition (cs.CV); Image and Video Processing (eess.IV)
We present an imaging and neural rendering technique that seeks to synthesize videos of light propagating through a scene from novel, moving camera viewpoints. Our approach relies on a new ultrafast imaging setup to capture a first-of-its kind, multi-viewpoint video dataset with picosecond-level temporal resolution. Combined with this dataset, we introduce an efficient neural volume rendering framework based on the transient field. This field is defined as a mapping from a 3D point and 2D direction to a high-dimensional, discrete-time signal that represents time-varying radiance at ultrafast timescales. Rendering with transient fields naturally accounts for effects due to the finite speed of light, including viewpoint-dependent appearance changes caused by light propagation delays to the camera. We render a range of complex effects, including scattering, specular reflection, refraction, and diffraction. Additionally, we demonstrate removing viewpoint-dependent propagation delays using a time warping procedure, rendering of relativistic effects, and video synthesis of direct and global components of light transport.
- [93] arXiv:2404.15968 (replaced) [pdf, other]
-
Title: Fast and Robust Expectation Propagation MIMO Detection via Preconditioned Conjugated GradientComments: Accepted for presentation at Asilomar Conference on Signals, Systems, and Computers 2024Subjects: Information Theory (cs.IT); Signal Processing (eess.SP)
We study the expectation propagation (EP) algorithm for symbol detection in massive multiple-input multiple-output (MIMO) systems. The EP detector shows excellent performance but suffers from a high computational complexity due to the matrix inversion, required in each EP iteration to perform marginal inference on a Gaussian system. We propose an inversion-free variant of the EP algorithm by treating inference on the mean and variance as two separate and simpler subtasks: We study the preconditioned conjugate gradient algorithm for obtaining the mean, which can significantly reduce the complexity and increase stability by relying on the Jacobi preconditioner that proves to fit the EP characteristics very well. For the variance, we use a simple approximation based on linear regression of the Gram channel matrix. Numerical studies on the Rayleigh-fading channel and on a realistic 3GPP channel model reveal the efficiency of the proposed scheme, which offers an attractive performance-complexity tradeoff and even outperforms the original EP detector in high multi-user inference cases where the matrix inversion becomes numerically unstable.
- [94] arXiv:2405.01437 (replaced) [pdf, html, other]
-
Title: Two competing populations with a common environmental resourceSubjects: Computer Science and Game Theory (cs.GT); Systems and Control (eess.SY); Populations and Evolution (q-bio.PE)
Feedback-evolving games is a framework that models the co-evolution between payoff functions and an environmental state. It serves as a useful tool to analyze many social dilemmas such as natural resource consumption, behaviors in epidemics, and the evolution of biological populations. However, it has primarily focused on the dynamics of a single population of agents. In this paper, we consider the impact of two populations of agents that share a common environmental resource. We focus on a scenario where individuals in one population are governed by an environmentally ``responsible" incentive policy, and individuals in the other population are environmentally ``irresponsible". An analysis on the asymptotic stability of the coupled system is provided, and conditions for which the resource collapses are identified. We then derive consumption rates for the irresponsible population that optimally exploit the environmental resource, and analyze how incentives should be allocated to the responsible population that most effectively promote the environment via a sensitivity analysis.
- [95] arXiv:2405.18118 (replaced) [pdf, html, other]
-
Title: An agent design with goal reaching guarantees for enhancement of learningSubjects: Artificial Intelligence (cs.AI); Systems and Control (eess.SY); Dynamical Systems (math.DS)
Reinforcement learning is commonly concerned with problems of maximizing accumulated rewards in Markov decision processes. Oftentimes, a certain goal state or a subset of the state space attain maximal reward. In such a case, the environment may be considered solved when the goal is reached. Whereas numerous techniques, learning or non-learning based, exist for solving environments, doing so optimally is the biggest challenge. Say, one may choose a reward rate which penalizes the action effort. Reinforcement learning is currently among the most actively developed frameworks for solving environments optimally by virtue of maximizing accumulated reward, in other words, returns. Yet, tuning agents is a notoriously hard task as reported in a series of works. Our aim here is to help the agent learn a near-optimal policy efficiently while ensuring a goal reaching property of some basis policy that merely solves the environment. We suggest an algorithm, which is fairly flexible, and can be used to augment practically any agent as long as it comprises of a critic. A formal proof of a goal reaching property is provided. Comparative experiments on several problems under popular baseline agents provided an empirical evidence that the learning can indeed be boosted while ensuring goal reaching property.
- [96] arXiv:2406.10932 (replaced) [pdf, html, other]
-
Title: Imperceptible Rhythm Backdoor Attacks: Exploring Rhythm Transformation for Embedding Undetectable Vulnerabilities on Speech RecognitionSubjects: Sound (cs.SD); Artificial Intelligence (cs.AI); Audio and Speech Processing (eess.AS)
Speech recognition is an essential start ring of human-computer interaction, and recently, deep learning models have achieved excellent success in this task. However, when the model training and private data provider are always separated, some security threats that make deep neural networks (DNNs) abnormal deserve to be researched. In recent years, the typical backdoor attacks have been researched in speech recognition systems. The existing backdoor methods are based on data poisoning. The attacker adds some incorporated changes to benign speech spectrograms or changes the speech components, such as pitch and timbre. As a result, the poisoned data can be detected by human hearing or automatic deep algorithms. To improve the stealthiness of data poisoning, we propose a non-neural and fast algorithm called Random Spectrogram Rhythm Transformation (RSRT) in this paper. The algorithm combines four steps to generate stealthy poisoned utterances. From the perspective of rhythm component transformation, our proposed trigger stretches or squeezes the mel spectrograms and recovers them back to signals. The operation keeps timbre and content unchanged for good stealthiness. Our experiments are conducted on two kinds of speech recognition tasks, including testing the stealthiness of poisoned samples by speaker verification and automatic speech recognition. The results show that our method has excellent effectiveness and stealthiness. The rhythm trigger needs a low poisoning rate and gets a very high attack success rate.
- [97] arXiv:2407.00258 (replaced) [pdf, html, other]
-
Title: Graph Simplification Solutions to the Street Intersection Miscount ProblemComments: Conference paperSubjects: Physics and Society (physics.soc-ph); Discrete Mathematics (cs.DM); Systems and Control (eess.SY); Computation (stat.CO)
Street intersection counts and densities are fundamental measures in transport geography and urban planning. Conventional street network data and analysis tools often lead to significant overcounting of these measures. This study investigates the causes of such overcounting and proposes remedies. It introduces algorithms to streamline urban street network models through edge simplification and node consolidation, which enhance computational efficiency and improve accuracy in network measures like intersection counts, street segment lengths, and node degrees. The algorithms are validated and subjected to a global empirical assessment to evaluate the extent of count bias. These findings highlight the prevalence of this bias and underscore the necessity for better methods to mitigate inaccuracies in intersection representation.
- [98] arXiv:2407.06888 (replaced) [pdf, html, other]
-
Title: A Complete Set of Quadratic Constraints for Repeated ReLU and GeneralizationsSubjects: Machine Learning (cs.LG); Systems and Control (eess.SY); Optimization and Control (math.OC)
This paper derives a complete set of quadratic constraints (QCs) for the repeated ReLU. The complete set of QCs is described by a collection of matrix copositivity conditions. We also show that only two functions satisfy all QCs in our complete set: the repeated ReLU and flipped ReLU. Thus our complete set of QCs bounds the repeated ReLU as tight as possible up to the sign invariance inherent in quadratic forms. We derive a similar complete set of incremental QCs for repeated ReLU, which can potentially lead to less conservative Lipschitz bounds for ReLU networks than the standard LipSDP approach. The basic constructions are also used to derive the complete sets of QCs for other piecewise linear activation functions such as leaky ReLU, MaxMin, and HouseHolder. Finally, we illustrate the use of the complete set of QCs to assess stability and performance for recurrent neural networks with ReLU activation functions. We rely on a standard copositivity relaxation to formulate the stability/performance condition as a semidefinite program. Simple examples are provided to illustrate that the complete sets of QCs and incremental QCs can yield less conservative bounds than existing sets.
- [99] arXiv:2408.08399 (replaced) [pdf, html, other]
-
Title: An Efficient and Explainable Transformer-Based Few-Shot Learning for Modeling Electricity Consumption Profiles Across Thousands of DomainsSubjects: Machine Learning (cs.LG); Systems and Control (eess.SY)
Electricity Consumption Profiles (ECPs) are crucial for operating and planning power distribution systems, especially with the increasing numbers of various low-carbon technologies such as solar panels and electric vehicles. Traditional ECP modeling methods typically assume the availability of sufficient ECP data. However, in practice, the accessibility of ECP data is limited due to privacy issues or the absence of metering devices. Few-shot learning (FSL) has emerged as a promising solution for ECP modeling in data-scarce scenarios. Nevertheless, standard FSL methods, such as those used for images, are unsuitable for ECP modeling because (1) these methods usually assume several source domains with sufficient data and several target domains. However, in the context of ECP modeling, there may be thousands of source domains with a moderate amount of data and thousands of target domains. (2) Standard FSL methods usually involve cumbersome knowledge transfer mechanisms, such as pre-training and fine-tuning, whereas ECP modeling requires more lightweight methods. (3) Deep learning models often lack explainability, hindering their application in industry. This paper proposes a novel FSL method that exploits Transformers and Gaussian Mixture Models (GMMs) for ECP modeling to address the above-described issues. Results show that our method can accurately restore the complex ECP distribution with a minimal amount of ECP data (e.g., only 1.6\% of the complete domain dataset) while it outperforms state-of-the-art time series modeling methods, maintaining the advantages of being both lightweight and interpretable. The project is open-sourced at this https URL.
- [100] arXiv:2408.10096 (replaced) [pdf, html, other]
-
Title: Convert and Speak: Zero-shot Accent Conversion with Minimum SupervisionComments: 9 pages, ACM MM2024(accepted)Subjects: Sound (cs.SD); Artificial Intelligence (cs.AI); Audio and Speech Processing (eess.AS)
Low resource of parallel data is the key challenge of accent conversion(AC) problem in which both the pronunciation units and prosody pattern need to be converted. We propose a two-stage generative framework "convert-and-speak" in which the conversion is only operated on the semantic token level and the speech is synthesized conditioned on the converted semantic token with a speech generative model in target accent domain. The decoupling design enables the "speaking" module to use massive amount of target accent speech and relieves the parallel data required for the "conversion" module. Conversion with the bridge of semantic token also relieves the requirement for the data with text transcriptions and unlocks the usage of language pre-training technology to further efficiently reduce the need of parallel accent speech data. To reduce the complexity and latency of "speaking", a single-stage AR generative model is designed to achieve good quality as well as lower computation cost. Experiments on Indian-English to general American-English conversion show that the proposed framework achieves state-of-the-art performance in accent similarity, speech quality, and speaker maintenance with only 15 minutes of weakly parallel data which is not constrained to the same speaker. Extensive experimentation with diverse accent types suggests that this framework possesses a high degree of adaptability, making it readily scalable to accommodate other accents with low-resource data. Audio samples are available at this https URL.
- [101] arXiv:2408.10287 (replaced) [pdf, other]
-
Title: Recognizing Beam Profiles from Silicon Photonics Gratings using Transformer ModelSubjects: Optics (physics.optics); Artificial Intelligence (cs.AI); Image and Video Processing (eess.IV)
Over the past decade, there has been extensive work in developing integrated silicon photonics (SiPh) gratings for the optical addressing of trapped ion qubits in the ion trap quantum computing community. However, when viewing beam profiles from infrared (IR) cameras, it is often difficult to determine the corresponding heights where the beam profiles are located. In this work, we developed transformer models to recognize the corresponding height categories of beam profiles of light from SiPh gratings. The model is trained using two techniques: (1) input patches, and (2) input sequence. For model trained with input patches, the model achieved recognition accuracy of 0.938. Meanwhile, model trained with input sequence shows lower accuracy of 0.895. However, when repeating the model-training 150 cycles, model trained with input patches shows inconsistent accuracy ranges between 0.445 to 0.959, while model trained with input sequence exhibit higher accuracy values between 0.789 to 0.936. The obtained outcomes can be expanded to various applications, including auto-focusing of light beam and auto-adjustment of z-axis stage to acquire desired beam profiles.
- [102] arXiv:2408.10500 (replaced) [pdf, html, other]
-
Title: SZTU-CMU at MER2024: Improving Emotion-LLaMA with Conv-Attention for Multimodal Emotion RecognitionZebang Cheng, Shuyuan Tu, Dawei Huang, Minghan Li, Xiaojiang Peng, Zhi-Qi Cheng, Alexander G. HauptmannComments: Ranked 1st in MER24@IJCAI and MRAC24@ACM MM (MER-NOISE & MER-OV (self-evaluated))Subjects: Multimedia (cs.MM); Computer Vision and Pattern Recognition (cs.CV); Sound (cs.SD); Audio and Speech Processing (eess.AS)
This paper presents our winning approach for the MER-NOISE and MER-OV tracks of the MER2024 Challenge on multimodal emotion recognition. Our system leverages the advanced emotional understanding capabilities of Emotion-LLaMA to generate high-quality annotations for unlabeled samples, addressing the challenge of limited labeled data. To enhance multimodal fusion while mitigating modality-specific noise, we introduce Conv-Attention, a lightweight and efficient hybrid framework. Extensive experimentation vali-dates the effectiveness of our approach. In the MER-NOISE track, our system achieves a state-of-the-art weighted average F-score of 85.30%, surpassing the second and third-place teams by 1.47% and 1.65%, respectively. For the MER-OV track, our utilization of Emotion-LLaMA for open-vocabulary annotation yields an 8.52% improvement in average accuracy and recall compared to GPT-4V, securing the highest score among all participating large multimodal models. The code and model for Emotion-LLaMA are available at this https URL.
- [103] arXiv:2408.11562 (replaced) [pdf, html, other]
-
Title: A Joint Noise Disentanglement and Adversarial Training Framework for Robust Speaker VerificationComments: 5 pages, accepted by Interspeech2024Subjects: Sound (cs.SD); Audio and Speech Processing (eess.AS)
Automatic Speaker Verification (ASV) suffers from performance degradation in noisy conditions. To address this issue, we propose a novel adversarial learning framework that incorporates noise-disentanglement to establish a noise-independent speaker invariant embedding space. Specifically, the disentanglement module includes two encoders for separating speaker related and irrelevant information, respectively. The reconstruction module serves as a regularization term to constrain the noise. A feature-robust loss is also used to supervise the speaker encoder to learn noise-independent speaker embeddings without losing speaker information. In addition, adversarial training is introduced to discourage the speaker encoder from encoding acoustic condition information for achieving a speaker-invariant embedding space. Experiments on VoxCeleb1 indicate that the proposed method improves the performance of the speaker verification system under both clean and noisy conditions.