Human-Computer Interaction
See recent articles
- [1] arXiv:2408.11821 [pdf, html, other]
-
Title: MIMA 2.0 -- Compact and Portable Multifunctional IoT integrated Menstrual AidJournal-ref: In Internet of Things (Vol. 25, p. 101075). Elsevier BV (2024)Subjects: Human-Computer Interaction (cs.HC)
The shredding intrauterine lining or the endometrium is known as Menstruation. It occurs every month and causes several issues like Menstrual Cramps and aches in the abdominal region, stains, menstrual malodor, rashes in intimate areas, and many more. In our research, almost all of the products available in the market do not cater to these problems single-handedly. There are few remedies available to cater to the cramps, among which heat therapy is the most commonly used. Our methodology, involved surveys regarding problems and the solutions to these problems that are deemed optimal. This inclusive approach helped us infer about the gaps in available menstrual aids which has become our guide towards developing MIMA (Multifunctional IoT Integrated Menstrual Aid). In this paper, we have featured an IOT incorporated multifunctional smart intimate wear that aims to provide for the multiple necessities of women during menstruation like leakproof, antibacterial, anti-odor, rash-free experience along with an integrated Bluetooth-controlled intimate heat-pad for relieving abdominal cramps. The entire process of product development has been done in phases according to feedback from target users in each stage. This paper is an extension to our paper [1] which serves as the proof of concept for our approach. The development has led us towards MIMA 2.0 featuring a completely concealed and integrated design that includes a safe Bluetooth-controlled heating system for the intimate area. The product has received incredibly positive feedback from survey participants.
- [2] arXiv:2408.11824 [pdf, html, other]
-
Title: AppAgent v2: Advanced Agent for Flexible Mobile InteractionsSubjects: Human-Computer Interaction (cs.HC); Artificial Intelligence (cs.AI)
With the advancement of Multimodal Large Language Models (MLLM), LLM-driven visual agents are increasingly impacting software interfaces, particularly those with graphical user interfaces. This work introduces a novel LLM-based multimodal agent framework for mobile devices. This framework, capable of navigating mobile devices, emulates human-like interactions. Our agent constructs a flexible action space that enhances adaptability across various applications including parser, text and vision descriptions. The agent operates through two main phases: exploration and deployment. During the exploration phase, functionalities of user interface elements are documented either through agent-driven or manual explorations into a customized structured knowledge base. In the deployment phase, RAG technology enables efficient retrieval and update from this knowledge base, thereby empowering the agent to perform tasks effectively and accurately. This includes performing complex, multi-step operations across various applications, thereby demonstrating the framework's adaptability and precision in handling customized task workflows. Our experimental results across various benchmarks demonstrate the framework's superior performance, confirming its effectiveness in real-world scenarios. Our code will be open source soon.
- [3] arXiv:2408.11831 [pdf, other]
-
Title: Web-based Visualization and Analytics of Petascale data: Equity as a Tide that Lifts All BoatsAashish Panta, Xuan Huang, Nina McCurdy, David Ellsworth, Amy Gooch, Giorgio Scorzelli, Hector Torres, Patrice Klein, Gustavo Ovando-Montejo, Valerio PascucciComments: 11 pages, 12 figures, conference paperSubjects: Human-Computer Interaction (cs.HC)
Scientists generate petabytes of data daily to help uncover environmental trends or behaviors that are hard to predict. For example, understanding climate simulations based on the long-term average of temperature, precipitation, and other environmental variables is essential to predicting and establishing root causes of future undesirable scenarios and assessing possible mitigation strategies. While supercomputer centers provide a powerful infrastructure for generating petabytes of simulation output, accessing and analyzing these datasets interactively remains challenging on multiple fronts. This paper presents an approach to managing, visualizing, and analyzing petabytes of data within a browser on equipment ranging from the top NASA supercomputer to commodity hardware like a laptop. Our novel data fabric abstraction layer allows user-friendly querying of scientific information while hiding the complexities of dealing with file systems or cloud services. We also optimize network utilization while streaming from petascale repositories through state-of-the-art progressive compression algorithms. Based on this abstraction, we provide customizable dashboards that can be accessed from any device with any internet connection, enabling interactive visual analysis of vast amounts of data to a wide range of users - from top scientists with access to leadership-class computing environments to undergraduate students of disadvantaged backgrounds from minority-serving institutions. We focus on NASA's use of petascale climate datasets as an example of particular societal impact and, therefore, a case where achieving equity in science participation is critical. We further validate our approach by deploying the dashboards and simplified training materials in the classroom at a minority-serving institution.
- [4] arXiv:2408.12121 [pdf, html, other]
-
Title: Emotion-Agent: Unsupervised Deep Reinforcement Learning with Distribution-Prototype Reward for Continuous Emotional EEG AnalysisComments: 11 pages, 4 figures, 4 tables, submitted to AAAI 2025Subjects: Human-Computer Interaction (cs.HC); Artificial Intelligence (cs.AI)
Continuous electroencephalography (EEG) signals are widely used in affective brain-computer interface (aBCI) applications. However, not all continuously collected EEG signals are relevant or meaningful to the task at hand (e.g., wondering thoughts). On the other hand, manually labeling the relevant parts is nearly impossible due to varying engagement patterns across different tasks and individuals. Therefore, effectively and efficiently identifying the important parts from continuous EEG recordings is crucial for downstream BCI tasks, as it directly impacts the accuracy and reliability of the results. In this paper, we propose a novel unsupervised deep reinforcement learning framework, called Emotion-Agent, to automatically identify relevant and informative emotional moments from continuous EEG signals. Specifically, Emotion-Agent involves unsupervised deep reinforcement learning combined with a heuristic algorithm. We first use the heuristic algorithm to perform an initial global search and form prototype representations of the EEG signals, which facilitates the efficient exploration of the signal space and identify potential regions of interest. Then, we design distribution-prototype reward functions to estimate the interactions between samples and prototypes, ensuring that the identified parts are both relevant and representative of the underlying emotional states. Emotion-Agent is trained using Proximal Policy Optimization (PPO) to achieve stable and efficient convergence. Our experiments compare the performance with and without Emotion-Agent. The results demonstrate that selecting relevant and informative emotional parts before inputting them into downstream tasks enhances the accuracy and reliability of aBCI applications.
- [5] arXiv:2408.12143 [pdf, html, other]
-
Title: Generative Artificial Intelligence and Human LearningSubjects: Human-Computer Interaction (cs.HC)
Generative artificial intelligence (GenAI) holds the potential to transform the delivery, cultivation, and evaluation of human learning. This Perspective examines the integration of GenAI as a tool for human learning, addressing its promises and challenges from a holistic viewpoint that integrates insights from learning sciences, educational technology, and human-computer interaction. GenAI promises to enhance learning experiences by scaling personalised support, diversifying learning materials, enabling timely feedback, and innovating assessment methods. However, it also presents critical issues such as model imperfections, ethical dilemmas, and the disruption of traditional assessments. We highlight the importance of cultivating AI literacy and advocate for informed engagement with GenAI technologies. Rigorous research across learning contexts is essential to evaluate GenAI's impact on human cognition, metacognition, and creativity. Humanity must learn with and about GenAI, ensuring it becomes a powerful ally in the pursuit of knowledge and innovation, rather than a crutch that undermines our intellectual abilities.
- [6] arXiv:2408.12169 [pdf, html, other]
-
Title: ReorderBench: A Benchmark for Matrix ReorderingComments: Submitted to IEEE TVCGSubjects: Human-Computer Interaction (cs.HC)
Matrix reordering permutes the rows and columns of a matrix to reveal meaningful visual patterns, such as blocks that represent clusters. A comprehensive collection of matrices, along with a scoring method for measuring the quality of visual patterns in these matrices, contributes to building a benchmark. This benchmark is essential for selecting or designing suitable reordering algorithms for specific tasks. In this paper, we build a matrix reordering benchmark, ReorderBench, with the goal of evaluating and improving matrix reordering techniques. This is achieved by generating a large set of representative and diverse matrices and scoring these matrices with a convolution- and entropy-based method. Our benchmark contains 2,835,000 binary matrices and 5,670,000 continuous matrices, each featuring one of four visual patterns: block, off-diagonal block, star, or band. We demonstrate the usefulness of ReorderBench through three main applications in matrix reordering: 1) evaluating different reordering algorithms, 2) creating a unified scoring model to measure the visual patterns in any matrix, and 3) developing a deep learning model for matrix reordering.
- [7] arXiv:2408.12170 [pdf, html, other]
-
Title: VoiceX: A Text-To-Speech Framework for Custom VoicesSilvan Mertes, Daksitha Withanage Don, Otto Grothe, Johanna Kuch, Ruben Schlagowski, Elisabeth AndréSubjects: Human-Computer Interaction (cs.HC); Sound (cs.SD); Audio and Speech Processing (eess.AS)
Modern TTS systems are capable of creating highly realistic and natural-sounding speech. Despite these developments, the process of customizing TTS voices remains a complex task, mostly requiring the expertise of specialists within the field. One reason for this is the utilization of deep learning models, which are characterized by their expansive, non-interpretable parameter spaces, restricting the feasibility of manual customization. In this paper, we present a novel human-in-the-loop paradigm based on an evolutionary algorithm for directly interacting with the parameter space of a neural TTS model. We integrated our approach into a user-friendly graphical user interface that allows users to efficiently create original voices. Those voices can then be used with the backbone TTS model, for which we provide a Python API. Further, we present the results of a user study exploring the capabilities of VoiceX. We show that VoiceX is an appropriate tool for creating individual, custom voices.
- [8] arXiv:2408.12238 [pdf, html, other]
-
Title: VR Cloud Gaming UX: Exploring the Impact of Network Quality on Emotion, Presence, Game Experience and CybersicknessComments: Accepted and will be presented in the ISMAR 2024 poster trackSubjects: Human-Computer Interaction (cs.HC)
This study explores the user experience (UX) of virtual reality (VR) cloud gaming under simulated network degradation conditions. Two contrasting games (Beat Saber, Cubism) were streamed via Meta Air Link to a Quest 3 device in a laboratory setup. Packet loss and delay were introduced into the streaming network using NetEm for WiFi traffic manipulation. In a within-subjects experiment, 16 participants played both games under three network conditions (Loss, Delay, Baseline), followed by post-game questionnaires assessing their emotions, perceived quality, game experience, sense of presence, and cybersickness. Friedman's test and Dunn's post-hoc test for pairwise comparisons revealed that packet loss had a greater impact on UX than delay across almost all evaluated aspects. Notably, packet loss in Beat Saber led to a significant increase in cybersickness, whereas in Cubism, players experienced a significant reduction in their sense of presence. Additionally, both games exhibited statistically significant variations between conditions in most game experience dimensions, perceived quality, and emotional responses. This study highlights the critical role of network stability in VR cloud gaming, particularly in minimizing packet loss. The different dynamics between the games suggest the possibility of genre-specific optimization and novel game design considerations for VR cloud games.
- [9] arXiv:2408.12365 [pdf, html, other]
-
Title: Enhancing Uncertainty Communication in Time Series Predictions: Insights and RecommendationsSubjects: Human-Computer Interaction (cs.HC); Machine Learning (cs.LG)
As the world increasingly relies on mathematical models for forecasts in different areas, effective communication of uncertainty in time series predictions is important for informed decision making. This study explores how users estimate probabilistic uncertainty in time series predictions under different variants of line charts depicting uncertainty. It examines the role of individual characteristics and the influence of user-reported metrics on uncertainty estimations. By addressing these aspects, this paper aims to enhance the understanding of uncertainty visualization and for improving communication in time series forecast visualizations and the design of prediction data this http URL the world increasingly relies on mathematical models for forecasts in different areas, effective communication of uncertainty in time series predictions is important for informed decision making. This study explores how users estimate probabilistic uncertainty in time series predictions under different variants of line charts depicting uncertainty. It examines the role of individual characteristics and the influence of user-reported metrics on uncertainty estimations. By addressing these aspects, this paper aims to enhance the understanding of uncertainty visualization and for improving communication in time series forecast visualizations and the design of prediction data dashboards.
- [10] arXiv:2408.12428 [pdf, html, other]
-
Title: VR4UrbanDev: An Immersive Virtual Reality Experience for Energy Data VisualizationSaeed Safikhani, Georg Arbesser-Rastburg, Anna Schreuer, Jürgen Suschek-Berger, Hermann Edtmayer, Johanna PirkerComments: 2 pages, 1 figure, to be published in the conference proceedings of VRST 2024Subjects: Human-Computer Interaction (cs.HC)
In this demonstration paper, we present our interactive virtual reality (VR) experience, which has been designed to facilitate interaction with energy-related information. This experience consists of two main modes: the world in miniature for large-scale and first-person for real-world scale visualizations. Additionally, we presented our approach to potential target groups in interviews. The results of these interviews can help developers for future implementation considering the requirements of each group.
- [11] arXiv:2408.12500 [pdf, html, other]
-
Title: WhisperMask: A Noise Suppressive Mask-Type Microphone for Whisper SpeechComments: 14 pages, 14 figuresJournal-ref: Proceedings of the Augmented Humans International Conference 2024Subjects: Human-Computer Interaction (cs.HC); Sound (cs.SD); Audio and Speech Processing (eess.AS)
Whispering is a common privacy-preserving technique in voice-based interactions, but its effectiveness is limited in noisy environments. In conventional hardware- and software-based noise reduction approaches, isolating whispered speech from ambient noise and other speech sounds remains a challenge. We thus propose WhisperMask, a mask-type microphone featuring a large diaphragm with low sensitivity, making the wearer's voice significantly louder than the background noise. We evaluated WhisperMask using three key metrics: signal-to-noise ratio, quality of recorded voices, and speech recognition rate. Across all metrics, WhisperMask consistently outperformed traditional noise-suppressing microphones and software-based solutions. Notably, WhisperMask showed a 30% higher recognition accuracy for whispered speech recorded in an environment with 80 dB background noise compared with the pin microphone and earbuds. Furthermore, while a denoiser decreased the whispered speech recognition rate of these two microphones by approximately 20% at 30-60 dB noise, WhisperMask maintained a high performance even without denoising, surpassing the other microphones' performances by a significant margin.WhisperMask's design renders the wearer's voice as the dominant input and effectively suppresses background noise without relying on signal processing. This device allows for reliable voice interactions, such as phone calls and voice commands, in a wide range of noisy real-world scenarios while preserving user privacy.
New submissions for Friday, 23 August 2024 (showing 11 of 11 entries )
- [12] arXiv:2408.11837 (cross-list from cs.LG) [pdf, html, other]
-
Title: MicroXercise: A Micro-Level Comparative and Explainable System for Remote Physical TherapyComments: Accepted by IEEE/ACM CHASE 2024Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Human-Computer Interaction (cs.HC); Signal Processing (eess.SP)
Recent global estimates suggest that as many as 2.41 billion individuals have health conditions that would benefit from rehabilitation services. Home-based Physical Therapy (PT) faces significant challenges in providing interactive feedback and meaningful observation for therapists and patients. To fill this gap, we present MicroXercise, which integrates micro-motion analysis with wearable sensors, providing therapists and patients with a comprehensive feedback interface, including video, text, and scores. Crucially, it employs multi-dimensional Dynamic Time Warping (DTW) and attribution-based explainable methods to analyze the existing deep learning neural networks in monitoring exercises, focusing on a high granularity of exercise. This synergistic approach is pivotal, providing output matching the input size to precisely highlight critical subtleties and movements in PT, thus transforming complex AI analysis into clear, actionable feedback. By highlighting these micro-motions in different metrics, such as stability and range of motion, MicroXercise significantly enhances the understanding and relevance of feedback for end-users. Comparative performance metrics underscore its effectiveness over traditional methods, such as a 39% and 42% improvement in Feature Mutual Information (FMI) and Continuity. MicroXercise is a step ahead in home-based physical therapy, providing a technologically advanced and intuitively helpful solution to enhance patient care and outcomes.
- [13] arXiv:2408.11860 (cross-list from cs.CL) [pdf, html, other]
-
Title: Risks and NLP Design: A Case Study on Procedural Document QANikita Haduong (1), Alice Gao (1), Noah A. Smith (1 and 2) ((1) Paul G. Allen School of Computer Science & Engineering, University of Washington, (2) Allen Institute for Artificial Intelligence)Journal-ref: Findings of the Association for Computational Linguistics ACL (2023) 1248-1269Subjects: Computation and Language (cs.CL); Human-Computer Interaction (cs.HC)
As NLP systems are increasingly deployed at scale, concerns about their potential negative impacts have attracted the attention of the research community, yet discussions of risk have mostly been at an abstract level and focused on generic AI or NLP applications. We argue that clearer assessments of risks and harms to users--and concrete strategies to mitigate them--will be possible when we specialize the analysis to more concrete applications and their plausible users. As an illustration, this paper is grounded in cooking recipe procedural document question answering (ProcDocQA), where there are well-defined risks to users such as injuries or allergic reactions. Our case study shows that an existing language model, applied in "zero-shot" mode, quantitatively answers real-world questions about recipes as well or better than the humans who have answered the questions on the web. Using a novel questionnaire informed by theoretical work on AI risk, we conduct a risk-oriented error analysis that could then inform the design of a future system to be deployed with lower risk of harm and better performance.
- [14] arXiv:2408.11910 (cross-list from cs.CY) [pdf, html, other]
-
Title: Why am I Still Seeing This: Measuring the Effectiveness Of Ad Controls and Explanations in AI-Mediated Ad Targeting SystemsComments: Accepted to the 7th AAAI Conference on AI, Ethics, and Society (AIES, 2024)Subjects: Computers and Society (cs.CY); Artificial Intelligence (cs.AI); Human-Computer Interaction (cs.HC); Machine Learning (cs.LG)
Recently, Meta has shifted towards AI-mediated ad targeting mechanisms that do not require advertisers to provide detailed targeting criteria, likely driven by excitement over AI capabilities as well as new data privacy policies and targeting changes agreed upon in civil rights settlements. At the same time, Meta has touted their ad preference controls as an effective mechanism for users to control the ads they see. Furthermore, Meta markets their targeting explanations as a transparency tool that allows users to understand why they saw certain ads and inform actions to control future ads.
Our study evaluates the effectiveness of Meta's "See less" ad control and the actionability of ad targeting explanations following the shift to AI-mediated targeting. We conduct a large-scale study, randomly assigning participants to mark "See less" to Body Weight Control or Parenting topics, and collecting the ads and targeting explanations Meta shows to participants before and after the intervention. We find that utilizing the "See less" ad control for the topics we study does not significantly reduce the number of ads shown by Meta on these topics, and that the control is less effective for some users whose demographics are correlated with the topic. Furthermore, we find that the majority of ad targeting explanations for local ads made no reference to location-specific targeting criteria, and did not inform users why ads related to the topics they marked to "See less" of continued to be delivered. We hypothesize that the poor effectiveness of controls and lack of actionability in explanations are the result of the shift to AI-mediated targeting, for which explainability and transparency tools have not yet been developed. Our work thus provides evidence for the need of new methods for transparency and user control, suitable and reflective of increasingly complex AI-mediated ad delivery systems. - [15] arXiv:2408.11935 (cross-list from cs.LG) [pdf, html, other]
-
Title: Explainable Anomaly Detection: Counterfactual driven What-If AnalysisLogan Cummins, Alexander Sommers, Sudip Mittal, Shahram Rahimi, Maria Seale, Joseph Jaboure, Thomas ArnoldComments: 8 pages, 6 figures, 3 tablesSubjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Human-Computer Interaction (cs.HC)
There exists three main areas of study inside of the field of predictive maintenance: anomaly detection, fault diagnosis, and remaining useful life prediction. Notably, anomaly detection alerts the stakeholder that an anomaly is occurring. This raises two fundamental questions: what is causing the fault and how can we fix it? Inside of the field of explainable artificial intelligence, counterfactual explanations can give that information in the form of what changes to make to put the data point into the opposing class, in this case "healthy". The suggestions are not always actionable which may raise the interest in asking "what if we do this instead?" In this work, we provide a proof of concept for utilizing counterfactual explanations as what-if analysis. We perform this on the PRONOSTIA dataset with a temporal convolutional network as the anomaly detector. Our method presents the counterfactuals in the form of a what-if analysis for this base problem to inspire future work for more complex systems and scenarios.
- [16] arXiv:2408.11936 (cross-list from cs.AI) [pdf, html, other]
-
Title: Estimating Contribution Quality in Online Deliberations Using a Large Language ModelSubjects: Artificial Intelligence (cs.AI); Human-Computer Interaction (cs.HC)
Deliberation involves participants exchanging knowledge, arguments, and perspectives and has been shown to be effective at addressing polarization. The Stanford Online Deliberation Platform facilitates large-scale deliberations. It enables video-based online discussions on a structured agenda for small groups without requiring human moderators. This paper's data comes from various deliberation events, including one conducted in collaboration with Meta in 32 countries, and another with 38 post-secondary institutions in the US.
Estimating the quality of contributions in a conversation is crucial for assessing feature and intervention impacts. Traditionally, this is done by human annotators, which is time-consuming and costly. We use a large language model (LLM) alongside eight human annotators to rate contributions based on justification, novelty, expansion of the conversation, and potential for further expansion, with scores ranging from 1 to 5. Annotators also provide brief justifications for their ratings. Using the average rating from other human annotators as the ground truth, we find the model outperforms individual human annotators. While pairs of human annotators outperform the model in rating justification and groups of three outperform it on all four metrics, the model remains competitive.
We illustrate the usefulness of the automated quality rating by assessing the effect of nudges on the quality of deliberation. We first observe that individual nudges after prolonged inactivity are highly effective, increasing the likelihood of the individual requesting to speak in the next 30 seconds by 65%. Using our automated quality estimation, we show that the quality ratings for statements prompted by nudging are similar to those made without nudging, signifying that nudging leads to more ideas being generated in the conversation without losing overall quality. - [17] arXiv:2408.12047 (cross-list from cs.CY) [pdf, html, other]
-
Title: Do Responsible AI Artifacts Advance Stakeholder Goals? Four Key Barriers Perceived by Legal and Civil StakeholdersSubjects: Computers and Society (cs.CY); Human-Computer Interaction (cs.HC)
The responsible AI (RAI) community has introduced numerous processes and artifacts (e.g., Model Cards, Transparency Notes, Data Cards) to facilitate transparency and support the governance of AI systems. While originally designed to scaffold and document AI development processes in technology companies, these artifacts are becoming central components of regulatory compliance under recent regulations such as the EU AI Act. Much prior work has explored the design of new RAI artifacts or their use by practitioners within technology companies. However, as RAI artifacts begin to play key roles in enabling external oversight, it becomes critical to understand how stakeholders--particularly those situated outside of technology companies who govern and audit industry AI deployments--perceive the efficacy of RAI artifacts. In this study, we conduct semi-structured interviews and design activities with 19 government, legal, and civil society stakeholders who inform policy and advocacy around responsible AI efforts. While participants believe that RAI artifacts are a valuable contribution to the broader AI governance ecosystem, many are concerned about their potential unintended, longer-term impacts on actors outside of technology companies (e.g., downstream end-users, policymakers, civil society stakeholders). We organize these beliefs into four barriers that help explain how RAI artifacts may (inadvertently) reconfigure power relations across civil society, government, and industry, impeding civil society and legal stakeholders' ability to protect downstream end-users from potential AI harms. Participants envision how structural changes, along with changes in how RAI artifacts are designed, used, and governed, could help redirect the role of artifacts to support more collaborative and proactive external oversight of AI systems. We discuss research and policy implications for RAI artifacts.
- [18] arXiv:2408.12103 (cross-list from cs.RO) [pdf, html, other]
-
Title: Control-Theoretic Analysis of Shared Control SystemsComments: Presented in the Variable Autonomy for Human-Robot Teaming (VAT) workshop at 33rd IEEE International Conference on Robot and Human Interactive Communication (RO-MAN) 2024Subjects: Robotics (cs.RO); Human-Computer Interaction (cs.HC)
Users of shared control systems change their behavior in the presence of assistance, which conflicts with assumpts about user behavior that some assistance methods make. In this paper, we propose an analysis technique to evaluate the user's experience with the assistive systems that bypasses required assumptions: we model the assistance as a dynamical system that can be analyzed using control theory techniques. We analyze the shared autonomy assistance algorithm and make several observations: we identify a problem with runaway goal confidence and propose a system adjustment to mitigate it, we demonstrate that the system inherently limits the possible actions available to the user, and we show that in a simplified setting, the effect of the assistance is to drive the system to the convex hull of the goals and, once there, add a layer of indirection between the user control and the system behavior. We conclude by discussing the possible uses of this analysis for the field.
- [19] arXiv:2408.12124 (cross-list from cs.LG) [pdf, html, other]
-
Title: Recording Brain Activity While Listening to Music Using Wearable EEG Devices Combined with Bidirectional Long Short-Term Memory NetworksComments: 15 pagesSubjects: Machine Learning (cs.LG); Human-Computer Interaction (cs.HC); Signal Processing (eess.SP)
Electroencephalography (EEG) signals are crucial for investigating brain function and cognitive processes. This study aims to address the challenges of efficiently recording and analyzing high-dimensional EEG signals while listening to music to recognize emotional states. We propose a method combining Bidirectional Long Short-Term Memory (Bi-LSTM) networks with attention mechanisms for EEG signal processing. Using wearable EEG devices, we collected brain activity data from participants listening to music. The data was preprocessed, segmented, and Differential Entropy (DE) features were extracted. We then constructed and trained a Bi-LSTM model to enhance key feature extraction and improve emotion recognition accuracy. Experiments were conducted on the SEED and DEAP datasets. The Bi-LSTM-AttGW model achieved 98.28% accuracy on the SEED dataset and 92.46% on the DEAP dataset in multi-class emotion recognition tasks, significantly outperforming traditional models such as SVM and EEG-Net. This study demonstrates the effectiveness of combining Bi-LSTM with attention mechanisms, providing robust technical support for applications in brain-computer interfaces (BCI) and affective computing. Future work will focus on improving device design, incorporating multimodal data, and further enhancing emotion recognition accuracy, aiming to achieve practical applications in real-world scenarios.
- [20] arXiv:2408.12463 (cross-list from cs.CV) [pdf, html, other]
-
Title: Smartphone-based Eye Tracking System using Edge Intelligence and Model OptimisationSubjects: Computer Vision and Pattern Recognition (cs.CV); Human-Computer Interaction (cs.HC); Machine Learning (cs.LG); Performance (cs.PF)
A significant limitation of current smartphone-based eye-tracking algorithms is their low accuracy when applied to video-type visual stimuli, as they are typically trained on static images. Also, the increasing demand for real-time interactive applications like games, VR, and AR on smartphones requires overcoming the limitations posed by resource constraints such as limited computational power, battery life, and network bandwidth. Therefore, we developed two new smartphone eye-tracking techniques for video-type visuals by combining Convolutional Neural Networks (CNN) with two different Recurrent Neural Networks (RNN), namely Long Short Term Memory (LSTM) and Gated Recurrent Unit (GRU). Our CNN+LSTM and CNN+GRU models achieved an average Root Mean Square Error of 0.955cm and 1.091cm, respectively. To address the computational constraints of smartphones, we developed an edge intelligence architecture to enhance the performance of smartphone-based eye tracking. We applied various optimisation methods like quantisation and pruning to deep learning models for better energy, CPU, and memory usage on edge devices, focusing on real-time processing. Using model quantisation, the model inference time in the CNN+LSTM and CNN+GRU models was reduced by 21.72% and 19.50%, respectively, on edge devices.
- [21] arXiv:2408.12579 (cross-list from cs.CL) [pdf, other]
-
Title: RuleAlign: Making Large Language Models Better Physicians with Diagnostic Rule AlignmentXiaohan Wang, Xiaoyan Yang, Yuqi Zhu, Yue Shen, Jian Wang, Peng Wei, Lei Liang, Jinjie Gu, Huajun Chen, Ningyu ZhangComments: Ongoing workSubjects: Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Human-Computer Interaction (cs.HC); Information Retrieval (cs.IR); Machine Learning (cs.LG)
Large Language Models (LLMs) like GPT-4, MedPaLM-2, and Med-Gemini achieve performance competitively with human experts across various medical benchmarks. However, they still face challenges in making professional diagnoses akin to physicians, particularly in efficiently gathering patient information and reasoning the final diagnosis. To this end, we introduce the RuleAlign framework, designed to align LLMs with specific diagnostic rules. We develop a medical dialogue dataset comprising rule-based communications between patients and physicians and design an alignment learning approach through preference learning. Experimental results demonstrate the effectiveness of the proposed approach. We hope that our work can serve as an inspiration for exploring the potential of LLMs as AI physicians.
Cross submissions for Friday, 23 August 2024 (showing 10 of 10 entries )
- [22] arXiv:2311.05457 (replaced) [pdf, html, other]
-
Title: Leveraging Large Language Models for Generating Mobile Sensing Strategies in Human Behavior ModelingComments: This manuscript has been accepted as a workshop paper on Ubicomp 2024Subjects: Human-Computer Interaction (cs.HC)
Mobile sensing plays a crucial role in generating digital traces to understand human daily lives. However, studying behaviours like mood or sleep quality in smartphone users requires carefully designed mobile sensing strategies such as sensor selection and feature construction. This process is time-consuming, burdensome, and requires expertise in multiple domains. Furthermore, the resulting sensing framework lacks generalizability, making it difficult to apply to different scenarios. In the research, we propose an automated mobile sensing strategy for human behaviour understanding. First, we establish a knowledge base and consolidate rules for data collection and effective feature construction. Then, we introduce the multi-granular human behaviour representation and design procedures for leveraging large language models to generate strategies. Our approach is validated through blind comparative studies and usability evaluation. Ultimately, our approach holds the potential to revolutionise the field of mobile sensing and its applications.
- [23] arXiv:2402.12989 (replaced) [pdf, html, other]
-
Title: Tactile Perception in Upper Limb Prostheses: Mechanical Characterization, Human Experiments, and Computational FindingsAlessia Silvia Ivani, Manuel G. Catalano, Giorgio Grioli, Matteo Bianchi, Yon Visell, Antonio BicchiSubjects: Robotics (cs.RO); Human-Computer Interaction (cs.HC)
Our research investigates vibrotactile perception in four prosthetic hands with distinct kinematics and mechanical characteristics. We found that rigid and simple socket-based prosthetic devices can transmit tactile information and surprisingly enable users to identify the stimulated finger with high reliability. This ability decreases with more advanced prosthetic hands with additional articulations and softer mechanics. We conducted experiments to understand the underlying mechanisms. We assessed a prosthetic user's ability to discriminate finger contacts based on vibrations transmitted through the four prosthetic hands. We also performed numerical and mechanical vibration tests on the prostheses and used a machine learning classifier to identify the contacted finger. Our results show that simpler and rigid prosthetic hands facilitate contact discrimination (for instance, a user of a purely cosmetic hand can distinguish a contact on the index finger from other fingers with 83% accuracy), but all tested hands, including soft advanced ones, performed above chance level. Despite advanced hands reducing vibration transmission, a machine learning algorithm still exceeded human performance in discriminating finger contacts. These findings suggest the potential for enhancing vibrotactile feedback in advanced prosthetic hands and lay the groundwork for future integration of such feedback in prosthetic devices.