Search NASA⌕ Search

SEARCH · Search NASA

Results for “Large data”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 613 records · Page 34

Application-driven Privacy-preserving Data Publishing with Correlated Attributes

Recent advances in computing have allowed for the possibility to collect large amounts of data on personal activities and private living spaces. To address the privacy concerns of users in this environment, we propose a novel framework called PR-GAN that offers privacy-preserving mechanism using generative adversarial networks. Given a target application, PR-GAN automatically modifies the data to hide sensitive attributes - which may be hidden and can be inferred by machine learning algorithms - while preserving the data utility in the target application. Unlike prior works, the public's possible knowledge of the correlation between the target application and sensitive attributes is built into our modeling. We formulate our problem as an optimization problem, show that an optimal solution exists and use generative adversarial networks (GAN) to create perturbations. We further show that our method provides privacy guarantees under the Pufferfish framework, an elegant generalization of the differential privacy that allows for the modeling of prior knowledge on data and correlations. Through experiments, we show that our method outperforms conventional methods in effectively hiding the sensitive attributes while guaranteeing high performance in the target application, for both property inference and training purposes. Finally, we demonstrate through further experiments that once our model learns a privacy-preserving task, such as hiding subjects' identity, on a group of individuals, it can perform the same task on a separate group with minimal performance drops.

97 MATHEMATICS AND COMPUTING↗

Fast Quasi-Static Time-Series Simulation for Accurate PV Inverter Semiconductor Fatigue Analysis with a Long-Term Solar Profile

Power system simulations with long-term data typically have large time steps varying from one second to a few minutes. However, for PV inverter semiconductors, the minimum thermal stress cycle occurs over the fundamental grid frequency (50 or 60 Hz). This requires the time step of the fatigue simulation to be around 100 µs. This small time step requires long computation times to process yearly power production profiles. This paper proposes a fast fatigue simulation for inverter semiconductors using the quasi-static time series (QSTS) simulation concept. The proposed simulation calculates the steady state of the semiconductor junction temperature by using a Fast Fourier Transform (FFT). The small thermal cycling during a switching period and even over the fundamental waveform is disregarded to further accelerate the simulation speed. The resulting time step of the fatigue simulation is 15 minutes, which is consistent with the solar dataset. The error of the proposed simulation is 0.16% compared to the fatigue simulation results using the complete thermal stress profile. A PV inverter that responds to a Transactive Energy System (TES) is simulated to demonstrate the use of the proposed fatigue simulation. The proposed simulation has the potential to co-simulate with system level simulation tools that also adopt the QSTS concept.

Liu, Yunting↗

Tunable Solid‐State Properties and Anisotropic Charge Mobility in Hydrogen‐Bonded Diketopyrrolopyrrole Polymers via Automated Device Fabrication and Characterization

The optoelectronic properties of semiconducting polymers and device performance rely on a delicate interplay of design and processing conditions. However, screening and optimizing the relationships between these parameters for reliably fabricating organic electronics can be an arduous task requiring significant time and resources. To overcome this challenge, Polybot is developed—a robotic platform within a self‐driving lab that can efficiently produce organic field‐effect transistors (OFETs) from various semiconducting polymers via high‐throughput blade coating deposition. Polybot not only handles the fabrication process but also can conduct characterization tests on the devices and autonomously analyze the data gathered, thus facilitating the rapid acquisition of data on a large scale. This work leverages the capabilities of this platform to investigate the fabrication of OFETs using hydrogen bonding‐containing semiconducting polymers. Through high‐throughput fabrication and characterization, various data trends are analyzed, and large extents of anisotropic charge mobility are observed in devices. The materials are thoroughly characterized to understand the role of processing conditions in solid state and electronic properties of these organic semiconductors. The findings demonstrate the effectiveness of automated fabrication and characterization platforms in uncovering novel structure–property relationships, facilitating refinement of rational chemical design, and processing conditions, ultimately leading to new semiconducting materials.

automated fabrication and characterization↗

PySIDT: Subgraph Isomorphic Decision Trees for Molecular Property Prediction

Accurate molecular property prediction is important across all fields of chemistry. Deep neural networks (DNNs) have become increasingly popular due to their ability to train automatically, avoiding the incredibly tedious process of constructing and extending traditional property estimation schemes. However, DNNs require large amounts of training data, are challenging to interpret, require large amounts of memory to load even during inference, and have severe difficulties incorporating qualitative chemical knowledge, which are often desired for molecular property prediction tasks. Here, in this study, we present PySIDT (https://github.com/zadorlab/PySIDT), a software for training and running inference on Subgraph Isomorphic Decision Trees (SIDTs). SIDTs are graph-based decision trees made of nodes associated with molecular substructures. Inference is done by descending target molecular structures down the decision tree to nodes with matching subgraph isomorphic substructures and making predictions based on the final (most specific) nodes matched. SIDTs scale down well to dataset sizes much smaller than is feasible for DNNs. As trees of molecular substructures, SIDTs are inherently readable and easy to visualize, making them easy to analyze. They are also straightforward to extend and retrain, facilitate uncertainty estimation, and enable easy integration of expert knowledge. We demonstrate the SIDT approach discussing its application to a diverse range of molecular prediction tasks: rate coefficient estimation, diffusion coefficient estimation, thermochemistry estimation, transition state bond stretch prediction, p K a prediction, stability of molecular structures, stability of surface structures, and prediction of surface lateral interaction energetics. Additionally, we demonstrate the power of the SIDT algorithms in two direct learning curve vanilla comparisons with the popular DNN-based software Chemprop and the popular gradient boosted trees-based software XGBoost on enthalpy of formation and rate coefficient prediction tasks. In particular, in the enthalpy of formation case, vanilla PySIDT is able to outperform vanilla Chemprop and XGBoost across the full range of training/validation set sizes out to 11,560 data points.

Johnson, Matthew Sean [Sandia National Laboratorie↗

Data augmentation for disruption prediction via robust surrogate models

The goal of this work is to generate large statistically representative data sets to train machine learning models for disruption prediction provided by data from few existing discharges. Such a comprehensive training database is important to achieve satisfying and reliable prediction results in artificial neural network classifiers. Here, we aim for a robust augmentation of the training database for multivariate time series data using Student t process regression. We apply Student t process regression in a state space formulation via Bayesian filtering to tackle challenges imposed by outliers and noise in the training data set and to reduce the computational complexity. Thus, the method can also be used if the time resolution is high. We use an uncorrelated model for each dimension and impose correlations afterwards via colouring transformations. We demonstrate the efficacy of our approach on plasma diagnostics data of three different disruption classes from the DIII-D tokamak. To evaluate if the distribution of the generated data is similar to the training data, we additionally perform statistical analyses using methods from time series analysis, descriptive statistics and classic machine learning clustering algorithms.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

The Role of Mobility Data Hubs in an Integrated Decarbonized Transportation Future

The landscape for connected mobility ecosystems is evolving rapidly as information and communication technologies lower the cost and complexity of connecting people to places, integrating transportation modes and collecting data regarding such movements. These developments have been key to unlocking new business opportunities, particularly through mobility services. While the mechanisms for data collection, processing, and transfer have made significant advances in the past decade, the broader landscape of mobility data architectures and data users remains largely unresolved. It is unclear as to whether the result will converge towards a framework that resembles a coherent quilt or a disjointed patchwork of competing visions. Initial approaches to mobility data collection and provisioning have been largely siloed - by mode or software - or held for exclusive use, however several key players are quickly realizing the need and opportunities enabled through integrated mobility data eco-systems, or mobility data hubs as referred to in this paper. As the business case for hosting mobility data hubs evolves, there is great uncertainty regarding their impact to either advance or exacerbate sustainable mobility (e.g., seamless connectivity across modes, decreased energy consumption and greenhouse gas emissions, etc.). Groups such as the United Nations and World Bank have identified data platforms as a key enabler of realizing environmental and social benefits. If designed with decarbonization in mind, we hypothesize that enhanced observability provided by these ever-expanding mobility data hubs can facilitate energy and emissions reductions that are otherwise limited by transactional barriers and knowledge asymmetry that is inherent to a more siloed approach. In this sense, integration of mobility data can help to create a competitive playing field where value is not determined by exclusivity of data, but rather the quality and uniqueness of a given service. The goals of this paper are to 1) identify key players and data architectures that are emerging in a service-based mobility market, 2) explore several use cases where mobility data hubs have enabled greater sustainability outcomes, and 3) discuss key issues that will need to be resolved to fully leverage emerging mobility data hubs towards a sustainable transportation future.

ADVANCED PROPULSION SYSTEMS,MATHEMATICS AND COMPUT↗

The Data Mine model for accessible partnerships in data science

Abstract The Data Mine at Purdue University is a pioneering experiential learning community for undergraduate and graduate students of any background to learn data science. The first data‐intensive experience embedded in a large learning community, The Data Mine had nearly 1300 students in academic year (AY) 2022–2023 and nearly 1700 students for AY 2023–2024. The Data Mine embodies data‐infused education, research, and collaboration. Students learn Python, R, SQL, and shell‐scripting, while working on weekly projects within a high‐performance computing (HPC) cluster. In the Corporate Partners cohort, students work on teams of 5–15 students, led by a paid student team leader. Each cohort follows an Agile approach, working on data‐intensive projects provided by industry partners and mentored by company employees. Students develop professional and data skills throughout the academic year, from August through April. Many students return in subsequent years to the program, increasing their tenure with a Corporate Partner. Student teams are inherently interdisciplinary; students from 133 different majors are involved in the program, ranging from new incoming students through PhD level students. These interdisciplinary teams of students bring new perspectives to challenging problems in which data science is a key part of the solution. The interdisciplinary teams foster an environment of synthesis with ideas and solutions. Students come together with different life experiences, different levels of technical skill, but also varying ways they navigate paths to solutions because of the variety of majors represented, resulting in a more creative and robust solution than a traditional data science program. This article is categorized under: Applications of Computational Statistics > Education in Computational Statistics

Betz, Margaret A.↗

Accelerating geostatistical modeling using geostatistics-informed machine Learning

Ordinary Kriging (OK) is a popular geostatistical algorithm for spatial interpolation and estimation. The computational complexity of OK changes quadratically and cubically for memory and speed, respectively, given the number of data. Therefore, it is computationally intensive and also challenging to process a large set of data, especially in three-dimensional (3D) cases. This paper develops a geostatistics-informed machine learning (GIML) model to improve the efficiency of OK by reducing the number of points required to be estimated using OK. Specifically, only a very few of the unknown points are estimated by OK to get the weights and estimations, which are used as the training dataset. Moreover, the governing equations of OK are used to guide our proposed machine learning to better reproduce the spatial distributions. Our results show that the proposed GIML can reduce the computational time of OK by at least one order of magnitude. The effectiveness of the GIML is evaluated and compared using a 2D case. Furthermore, we demonstrate its efficiency and robustness by considering a different number of training samples on various 3D simulation grids.

58 GEOSCIENCES↗

Roadmap and Benchmarking: Privacy in Federated Load Forecasting

Data-driven techniques for energy demand forecasting continue to emerge with promising impacts on distribution grid planning. However, the development of robust and generalizable machine learning models requires that representative high quality training data are available. Distributed energy resources have begun to embed intelligence, gathering large amounts of data on customer demand, behavior, and household devices that are connected to the grid. Though utilities aggregate meter-level demand data for load shaping, demand response, outage management, reliability planning, and billing applications, there lies an inherent privacy concern in sharing consumption data that may identify individual consumer behavioral patterns. Hence, while sharing the data is crucial, the private sensitive customer data must be safeguarded from being exposed or manipulated. In this study, we propose a roadmap for implementing a based privacy preserving framework to support the advancement of data-driven analytics in data-sensitive distributed energy resources environments. The roadmap incorporates federated learning–a distributed training framework, differential privacy–a statistical framework that provides guarantees to safeguard the leakage of sensitive data, secure multiparty computation and homomorphic encryption– techniques for encrypting model gradients and applying secure aggregation on the server. Moreover, we perform baseline experiments on the federated short-term load forecasting (STLF) task using open-source residential load profile datasets, offering insights into the challenges of integrating differential privacy into federated learning.

Abebe, Waqwoya [Oak Ridge National Laboratory (ORN↗

Automatic recognition system for document digitization in nuclear power plants

With the increasing number of data-driven models in nuclear applications, large volumes of numerical data are required to accurately model and predict the health status of a plant component. However, many historical operation logs that contain useful information are not fully utilized due to the lack of a systematic approach of digitization. To overcome this issue, this study proposes an automatic pipeline for extracting information from handwritten tabular documents collected from nuclear power plants. In our pipeline, we first denoise scanned documents with morphological operations, and then extract relevant parts from individual pages using both traditional computer vision and neural network methods. Handwriting recognition is applied to obtain text and numbers. As the most challenging step is how to crop only relevant information, the main focus of our paper is to detect tables and cells from scanned handwritten documents. Here we evaluate the efficiency and accuracy of our proposed method on handwritten operational reports obtained from a real-world case study. The results demonstrate the high accuracy and practicality of our proposed method.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗

Electron Tomography and Machine Learning for Understanding the Highly Ordered Structure of Leafhopper Brochosomes

Insects known as leafhoppers (Hemiptera: Cicadellidae) produce hierarchically structured nanoparticles known as brochosomes that are exuded and applied to the insect cuticle, thereby providing camouflage and anti-wetting properties to aid insect survival. Although the physical properties of brochosomes are thought to depend on the leafhopper species, the structure–function relationships governing brochosome behavior are not fully understood. Brochosomes have complex hierarchical structures and morphological heterogeneity across species, due to which a multimodal characterization approach is required to effectively elucidate their nanoscale structure and properties. In this work, we study the structural and mechanical properties of brochosomes using a combination of atomic force microscopy (AFM), electron microscopy (EM), electron tomography, and machine learning (ML)-based quantification of large and complex scanning electron microscopy (SEM) image data sets. This suite of techniques allows for the characterization of internal and external brochosome structures, and ML-based image analysis methods of large data sets reveal correlations in the structure across several leafhopper species. Our results show that brochosomes are relatively rigid hollow spheres with characteristic dimensions and morphologies that depend on leafhopper species. Nanomechanical mapping AFM is used to determine a characteristic compression modulus for brochosomes on the order of 1–3 GPa, which is consistent with crystalline proteins. Altogether, this work provides an improved understanding of the structural and mechanical properties of leafhopper brochosomes using a new set of ML-based image classification tools that can be broadly applied to nanostructured biological materials.

Chemical structure↗

Power System Event Identification with Transfer Learning Using Large-scale Real-world Synchrophasor Data in the United States

The lack of sufficient labeled events and long training time limit the applicability of deep neural network-based power system event identification using synchrophasor data. In this paper, we propose to leverage transfer learning technique to boost the reliability and reduce the required training time of neural classifier for power system event identification. We use the weights of a neural classifier trained on one transmission system as the initial parameters of another neural classifier for a different transmission system. Numerical tests with real-world synchrophasor data from the Eastern and Western Interconnections of the United States show that the proposed transfer learning approach is very effective in not only improving the training reliability but also reducing the training time.

24 POWER TRANSMISSION AND DISTRIBUTION↗

An Integrated Assessment of a G3 GMD Event on Large-Scale Power Grids: From Magnetometer Data to Geomagnetically Induced Current Analysis

Solar activities can cause geomagnetic disturbances (GMDs) that give rise to geomagnetically induced currents (GICs) which may compromise the reliability of the power system. In order to build more reliable models representing GMD interactions with the power grid, the power system’s detailed electrical model must be considered along with fluctuations in the earth’s magnetic and induced surface electric fields. Here, this study investigates the impact of incorporating spatially varying magnetic fields into surface electric field models on GMD risk metrics. A spatially independent magnetic field model and a spatially varying model are compared through simulations. To perform this analysis, the earth’s magnetic field disturbances are transformed into surface electric fields using respective one-dimensional earth conductivity models. Then, the modeling impact of these electric fields is studied using a 2,000-bus grid for Texas and a 25,000-bus grid for the northeast and mid- Atlantic regions of the United States. Simulation results reveal that the inclusion of spatially varying magnetic fields results in considerable differences in GMD risk metrics, highlighting the importance of accounting for spatial variability when assessing GMD risks in the power system.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Efficient and Flexible Hierarchical Data Layouts for a Unified Encoding of Scalar Field Precision and Resolution

To address the problem of ever-growing scientific data sizes making data movement a major hindrance to analysis, we introduce a novel encoding for scalar fields: a unified tree of resolution and precision, specifically constructed so that valid cuts correspond to sensible approximations of the original field in the precision-resolution space. Furthermore, we introduce a highly flexible encoding of such trees that forms a parameterized family of data hierarchies. We discuss how different parameter choices lead to different trade-offs in practice, and show how specific choices result in known data representation schemes such as zfp[52], idx[58], and jpeg2000 [76]. Lastly, we provide system-level details and empirical evidence on how such hierarchies facilitate common approximate queries with minimal data movement and time, using real-world data sets ranging from a few gigabytes to nearly a terabyte in size. Experiments suggest that our new strategy of combining reductions in resolution and precision is competitive with state-of-the-art compression techniques with respect to data quality, while being significantly more flexible and orders of magnitude faster, and requiring significantly reduced resources.

97 MATHEMATICS AND COMPUTING↗

The Edge of Exploration: An Edge Storage and Computing Framework for Ambient Noise Seismic Interferometry Using Internet of Things Based Sensor Networks

Recent technological advances have reduced the complexity and cost of developing sensor networks for remote environmental monitoring. However, the challenges of acquiring, transmitting, storing, and processing remote environmental data remain significant. The transmission of large volumes of sensor data to a centralized location (i.e., the cloud) burdens network resources, introduces latency and jitter, and can ultimately impact user experience. Edge computing has emerged as a paradigm in which substantial storage and computing resources are located at the “edge” of the network. In this paper, we present an edge storage and computing framework leveraging commercially available components organized in a tiered architecture and arranged in a hub-and-spoke topology. The framework includes a popular distributed database to support the acquisition, transmission, storage, and processing of Internet-of-Things-based sensor network data in a field setting. We present details regarding the architecture, distributed database, embedded systems, and topology used to implement an edge-based solution. Lastly, a real-world case study (i.e., seismic) is presented that leverages the edge storage and computing framework to acquire, transmit, store, and process millions of samples of data per hour.

58 GEOSCIENCES↗

Computing for the DUNE Long-Baseline Neutrino Oscillation Experiment

This is a talk given at Computers in High Energy Physics in Adelaide, South Australia, Australia in November 2019. It is partially intended to explain the context of DUNE Computing for computing specialists. The DUNE collaboration consists of over 180 institutions from 33 countries. The experiment is in preparation now with commissioning of the first 10kT fiducial volume Liquid Argon TPC expected over the period 2025-2028 and a long data taking run with 4 modules expected from 2029 and beyond. An active prototyping program is already in place with a short test beam run with a 700T, 15,360 channel prototype of single-phase readout at the neutrino platform at CERN in late 2018 and tests of a similar sized dual-phase detector scheduled for mid-2019. The 2018 test beam run was a valuable live test of our computing model. The detector produced raw data at rates of up to ~2GB/s. These data were stored at full rate on tape at CERN and Fermilab and replicated at sites in the UK and Czech Republic. In total 1.2 PB of raw data from beam and cosmic triggers were produced and reconstructed during the six week test beam run. Baseline predictions for the full DUNE detector data, starting in the late 2020's are 30-60 PB of raw data per year. In contrast to traditional HEP computational problems, DUNE's Liquid Argon TPC data consist of simple but very large (many GB) 2D data objects which share many characteristics with astrophysical images. This presents opportunities to use advances in machine learning and pattern recognition as a frontier user of High Performance Computing facilities capable of massively parallel processing.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗