Search NASA⌕ Search

SEARCH · Search NASA

Results for “Data analytics”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 649 records · Page 36

Build and Execute Environment

At exascale, the challenge becomes to develop applications that run at scale and use exascale platforms reliably, efficiently, and flexibly. Workflows become much more complex because they must seamlessly integrate simulation and data analytics. They must include down-sampling, post-processing, feature extraction, and visualization. Power and data transfer limitations require these analysis tasks to be run in-situ or in-transit. We expect successful workflows will comprise multiple linked simulations along with tens of analysis routines. Users will have limited development time at scale and, therefore, must have rich tools to develop, debug, test, and deploy applications. At this scale, successful workflows will compose linked computations from an assortment of reliable, well-defined computation elements, ones that can come and go as required, based on the needs of the workflow over time. We propose a novel framework that utilizes both virtual machines (VMs) and software containers to create a workflow system that establishes a uniform build and execution environment (BEE) beyond the capabilities of current systems. In this environment, applications will run reliably and repeatably across heterogeneous hardware and software. Containers, both commercial (Docker and Rocket) and open-source (LXC and LXD), define a runtime that isolates all software dependencies from the machine operating system. Workflows may contain multiple containers that run different operating systems, different software, and even different versions of the same software. We will run containers in open-source virtual machines (KVM) and emulators (QEMU) so that workflows run on any machine entirely in user-space. On this platform of containers and virtual machines, we will deliver workflow software that provides services, including repeatable execution, provenance, checkpointing, and future proofing. We will capture provenance about how containers were launched and how they interact to annotate workflows for repeatable and partial re-execution. We will coordinate the physical snapshots of virtual machines with parallel programming constructs, such as barriers, to automate checkpoint and restart. We will also integrate with HPC-specific container runtimes to gain access to accelerators and other specialized hardware to preserve native performance. Containers will link development to continuous integration. When application developers check code in, it will automatically be tested on a suite of different software and hardware architectures.

Guan, Qiang↗

Towards an Open (Data) Science Analytics-Hub for Reproducible Multi-Model Climate Analysis at Scale

Open Science is key to future scientific research and promotes a deep transformation in the whole scientific research process encouraging the adoption of transparent and collaborative scientific approaches aimed at knowledge sharing. Open Science is increasingly gaining attention in the current and future research agenda worldwide. To effectively address Open Science goals, besides Open Access to results and data, it is also paramount to provide tools or environments to support the whole research process, in particular the design, execution and sharing of transparent and reproducible experiments, including data provenance (or lineage) tracking. This work introduces the Climate Analytics-Hub, a new component on top of the Earth System Grid Federation (ESGF), which joins big data approaches and parallel computing paradigms to provide an Open Science environment for reproducible multi-model climate change data analytics experiments at scale. An operational implementation has been set up at the SuperComputing Centre of the Euro- Mediterranean Center on Climate Change, with the main goal of becoming a reference Open Science hub in the climate community regarding the multi-model analysis based on the Coupled Model Intercomparison Project (CMIP).

analytics-hub↗

International interlaboratory compilation of trace element concentrations in the CUP-2 uranium ore concentrate standard

Herein, a nuclear forensics investigation involving a uranium ore concentrate relies on accurate and precise analysis of impurities. Analytical data defensibility requires the use of reference materials as part of quality control. This study presents a compilation of trace element concentration results of the CUP-2 Uranium Ore Concentrate Standard measured by 11 different laboratories. The laboratories employed various dissolution methods, analytical preparation methods, and instrumental platforms. The data presented here contain concentrations of 66 impurities with up to 138 individual data points for each impurity. Consensus values have been assigned to each impurity following a statistical analysis of the data set.

38 RADIATION CHEMISTRY, RADIOCHEMISTRY, AND NUCLEA↗

Data Confidentiality Challenges in Big Data Applications

In this paper, we address the problem of data confidentiality in big data analytics. In many fields, much useful patterns can be extracted by applying machine learning techniques to big data. However, data confidentiality must be protected. In many scenarios, data confidentiality could well be a prerequisite for data to be shared. We present a scheme to provide provable secure data confidentiality and discuss various techniques to optimize performance of such a system.

data confidentiality↗

GraphAide: Advanced Graph-Assisted Query and Reasoning System

Curating knowledge from multiple siloed sources that contain both structured and unstructured data is a major challenge in many real-world applications. Pattern matching and querying represent fundamental tasks in modern data analytics that leverage this curated knowledge. The development of such applications necessitates overcoming several research challenges, including data extraction, named entity recognition, data modeling, and designing query interfaces. Moreover, the explainability of these functionalities is critical for their broader adoption. The emergence of Large Language Models (LLMs) has accelerated the development lifecycle of new capabilities. Nonetheless, there is an ongoing need for domain-specific tools tailored to user activities. The creation of digital assistants has gained considerable traction in recent years, with LLMs offering a promising avenue to develop such assistants utilizing domain-specific knowledge and assumptions. In this context, we introduce an advanced query and reasoning system, GraphAide, which constructs a knowledge graph (KG) from diverse sources and allows to query and reason over the resulting KG. GraphAide harnesses both the KG and LLMs to rapidly develop domain-specific digital assistants. It integrates design patterns from retrieval augmented generation (RAG) and the semantic web to create an agentic LLM application. GraphAide underscores the potential for streamlined and efficient development of specialized digital assistants, thereby enhancing their applicability across various domains.

Purohit, Sumit [BATTELLE (PACIFIC NW LAB)] (ORCID:↗

Streaming Statistics

In the context of a larger effort for in situ data analytics, there is a need to calculate basic statistics metrics (e.g., count, mean, median) online as new data points become available. Originally, the code for such online, or in other words streaming, statistics was part of the TALASS (Topological Analysis of Large- Scale Simulations) library. We isolated the relevant code and created a standalone library from it called Streaming Statistics. We also added an ability to serialize and deserialize the statistics objects so that the library can be used in distributed, task-based processing. To use the Streaming Statistics library, the user chooses a statistic, constructs an object for it, and then "adds" values to it, which means the statistic is augmented.

Shudler, Sergei↗

Data Fusion to Enhance Quality Control and Analysis with Instruments at the Marine and Coastal Research Laboratory

Deploying environmental monitoring instruments in the marine environment can be challenging, facing challenges around device survivability, biofouling and corrosion, and consistent data collection. This project explores the use of data fusion – the process of integrating multiple data sources to produce more consistent, accurate, and useful information – to build a consistent long-term monitoring system at the Marine and Coastal Research Laboratory (MCRL) in Sequim, Washington. Unused instruments that had been acquired from past projects were inventoried and deployments planned on the MCRL pier and floating dock. A total of 8 instruments were deployed including a tide gauge, hydrophone, acoustic Doppler current profiler (ADCP), photosynthetically active radiation (PAR) sensors, meteorological station, and three water quality sensors. Deployments were planned to be well-protected around the pier structure and a maintenance schedule was created for cleaning and recalibration. An automated data pipeline was created to aggregate data on edge computers that push data to Amazon Web Services (AWS) cloud storage every 15 minutes, performing automated quality control and data transformations using the Time Series Data Analytical Toolkit (TSDAT). Continued efforts are underway to maintain this system into the future, take a data-driven approach to maintenance scheduling, improve the reliability of the system, and share the data with a variety of end-users.

54 ENVIRONMENTAL SCIENCES↗

Fiscal Year 2010 Phased Construction Completion Report for EU Z2-32 in Zone 2, East Tennessee Technology Park, Oak Ridge, Tennessee

The Record of Decision for Soil, Buried Waste, and Subsurface Structure Actions in Zone 2, East Tennessee Technology Park, Oak Ridge, Tennessee (DOEIORJO 1-2161 &D2) (Zone 2 ROD) acknowledged that most of the 800 acres in Zone 2 were contaminated, but that sufficient data to confirm the levels of contamination were lacking. The Zone 2 ROD further specified that a sampling strategy for filling the data gaps would be developed. The Remedial Design Report/Remedial Action Work Plan for Zone 2 Soils, Slabs, and Subsurface Structures, East Tennessee Technology Park, Oak Ridge, Tennessee (DOEIORIO 1 -2224&D3) (RDRJRAWP) defined the sampling strategy as the Dynamic Verification Strategy (DVS), generally following the approach used for characterization of the Zone I exposure units (EUs). The Zone 2 ROD divided the Zone 2 area into seven geographic areas and 44 EUs. To facilitate the data quality objectives (DQOs) of the DVS process, the RDR/RAWP regrouped the 44 EUs into 12 DQO scoping EU groups. These groups facilitated the DQO process by placing similar facilities and their support facilities together, which allowed identification of data gaps. The EU groups were no longer pertinent after DQO planning was completed and characterization was conducted as areas became accessible. As the opportunity to complete characterization became available, the planned DVS program was completed for the EU addressed in this document (EU Z2-32). The purpose of this Phased Construction Completion Report (PCCR) is to address the following: (1) Document DVS characterization results for EU Z2-32. (2) Describe and document the risk evaluation and determine if the EU meets the Zone 2 ROD requirements for unrestricted industrial use to 10 ft bgs. (3) Identify additional areas not defined in the Zone 2 ROD that require remediation based on the DVS evaluation results. (4) Describe the remedial action performed in the K-1066-G Yard in EU Z2-32. Approximately 18.4 acres are included in the EU addressed in this PCCR. Based on results of the DVS evaluation, all 18.4 acres are recommended for unrestricted industrial use to 10 ft bgs. There are no Federal Facility Agreement Sites included in Appendix A of the Zone 2 ROD in EU Z2-32. The Zone 2 ROD requires land use controls to prevent disturbance of soils below 10 ft deep and to restrict future land use to industrial/commercial activities. In response to stakeholder comments, the U.S. Department of Energy agreed to re-evaluate the need for such land use restrictions. This document includes a screening evaluation to determine the likelihood of land use controls in EU Z2-32 being modified to: (1) eliminate the restriction on disturbance of soils below 10 ft bgs where data indicate the absence of residual contamination at any depth that would result in an unacceptable risk to the future industrial worker, and (2) permit alternative land uses that would be protective of future site occupants. Results of this screening evaluation indicate a high probability that restrictions on disturbing soil below 10 ft bgs could be safely eliminated for EU Z2-32. A qualitative screening evaluation considered the likelihood of unrestricted land use being protective of future site occupants. Based on this qualitative assessment, all 18.4 acres addressed in this PCCR were assigned a high probability for consideration of release for unrestricted land use. This document contains the main text (Sects. 1 through 13) and one appendix. The main text addresses the purpose for this PCCR as described above. Additional supporting detail (e.g., field work and data summaries, graphics) is provided in the EU Z2-32 technical memorandum (Appendix A). Historical and DVS analytical data used in this PCCR are provided on a compact disc accompanying this document and can be accessed through the Oak Ridge Environmental Information System.

54 ENVIRONMENTAL SCIENCES↗

Thermal conductivity of Rene 41 honeycomb panels

Effective thermal conductivities of Rene 41 panels suitable for advanced space transportation vehicle structures were determined analytically and experimentally for temperature ranges between 20.4K (423 F) and 1186K (1675 F). The cryogenic data were obtained using a cryostat whereas the high temperature data were measured using a heat flow meter and a comparative thermal conductivity instrument respectively. Comparisons were made between analysis and experimental data. Analytical methods appear to provide reasonable definition of the honeycomb panel effective thermal conductivities.

Deriugin, V.↗

Analytic model approach to the inversion of scattering data

An analytic model approach is applied to several simple atmospheric inversion problems. This method gives a sharp determination of aerosol size distribution parameters. It is shown that this analytic approach, together with ground level point sampling data measurements, can be used to infer information on the tropospheric ozone profile.

Green, A. E. S.↗

DataFed

There is a need to easily and uniformly capture, access, enrich, organize, search, and collaborate on working scientific data sets across geographically distributed experiment, computation, and data analytics facilities. DataFed provides a set of services and applications enabling simple and uniform cross-facility scientific data management with federated (distributed) data repositories. This solution greatly reduces effort required by scientist to organize and/or move data between facilities/institutes, Reduces likelihood of misidentification and subsequent misuse of data, and improves scientific reproducibility through deep, domain-specific context and provenance capture.

Stansberry, DaleV.↗

ShareAnalytics

ShareAnalytics is a lightweight extensible platform for collaborative data analytics and facility characterization via Non-intrusive Load Monitoring (NILM). It consists of three core components: a full stack web application, a dashboard tool for analyzing streaming data and a High Performance Computing (HPC) cluster leveraging slurm for resource management to perform real time analysis. The entire framework is implemented in Docker for ease of deployment and to allow the user to test the platform before making a significant investment in hardware if they require additional computing resources. While many other platforms for analyzing streaming data exists, ShareAnalytics is unique in that HPC capabilities and facility characterization via scored analytics are fully integrated into the platform.

Oesch, TimothyS↗

Chemical Composition Analysis of Simulated Waste Glass T10-G-16A

In this report, SRNL provides chemical composition analyses of a simulated LAW glass designated T10-G-16A.The measured chemical composition data are reported and compared with the targeted values for each component. No issues were identified in reviewing the analytical data.

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗

Chemical Composition Analysis of Simulated Waste Glass T10-G-16A

In this report, SRNL provides chemical composition analyses of a simulated LAW glass designated T10-G-16A. The measured chemical composition data are reported and compared with the targeted values for each component. No issues were identified in reviewing the analytical data. The measured concentrations of Al 2 O 3 , CaO, Na 2 O, and SiO 2 in T10-G-16A deviated from the targeted composition. It is of potential interest to note that these deviations result in a glass composition that is shifted toward the SiO 2 corner of the Al 2 O 3 -Na 2 O-SiO 2 phase diagram, away from the nepheline phase field. The results of these analyses will be used in developing a better understanding of the properties and performance of this glass.

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗

EDX ClaiMM: Digital Resources for the Critical Minerals and Materials Community

Securing critical mineral supply chains is essential for transitioning to a clean energy economy and for maintaining national security. Big-data analytics can serve as a cost-effective means of identifying new domestic critical mineral resources but only if data can be easily located and digested. Using ArcGIS Enterprise Sites, EDX ClaiMM was developed to increase the accessibility of critical minerals data, reducing time spent on data collection and integration. Hosted tools provide rapid visualization and exploration of key datasets, unlocking insights to support resource assessments.

Yesenchak, Rachel↗

Utilizing HDF4 File Content Maps for the Cloud

We demonstrate a prototype study that HDF4 file content map can be used for efficiently organizing data in cloud object storage system to facilitate cloud computing. This approach can be extended to any binary data formats and to any existing big data analytics solution powered by cloud computing because HDF4 file content map project started as long term preservation of NASA data that doesn't require HDF4 APIs to access data.

Elastic Search↗

Spatio-Temporal Denoising Graph Autoencoders with Data Augmentation for Missing Photovoltaic Data Imputation

The integration of the global Photovoltaic (PV) market with real time data-loggers has enabled large scale PV data analytical pipelines for power forecasting and long-term reliability assessment of PV fleets. Nevertheless, the performance of PV data analysis heavily depends on the quality of PV timeseries data. This paper proposes a novel Spatio-Temporal Denoising Graph Autoencoder (STD-GAE) framework to impute missing PV Power Data. STDGAE exploits temporal correlation, spatial coherence, and value dependencies from domain knowledge to recover missing data. It is empowered by two modules. (1) To cope with sparse yet various scenarios of missing data, STD-GAE incorporates a domain-knowledge aware data augmentation module that creates plausible variations of missing data patterns. This generalizes STD-GAE to robust imputation over different seasons and environment. (2) STD-GAE nontrivially integrates spatiotemporal graph convolution layers (to recover local missing data by observed “neighboring” PV plants) and denoising autoencoder (to recover corrupted data from augmented counterpart) to improve the accuracy of imputation accuracy at PV fleet level. We have evaluated our proposed model on two realworld PV datasets. Experimental results show that STD-GAE can achieve a gain of 43.14% in imputation accuracy and remains less sensitive to missing rate, different seasons, and missing scenarios, compared with state-of-the-art data imputation methods such as MIDA and LRTC-TNN.

Fan, Yangxin↗

Karlsruhe Database for Radioactive Wastes (KADABRA) - Accounting and Management System for Radioactive Waste Treatment - 12275

The data management system KADABRA was designed according to the purposes of the Cen-tral Decontamination Department (HDB) of the Wiederaufarbeitungsanlage Karlsruhe Rueckbau- und Entsorgungs-GmbH (WAK GmbH), which is specialized in the treatment and conditioning of radioactive waste. The layout considers the major treatment processes of the HDB as well as regulatory and legal requirements. KADABRA is designed as an SAG ADABAS application on IBM system Z mainframe. The main function of the system is the data management of all processes related to treatment, transfer and storage of radioactive material within HDB. KADABRA records the relevant data concerning radioactive residues, interim products and waste products as well as the production parameters relevant for final disposal. Analytical data from the laboratory and non destructive assay systems, that describe the chemical and radiological properties of residues, production batches, interim products as well as final waste products, can be linked to the respective dataset for documentation and declaration. The system enables the operator to trace the radioactive material through processing and storage. Information on the actual sta-tus of the material as well as radiological data and storage position can be gained immediately on request. A variety of programs accessed to the database allow the generation of individual reports on periodic or special request. KADABRA offers a high security standard and is constantly adapted to the recent requirements of the organization. (authors)

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗