Search NASA⌕ Search

SEARCH · Search NASA

Results for “Data analytics”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 559 records · Page 31

From Dr. Steven Ashby, Director of PNNL

Powered by the creativity and imagination of more than 4,000 exceptional scientists, engineers and support professionals, at PNNL we advance the frontiers of science and address some of the most challenging problems in energy, the environment and national security. As DOE’s premier chemistry, environmental sciences and data analytics laboratory, we provide national leadership in four areas: deepening our understanding of climate science; inventing the future power grid; preventing nuclear proliferation; and speeding environmental remediation. Other areas where we make important contributions include energy storage, microbial biology and cyber security. PNNL also is home to EMSL (the Environmental Molecular Sciences Laboratory), one of DOE’s scientific user facilities. We apply these science strengths to address both national and international problems in complex adaptive systems that are too difficult for one institution to tackle alone. Take earth systems, for instance. The earth is a complex adaptive system because it involves everything from climate and microbial communities in the soil to emissions from cars and coal-powered industrial plants. All of these factors and others ultimately influence not only our environment and overall quality of life, but cause the earth to adapt in ways that must be further addressed. PNNL researchers are playing a vital role in finding solutions across every area of this complex adaptive system.

99 GENERAL AND MISCELLANEOUS↗

Quantifying Uncertainty in HPC Job Queue Time Predictions

High Performance Computing (HPC) has developed at an unprecedented pace in recent decades. This growth has demanded corresponding development in the area of HPC Operational Data Analytics (ODA), which encompasses a wide range of data analysis techniques, ML/AI efforts, tools, and visualizations. Published studies in ODA offer a variety of practical ways to inform HPC users, administrators, procurement managers, and other stakeholders. Uncertainty analysis, however, is rare in the related published literature. For instance, we identify only 1 out of 14 existing studies focused on job queue time prediction that investigates the uncertainty aspect of their proposed predictions. We recognize the utmost importance uncertainty quantification can have in such predictive analytics solutions, with consequences in how users interpret information they receive, and attempt to bridge this gap. With the goal of improving access to such insights, we develop a process for determining upper and lower bounds of the predicted queue times of a regression model at a specified confidence level. Our current research is focused on the uncertainty in predicting job queue times, yet our approach may be employed in predicting other metrics.

HPC↗

Comparison of Laboratory Analytical Methods for Uranium Analysis Superior Steel FUSRAP Site - 16606

The Superior Steel Site processed uranium metal in support of the Atomic Energy Commission's fuel-element development program in the 1950's, and was subsequently classified as a Formerly Utilized Sites Remedial Action Program (FUSRAP) Site. Previous investigations at the site identified uranium contamination both inside and outside the buildings associated with historic uranium metalworking activities. The U.S. Army Corps of Engineers (USACE) is currently engaged in the Remedial Investigation (RI) phase in accordance with the Comprehensive Environmental Response, Compensation, and Liability Act (CERCLA) of 1980, as amended. As part of the RI, gamma spectroscopy analysis of environmental samples was performed on-site using a laboratory-grade, high-purity germanium (HPGe) spectrometer. The gamma spectroscopy analysis included isotopes of uranium and their progeny. As a quality assurance measure, a subset of samples already analyzed on-site were sent to a certified, commercial, off-site laboratory for analysis using a suite of analytical methods including gamma spectroscopy, alpha spectroscopy, and mass spectroscopy. As an element of the quality assurance program implemented for the RI, an assessment of the correlations between the various analytical methods employed by both the on-site and off-site laboratory was performed. This assessment revealed inconsistencies between on-site and off-site gamma spectroscopy analytical results, which raised concerns about the appropriateness and suitability of the analytical data. To resolve the data quality concerns and better understand the source(s) of analytical results variability, the Superior Steel RI team (USACE together with its contractor, Amec Foster Wheeler) developed a quality control test whose design objective was to control or practically eliminate sampling and sample preparation variability in order to focus more clearly on analytical variability. This paper discusses the results of this supplemental quality control test, showing the differences in the various analytical methods (both on-site and off-site). It demonstrates in a practical and real world setting how sample preparation and analytical variability, if not controlled and understood can impact data quality and data defensibility. (authors)

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗

The role of quantum computing in advancing scientific high-performance computing: A perspective from the ADAC institute

Quantum computing (QC) has gained significant attention over the past two decades due to its potential for speeding up classically demanding tasks. This transition from an academic focus to a thriving commercial sector is reflected in substantial global investments. While advancements in qubit counts and functionalities continue at a rapid pace, current quantum systems still lack the scalability for practical applications, facing challenges such as too high error rates and limited coherence times. Here, this perspective paper examines the relationship between QC and high-performance computing (HPC), highlighting their complementary roles in enhancing computational efficiency. It is widely acknowledged that even fully error-corrected QC will not be suited for all computational tasks. Rather, future compute infrastructures are anticipated to employ quantum acceleration within hybrid systems that integrate HPC and QC. While QC can enhance classical computing, traditional HPC remains essential for maximizing quantum acceleration. This integration is a priority for supercomputing centers and companies, sparking innovation to address the challenges of merging these technologies. The novelty of this work lies in its unique perspective, reflecting the collective insights of the Accelerated Data Analytics and Computing (ADAC) Institute, a global consortium of over 20 leading HPC centers. Recognizing the growing importance of QC, ADAC established a Quantum Computing Working Group in 2023 to foster collaboration and knowledge-sharing among its members. This paper synthesizes insights from the group’s collaborative efforts and incorporates findings from a member survey that captures shared experiences, ongoing projects, and strategic directions. By outlining the current landscape and challenges of QC integration into HPC ecosystems, this work offers HPC specialists practical and forward-looking guidance on the opportunities and implications of QC in computationally intensive endeavors.

Accelerated Data Analytics and↗

Combustion machine learning: Principles, progress and prospects

Progress in combustion science and engineering has led to the generation of large amounts of data from large-scale simulations, high-resolution experiments, and sensors. This corpus of data offers enormous opportunities for extracting new knowledge and insights—if harnessed effectively. Machine learning (ML) techniques have demonstrated remarkable success in data analytics, thus offering a new paradigm for data-intense analyses and scientific investigations through combustion machine learning (CombML). While data-driven methods are utilized in various combustion areas, recent advances in algorithmic developments, the accessibility of open-source software libraries, the availability of computational resources, and the abundance of data have together rendered ML techniques ubiquitous in scientific analysis and engineering. This article examines ML techniques for applications in combustion science and engineering. Starting with a review of sources of data, data-driven techniques, and concepts, we examine supervised, unsupervised, and semi-supervised ML methods. Various combustion examples are considered to illustrate and to evaluate these methods. Next, we review past and recent applications of ML approaches to problems in combustion, spanning fundamental combustion investigations, propulsion and energy-conversion systems, and fire and explosion hazards. Challenges unique to CombML are discussed and further opportunities are identified, focusing on interpretability, uncertainty quantification, robustness, consistency, creation and curation of benchmark data, and the augmentation of ML methods with prior combustion-domain knowledge.

33 ADVANCED PROPULSION SYSTEMS↗

Multivariate Predictive Analytics of Wind Power Data for Robust Control of Energy Storage

Short-term forecasting is frequently identified as an important tool for the effective management of wind generation. However, forecasting errors, inherent to the point forecasts, increase requirements for energy storage and can affect optimal system operation. Probabilistic forecasts can help tackle this issue by providing a proper characterization of forecasting errors in the optimization process. This paper proposes a multivariate model of forecasting data for wind generation. Predictive uncertainty intervals of wind power can be obtained by sampling from the proposed model. The main goal is to use empirical data models without linear or Gaussian approximations of the distributional or temporal variations. The predictive modeling is utilized within a case study of an energy storage system. Finally, a modified robust convex programming is used to maintain the practical robustness and feasibility of the solution based on the sampled scenarios from the model.

17 WIND ENERGY↗

Evaluation of the Need for and Design of a City Energy Operations System

There is growing interest in city and community scale energy efficiency goals across the US and around the world [1]. Forward thinking developers are expressing interest in developing methods to track the actual operations of new urban developments and campuses. Local governments and developers are exploring how to ensure that the built infrastructure, including buildings, transportation, energy generation and storage assets, is not only designed for energy efficiency, but operates efficiently as well. There is a need for city and district scale tools to support energy analysis from design through operations. Smart city data analytics is a growing area of interest in the smart city technology space. However, while there is enthusiasm for the concept of smart city energy analytics, there is a lack of common semantic data models, interoperable systems, and methods to collect data and support decision-making for operating multi-building city- and district-scale energy systems. Urban developers are asking broad questions about how such a system might be designed, built, valued, maintained, and optimized. This paper describes ongoing research to explore and define a City or District Energy Operations System (CityEOS).

Piette, Mary Ann↗

Data Intensive Computing on Amazon Web Services

The Geophysical Monitoring Program (GMP) has spent the past few years building up the capability to perform data intensive computing using what have been referred to as “big data” tools. These big data tools would be used against massive archives of seismic signals (>300 TB) to conduct research not previously possible. Examples of such tools include Hadoop (HDFS, MapReduce), HBase, Hive, Storm, Spark, Solr, and many more by the day. These tools are useful for performing data analytics on datasets that exceed the resources of traditional analytic approaches. To this end, a research big data cluster (“Cluster A”) was set up as a collaboration between GMP and Livermore Computing (LC).

97 MATHEMATICS AND COMPUTING↗

Air Traffic Management TestBed Simulation Architect: User's Guide

The Air Traffic Management (ATM) TestBed is a Platform as a Service that is being developed by the National Aeronautics and Space Administration (NASA) to help design, configure, integrate, run, and monitor air traffic simulations. The platform provides cloud services including back-end big-data analytics tools, on-demand computing resource management, data storage, and communication middleware. The ATM TestBed reduces the time to test concepts and technologies, supports interactions among various concepts such as human-in-the-loop and automation-in-the-loop simulations, and enables collaborative simulations by sharing technologies and tools in the ATM community. The Simulation Architect application provides a graphical user interface tool for designing traffic scenarios and simulations using blocks representing components and links representing message channels linking them. This guide describes a high-level user interface design of Simulation Architect and provides information for a new user to compose traffic scenarios and simulations.

Software User Guide↗

Air Traffic Management TestBed Traffic Viewer: Developer's Guide

The Air Traffic Management (ATM) TestBed is a Platform as a Service that is being developed by the National Aeronautics and Space Administration (NASA) to help design, configure, integrate, run, and monitor air traffic simulations. The platform is designed to provide cloud services including back-end, big-data analytics tools, on-demand computing resource management, data storage, and communication middleware. The ATM TestBed reduces the time to test concepts and technologies, supports interactions among various methods such as human-in-the-loop and automation-in-the-loop simulations, and enables collaborative simulations by sharing technologies and tools in the ATM community. The Traffic Viewer application provides a graphical user interface tool for visualizing real and simulated air traffic as well as airspace definition in two-dimensional space. This guide describes a high-level design and implementation of Traffic Viewer and provides information for a new developer or a user to add new capabilities by following the software design and leveraging existing capabilities.

Lai, Chok Fung↗

Air Traffic Management TestBed Data Exchange Model

The Air Traffic Management (ATM) TestBed is a Platform as a Service that is being developed by the National Aeronautics and Space Administration (NASA) to help design, configure, integrate, run, and monitor air traffic simulations. The platform is designed to provide cloud services including back-end, big-data analytics tools, on-demand computing resource management, data storage, and communication middleware. The ATM TestBed reduces the time to test concepts and technologies, supports interactions among various methods such as human-in-the-loop and automation-in-the-loop simulations, and enables collaborative simulations by sharing technologies and tools in the ATM community. In order to allow easier access to simulation components, TestBed provides a messaging support layer for connectivity using a consistent set of input/output interfaces. In addition, a standard data format is introduced to facilitate communication between the components. The data exchange model, supported in the messaging support layer, standardizes the format of the information to be exchanged among the components. This document describes the messaging data model currently developed in TestBed and provides data dictionaries for references to component developers as well as simulation engineers.

air traffic simulation↗

Web-based Visualization and Analytics of Petascale Data: Equity as a Tide that Lifts All Boats

Scientists generate petabytes of data daily to help uncover environmental trends or behaviors that are hard to predict. For example, understanding climate simulations based on the long-term average of temperature, precipitation, and other environmental variables is essential to predicting and establishing root causes of future undesirable scenarios and assessing possible mitigation strategies. While supercomputer centers provide a powerful infrastructure for generating petabytes of simulation output, accessing and analyzing these datasets interactively remains challenging on multiple fronts. This paper presents an approach to managing, visualizing, and analyzing petabytes of data within a browser on equipment ranging from the top NASA supercomputer to commodity hardware like a laptop. Our novel data fabric abstraction layer allows user-friendly querying of scientific information while hid-ing the complexities of dealing with file systems or cloud services.We also optimize network utilization while streaming from petas-cale repositories through state-of-the-art progressive compression algorithms. Based on this abstraction, we provide customizable dashboards that can be accessed from any device with any inter-net connection, enabling interactive visual analysis of vast amounts of data to a wide range of users - from top scientists with access to leadership-class computing environments to undergraduate students of disadvantaged backgrounds from minority-serving institutions. We focus on NASA’s use of petascale climate datasets as an example of particular societal impact and, therefore, a case where achieving equity in science participation is critical. We validate our approach by improving the ability of climate scientists to visually explore their data via two fully interactive dashboards. We further validate our approach by deploying the dashboards and simplified training materials in the classroom at a minority-serving institution.These dashboards, released in simplified form to the general public, contribute significantly to a broader push to democratize the access and use of climate data.

Data visualization↗

Visualizing the NIOSH Pocket Guide: Open-source web application for accessing and exploring the NIOSH Pocket Guide to Chemical Hazards

The NIOSH Pocket Guide to Chemical Hazards is a trusted resource that displays key information for a collection of chemicals commonly encountered in the workplace. Entries contain chemical structures—occupational exposure limit information ranging from limits based on full-shift time-weighted averages to acute limits such as short-term exposure limits and immediately dangerous to life or health values, as well as a variety of other data such as chemical-physical properties and symptoms of exposure. The NIOSH Pocket Guide (NPG) is available as a printed, hardcopy book, a PDF version, an electronic database, and a downloadable application for mobile phones. All formats of the NIOSH Pocket Guide allow users to access the data for each chemical separately, however, the guide does not support data analytics or visualization across chemicals. This project reformatted existing data in the NPG to make it searchable and compatible with exploration and analysis using a web application. The resulting application allows users to investigate the relationships between occupational exposure limits, the range and distribution of occupational exposure limits, and the specialized sorting of chemicals by health endpoint or to summarize information of particular interest. These tasks would have previously required manual extraction of the data and analysis. The usability of this application was evaluated among industrial hygienists and researchers and while the existing application seems most relevant to researchers, the open-source code and data are amenable to modification by users to increase customization.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

SAAF: SANS data Analysis using Analytical Functions

The recently completed Extended Q-Range Small Angle Scattering Diffractometer (EQ-SANS) has put the focus on its software needs with renewed urgency. In a series of efforts, we aim at providing a complete set of software solutions on the EQ-SANS instrument. These programs include initial data processing, data correction and reduction, analytical model fitting to the scattering data, Monte Carlo simulation for structure determination, and virtual instrument simulation for experiment planning. SAAF is one such program for analytical data modeling. It takes the reduced EQ-SANS data and allows users to fit the data to analytical models. These models are easy to write. They can either be user written, or from the pre-supplied model library.

Data Analysis↗

NASA’s Advance Information Systems Technology (AIST) Program

NASA’s Earth Science Technology Office (ESTO) develops early-stage technologies to enhance scientific understanding. ESTO’s technologies can be reliably and confidently applied to a broad range of science measurements and missions, as well as facilitate practical applications to benefit society at large. As the lead technology office within the Earth Science Division of the NASA Science Mission Directorate, ESTO is focused on the technological challenges inherent to space-based investigations of our planet's dynamic, interrelated systems. Through flexible, science-driven technology strategies and a competitive selection process, ESTO-funded technologies have supported numerous Earth science missions as well as commercial applications. From next generation sensors and instruments to communication and information systems, ESTO technologies enable many NASA missions and data services. The Advanced Information Systems Technology (AIST) Program is one focal area of ESTO. AIST innovates information system technologies that enable the development of new observing systems as well as agile science investigations through data analytics and artificial intelligence tools and algorithms. AIST utilizes an end-to-end development approach with the goal of infusing mature technologies into future missions, measurements, and analysis. One thrust of AIST is data exploitation and analysis using an analytic center framework to make observational data and model output more accessible and usable to scientists conducting specific investigations as well as to extract higher-level science content and information from the data. The framework accelerates scientific discovery by harmonizing the data, tools, and computational resources. NASA’s upcoming Surface, Biology and Geology (SBG) mission is one such mission whose data architecture will leverage many of the technologies developed through the AIST program. This talk will highlight current and future program development goals.

Laura Rogers↗

A Magnetoencephalographic/Encephalographic (MEG/EEG) Brain-Computer Interface Driver for Interactive iOS Mobile Videogame Applications Utilizing the Hadoop Ecosystem, MongoDB, and Cassandra NoSQL Databases

In Phase I, we collected data on five subjects yielding over 90% positive performance in Magnetoencephalographic (MEG) mid-and post-movement activity. In addition, a driver was developed that substituted the actions of the Brain Computer Interface (BCI) as mouse button presses for real-time use in visual simulations. The process was interfaced to a flight visualization demonstration utilizing left or right brainwave thought movement, the user experiences, the aircraft turning in the chosen direction, or on iOS Mobile Warfighter Videogame application. The BCI’s data analytics of a subject’s MEG brain waves and flight visualization performance videogame analytics were stored and analyzed using the Hadoop Ecosystem as a quick retrieval data warehouse. In Phase II portion of the project involves the Emotiv Encephalographic (EEG) Wireless Brain–Computer interfaces (BCIs) allow for people to establish a novel communication channel between the human brain and a machine, in this case, an iOS Mobile Application(s). The EEG BCI utilizes advanced and novel machine learning algorithms, as well as the Spark Directed Acyclic Graph (DAG), Cassandra NoSQL database environment, and also the competitor NoSQL MongoDB database for housing BCI analytics of subject’s response and users’ intent illustrated for both MEG/EEG brainwave signal acquisition. The wireless EEG signals that were acquired from the OpenVibe and the Emotiv EPOC headset can be connected via Bluetooth to an iPhone utilizing a thin Client architecture. The use of NoSQL databases were chosen because of its schema-less architecture and Map Reduce computational paradigm algorithm for housing a user’s brain signals from each referencing sensor. Thus, in the near future, if multiple users are playing on an online network connection and an MEG/EEG sensor fails, or if the connection is lost from the smartphone and the webserver due to low battery power or failed data transmission, it will not nullify the NoSQL document-oriented (MongoDB) or column-oriented Cassandra databases. Additionally, NoSQL databases have fast querying and indexing methodologies, which are perfect for online game analytics and technology. In Phase II, we collected data on five MEG subjects, yielding over 90% positive performance on iOS Mobile Applications with Objective-C and C++, however on EEG signals utilized on three subjects with the Emotiv wireless headsets and (n < 10) subjects from the OpenVibe EEG database the Variational Bayesian Factor Analysis Algorithm (VBFA) yielded below 60% performance and we are currently pursuing extending the VBFA algorithm to work in the time-frequency domain referred to as VBFA-TF to enhance EEG performance in the near future. The novel usage of NoSQL databases, Cassandra and MongoDB, were the primary main enhancements of the BCI Phase II MEG/EEG brain signal data acquisition, queries, and rapid analytics, with MapReduce and Spark DAG demonstrating future implications for next generation biometric MEG/EEG NoSQL databases.

McClay, Wilbert↗