Search NASA⌕ Search

SEARCH · Search NASA

Results for “Data Challenge”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 541 records · Page 30

Scaling the Earth System Grid to 100Gbps Networks

The SC11 demonstration, titled Scaling the Earth System Grid to 100Gbps Networks, showed the ability to use underlying infrastructure for the movement of climate data over 100Gbps network. Climate change research is one of the critical data intensive sciences, and the amount of data is continuously growing. Climate simulation data is geographically distributed over the world, and it needs to be accessed from many sources for fast and efficient analysis and inter-comparison of simulations. We used a 100Gbps link connecting National Energy Research Scientific Computing Center (NERSC) at Lawrence Berkeley National Laboratory (LBNL), Argonne National Laboratory (ANL) and Oak Ridge National Laboratory (ORNL). In the demo, the Intergovernmental Panel on Climate Change (IPCC) Fourth Assessment Report (AR4) phase 3 of the Coupled Model Intercomparison Project (CMIP-3) dataset was staged into the memory of computing nodes at ANL and ORNL from NERSC over the 100Gbps network for analysis and visualization. In general, climate simulation data consists of relatively small and large files with irregular file size distribution in each dataset. In this demo, we addressed challenges on data management in terms of high bandwidth networks, usability of existing protocols and middleware tools, and how applications can adapt and benefit from next generation networks.

97 MATHEMATICS AND COMPUTING↗

Single-particle structure determination by X-ray free-electron lasers: Possibilities and challenges

Single-particle structure recovery without crystals or radiation damage is a revolutionary possibility offered by X-ray free-electron lasers, but it involves formidable experimental and data-analytical challenges. Many of these difficulties were encountered during the development of cryogenic electron microscopy of biological systems. Electron microscopy of biological entities has now reached a spatial resolution of about 0.3 nm, with a rapidly emerging capability to map discrete and continuous conformational changes and the energy landscapes of biomolecular machines. Nonetheless, single-particle imaging by X-ray free-electron lasers remains important for a range of applications, including the study of large “electron-opaque” objects and time-resolved examination of key biological processes at physiological temperatures. After summarizing the state of the art in the study of structure and conformations by cryogenic electron microscopy, we identify the primary opportunities and challenges facing X-ray-based single-particle approaches, and possible means for circumventing them.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Kinetics of silicide formation over a wide range of heating rates spanning six orders of magnitude

Kinetic processes involving intermediate phase formation are often assumed to follow an Arrhenius temperature dependence. This behavior is usually inferred from limited data over narrow temperature intervals, where the exponential dependence is generally fully satisfied. However, direct evidence over wide temperature intervals is experimentally challenging and data are scarce. Here, we report a study of silicide formation between a 12 nm film of palladium and 15 nm of amorphous silicon in a wide range of heating rates, spanning six orders of magnitude, from 0.1 to 10{sup 5 }K/s, or equivalently more than 300 K of variation in reaction temperature. The calorimetric traces exhibit several distinct exothermic events related to interdiffusion, nucleation of Pd{sub 2}Si, crystallization of amorphous silicon, and vertical growth of Pd{sub 2}Si. Interestingly, the thickness of the initial nucleation layer depends on the heating rate revealing enhanced mass diffusion at the fastest heating rates during the initial stages of the reaction. In spite of this, the formation of the silicide strictly follows an Arrhenius temperature dependence over the whole temperature interval explored. A kinetic model is used to fit the calorimetric data over the complete heating rate range. Calorimetry is complemented by structural analysis through transmission electron microscopy and both standard and in-situ synchrotron X-ray diffraction.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗

An Intelligent Adaptable Monitoring Package. Final Report

The “Intelligent Adaptable Monitoring Package” project was a four-year effort that demonstrated the feasibility of integrated sensing packages at tidal and wave energy sites. Such integration is generally required by the breadth of sensors required to understand environmental effects at marine energy sites and the operational difficulty of deploying, maintaining, and recovering such sensors. Over the course of the project, the Adaptable Monitoring Package (AMP) was deployed in multiple settings, each corresponding to a project budget period: - Budget Period 1: Demonstration of cabled deployment at Pacific Northwest National Laboratory’s Marine Science Laboratory. The deployment highlighted AMP hardware endurance over a 4-month deployment in a tidally-dominated environment and laid the groundwork for machine learning algorithms to detect and classify targets present in active sonar data. - Budget Period 2: Demonstration of an autonomous deployment at PacWave South off the coast of Newport, Oregon. The deployment highlighted the stability of AMP hardware and software, with the autonomous package collecting data on a duty cycle over a 1.5-month deployment. - Budget Period 3: Demonstration of an autonomous deployment powered by a wave energy converter at the U.S. Navy’s Wave Energy Test Site. The deployment highlighted the potential of wave energy to power ocean observatories and led to the development of machine learning algorithms to detect and classify targets in optical camera data. In aggregate, this project’s greatest success was demonstrating the AMP’s flexibility in a range of deployment scenarios. Each budget period represented a “first of a kind” demonstration of integrated instrumentation – cabled AMP, autonomous AMP, wave-energy powered AMP – and each deployment helped to identify and set goals for the next. Further, despite the exploratory nature of these deployments, each one achieved high system up-time and proved that flexible integration of multiple sensors in a single package represents a viable strategy for marine energy environmental monitoring. The key lessons learned from the project are: - Without continuous power, either from a shore cable or in situ source, many of the benefits of integration are lost (If continuous power is not available, the ability to detect rare events is lost, as is the ability to minimize the risk of behavioral changes through adaptive sensing. However, even on a duty cycle, there is still value in being able to acquire synchronous data from multiple sensors.); and - Observations from a moving platform present substantially greater data processing challenges than those from stationary platforms. Finally, these deployments also demonstrate an important truth: successful integration alone does not guarantee that relevant data are collected. To grow the knowledge base about environmental interactions with marine energy converters, integrated systems, like the AMP, need to include the right sensor mix and connect the data pipelines to effective processing algorithms. These deployments establish a strong foundation for future collaborations with the environmental research community: not only to understand the environmental effects of marine energy, but also to improve our general ability to study life in the sea.

16 TIDAL AND WAVE POWER↗

Visualization of the electrostatic potential distribution in both polar ionospheres using multiple satellite measurements

During the time from December 1991 through March 1992, there were four operational DMSP satellites in polar orbit. All four satellites carried the Special Sensor-Ions, Electrons, Scintillation (SSIES) plasma package which included an ion drift meter. Data from the drift meter, combined with the magnetic field data, allowed the calculation of the electrostatic potential in the ionosphere along the satellite's path. Simultaneous polar coverage by four satellites was unprecedented, providing researchers with almost continuous monitoring of the potential distribution in both hemispheres for the four month period. Combining the magnitude and location of the potential data from each of the four satellites in order to examine the varying potential distribution pattern in both hemispheres presented a major challenge in data visualization. The problem was solved by developing a three-dimensional presentation of the data where the potentials are color coded and represented by the vertical dimension. This paper presents examples from a computer animation of several days of data demonstrating evolution of the size and shape of the potential distribution, along with how these changes correspond to variations in other geophysical parameters, such as the IMF orientation and the K(sub p) index.

Hairston, Marc R.↗

Roundtable on Producing and Managing Large Scientific Data with Artificial Intelligence and Machine Learning (Brochure)

The US Department of Energy’s (DOE’s) twelve Basic Energy Sciences (BES) scientific user facilities provide access to the world’s most advanced research instruments, annually serving over 16,000 users with impact reported in nearly 7,000 publications and producing unprecedented quantities of scientific data. While impressive, reaching the full potential of these rapidly growing facilities will require new innovations to solve a variety of technical challenges in data acquisition, control, modeling, and analysis. Artificial intelligence and machine learning (AI/ML) have opened corresponding new avenues in optimization, efficient surrogate models, data analytics, and inverse problems. These intriguing capabilities suggest that AI/ML can greatly accelerate the quest to probe and understand fundamental phenomena across a vast range of lengths and timescales, potentially leading to transformative advances across scientific disciplines.

97 MATHEMATICS AND COMPUTING↗

Towards a data-centric research and development roadmap for large-scale science user facilities

© 2017 IEEE. The U.S. Department of Energy (DOE) Office of Science (SC) operates approximately four dozen large-scale science user facilities (SUFs), each of which generates a tremendous amount of scientific data from experiments, observations and computations. To better understand the data needs and challenges, DOE has run many workshops in recent years to identify and articulate data-centric challenges and opportunities at varying resolution, from facility to community scale. Building on those workshop reports, as well as others from elsewhere in the community, this article goes beyond the findings-recommendations typical of workshop reports to consider how one might structure a broad, technology- and data-centric, coordinated research effort that would realize progress towards solutions that address the well documented challenges and opportunities. We focus on identifying practical issues of strategic relevance, along with offering a view about the focal points for a coordinated research and development effort that would target meeting data-centric needs of a broad set of science users and SUFs. These focal points would, by their nature, engage a spectrum of researchers from computer science, computational and experimental sciences, and data science in a coordinated fashion.

Bethel, EW↗

Benchmark Tracking System for Performance Monitoring

Benchmarking is essential for high-performance software development, particularly for monitoring performance across code iterations. This project focused on enhancing the benchmarking process for Lamellar, an asynchronous runtime for High-Performance Computing (HPC) systems developed at Pacific Northwest National Laboratory. Prior to this work, benchmark results were difficult to track and compare across code versions, presenting significant challenges in identifying performance regressions and long-term trends. The primary objective was to establish a systematic, reproducible approach for measuring performance and detecting regressions following code commits. Our methodology involved three key components: standardizing benchmark outputs, implementing data versioning, and developing analysis tools. We standardized the benchmark output format to JSON Line records containing specific fields (execution time, hardware specifications, and environmental variables). To address data management challenges, we evaluated several options and eventually chose a git repository dedicated to benchmark data. We developed a suite of Python tools that processed benchmark results, enriched them with metadata, and facilitated search in the repository. The resulting system enables more efficient filtering and comparison of performance metrics across commit histories, hardware configurations, and benchmark variants through a unified query interface. Our implementation reduces computational overhead by first checking for existing results through configuration matching before initiating new benchmark runs, thereby conserving resources. The system has been validated by Lamellar developers. It organizes results by benchmark type and build configurations for efficient retrieval. Future developments include a planned Large Language Model interface for predicting benchmark performance, incorporating the criterion package for statistical analysis, which will enable automated detection of statistically significant performance changes, and integration with continuous integration pipelines. Despite these enhancements being reserved for future work, this project has successfully provided the Lamellar development team with a framework for maintaining consistent performance standards and identifying optimization opportunities across workloads and hardware environments.

97 MATHEMATICS AND COMPUTING↗

Quantifying Annual Industrial Locomotive Energy Consumption in the United States

While US Class 1 railroad locomotive rosters and annual fuel consumption are well-documented, considerably less is known regarding the overall energy consumption of operations involving industrial locomotives. To determine the energy savings potential of this rail operating sector, the objective of this research is to develop an inventory of US industrial locomotives and a baseline estimate of their annual energy consumption. Creating an industrial locomotives roster from public data is challenging given their diverse ownership by shippers or leasing companies, and operating locales largely out of public view. By cross-referencing public data on locomotive reporting marks, serial numbers, online images and aerial images, the project team confirmed the age, model and horsepower of over one thousand industrial locomotives. Estimating energy consumption is complicated by the variability in industrial locomotive types and power ratings, and extreme differences in duty cycles and utilization. Given these limitations, using quantified case study examples and adjustments to standard EPA line-haul and switching duty cycles, bounds on the magnitude of annual US industrial locomotive energy consumption were estimated.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗

DATA-DRIVEN ALGORITHMS FOR DISTRIBUTION SYSTEM OPERATION AND CONTROL

Transactive energy systems are an alternative solution for the problems that distribution system operators face due to an increase in the use of distributed energy resources(DERs) and resulting much required scalability in managing active distribution system. To meet the reliability, sustainability and security requirement, ecient management and control of DERs including distributed generators, energy storage systems, and demand response is necessary. DERs can provide services such as reactive power compensation, voltage control, energy balancing, peak load reduction. These edge resources can also be used for system restoration after the extreme events within regulatory bound and enabling resiliency of the electric grid. While there are numbers of related work focused on solving these problems, availability of additional data resources has not been utilized to its potential to enable the reliable, sustainable and resilient electric grid. In order to address these challenges, data-driven algorithms have been proposed in this dissertation, including a) Volt/Var control, b) activity aware demand response and c) estimation of behind the meter distributed energy resources. Results indicate satisfactory performance of the developed algorithms.

ZHANG, YUE↗

Uncertainty Visualization Challenges in Decision Systems with Ensemble Data & Surrogate Models

Uncertainty visualization is a key component in translating important insights from ensemble simulation data into actionable decision-making by visually conveying various aspects of uncertainty within a system. With the recent advent of fast surrogate models trained on ensemble data, we can substitute computationally expensive simulations, which allows users to interact with more aspects of data spaces than ever before. However, the use of ensemble data with surrogate models in a decision-making tool brings up new challenges for uncertainty visualization, namely how to reconcile and communicate the new and different types of uncertainties brought in by surrogates and how to utilize these new data estimates in actionable ways. In this work, we examine these issues as they relate to high-dimensional data visualization, the integration of discrete datasets and the continuous representations of those datasets, and the unique difficulties associated with systems that allow users to iterate between input and output spaces. We assess the role of uncertainty visualization in facilitating intuitive and actionable interaction with ensemble data and surrogate models, and highlight key challenges in this new frontier of computational simulation.

ensemble data↗

Manifold learning of four-dimensional scanning transmission electron microscopy

Four-dimensional scanning transmission electron microscopy (4D-STEM) of local atomic diffraction patterns is emerging as a powerful technique for probing intricate details of atomic structure and atomic electric fields. However, efficient processing and interpretation of large volumes of data remain challenging, especially for two-dimensional or light materials because the diffraction signal recorded on the pixelated arrays is weak. Here we employ data-driven manifold leaning approaches for straightforward visualization and exploration analysis of 4D-STEM datasets, distilling real-space neighboring effects on atomically resolved deflection patterns from single-layer graphene, with single dopant atoms, as recorded on a pixelated detector. These extracted patterns relate to both individual atom sites and sublattice structures, effectively discriminating single dopant anomalies via multi-mode views. We believe manifold learning analysis will accelerate physics discoveries coupled between data-rich imaging mechanisms and materials such as ferroelectric, topological spin, and van der Waals heterostructures.

47 OTHER INSTRUMENTATION↗

Effect of fluid countermeasures of varying osmolarity on cardiovascular responses to orthostatic stress

Current operational procedures for shuttle crewmembers include the ingestion of a fluid countermeasure approximately 2 hours before reentry into the earth's gravitational field. The ingestion of the fluid countermeasure is thought to restore plasma volume and improve orthostatic responses upon reentry. The present countermeasure consists of ingesting salt tablets and water to achieve an isotonic solution. It has yet to be determined whether this is the optimal drink to restore orthostatic tolerance. It is also not known whether the drink solution is effective in increasing plasma volume. The purpose here is to evaluate the effectiveness of drink solutions of different osmolarity on restoring plasma volume and orthostatic responses. A hypertonic drink solution was more effective in restoring plasma volume after dehydration than an isotonic solution. However, there were no differences in their effects on an orthostatic challenge. These data suggest that the plasma volume differences produced in this study were not sufficient to produce differences in the cardiovascular responses to an orthostatic challenge, or there are other changes that occur during space flight that are more important in determining orthostatic intolerance.

Davis, John E.↗

Discrete Data Qualification System and Method Comprising Noise Series Fault Detection

A Sensor Data Qualification (SDQ) function has been developed that allows the onboard flight computers on NASA s launch vehicles to determine the validity of sensor data to ensure that critical safety and operational decisions are not based on faulty sensor data. This SDQ function includes a novel noise series fault detection algorithm for qualification of the output data from LO2 and LH2 low-level liquid sensors. These sensors are positioned in a launch vehicle s propellant tanks in order to detect propellant depletion during a rocket engine s boost operating phase. This detection capability can prevent the catastrophic situation where the engine operates without propellant. The output from each LO2 and LH2 low-level liquid sensor is a discrete valued signal that is expected to be in either of two states, depending on whether the sensor is immersed (wet) or exposed (dry). Conventional methods for sensor data qualification, such as threshold limit checking, are not effective for this type of signal due to its discrete binary-state nature. To address this data qualification challenge, a noise computation and evaluation method, also known as a noise fault detector, was developed to detect unreasonable statistical characteristics in the discrete data stream. The method operates on a time series of discrete data observations over a moving window of data points and performs a continuous examination of the resulting observation stream to identify the presence of anomalous characteristics. If the method determines the existence of anomalous results, the data from the sensor is disqualified for use by other monitoring or control functions.

Fulton, Christopher↗

Big data analytics in smart grids: state-of-the-art, challenges, opportunities, and future directions

Big data has a potential to unlock novel groundbreaking opportunities in the power grid sector that enhances a multitude of technical, social, and economic gains. The currently untapped potential of applying the science of big data for better planning and operation of the power grid is a very challenging task and needs significant efforts all-around. As power grid technologies evolve in conjunction with measurement and communication technologies, this results in unprecedented amount of heterogeneous big data sets from diverse sources. In particular, computational complexity, data security, and operational integration of big data into utility decision frameworks are the key challenges to transform the heterogeneous large dataset into actionable outcomes. Moreover, due to the complex nature of power grids along with the need to balance power in real time, seamless integration of big data into utility operations is very critical. In this context, big data analytics combined with grid visualization can lead to better situational awareness and predictive decisions. This paper presents a comprehensive state-of-the-art review of big data analytics and its applications in power grids, and also identifies challenges and opportunities from utility, industry, and research perspectives. The paper analyzes research gaps and presents insights on future research directions to integrate big data analytics into electric utility decision framework. Detailed information for utilities looking to apply big data analytics and details insights on how utilities can enhance revenue streams and bring disruptive innovation in the industry are discussed. More importantly, general guidelines for utilities to make the right investment in the adoption of big data analytics by unveiling interdependencies among critical infrastructures and operations are also provided.

13 HYDRO ENERGY↗

Acquisition of Real-Time Operation Analytics for an Automated Serial Sectioning System

Mechanical serial sectioning is a highly repetitive technique employed in metallography for the rendering of 3D reconstructions of microstructure. While alternate techniques such as ultrasonic detection, micro-computed tomography, and focused ion beam milling have progressed much in recent years, few alternatives provide equivalent opportunities for comparatively high resolutions over significantly sized cross-sectional areas and volumes. To that end, the introduction of automated serial sectioning systems has greatly heightened repeatability and increased data collection rates while diminishing opportunity for mishandling and other user-introduced errors. Unfortunately, even among current, state-of-the-art automated serial sectioning systems, challenges in data collection have not been fully eradicated. Therefore, this paper highlights two specific advances to assist in this area; a non-contact laser triangulation method for assessment of material removal rates and a newly developed graphical user interface providing real-time monitoring of experimental progress. Furthermore, both are shown to be helpful in the rapid identification of anomalies and interruptions, while also providing comparable and less error-prone measures of removal rate over the course of these long-term, challenging, and innately destructive characterization experiments.

36 MATERIALS SCIENCE↗

Harvesting the low-hanging fruit of high energy savings -- Virtual Occupancy using Wi-Fi Data

Approximately 20% of primary energy consumed in the U.S. is attributed to HVAC use. Ideally, HVAC operation would be driven by actual building occupancy, but lack of reliable occupancy information often results in the use of conservative static schedules. This disparity is even more pronounced in a college campus, where the function of each space differs by building (classrooms, offices, libraries) and the class schedules change frequently -- every semester, day of week, and hour. While several research papers propose the use of counts of the Wi-Fi connections (e.g., phones, computers) as a proxy for occupancy, few real-world implementations exist. This paper describes the development and deployment of an open-source Wi-Fi-to-Occupancy software library in 65 buildings of a college campus, and the planned integration with the building energy management and control system at the building scale. Over a year of Wi-Fi data was gathered into distinct academic periods, including fall and spring semester, academic breaks, and summer sessions. Patterns such as students moving between classrooms, closing laptops before exams, etc., can be visualized from the data. Approximating occupancy from Wi-Fi data presents challenges which we address in this project -- for example, identifying static devices, or estimating the ratio of devices per person. Utilizing real-time occupancy data to inform optimal HVAC schedules and ventilation rates creates the potential to identify and reduce energy waste. Other potential applications include forecasting occupancy, and using Wi-Fi data to predict peak demands. Finally, the paper discusses how to easily scale these tools to other buildings.

Pritoni, Marco↗

NASA/SPoRT's GOES-R Activities in Support of Product Development, Management, and Training

The NASA Short-term Prediction Research and Transition (SPoRT) Center supports many activities within the GOES-R Proving Grounds (PG). These include the development of imagery from existing instrumentation as a proxy to future Advanced Baseline Imager (ABI) capabilities on GOES-R. The Moderate Resolution Imaging Spectroradiometer (MODIS) and the Visible/Infrared Imager/Radiometer Suite (VIIRS) instruments are used to provide a glimpse of the multi-spectral capabilities that will become the norm as the number of channels and data rate dramatically increase with GOES-R. The NOAA/NWS has plans to provide operational users with all ABI channels at the highest resolution. Data fusion of individual channels into composite red, green, and blue imagery products will assist the end user with this future wave of information. While increasing the efficiency in the operational use of ABI channels, these composites provide only qualitative information. Within the GOES-R PG, SPoRT and other partners are exploring ways to include quantitative information as part of the composite imagery. However, limitations in local hardware processing and/or data bandwidth for users of the GOES-R data stream are challenges to overcome. This presentation will discuss the creation of these composite images as well as possible solutions to address these processing challenges. In a similar manner the Geostationary Lightning Mapper (GLM) to be launched on GOES-R presents several data management challenges. The GLM is a pioneering instrument to quantify total lightning from a geostationary platform. The expected data frequency from the GLM is to be at a sub-minute interval. Users of such a data set may have little experience in handling such a rapid update of information. To assist users, SPoRT is working with the NWS to develop tools within the user fs decision support system to allow tracking and analysis of total lightning from a storm-based perspective. This presentation will discuss the challenges and progress of this tool development work. With new data and products comes the need for user Training. Within the GOES-R PG SPoRT is supporting the demonstration of these future products by providing various training materials to end users. A summary of training provided to operational users will be discussed.

Fuell, Kevin K.↗