Search NASA⌕ Search

SEARCH · Search NASA

Results for “DATA”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 289 records · Page 16

Data-Driven Modeling and Optimization of Building Energy Consumption: a Case Study

Installing sensors and Building Automation Systems (BAS) allows controlling the facility operations while generating data that can be analyzed for model development. This work focuses on data-driven modeling of the building to optimize energy consumption. The City of Orlando aims to reduce its energy consumption so, they provided us access to their BAS for data and studying the operation of its facilities. We selected a mid-size pilot building to conduct data analysis and modeling. We develop an Application Programming Interface (API) to login to the servers and scrape data. The scraped data contains features ranging from environmental conditions to equipment activity. This dataset is a time series so, it's handled in accordance and analyzed to investigate patterns and relations between data points that help choose parameters for predictive models for building and equipment. Finally, the models are optimized to reduce the energy consumption of the facility.

Grover, Divas↗

A data variance technique for automated despiking of magnetotelluric data with a remote reference

The magnetotelluric method employs co-located surface measurements of electric and magnetic fields to infer the local electrical structure of the earth. The frequency-dependent 'apparent resistivity' curves can be inaccurate at long periods if input data are contaminated - even when robust remote reference techniques are employed. Data despiking prior to processing can result in significantly more reliable estimates of long period apparent resistivities. This paper outlines a two-step method of automatic identification and replacement for spike-like contamination of magnetotelluric data; based on the simultaneity of natural electric and magnetic field variations at distant sites. This simultaneity is exploited both to identify windows in time when the array data are compromised, and to generate synthetic data that replace observed transient noise spikes. In the first step, windows in data time series containing spikes are identified via intersite comparison of channel 'activity' - such as the variance of differenced data within each window. In the second step, plausible data for replacement of flagged windows is calculated by Wiener filtering coincident data in clean channels. The Wiener filters - which express the time-domain relationship between various array channels - are computed using an uncontaminated segment of array training data. Examples are shown where the algorithm is applied to artificially contaminated data, and to real field data. In both cases all spikes are successfully identified. In the case of implanted artificial noise, the synthetic replacement time series are very similar to the original recording. In all cases, apparent resistivity and phase curves obtained by processing the despiked data are much improved over curves obtained from raw data.

54 ENVIRONMENTAL SCIENCES↗

Early arrival waveform inversion using data uncertainties and matching filters with application to near-surface seismic refraction data

We develop an early arrival waveform inversion (EAWI) technique for high-resolution near-surface velocity estimation by iteratively updating the P-wave velocity model to minimize the difference between the observed and calculated seismic refraction data. Traditional EAWI uses a least-squares penalty function and an acoustic forward-modeling engine. Conventional least-squares error is sensitive to data with low signal-to-noise ratio (S/N) and iterations of EAWI stop at a local-minimum data misfit or at the preassigned maximum number of iterations. These stopping criteria can result in overfitting the data. In addition, fitting the elastic field data with an acoustic modeling engine can introduce artifacts in velocity estimation, especially in land data with significant elastic effects. To overcome these challenges, we develop a robust EAWI (REAWI) method by (1) incorporating the data uncertainties into the penalty function and (2) mitigating the elastic effects using a matching filter workflow. The data uncertainties are estimated from waveform reciprocal errors. When full-waveform reciprocity is not available, trace interpolation is applied. The proposed method prevents closely fitting data with low S/N, avoids overall overfitting by stopping the iterations when a normalized chi-square ([Formula: see text]) waveform misfit of one is achieved, and is less affected by elastic effects. Numerical examples and application to near-surface refraction data at a groundwater contamination site suggest that the final REAWI models are more accurate than the corresponding EAWI models, at the same level of misfit. This is the first known application of a matching filter workflow to real land data. The final REAWI models satisfy an appropriate misfit between the real data and predicted elastic P-wave data, making this approach in this respect equivalent to elastic waveform inversion. We also develop a method to analyze model constraint by examining the energy of the wavefield Fréchet derivative thereby avoiding the influence of the data residuals in traditional Fréchet kernels.

Geochemistry & Geophysics↗

ASCR Workshop on In Situ Data Management: Enabling Scientific Discovery from Diverse Data Sources

In January 2019, the U.S. Department of Energy, Office of Science program in Advanced Scientific Computing Research, convened a workshop to identify priority research directions for in situ data management (ISDM). The workshop defined ISDM as the practices, capabilities, and procedures to control the organization of data and enable the coordination and communication among heterogeneous tasks, executing simultaneously in a high-performance computing system, cooperating toward a common objective. The workshop revealed two primary, interdependent motivations for processing and managing data in situ. The first motivation is that the in situ methodology enables scientific discovery from a broad range of data sources over a wide scale of computing platforms: leadership-class systems, clusters, clouds, workstations, and embedded devices at the edge. The successful development of ISDM capabilities will benefit real-time decision-making, design optimization, and data-driven scientific discovery. The second motivation is the need to decrease data volumes. ISDM can make critical contributions to managing large data volumes from computations and experiments to minimize data movement, save storage space, and boost resource efficiency, often while simultaneously increasing scientific precision. A fundamental finding of this workshop is that the methodologies used to manage data among a variety of tasks in situ can be used to facilitate scientific discovery from many different data sources—simulation, experiment, and sensors, for example—and that being able to do so at numerous computing scales will benefit real-time decision-making, design optimization, and data-driven scientific discovery across the Office of Science mission space. Applications wanting to use the in situ capabilities include those where data analysis feeds back to the simulation, decisions are made autonomously, big data or machine learning is among the tasks to be coordinated, and computations need to be completed in real time. The workshop identified six priority research directions that highlight the components and capabilities needed for ISDM to be successful for the wide variety of applications discussed: making ISDM capabilities more pervasive, controllable, composable, and transparent, with a focus on greater coordination with the software stack and a diversity of fundamentally new data algorithms.

97 MATHEMATICS AND COMPUTING↗

MST data exchange through the NCAR incoherent-scatter radar data base

One means of making MST (mesosphere stratosphere troposphere) radar data more easily accessible for scientific research by the general scientific community is through a centralized data base. Such a data base can be designed to readily provide information on data availability and quality, and to provide copies of data from any radar in a common format to the user. The ionospheric incoherent scatter community has established a centralized data base at NCAR that may serve not only as a model for a possible MST data base, but also as a catalyst for getting an MST data base started. (Some key elements of the NCAR data base are given.) The NCAR data base can include MST data in the same framework with relatively little extra effort. They are willing to handle MST data on a limited basis in order to permit assessment of community interest and in order to provide some experience with a centralized data base for MST data.

Richmond, A. D.↗

Marine Magnetic Data Holdings of World Data Center-a for Marine Geology and Geophysics

The World Data Center-A for Marine Geology and Geophysics is co-located with the Marine Geology & Geophysical Data Center, Boulder, CO. Fifteen million digital marine magnetic trackline measurements are managed within the GEOphysical DAta System (GEODAS). The bulk of these data were collected with proton precision magnetometers under Transit Satellite navigational control. Along-track sampling averages about 1 sample per kilometer, while spatial density, a function of ship's track and survey pattern, range from 4 to 0.02 data points/sq. km. In the near future, the entire geophysical data set will be available on CD-ROM. The Marine Geology and Geophysics Division (World Data Center-A for MGG), of the National Geophysical Data Center, handles a broad spectrum of marine geophysical data, including measurements of bathymetry, magnetics, gravity, seismic reflection subbottom profiles, and side-scan images acquired by ships throughout the world's oceans. Digital data encompass the first three, while the latter two are in analog form, recorded on 35mm microfilm. The marine geophysical digital trackline data are contained in the GEODAS data base which includes 11.6 million nautical miles of cruise trackline coverage contributed by more than 70 organizations worldwide. The inventory includes data from 3206 cruises with 33 million digital records and indexing to 5.3 million track miles of analog data on microfilm.

Sharman, George F.↗

Real Data and Rapid Results: Ocean Color Data Analysis with Giovanni (GES DISC Interactive Online Visualization and ANalysis Infrastructure)

The NASA Goddard Earth Sciences Data and Information Services Center (GES DISC) has taken a major step addressing the challenge of using archived Earth Observing System (EOS) data for regional or global studies by developing an infrastructure with a World Wide Web interface which allows online, interactive, data analysis: the GES DISC Interactive Online Visualization and ANalysis Infrastructure, or "Giovanni." Giovanni provides a data analysis environment that is largely independent of underlying data file format. The Ocean Color Time-Series Project has created an initial implementation of Giovanni using monthly Standard Mapped Image (SMI) data products from the Sea-viewing Wide Field-of-view Sensor (SeaWiFS) mission. Giovanni users select geophysical parameters, and the geographical region and time period of interest. The system rapidly generates a graphical or ASCII numerical data output. Currently available output options are: Area plot (averaged or accumulated over any available data period for any rectangular area); Time plot (time series averaged over any rectangular area); Hovmeller plots (image view of any longitude-time and latitude-time cross sections); ASCII output for all plot types; and area plot animations. Future plans include correlation plots, output formats compatible with Geographical Information Systems (GIs), and higher temporal resolution data. The Ocean Color Time-Series Project will produce sensor-independent ocean color data beginning with the Coastal Zone Color Scanner (CZCS) mission and extending through SeaWiFS and Moderate Resolution Imaging Spectroradiometer (MODIS) data sets, and will enable incorporation of Visible/lnfrared Imaging Radiometer Suite (VIIRS) data, which will be added to Giovanni. The first phase of Giovanni will also include tutorials demonstrating the use of Giovanni and collaborative assistance in the development of research projects using the SeaWiFS and Ocean Color Time-Series Project data in the online Laboratory for Ocean Color Users (LOCUS). The synergy of Giovanni with high-quality ocean color data provides users with the ability to investigate a variety of important oceanic phenomena, such as coastal primary productivity related to pelagic fisheries, seasonal patterns and interannual variability, interdependence of atmospheric dust aerosols and harmful algal blooms, and the potential effects of climate change on oceanic productivity.

Acker, J. G.↗

Combining Satellite and in Situ Data with Models to Support Climate Data Records in Ocean Biology

The satellite ocean color data record spans multiple decades and, like most long-term satellite observations of the Earth, comes from many sensors. Unfortunately, global and regional chlorophyll estimates from the overlapping missions show substantial biases, limiting their use in combination to construct consistent data records. SeaWiFS and MODIS-Aqua differed by 13% globally in overlapping time segments, 2003-2007. For perspective, the maximum change in annual means over the entire Sea WiFS mission era was about 3%, and this included an El NinoLa Nina transition. These discrepancies lead to different estimates of trends depending upon whether one uses SeaWiFS alone for the 1998-2007 (no significant change), or whether MODIS is substituted for the 2003-2007 period (18% decline, P less than 0.05). Understanding the effects of climate change on the global oceans is difficult if different satellite data sets cannot be brought into conformity. The differences arise from two causes: 1) different sensors see chlorophyll differently, and 2) different sensors see different chlorophyll. In the first case, differences in sensor band locations, bandwidths, sensitivity, and time of observation lead to different estimates of chlorophyll even from the same location and day. In the second, differences in orbit and sensitivities to aerosols lead to sampling differences. A new approach to ocean color using in situ data from the public archives forces different satellite data to agree to within interannual variability. The global difference between Sea WiFS and MODIS is 0.6% for 2003-2007 using this approach. It also produces a trend using the combination of SeaWiFS and MODIS that agrees with SeaWiFS alone for 1998-2007. This is a major step to reducing errors produced by the first cause, sensor-related discrepancies. For differences that arise from sampling, data assimilation is applied. The underlying geographically complete fields derived from a free-running model is unaffected by solar zenith angle requirements and obscuration from clouds and aerosols. Combined with in situ dataenhanced satellite data, the model is forced into consistency using data assimilation. This approach eliminates sampling discrepancies from satellites. Combining the reduced differences of satellite data sets using in situ data, and the removal of sampling biases using data assimilation, we generate consistent data records of ocean color. These data records can support investigations of long-term effects of climate change on ocean biology over multiple satellites, and can improve the consistency of future satellite data sets.

Gregg, Watson↗

Usage of Data-Encoded Web Maps with Client Side Color Rendering for Combined Data Access, Visualization and Modeling Purposes

Current approaches to satellite observation data storage and distribution implement separate visualization and data access methodologies which often leads to the need in time consuming data ordering and coding for applications requiring both visual representation as well as data handling and modeling capabilities. We describe an approach we implemented for a data-encoded web map service based on storing numerical data within server map tiles and subsequent client side data manipulation and map color rendering. The approach relies on storing data using the lossless compression Portable Network Graphics (PNG) image data format which is natively supported by web-browsers allowing on-the-fly browser rendering and modification of the map tiles. The method is easy to implement using existing software libraries and has the advantage of easy client side map color modifications, as well as spatial subsetting with physical parameter range filtering. This method is demonstrated for the ASTER-GDEM elevation model and selected MODIS data products and represents an alternative to the currently used storage and data access methods. One additional benefit includes providing multiple levels of averaging due to the need in generating map tiles at varying resolutions for various map magnification levels. We suggest that such merged data and mapping approach may be a viable alternative to existing static storage and data access methods for a wide array of combined simulation, data access and visualization purposes.

Pliutau, Denis↗

Constructing the 'Best' Reliability Data for the Job - Developing Generic Reliability Data from Alternative Sources Early in a Product's Development Phase

Reliability practitioners advocate getting reliability involved early in a product development process. However, when assigned to estimate or assess the (potential) reliability of a product or system early in the design and development phase, they are faced with lack of reasonable models or methods for useful reliability estimation. Developing specific data is costly and time consuming. Instead, analysts rely on available data to assess reliability. Finding data relevant to the specific use and environment for any project is difficult, if not impossible. Instead, analysts attempt to develop the "best" or composite analog data to support the assessments. Industries, consortia and vendors across many areas have spent decades collecting, analyzing and tabulating fielded item and component reliability performance in terms of observed failures and operational use. This data resource provides a huge compendium of information for potential use, but can also be compartmented by industry, difficult to find out about, access, or manipulate. One method used incorporates processes for reviewing these existing data sources and identifying the available information based on similar equipment, then using that generic data to derive an analog composite. Dissimilarities in equipment descriptions, environment of intended use, quality and even failure modes impact the "best" data incorporated in an analog composite. Once developed, this composite analog data provides a "better" representation of the reliability of the equipment or component. It can be used to support early risk or reliability trade studies, or analytical models to establish the predicted reliability data points. It also establishes a baseline prior that may updated based on test data or observed operational constraints and failures, i.e., using Bayesian techniques. This tutorial presents a descriptive compilation of historical data sources across numerous industries and disciplines, along with examples of contents and data characteristics. It then presents methods for combining failure information from different sources and mathematical use of this data in early reliability estimation and analyses.

Kleinhammer, Roger K.↗

Using Big Data Technologies with Earth Science Data in HDF5: HDF5 Scalable Solutions

HDF5 (Hierarchical Data Format 5) is open-source, high-performance software that consists of an abstract data model, library, and fileformat used for storing and managing extremely large and/or complex data collections. NASA Earth Observing System (EOS) Data and Information Systems use HDF5 as an archival format to store remote sensing data from EOS satellites. HDF5 is also used to store other types of Geoscience and Strophysical data, e.g., seismic data and data from Low-Frequency Array (LOFAR) radio telescopes. Data stored in HDF5 has reached tens of petabytes and is growing at an accelerated rate.With the growing amout of HDF5 Earth Science data to analyze and process, scientists need to adopt big data technologies including new storage paradigms such as cloud and object storage. To run models and perform data analysis they also need to utilizied efficient and diverse ways to access data, from high-performance computing's (HPC) Message Passing Interface (MPI) I/O and deep memory hierarchies (DMH) to non-HPC frameworks such as Apache Hadoop, Spark, and Drill. The HDF Group continually works to enable usage of big data technologies in HDF software.

Knox, Larry↗

Medical Data Architecture Platform and Recommended Requirements for a Medical Data System for Exploration Missions

The Medical Data Architecture (MDA) project supports the Exploration Medical Capability (ExMC) risk to minimize or reduce the risk of adverse health outcomes and decrements in performance due to in-flight medical capabilities on human exploration missions. To mitigate this risk, the ExMC MDA project addresses the technical limitations identified in ExMC Gap Med 07: We do not have the capability to comprehensively process medically- relevant information to support medical operations during exploration missions. This gap identifies that the current in-flight medical data management includes a combination of data collection and distribution methods that are minimally integrated with on-board medical devices and systems. Furthermore, there are a variety of data sources and methods of data collection. For an exploration mission, the seamless management of such data will enable a more medically autonomous crew than the current paradigm of medical data management on the International Space Station. ExMC has recognized that in order to make informed decisions about a medical data architecture framework, current methods for medical data management must not only be understood, but an architecture must also be identified that provides the crew with actionable insight to medical conditions. This medical data architecture will provide the necessary functionality to address the challenges of executing a self-contained medical system that approaches crew health care delivery without assistance from ground support. Hence, the products derived from the third MDA prototype development will directly inform exploration medical system requirements for Level of Care IV in Gateway missions. In fiscal year 2019, the MDA project developed Test Bed 3, the third iteration in a series of prototypes, that featured integrations with cognition tool data, ultrasound image analytics and core Flight Software (cFS). Maintaining a layered architecture design, the framework implemented a plug-in, modular approach in the integration of these external data sources. An early version of MDA Test Bed 3 software was deployed and operated in a simulated analog environment that was part of the Next Space Technologies for Exploration Partnerships (NextSTEP) Gateway tests of multiple habitat prototypes. In addition, the MDA team participated in the Gateway Test and Verification Demonstration, where the MDA cFS applications was integrated with Gateway-in-a-Box software to send and receive medically relevant data over a simulated vehicle network. This software demonstration was given to ExMC and Gateway Program stakeholders at the NASA Johnson Space Center Integrated Power, Avionics and Software (iPAS) facility. Also, the integrated prototypes served as a vehicle to provide Level 5 requirements for the Crew Health and Performance Habitat Data System for Gateway Missions (Medical Level of Care IV). In the upcoming fiscal year, the MDA project will continue to provide systems engineering and vertical prototypes to refine requirements for medical Level of Care IV and inform requirements for Level of Care V.

Krihak, M.↗

Minimally buffered data transfers between nodes in a data communications network

Methods, apparatus, and products for minimally buffered data transfers between nodes in a data communications network are disclosed that include: receiving, by a messaging module on an origin node, a storage identifier, a origin data type, and a target data type, the storage identifier specifying application storage containing data, the origin data type describing a data subset contained in the origin application storage, the target data type describing an arrangement of the data subset in application storage on a target node; creating, by the messaging module, origin metadata describing the origin data type; selecting, by the messaging module from the origin application storage in dependence upon the origin metadata and the storage identifier, the data subset; and transmitting, by the messaging module to the target node, the selected data subset for storing in the target application storage in dependence upon the target data type without temporarily buffering the data subset.

97 MATHEMATICS AND COMPUTING↗

SciDAC-Data, A Project to Enabling Data Driven Modeling of Exascale Computing

The SciDAC-Data project is a DOE funded initiative to analyze and exploit two decades of information and analytics that have been collected by the Fermilab Data Center on the organization, movement, and consumption of High Energy Physics data. The project will analyze the analysis patterns and data organization that have been used by the NOvA, MicroBooNE, MINERvA and other experiments, to develop realistic models of HEP analysis workflows and data processing. The SciDAC-Data project aims to provide both realistic input vectors and corresponding output data that can be used to optimize and validate simulations of HEP analysis. These simulations are designed to address questions of data handling, cache optimization and workflow structures that are the prerequisites for modern HEP analysis chains to be mapped and optimized to run on the next generation of leadership class exascale computing facilities. We will address the use of the SciDAC-Data distributions acquired from Fermilab Data Center’s analysis workflows and corresponding to around 71,000 HEP jobs, as the input to detailed queuing simulations that model the expected data consumption and caching behaviors of the work running in HPC environments. In particular we describe in detail how the Sequential Access via Metadata (SAM) data handling system in combination with the dCache/Enstore based data archive facilities have been analyzed to develop the radically different models of the analysis of HEP data. We present how the simulation may be used to analyze the impact of design choices in archive facilities.

Mubarak, M.↗

Enriching the physics program of the CMS experiment via data scouting and data parking

Specialized data-taking and data-processing techniques were introduced by the CMS experiment in Run 1 of the CERN LHC to enhance the sensitivity of searches for new physics and the precision of standard model measurements. These techniques, termed data scouting and data parking, extend the data-taking capabilities of CMS beyond the original design specifications. The novel data-scouting strategy trades complete event information for higher event rates, while keeping the data bandwidth within limits. Data parking involves storing a large amount of raw detector data collected by algorithms with low trigger thresholds to be processed when sufficient computational power is available to handle such data. The research program of the CMS Collaboration is greatly expanded with these techniques. The implementation, performance, and physics results obtained with data scouting and data parking in CMS over the last decade are discussed in this Report, along with new developments aimed at further improving low-mass physics sensitivity over the next years of data taking.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

Tightly integrated single- and multi-crystal data collection strategy calculation and parallelized data processing in JBluIce beamline control system

The calculation of single- and multi-crystal data collection strategies and a data processing pipeline have been tightly integrated into the macromolecular crystallographic data acquisition and beamline control software JBluIce . Both tasks employ wrapper scripts around existing crystallographic software. JBluIce executes scripts through a distributed resource management system to make efficient use of all available computing resources through parallel processing. The JBluIce single-crystal data collection strategy feature uses a choice of strategy programs to help users rank sample crystals and collect data. The strategy results can be conveniently exported to a data collection run. The JBluIce multi-crystal strategy feature calculates a collection strategy to optimize coverage of reciprocal space in cases where incomplete data are available from previous samples. The JBluIce data processing runs simultaneously with data collection using a choice of data reduction wrappers for integration and scaling of newly collected data, with an option for merging with pre-existing data. Data are processed separately if collected from multiple sites on a crystal or from multiple crystals, then scaled and merged. Results from all strategy and processing calculations are displayed in relevant tabs of JBluIce .

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗

Integrated Large-Scale Data Management Platform for Photovoltaic Power Conversion Equipment (PCE) Reliability Data

To meet the demand for accuracy and real-time capability of PV system degradation evaluation, massive volume data is needed to run high-fidelity and high-efficiency simulations and perform advanced data analysis. However, PV farm operators have a series of difficulties with PV inverter data, such as data collection from multiple channels, massive data storage, data management and massive data analysis. To address these challenges, we developed an integrated data management platform capable of data acquisition, processing, storage, query, and performing big data analysis utilizing AI algorithms. The platform can also achieve data correctness verification and provide an effective distributed data management solution to retrieve massive data and establish a connection to distributed computational frameworks.

data management platform↗

Making Invisible Visible: Data-Driven Seismic Inversion With Spatio-Temporally Constrained Data Augmentation

Deep learning and data-driven approaches have shown great potential in scientific domains. The promise of data-driven techniques relies on the availability of a large volume of high-quality training datasets. Due to the high cost of obtaining data through expensive physical experiments, instruments, and simulations, data augmentation techniques for scientific applications have emerged as a new direction for obtaining scientific data recently. However, existing data augmentation techniques originating from computer vision yield physically unacceptable data samples that are not helpful for the domain problems that we are interested in. In this article, we develop new data augmentation techniques based on convolutional neural networks. Specifically, our generative models leverage different physics knowledge (such as governing equations, observable perception, and physics phenomena) to improve the quality of the synthetic data. To validate the effectiveness of our data augmentation techniques, we apply them to solve a subsurface seismic full-waveform inversion using simulated CO 2 leakage data. Our interest is to invert for subsurface velocity models associated with very small CO 2 leakage. We validate the performance of our methods using comprehensive numerical tests. Here via comparison and analysis, we show that data-driven seismic imaging can be significantly enhanced by using our data augmentation techniques. Particularly, the imaging quality has been improved by 15% in test scenarios of general-sized leakage and 17% in small-sized leakage when using an augmented training set obtained with our techniques.

58 GEOSCIENCES↗