Search NASA⌕ Search

SEARCH · Search NASA

Results for “data diversity”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 73 records · Page 4

Data Science and Urban Air Mobility: Challenges and Opportunities

Aviation is broadly a combination of aircraft, airspace and airports. The data science life cycle comprises of five steps - capture, maintain, process, analyze and communicate. The presentation introduces the legacy of conventional aviation research in the context of the data science life cycle to motivate the challenges with Urban Air Mobility, a field that is quite nascent. A summary of recent research will be presented to highlight the innovative ways to address the challenges. Examples provided will include the generation of synthetic data, encounter models from simulations, and leveraging novel and diverse data sets from traditional transportation and non-aviation sources, to analyze problems of operation in urban airspace. Finally, opportunities will be identified for further exploration, niche development and filling the gaps in the field of data science for UAM.

Urban Air Mobility↗

Data Science Challenges for Urban Air Mobility

Aviation is a combination of aircraft, airspace and airports. The data science life cycle comprises of five steps - capture, maintain, process, analyze and communicate. The presentation introduces the legacy of conventional aviation research in the context of the data science life cycle to motivate the challenges with Urban Air Mobility, a field that is quite nascent. A summary of recent research will be presented to highlight the innovative ways to address the challenges. Examples provided will include the generation of synthetic data, encounter models from simulations, and leveraging novel and diverse data sets from traditional transportation and non-aviation sources, to analyze problems of operation in urban airspace. Finally, opportunities will be identified for further exploration, niche development and filling the gaps in the field of data science for UAM.

Data Science↗

Data Science Challenges for Urban Air Mobility

Aviation is a combination of aircraft, airspace and airports. The data science life cycle comprises of five steps - capture, maintain, process, analyze and communicate. The presentation introduces the legacy of conventional aviation research in the context of the data science life cycle to motivate the challenges with Urban Air Mobility, a field that is quite nascent. A summary of recent research will be presented to highlight the innovative ways to address the challenges. Examples provided will include the generation of synthetic data, encounter models from simulations, and leveraging novel and diverse data sets from traditional transportation and non-aviation sources, to analyze problems of operation in urban airspace. Finally, opportunities will be identified for further exploration, niche development and filling the gaps in the field of data science for UAM.

Data Science↗

Upper Atmosphere Data Program (UADP)

Rapid, effective access to the increasing volume of information on stratospheric trace gases, both from measurements and model calculations, is important in advancing our knowledge of the Earth's atmosphere. The data have their origins in a variety of sources, ranging from satellite experiments and 2-D model predictions to individual balloon measurements, and are found in a variety of locations and formats. This task focuses on development and operation of an electronic data base for such trace gas data. The objectives are to provide effective access to these diverse data sets; to foster electronic data access, manipulation, and display; and to support periodic assessment and intercomparison activities.

Seals, R. K., Jr.↗

Meteorological factors in Earth-Satellite propagation

A 5-meter paraboloidal antenna operated at 28 GHz showed gain changes of 2 dB due to rain. While precise estimation of the corresponding angle of arrival changes is difficult, they appear to have been on the order of 0.02 degrees. The attenuation at 28.6 GHz inferred from radiometry agreed well with that measured simultaneously over a satellite/Earth link at the same frequency. The radiometers so calibrated have been used to add to the available site diversity data base using a 9 km baseline. An improved empirical model of site diversity gain was obtained by applying regression techniques to available published data. A brief review of the literature has led to suggestions for two experimental programs, one dealing with multifrequency radiometry and the other with the effects of the stochastic properties of precipitation on wideband data transmission.

Levis, C. A.↗

Development of city buildings dataset for urban building energy modeling

Urban building energy modeling (UBEM) is becoming a proven tool to support energy efficiency programs for buildings in cities. Development of a city-scale dataset of the existing building stock is a critical step of UBEM to automatically generate energy models of urban buildings and simulate their performance. This study introduces data needs, data standards, and data sources to develop city building datasets for UBEM. First, a literature review of data needs for UBEM was conducted. Then, the capabilities of the current data standards for city building datasets were reviewed. Moreover, the existing public data sources from several pioneer cites were studied to evaluate whether they are adequate to support UBEM. The results show that most cities have adequate public data to support UBEM; however, the data are represented in different formats without standardization, and there is a lack of common keys to make the data mapping easier. Finally, a case study is presented to integrate the diverse data sources from multiple city departments of San Francisco. The data mapping process is introduced and discussed. It is recommended to use the unique building identifiers as the common keys in the data sources to simplify the data mapping process. Furthermore, the integration methods and workflow are applied to other U.S. cities for developing the city-scale datasets of their existing building stock, including San Jose, Los Angeles, and Boston.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

Integration of transcriptomic and proteomic data identifies biological functions in cell populations from human infant lung

Systems biology uses computational approaches to integrate diverse data types to understand cell and organ behavior. Data derived from complementary technologies, for example transcriptomic and proteomic analyses, are providing new insights into development and disease. We compared mRNA and protein profiles from purified endothelial, epithelial, immune, and mesenchymal cells from normal human lung tissue. Signatures for each cell type were identified and compared at both mRNA and protein levels. Cell specific biological processes and pathways were predicted by analysis of concordant and discordant RNA-protein pairs. Cell clustering and gene set enrichment comparisons identified shared versus unique processes associated with transcriptomic and/or proteomic data. Clear cell-cell correlations between mRNA and protein data were obtained from each cell type. Approximately 40% of RNA-protein pairs were coherently expressed. While the correlation between RNA and their protein products was relatively low (approximately 0.4), cell specific signature genes involved in functional processes characteristic of each cell type, were more highly correlated with their protein products. Consistency of cell specific RNA-protein signatures indicated an essential framework for the function of each cell type. Visualization and reutilization of the protein and RNA profiles are supported by a new web application, "LungProteomics," which is incorporated into the LGEA web portal and freely accessible at (https://research.cchmc.org/pbge/lunggens/lungProteomics/human_sorted_profiles.html).

human lung, sorted cells, integrated omics, Transc↗

The ANI-1ccx and ANI-1x data sets, coupled-cluster and density functional theory properties for molecules

Abstract Maximum diversification of data is a central theme in building generalized and accurate machine learning (ML) models. In chemistry, ML has been used to develop models for predicting molecular properties, for example quantum mechanics (QM) calculated potential energy surfaces and atomic charge models. The ANI-1x and ANI-1ccx ML-based general-purpose potentials for organic molecules were developed through active learning; an automated data diversification process. Here, we describe the ANI-1x and ANI-1ccx data sets. To demonstrate data diversity, we visualize it with a dimensionality reduction scheme, and contrast against existing data sets. The ANI-1x data set contains multiple QM properties from 5 M density functional theory calculations, while the ANI-1ccx data set contains 500 k data points obtained with an accurate CCSD(T)/CBS extrapolation. Approximately 14 million CPU core-hours were expended to generate this data. Multiple QM calculated properties for the chemical elements C, H, N, and O are provided: energies, atomic forces, multipole moments, atomic charges, etc. We provide this data to the community to aid research and development of ML models for chemistry.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Object detection with deep learning for rare event search in the GADGET II TPC

In the pursuit of identifying rare two-particle events within the GADGET II Time Projection Chamber (TPC), this paper presents a comprehensive approach for leveraging Convolutional Neural Networks (CNNs) and various data processing methods. To address the inherent complexities of 3D TPC track reconstructions, the data is expressed in 2D projections and 1D quantities. This approach capitalizes on the diverse data modalities of the TPC, allowing for the efficient representation of the distinct features of the 3D events, with no loss in topology uniqueness. Additionally, it leverages the computational efficiency of 2D CNNs and benefits from the extensive availability of pre-trained models. Given the scarcity of real training data for the rare events of interest, simulated events are used to train the models to detect real events. To account for potential distribution shifts when predominantly depending on simulations, significant perturbations are embedded within the simulations. This produces a broad parameter space that works to account for potential physics parameter and detector response variations and uncertainties. These parameter-varied simulations are used to train sensitive 2D CNN object detectors. When combined with 1D histogram peak detection algorithms, this multi-modal detection framework is highly adept at identifying rare, two-particle events in data taken during experiment 21072 at the Facility for Rare Isotope Beams (FRIB), demonstrating a 100% recall for events of interest. Here, we present the methods and outcomes of our investigation and discuss the potential future applications of these techniques.

Convolutional neural network↗

CoREC: Scalable and Resilient In-memory Data Staging for In-situ Workflows

The dramatic increase in the scale of current and planned high-end HPC systems is leading new challenges, such as the growing costs of data movement and IO, and the reduced mean time between failures (MTBF) of system components. In-situ workflows, i.e., executing the entire application workflows on the HPC system, have emerged as an attractive approach to address data-related challenges by moving computations closer to the data, and staging-based frameworks have been effectively used to support in-situ workflows at scale. However, the resilience of these staging-based solutions has not been addressed, and they remain susceptible to expensive data failures. Furthermore, naive use of data resilience techniques such as n-way replication and erasure codes can impact latency and/or result in significant storage overheads. In this article, we present CoREC, a scalable and resilient in-memory data staging runtime for large-scale in-situ workflows. CoREC uses a novel hybrid approach that combines dynamic replication with erasure coding based on data access patterns. It also leverages multiple levels of replications and erasure coding to support diverse data resiliency requirements. Furthermore, the article presents optimizations for load balancing and conflict-avoiding encoding, and a low overhead, lazy data recovery scheme. We have implemented the CoREC runtime and have deployed with the DataSpaces staging service on leadership class computing machines and present an experimental evaluation in the article. Here, the experiments demonstrate that CoREC can tolerate in-memory data failures while maintaining low latency and sustaining high overall storage efficiency at large scales.

97 MATHEMATICS AND COMPUTING↗

Well Stimulation Treatment in California: Evaluation of Disclosure Data, May 2015 – October 2019

In this document, disclosure data from 1,228 well stimulations occurring May 2015 to October 2019 in California are evaluated. This evaluation updates a previous study that was based on 618 well stimulations occurring May 2015 to June 2016 (Stringfellow, W.T., Camarillo, M.K., and Jordan, P. 2017, Status of Well Stimulation in California Since Implementation of SB-4 Regulations, Berkeley National Laboratory, Berkeley, CA). While the goal of the previous study was to provide a summary of information obtained since passage of California Senate Bill No. 4—Oil and Gas, the goal of the current evaluation is to provide a basis for studying chemical indicators that could potentially be used to detect aquifer contamination. We evaluate chemical indicators is a separate document using the data contained herein as well as data from monitoring wells located within oil fields. Compared with the data used by Stringfellow et al. 2017, the current evaluation is based on a more diverse data set. While the previous study contained data for only two oil fields with more than 30 well stimulations, the current data set contains four oil fields with more than 30 well stimulations, making these data more ideal for comparisons using statistical tests. More producers and a new oil field, Buena Vista Nose, are represented in the current data set. Some well stimulation practices remain relatively unchanged since the previous study. Well stimulation is mostly occurring in Kern County with the exception of a single well stimulation in Orange County. Almost all well stimulations are hydraulic fracturing. The current data set contains two acid fracturing treatments and one matrix acidizing treatment. The median number of chemicals added per well stimulation—excluding water and proppant—was previously 21; the number in the expanded data set is 20. Median water added per well stimulation was previously 89,000 gallons and is now 98,000 gallons. While this water use represents a 10% increase, water use for hydraulic fracturing in California is still lower than water use in other oil and gas fields. Chemical formulations appear to have shifted since the previous study. This shift may be related to a change in the predominant service company. We identified 26 chemicals for which the frequency of use has changed by more than 30%. The total number of chemicals used has also expanded. Previously, 178 unique chemicals were identified as being added to well stimulation fluids. The total number of unique chemicals is now 205. Analytical data available for base and recovered fluids has expanded. The number of base fluid samples has increased from 12 to 35 and the number of recovered fluid samples has increased from 1,078 to 2,166. The expanded data set is more ideal for an evaluation of indicator chemicals and comparisons across different formations. The analytical data for recovered fluid samples indicates that many ions, radioactive constituents, and organics are consistently observed, making these chemicals ideal for consideration as potential indicators of aquifer contamination.

54 ENVIRONMENTAL SCIENCES↗

PlasmoData.jl — A Julia framework for modeling and analyzing complex data as graphs

Datasets encountered in scientific and engineering applications appear in complex formats (e.g., images, multivariate time series, molecules, video, text strings, networks). Graph theory provides a unifying framework to model such datasets and enables the use of powerful tools that can help analyze, visualize, and extract value from data. In this work, we present PlasmoData.jl, an open-source, Julia framework that uses concepts of graph theory to facilitate the modeling and analysis of complex datasets. The core of our framework is a general data modeling abstraction, which we call a DataGraph. We show how the abstraction and software implementation can be used to represent diverse data objects as graphs and to enable the use of tools from topology, graph theory, and machine learning (e.g., graph neural networks) to conduct a variety of tasks. We illustrate the versatility of the framework by using real datasets: (i) an image classification problem using topological data analysis to extract features from the graph model to train machine learning models; (ii) a disease outbreak problem where we model multivariate time series as graphs to detect abnormal events; and (iii) a technology pathway analysis problem where we highlight how we can use graphs to navigate connectivity. Further, our discussion also highlights how PlasmoData.jl leverages native Julia capabilities to enable compact syntax, scalable computations, and interfaces with diverse packages. Overall, we show that the DataGraph abstraction and PlasmoData.jl Julia package are able to model data within graphs and enable useful analysis.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗

Packaging and distributing ecological data from multisite studies

Studies of global change and other regional issues depend on ecological data collected at multiple study areas or sites. An information system model is proposed for compiling diverse data from dispersed sources so that the data are consistent, complete, and readily available. The model includes investigators who collect and analyze field measurements, science teams that synthesize data, a project information system that collates data, a data archive center that distributes data to secondary users, and a master data directory that provides broader searching opportunities. Special attention to format consistency is required, such as units of measure, spatial coordinates, dates, and notation for missing values. Often data may need to be enhanced by estimating missing values, aggregating to common temporal units, or adding other related data such as climatic and soils data. Full documentation, an efficient data distribution mechanism, and an equitable way to acknowledge the original source of data are also required.

Information Systems↗

The IGS Global Data Center at the CDDIS: An Update

The Crustal Dynamics Data Information System (CDDIS) has served as a global data center for the International GPS Service (IGS) since its start in June 1992, providing on-line access to data from nearly 300 sites on a daily basis. The CDDIS provides easy and ready access to a variety of data sets, products, and information about these data. The specialized nature of the CDDIS lends itself well to enhancement and thus can accommodate diverse data sets and user requirements. This poster paper will present information about the GPS and GLONASS data and products archive at the CDDIS. General information about the system, the computer architecture, archive contents, and future plans, and its support of other international space geodesy services (the ILRS, IVS, and IDS) will be discussed.

Noll, Carey↗

Archiving Space Geodesy Data for 20+ Years at the CDDIS

Since 1982, the Crustal Dynamics Data Information System (CDDIS) has supported the archive and distribution of geodetic data products acquired by NASA programs. These data include GPS (Global Positioning System), GLONASS (GLObal NAvigation Satellite System), SLR (Satellite Laser Ranging), VLBI (Very Long Baseline Interferometry), and DORIS (Doppler Orbitography and Radiolocation Integrated by Satellite). The data archive supports NASA's space geodesy activities through the Solid Earth and Natural Hazards (SENH) program. The CDDIS data system and its archive have become increasingly important to many national and international programs, particularly several of the operational services within the International Association of Geodesy (IAG), including the International GPS Service (IGS), the International Laser Ranging Service (ILRS), the International VLBI Service for Geodesy and Astrometry (IVS), the International DORIS Service (IDS), and the International Earth Rotation Service (IERS). The CDDIS provides easy and ready access to a variety of data sets, products, and information about these data. The specialized nature of the CDDIS lends itself well to enhancement and thus can accommodate diverse data sets and user requirements. All data sets and metadata extracted from these data sets are accessible to scientists through ftp and the web; general information about each data set is accessible via the web. The CDDIS, including background information about the system and its user communities, the computer architecture, archive contents, available metadata, and future plans will be discussed.

Noll, Carey E.↗

DataHub--Knowledge-Based Science Data Management for Exploratory Data Analysis

It is our belief that new modes of research and new tools will be required to handle the massive amount of diverse data that is to be stored, organized, accessed, distributed, visualized, and analyzed. The fundamental innovation required is the integration of three automation technologies, videlicet knowledge-based expert systems, science visualization and science data management. This integration is based on a concept caled the DataHub, which we describe here.

DataHub↗

Iterative-Transform Phase Retrieval Using Adaptive Diversity

A phase-diverse iterative-transform phase-retrieval algorithm enables high spatial-frequency, high-dynamic-range, image-based wavefront sensing. [The terms phase-diverse, phase retrieval, image-based, and wavefront sensing are defined in the first of the two immediately preceding articles, Broadband Phase Retrieval for Image-Based Wavefront Sensing (GSC-14899-1).] As described below, no prior phase-retrieval algorithm has offered both high dynamic range and the capability to recover high spatial-frequency components. Each of the previously developed image-based phase-retrieval techniques can be classified into one of two categories: iterative transform or parametric. Among the modifications of the original iterative-transform approach has been the introduction of a defocus diversity function (also defined in the cited companion article). Modifications of the original parametric approach have included minimizing alternative objective functions as well as implementing a variety of nonlinear optimization methods. The iterative-transform approach offers the advantage of ability to recover low, middle, and high spatial frequencies, but has disadvantage of having a limited dynamic range to one wavelength or less. In contrast, parametric phase retrieval offers the advantage of high dynamic range, but is poorly suited for recovering higher spatial frequency aberrations. The present phase-diverse iterative transform phase-retrieval algorithm offers both the high-spatial-frequency capability of the iterative-transform approach and the high dynamic range of parametric phase-recovery techniques. In implementation, this is a focus-diverse iterative-transform phaseretrieval algorithm that incorporates an adaptive diversity function, which makes it possible to avoid phase unwrapping while preserving high-spatial-frequency recovery. The algorithm includes an inner and an outer loop (see figure). An initial estimate of phase is used to start the algorithm on the inner loop, wherein multiple intensity images are processed, each using a different defocus value. The processing is done by an iterative-transform method, yielding individual phase estimates corresponding to each image of the defocus-diversity data set. These individual phase estimates are combined in a weighted average to form a new phase estimate, which serves as the initial phase estimate for either the next iteration of the iterative-transform method or, if the maximum number of iterations has been reached, for the next several steps, which constitute the outerloop portion of the algorithm. The details of the next several steps must be omitted here for the sake of brevity. The overall effect of these steps is to adaptively update the diversity defocus values according to recovery of global defocus in the phase estimate. Aberration recovery varies with differing amounts as the amount of diversity defocus is updated in each image; thus, feedback is incorporated into the recovery process. This process is iterated until the global defocus error is driven to zero during the recovery process. The amplitude of aberration may far exceed one wavelength after completion of the inner-loop portion of the algorithm, and the classical iterative transform method does not, by itself, enable recovery of multi-wavelength aberrations. Hence, in the absence of a means of off-loading the multi-wavelength portion of the aberration, the algorithm would produce a wrapped phase map. However, a special aberration-fitting procedure can be applied to the wrapped phase data to transfer at least some portion of the multi-wavelength aberration to the diversity function, wherein the data are treated as known phase values. In this way, a multiwavelength aberration can be recovered incrementally by successively applying the aberration-fitting procedure to intermediate wrapped phase maps. During recovery, as more of the aberration is transferred to the diversity function following successive iterations around the ter loop, the estimated phase ceases to wrap in places where the aberration values become incorporated as part of the diversity function. As a result, as the aberration content is transferred to the diversity function, the phase estimate resembles that of a reference flat.

Dean, Bruce H.↗

Biology-inspired Architecture for Situation Management

Situation Management is a rapidly developing science combining new techniques for data collection with advanced methods of data fusion to facilitate the process leading to correct decisions prescribing action. Current research focuses on reducing increasing amounts of diverse data to knowledge used by decision makers and on reducing time between observations, decisions and actions. No new technology is more promising for increasing the diversity and fidelity of observations than sensor networks. However, current research on sensor networks concentrates on a centralized network architecture. We believe this trend will not realize the full potential of situation management. We propose a new architecture modeled after biological ecosystems where motes are autonomous and intelligent, yet cooperate with local neighborhoods. Providing a layered approach, they sense and act independently when possible, and cooperate with neighborhoods when necessary. The combination of their local actions results in global effects. While situation management research is currently dominated by military applications, advances envisioned for industrial and business applications have similar requirements. NASA has requirements for intelligent and autonomous systems in future missions that can benefit from advances in situation management. We describe requirements for the Integrated Vehicle Health Management program where our biology-inspired architecture provides a layered approach and decisions can be made at the proper level to improve safety, reduce costs, and improve efficiency in making diagnostic and prognostic assessments of the structural integrity, aerodynamic characteristics, and operation of aircraft.

Jones, Kennie H.↗