Search NASA⌕ Search

SEARCH · Search NASA

Results for “Data Challenge”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,045 records · Page 58

Giovanni - The Bridge Between Data and Science

This article describes new features in the Geospatial Interactive Online Visualization ANd aNalysis Infrastructure (Giovanni), a user-friendly online tool that enables visualization, analysis, and assessment of NASA Earth science data sets without downloading data and software. Since the satellite era began, data collected from Earth-observing satellites have been widely used in research and applications; however, using satellite-based data sets can still be a challenge to many. To facilitate data access and evaluation, as well as scientific exploration and discovery, the NASA Goddard Earth Sciences (GES) Data and Information Services Center (DISC) has developed Giovanni for a wide range of users around the world. This article describes the latest capabilities of Giovanni with examples, and discusses future plans for this innovative system.

GPM↗

Serving NASA GES DISC Multi-Spatiotemporal Earth Science Data to the GIS community

NASA Earth Science (ES) data is essential to a wide range of GIS research and applications. However, for many GIS users, searching, accessing, using and analyzing NASA ES data can be of a great challenge- ranging from the sheer data volumes, types of science parameters, and to the complexity of data encoding formats. As one of the twelve NASA Science Mission Directorate (SMD) Data Centers, Goddard Earth Sciences (GES) Data and Information Services Center (DISC) archives and distributes petabytes of ES parameters covering atmosphere, land, and ocean fields. Most data are multidimensional and multi-spatiotemporal in nature and are encoded in different science data formats (e.g, HDF, HDF-EOS, netCDF, GRIB, binary), which usually contain multiple variables and different metadata information. By far, GES DISC has been developing a number of services and online tools to help GIS users to easily explore our data products. In this presentation, we will describe our ArcGIS-based data accessing and visualization services and portals, which allow users directly exploring the multi-spatiotemporal ES data in ArcGIS clients without having to pre-download/import the data. The ArcGIS services are also compliant with the Open Geospatial Consortium (OGC) Web Coverage Service (WCS) and Web Map Service (WMS) protocols and can be accessed by any other WCS/WMS clients to get customized GES DISC EO data on-the-fly from such services.

Wei, Jennifer↗

An Evaluation of Remotely Sensed and In-Situ Data Sufficiency for SGMA-Scale Groundwater Studies in the Central Valley, California

California’s Central Valley aquifer is a critical freshwater resource for the state, providing drinking water to 6.5 million residents and irrigation water for more than half of the nation’s produce. Increasing demands have led to a steady decline of aquifer water levels, leading to irreversible compaction and land subsidence. The Sustainable Groundwater Management Act of California (SGMA), instituted in 2014, requires high-priority regions to establish sustainability plans by 2020-2022 that mitigate groundwater depletion and land subsidence. Many regions within the Central Valley lack groundwater monitoring wells or continuous well data, rendering in-situ monitoring challenging. When combined with other data, NASA’s Gravity Recovery and Climate Experiment (GRACE) and GRACE Follow On (GRACE-FO) missions provide valuable information about groundwater storage changes at a sub-basin scale. Interferometric Surface Aperture Radar (InSAR) measurements can map land subsidence, and Global Positioning System (GPS) can be used to estimate crustal uplift. Yet, all of these measurements have differences in resolution, coverage, discretization, and record length. Here we assess the various datasets that are potentially useful for Central Valley sub-basin groundwater monitoring and provide an analysis of gaps and other issues in regard to their suitability for SGMA-related analysis. Finally, we offer next steps and recommendations on data treatment and integration.

Kyra H Kim↗

2.2.5.401 - Ocean Observing Prize

The Powering the Blue Economy: Ocean Observing Prize is a $2.4M contest that challenges innovators to develop solutions that integrate marine renewable energy with ocean observation platforms to revolutionize our ability to monitor, manage, and understand the ocean. Competitors are challenged to design, build, and test novel, wave-powered, self-charging autonomous underwater vehicle (AUV) systems that could be suitable for a 6-month deployment in the Atlantic Ocean to monitor hurricane formation. Through engagement with the end user ocean observing community, the team identified challenges in a lack of data on storm intensity, endangering coastal communities. This challenge could be addressed by collecting data before, during, and after a hurricane develops and strengthens at sea. Amassing this data, however, requires an ocean observing system that can be deployed at sea for long periods, waiting for approaching storms. To help solve this problem, competitors are working to integrate wave energy capture with AUV systems to collect this data, helping to better protect coastal communities from oncoming storms. Solutions developed through this prize aim to enable long-term collection of data before, during, and after a hurricane that will help better forecast storm intensity. The prize mechanism provides an environment to accelerate technology development for innovators and helps to improve awareness of marine energy and its ability to provide power to blue economy applications.

coastal communities↗

Types of data: terminology and examples

Data produced at wastewater utilities are obtained through a variety of devices, including actuators, control devices, and sensors. This results in data that can be highly variable in its structure. Dealing with the resulting heterogeneity of data formats can be a challenge when storing or interpreting the data. For this reason, this chapter serves as an overview for the most important structural aspects of data typically found at a utility. The specific aims of this chapter are to:

Plana, Queralt↗

Analyzing Electric Field Morphology Through Data-Model Comparisons of the GEM IM/S Assessment Challenge Events

The storm-time inner magnetospheric electric field morphology and dynamics are assessed by comparing numerical modeling results of the plasmasphere and ring current with many in situ and remote sensing data sets. Two magnetic storms are analyzed, April 22,2001 and October 21-23,2001, which are the events selected for the Geospace Environment Modeling (GEM) Inner Magnetosphere/Storms (IM/S) Assessment Challenge (IMSAC). The IMSAC seeks to quantify the accuracy of inner magnetospheric models as well as synthesize our understanding of this region. For each storm, the ring current-atmosphere interaction model (RAM) and the dynamic global core plasma model (DGCPM) were run together with various settings for the large-scale convection electric field and the nightside ionospheric conductance. DGCPM plasmaspheric parameters were compared with IMAGE-EUV plasmapause extractions and LANL-MPA plume locations and velocities. RAM parameters were compared with Dst*, LANL-MPA fluxes and moments, IMAGE-MENA images, and IMAGE-HENA images. Both qualitative and quantitative comparisons were made to determine the electric field morphology that allows the model results to best fit the plasma data at various times during these events. The simulations with self-consistent electric fields were, in general, better than those with prescribed field choices. This indicates that the time-dependent modulation of the inner magnetospheric electric fields by the nightside ionosphere is quite significant for accurate determination of these fields (and their effects). It was determined that a shielded Volland-Stern field description driven by the 3-hour Kp index yields accurate results much of the time, but can be quite inconsistent. The modified Mcllwain field description clearly lagged in overall accuracy compared to the other fields, but matched some data sets (like Dst*) quite well. The rankings between the simulations varied depending on the storm and the individual data sets, indicating that each field description did well for some place, time, and energy range during the events, as well as doing less well in other places, times, and energies. Several unresolved issues regarding the storm-time inner magnetospheric electric field are discussed.

Liemohn, Michael W.↗

The AmeriFlux Data Activity and Data System: An Evolving Collection of Data Management Techniques, Tools, Products and Services

The Carbon Dioxide Information Analysis Center (CDIAC) at Oak Ridge National Laboratory (ORNL), USA has provided scientific data management support for the U.S. Department of Energy and international climate change science since 1982. Over this period, climate change science has expanded from research focusing on basic understanding of geochemical cycles, particularly the carbon cycle, to integrated research addressing climate change impacts, vulnerability, adaptation, and mitigation. Interests in climate change data and information worldwide have grown remarkably and, as a result, so have demands and expectations for CDIAC’s data systems. To meet the growing demands, CDIAC’s strategy has been to design flexible data systems using proven technologies blended with new, evolving technologies and standards. CDIAC development teams are multidisciplinary and include computer science and information technology expertise, but also scientific expertise necessary to address data quality and documentation issues and to identify data products and system capabilities needed by climate change scientists. CDIAC has learned there is rarely a single commercial tool or product readily available to satisfy long-term scientific data system requirements (i.e., one size does not fit all and the breadth and diversity of environmental data are often too complex for easy use with commercial products) and typically deploys a variety of tools and data products in an effort to provide credible data freely to users worldwide. Like many scientific data management applications, CDIAC’s data systems are highly customized to satisfy specific scientific usage requirements (e.g., developing data products specific for model use) but are also designed to be flexible and interoperable to take advantage of new software engineering techniques, standards (e.g., metadata standards) and tools and to support future Earth system data efforts (e.g., ocean acidification). CDIAC has provided data management support for numerous long-term measurement projects crucial to climate change science. One current example is the AmeriFlux measurement network. AmeriFlux provides continuous measurements from forests, grasslands, wetlands, and croplands in North, Central, and South America and offers important insight about carbon cycling in terrestrial ecosystems. We share our approaches in satisfying the challenges of delivering AmeriFlux data worldwide to benefit others with similar challenges handling climate change data, further heighten awareness and use of an outstanding ecological data resource, and highlight expanded software engineering applications being used for climate change measurement data.

Boden, Thomas A.↗

SONET: a semantic ontological network graph for managing points of interest data heterogeneity

Scalability, standardization, and management are important issues when working with very large Volunteered Geographic Information (VGI). VGI is a rich and valuable source of Points of Interest (POI) information, but its inherent heterogeneity in content, structure, and scale across sources present major challenges for interlinking data sources for analysis. To be useful at scale, the raw information needs to be transformed into a standardized schema that can be easily and reliably used by data analysts. In this work, we tackle the problem of unifying POI categories (e.g. restaurants, temple, and hotel) across multiple data sources to aid in improving land use maps and population distribution estimation as well as support data analysts wishing to fuse multiple data sources with the OpenStreetMap (OSM) mapping platform or working with projects that are already configured in the OSM schema and wish to add additional sources of information. Graph theory and its implementation through the SONET graph database, provides a programmatic way to organize, store, and retrieve standardized POI categories at multiple levels of abstraction. Additionally, it addresses category heterogeneity across data sources by standardizing and managing categories in a way that makes cross-domain analysis possible.

Palumbo, Rachel↗

Comparability of Liquid Chromatography Tandem Mass Spectrometry Analysis of Dissolved Organic Matter across Laboratories

Non-targeted liquid chromatography tandem highresolution mass spectrometry (LC−MS/MS) is increasingly applied for the structure-resolved chemical analysis of dissolved organic matter (DOM). With new developments in MS instrumentation and analysis software, the approach has gained substantial momentum over the past decade. However, achieving high-quality analytical data that is reproducible and comparable across laboratories can be a bottleneck in non-targeted metabolomics and organic matter chemical analysis, especially for data reuse in repository-scale analyses. Understanding the capabilities as well as challenges of comparing LC−MS/MS data from different laboratories is necessary for inferring global trends from public data sets. To illuminate instrumentation factors that drive differences and variability, we used a standardized data analysis pipeline, including classical (CMN) and featurebased molecular networking (FBMN), to analyze data from a ring trial by 24 laboratories on identical sample sets of algal and DOM extracts that were mixed in predefined concentrations and spiked with standards. Our results showed that data sets from similar mass spectrometer types with unified instrument parameters were qualitatively comparable, resolving the same general trends and shared mass spectral features. Interlaboratory comparability was best for high-intensity features, while low-intensity features showed greater detection variability. Our analysis also highlights challenges when comparing data from instruments with different acquisition rates or operating with less standardized methods. Lastly, we provide recommendations for data integration, public data sharing, standardization, and best practices for standardized LC−MS/MS data acquisition, which will be critical for long-term time series and intercomparability of DOM chemical analyses.

DOM↗

Quantifying Interannual Variability of the UTLS Ozone Using Assimilation of Satellite Data

An accurate representation of spatial and temporal variability of the Upper Troposphere Lower Stratosphere (UTLS) ozone is essential for understanding both the tropospheric ozone budget and ozone s contribution to radiative forcing. The complex, dynamically driven structure of trace gas fields in the UTLS presents a challenge to data-based and modelling studies. Small features are not fully resolved in data from limb-sounding instruments such as the Microwave Limb Sounder on EOS-Aura (the EOS-MLS), but are captured in assimilation of those data as vertical structure is added from the assimilated meteorology. This will be demonstrated using a multi-year assimilation of EOS-MLS observations in the Goddard Earth Observing System, Version 5 (GEOS-5) data assimilation system. The results demonstrate the realism of the seasonal and year to year variability of laminar structures in the mid-latitudinal ozone field between years 2005-2007, for which independent validation data are available from the HIRDLS instrument. The analysis is done in the context of the underlying large scale dynamics. The lifetimes of most research instruments are too short for them to be used throughout the duration of long-term (at least 3 decades) reanalyses. For example, the EOS-MLS instrument has operated since mid-2004 until present. By contrast, Solar Backscatter Ultra Violet (SBUV) measurements provide continuous data since late 1978, but their vertical resolution is insufficient to represent the profile shape in the UTLS. Assimilation of these SBUV/2 observations in the GEOS-5 data assimilation system has hitherto not captured a realistic ozone structure in the UTLS, even though transport studies using GEOS-5 wind fields do show such structures. We show that careful construction of the background error covariance structure in GEOS-5 can lead to more realistic UTLS ozone fields when assimilating SBUV/2 observations. The reasoning behind this will be discussed, emphasizing the need to retain the sharp gradient of ozone concentrations across the tropopause. We analyze the UTLS ozone distributions in multi-year SBUV/2 assimilation experiments, comparing the results against the independent HIRDLSdataset and, for a longer period, with the MLS assimilation and discuss the consequences for tropospheric ozone and radiative forcing.

Wargan, K.↗

LADS: Optimizing Data Transfers using Layout-Aware Data Scheduling

While future terabit networks hold the promise of signifi- cantly improving big-data motion among geographically distributed data centers, significant challenges must be overcome even on today s 100 gigabit networks to real- ize end-to-end performance. Multiple bottlenecks exist along the end-to-end path from source to sink. Data stor- age infrastructure at both the source and sink and its in- terplay with the wide-area network are increasingly the bottleneck to achieving high performance. In this paper, we identify the issues that lead to congestion on the path of an end-to-end data transfer in the terabit network en- vironment, and we present a new bulk data movement framework called LADS for terabit networks. LADS ex- ploits the underlying storage layout at each endpoint to maximize throughput without negatively impacting the performance of shared storage resources for other users. LADS also uses the Common Communication Interface (CCI) in lieu of the sockets interface to use zero-copy, OS-bypass hardware when available. It can further im- prove data transfer performance under congestion on the end systems using buffering at the source using flash storage. With our evaluations, we show that LADS can avoid congested storage elements within the shared stor- age resource, improving I/O bandwidth, and data transfer rates across the high speed networks.

Data Management↗

A genoscape–network model for conservation prioritization in a migratory bird

Migratory animals are declining worldwide and coordinated conservation efforts are needed to reverse current trends. We devised a novel genoscape-network model that combines genetic analyses with species distribution modeling and demographic data to overcome challenges with conceptualizing alternative risk factors in migratory species across their full annual cycle. We applied our method to the long distance, Neotropical migratory bird, Wilson's Warbler (Cardellina pusilla). Despite a lack of data from some wintering locations, we demonstrated how the results can be used to help prioritize conservation of breeding and wintering areas. For example, we showed that when genetic, demographic, and network modeling results were considered together it became clear that conservation recommendations will differ depending on whether the goal is to preserve unique genetic lineages or the largest number of birds per unit area. More specifically, if preservation of genetic lineages is the goal, then limited resources should be focused on preserving habitat in the California Sierra, Basin Rockies, or Coastal California, where the 3 most vulnerable genetic lineages breed, or in western Mexico, where 2 of the 3 most vulnerable lineages overwinter. Alternatively, if preservation of the largest number of individuals per unit area is the goal, then limited conservation dollars should be placed in the Pacific Northwest or Central America, where densities are estimated to be the highest. Overall, our results demonstrated the utility of adopting a genetically based network model for integrating multiple types of data across vast geographic scales and better inform conservation decision-making for migratory animals.

60 APPLIED LIFE SCIENCES↗

High-performance epistasis detection in quantitative trait GWAS

epiSNP is a program for identifying pairwise single nucleotide polymorphism (SNP) interactions (epistasis) in quantitative-trait genome-wide association studies (GWAS). A parallel MPI version (EPISNPmpi) was created in 2008 to address this computationally expensive analysis on large data sets with many quantitative traits and SNP markers. However, the falling cost of genotyping has led to an explosion of large-scale GWAS data sets that challenge EPISNPmpi’s ability to compute results in a reasonable amount of time. Therefore, we optimized epiSNP for modern multi-core and highly parallel many-core processors to efficiently handle these large data sets. This paper describes the serial optimizations, dynamic load balancing using MPI-3 RMA operations, and shared-memory parallelization with OpenMP to further enhance load balancing and allow execution on the Intel Xeon Phi coprocessor (MIC). For a large GWAS data set, our optimizations provided a 38.43× speedup over EPISNPmpi on 126 nodes using 2 MICs on TACC’s Stampede Supercomputer. We also describe a Coarray Fortran (CAF) version that demonstrates the suitability of PGAS languages for problems with this computational pattern. We show that the Coarray version performs competitively with the MPI version on the NERSC Edison Cray XC30 supercomputer. Finally, the performance benefits of hyper-threading for this application on Edison (average 1.35× speedup) are demonstrated.

97 MATHEMATICS AND COMPUTING↗

Solar Thermal Upper Stage Liquid Hydrogen Pressure Control Testing and Analytical Modeling

The demonstration of a unique liquid hydrogen (LH2) storage and feed system concept for solar thermal upper stage was cooperatively accomplished by a Boeing/NASA Marshall Space Flight Center team. The strategy was to balance thermodynamic venting with the engine thrusting timeline during a representative 30-day mission, thereby, assuring no vent losses. Using a 2 cubic m (71 cubic ft) LH2 tank, proof-of-concept testing consisted of an engineering checkout followed by a 30-day mission simulation. The data were used to anchor a combination of standard analyses and computational fluid dynamics (CFD) modeling. Dependence on orbital testing has been incrementally reduced as CFD codes, combined with standard modeling, continue to be challenged with test data such as this.

Olsen, A. D.↗

Research Activities at Fermilab for Big Data Movement

Adaptation of 100GE Networking Infrastructure is the next step towards management of Big Data. Being the US Tier-1 Center for the Large Hadron Collider's (LHC) Compact Muon Solenoid (CMS) experiment and the central data center for several other large-scale research collaborations, Fermilab has to constantly deal with the scaling and wide-area distribution challenges of the big data. In this paper, we will describe some of the challenges involved in the movement of big data over 100GE infrastructure and the research activities at Fermilab to address these challenges.

Mhashilkar, Parag↗

Improving Scientific Data Analysis Through Multi-touch Enabled Interactive Data Visualization with Applications to Neutron Science

Scientific data analysis is an inherently exploratory process that requires flexible human interaction with data visualization and anal- ysis tools. In this paper, we describe challenges in neutron scattering data analysis and present our approach for allowing more effective human interaction. In addition to multi-touch enabled data visualiza- tion techniques, we describe color scale enhancements, layouts, and visual analytics approaches for 3D volume slicing. Since challenges in neutron science are echoed in most scientific fields, we generalize the project objective and present a case for bridging the gap between the latest advances in interactive data visualization and practical scientific scenarios where new critical knowledge is buried in large volumes of complex data.

Steed, Chad↗

Software Engineering Support of the Third Round of Scientific Grand Challenge Investigations: An Earth Modeling System Software Framework Strawman Design that Integrates Cactus and UCLA/UCB Distributed Data Broker

One of the most significant challenges in large-scale climate modeling, as well as in high-performance computing in other scientific fields, is that of effectively integrating many software models from multiple contributors. A software framework facilitates the integration task. both in the development and runtime stages of the simulation. Effective software frameworks reduce the programming burden for the investigators, freeing them to focus more on the science and less on the parallel communication implementation, while maintaining high performance across numerous supercomputer and workstation architectures. This document proposes a strawman framework design for the climate community based on the integration of Cactus, from the relativistic physics community, and UCLA/UCB Distributed Data Broker (DDB) from the climate community. This design is the result of an extensive survey of climate models and frameworks in the climate community as well as frameworks from many other scientific communities. The design addresses fundamental development and runtime needs using Cactus, a framework with interfaces for FORTRAN and C-based languages, and high-performance model communication needs using DDB. This document also specifically explores object-oriented design issues in the context of climate modeling as well as climate modeling issues in terms of object-oriented design.

Talbot, Bryan↗

CryoDRGN-AI: neural ab initio reconstruction of challenging cryo-EM and cryo-ET datasets

Proteins and other biomolecules form dynamic macromolecular machines that are tightly orchestrated to move, bind, and perform chemistry. Cryo-electron microscopy (cryo-EM) and cryo-electron tomography (cryo-ET) can access the intrinsic heterogeneity of these complexes and are therefore key tools for understanding their function. However, 3D reconstruction of the collected imaging data presents a challenging computational problem, especially without any starting information, a setting termed ab initio reconstruction. Here, in this study, we introduce cryoDRGN-AI, a method leveraging an expressive neural representation and combining an exhaustive search strategy with gradient-based optimization to process challenging heterogeneous datasets. Using cryoDRGN-AI, we reveal new conformational states in large datasets, reconstruct previously unresolved motions from unfiltered datasets, and demonstrate ab initio reconstruction of biomolecular complexes from in situ data. With this expressive and scalable model for structure determination, we hope to unlock the full potential of cryo-EM and cryo-ET as a high-throughput tool for structural biology and discovery.

Levy, Axel [Stanford Univ., CA (United States); SL↗