Search NASA⌕ Search

SEARCH · Search NASA

Results for “Data Challenge”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 433 records · Page 24

NASA's Earth Observing Data and Information System - Near-Term Challenges

NASA's Earth Observing System Data and Information System (EOSDIS) has been a central component of the NASA Earth observation program since the 1990's. EOSDIS manages data covering a wide range of Earth science disciplines including cryosphere, land cover change, polar processes, field campaigns, ocean surface, digital elevation, atmosphere dynamics and composition, and inter-disciplinary research, and many others. One of the key components of EOSDIS is a set of twelve discipline-based Distributed Active Archive Centers (DAACs) distributed across the United States. Managed by NASA's Earth Science Data and Information System (ESDIS) Project at Goddard Space Flight Center, these DAACs serve over 3 million users globally. The ESDIS Project provides the infrastructure support for EOSDIS, which includes other components such as the Science Investigator-led Processing systems (SIPS), common metadata and metrics management systems, specialized network systems, standards management, and centralized support for use of commercial cloud capabilities. Given the long-term requirements, and the rapid pace of information technology and changing expectations of the user community, EOSDIS has evolved continually over the past three decades. However, many challenges remain. Challenges addressed in this paper include: growing volume and variety, achieving consistency across a diverse set of data producers, managing information about a large number of datasets, migration to a cloud computing environment, optimizing data discovery and access, incorporating user feedback from a diverse community, keeping metadata updated as data collections grow and age, and ensuring that all the content needed for understanding datasets by future users is identified and preserved.

Remote Sensing↗

A Kriging based spatiotemporal approach for traffic volume data imputation

Along with the rapid development of Intelligent Transportation Systems, traffic data collection technologies have progressed fast. The emergence of innovative data collection technologies such as remote traffic microwave sensor, Bluetooth sensor, GPS-based floating car method, and automated license plate recognition, has significantly increased the variety and volume of traffic data. Despite the development of these technologies, the missing data issue is still a problem that poses great challenge for data based applications such as traffic forecasting, real-time incident detection, dynamic route guidance, and massive evacuation optimization. A thorough literature review suggests most current imputation models either focus on the temporal nature of the traffic data and fail to consider the spatial information of neighboring locations or assume the data follow a certain distribution. These two issues reduce the imputation accuracy and limit the use of the corresponding imputation methods respectively. As a result, this paper presents a Kriging based data imputation approach that is able to fully utilize the spatiotemporal correlation in the traffic data and that does not assume the data follow any distribution. A set of scenarios with different missing rates are used to evaluate the performance of the proposed method. The performance of the proposed method was compared with that of two other widely used methods, historical average and K-nearest neighborhood. Comparison results indicate that the proposed method has the highest imputation accuracy and is more flexible compared to other methods.

Science & Technology - Other Topics↗

Center for Plasma Edge Simulation (CPES) (Rutgers University Final Report)

The CPES scientific simulations run at scale on leadership class machines, collaborate at runtime and produce and exchange large data sizes, which present multiple I/O and data management challenges. During the CPES project, the Rutgers team worked with the rest of the CPES team to address these challenges at different levels, and specifically (1) at the data transport and communication level through the DART (Decoupled and Asynchronous Remote Data Transfers) framework, and (2) at the data management and services level through the DataSpaces and ActiveSpaces frameworks. These frameworks and their impact are briefly described.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

Large Aperture Scanning Lidar Based on Holographic Optical Elements

Lidar remote sensing instruments can make a significant contribution to satisfying many of the required measurements of atmospheric and surface parameters for future spaceborne platforms, including topographic altimeters, atmospheric profiles of, wind, humidity, temperature, trace molecules, aerosols, and clouds. It is highly desirable to have wide measurement swaths for rapid coverage rather than just the narrow ribbon of data that is obtained with a nadir only observation. For most applications global coverage is required, and for wind measurements scanning or pointing is required in order to retrieve the full 3-D wind vector from multiple line-of-sight Doppler measurements. Conventional lidar receivers make up a substantial portion of the instrument's size and weight. Wide angle scanning typically requires a large scanning mirror in front of the receiver telescope, or pointing the entire telescope and aft optics assembly, Either of these methods entails the use of large bearings, motors, gearing and their associated electronics. Spaceborne instruments also need reaction wheels to counter the torque applied to the spacecraft by these motions. NASA has developed simplified conical scanning telescopes using Holographic Optical Elements (HOEs) to reduce the size, mass, angular momentum, and cost of scanning lidar systems. NASA has developed two operating lidar systems based on 40 cm diameter HOEs. The first such system, named Prototype Holographic Atmospheric Scanner for Environmental Remote Sensing (PHASERS) was a joint development between NASA Goddard Space Flight Center (GSFC) and the University of Maryland College Park. PHASERS is based on a reflection HOE for use at the doubled Nd:YAG laser wavelength of 532 nm and has recently undergone a number of design changes in a collaborative effort between GSFC and Saint Anselm College in New Hampshire. The next step was to develop IR transmission HOEs for use with the Nd:YAG fundamental in the Holographic Airborne Rotating Lidar Instrument Experiment (HARLIE). The HOE spins like a compact disk in a large ring ball bearing. In an aircraft the HOE faces down, looking out through a window at an angle of 45 degrees off-nadir. The HOE diffracts 85% of the incident 532 nm light into a 160 micron spot at a focal length of 1 meter. HARLIE is a field deployable lidar measuring aerosol, cloud, and boundary layer backscatter for atmospheric research. It has flown several times and is also used from a ground-based trailer in an upward-looking mode. The HOE generates a 45 degree conical scan pattern by rotating at speeds up to 30 rpm. Like PHASERS, the HOE in HARLIE serves both as the laser collimating lens as well as the receiver telescope primary optic. The telescope is coupled to the receiver package via fiber optic. The transmitter is a diode pumped Nd:YAG laser operating at 1064 nm, delivering 1 mJ pulses at a 5 KHz rep-rate. The receiver has a 200 microradian field-of-view and a 0.5 nm optical bandpass. The photon counting data system utilizes a single Geiger-mode silicon avalanche photodiode detector, This new technology has also presented us with new data visualization challenges as well as new measurement techniques. The backscatter data obtained from a stationary (i.e. ground-based) scanning HOE lidar is on the surface of a cone, which when viewed over many consecutive scans can reveal atmospheric motions on this surface over time as the atmosphere advects over the site. In a moving platform such as an airplane or satellite, the data from consecutive scans cover different areas under the flight path, revealing atmospheric structure in 3-dimensions. An example of a visualization of HARLIE ground-based data is presented, showing aerosol backscatter on a 90 degree conical surface generated from one 360 degree scan of the lidar during the HOLO-1 field campaign on the afternoon of 10 March 1999. Higher backscatter levels are rendered as lighter signal against a dark background. Breaking Kelvin-Helmholtz waves are evident on the north side of the scan at an altitude of 10-11 km. Time series of successive scans made at regular intervals render unique views of atmospheric motions, from which vertical profiles of atmospheric wind vectors can be obtained using a unique data analysis approach. Wind vectors obtained from the lidar were compared with co-located radiosonde wind profiles during an intensive operating period in September-October 2000 at the Atmospheric Radiation Measurement Program's Southern Great Plains Central Facility.

Schwemmer, Geary K.↗

Supporting Building Portfolio Investment and Policy Decision Making through an Integrated Building Utility Data Platform

The American Recovery and Reinvestment Act stimulus funding of 2009 for smart grid projects resulted in the tripling of smart meters deployment. In 2012, the Green Button initiative provided utility customers with access to their real-time1 energy usage. The availability of finely granular data provides an enormous potential for energy data analytics and energy benchmarking. The sheer volume of time-series utility data from a large number of buildings also poses challenges in data collection, quality control, and database management for rigorous and meaningful analyses. In this paper, we will describe a building portfolio-level data analytics tool for operational optimization, business investment and policy assessment using 15-minute to monthly intervals utility data. The analytics tool is developed on top of the U.S. Department of Energy’s Standard Energy Efficiency Data (SEED) platform, an open source software application that manages energy performance data of large groups of buildings. To support the significantly large volume of granular interval data, we integrated a parallel time-series database to the existing relational database. The time-series database improves on the current utility data input, focusing on real-time data collection, storage, analytics and data quality control. The fully integrated data platform supports APIs for utility apps development by third party software developers. These apps will provide actionable intelligence for building owners and facilities managers. Unlike a commercial system, this platform is an open source platform funded by the U.S. Government, accessible to the public, researchers and other developers, to support initiatives in reducing building energy consumption.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

Evolving the Technical Infrastructure of the Planetary Data System for the 21st Century

The Planetary Data System (PDS) was established in 1989 as a distributed system to assure scientific oversight. Initially the PDS followed guidelines recommended by the National Academies Committee on Data Management and Computation (CODMAC, 1982) and placed emphasis on archiving validated datasets. But overtime user demands, supported by increased computing capabilities and communication methods, have placed increasing demands on the PDS. The PDS must add additional services to better enable scientific analysis within distributed environments and to ensure that those services integrate with existing systems and data. To face these challenges the Planetary Data System (PDS) must modernize its architecture and technical implementation. The PDS 2010 project addresses these challenges. As part of this project, the PDS has three fundamental project goals that include: (1) Providing more efficient client delivery of data by data providers to the PDS (2) Enabling a stable, long-term usable planetary science data archive (3) Enabling services for the data consumer to find, access and use the data they require in contemporary data formats. In order to achieve these goals, the PDS 2010 project is upgrading both the technical infrastructure and the data standards to support increased efficiency in data delivery as well as usability of the PDS. Efforts are underway to interface with missions as early as possible and to streamline the preparation and delivery of data to the PDS. Likewise, the PDS is working to define and plan for data services that will help researchers to perform analysis in cost-constrained environments. This presentation will cover the PDS 2010 project including the goals, data standards and technical implementation plans that are underway within the Planetary Data System. It will discuss the plans for moving from the current system, version PDS 3, to version PDS 4.

Beebe, Reta F.↗

OmicsMLMentor: A Web Application for Guided Machine Learning Analysis of Omics Data

Expression-based omics technologies (e.g. proteomics, metabolomics, transcriptomics, etc.) increasingly rely on supervised and unsupervised machine learning (ML) models to find key biomolecules distinguishing conditions, identify natural groupings in biological data, or generate predictions for outcomes of interest. Fitting ML models to omics data presents several challenges, including handling missing data, selecting a normalization method, choosing a valid model, and optimizing hyperparameters, all requiring statistical programming skills to address these challenges. Thus, the open-source web application SLOPE was designed to lower the barrier to ML modeling for omics data. SLOPE supports the fitting of 15 ML models (10 supervised and 5 unsupervised) tailored to omics datasets, such as proteomics, metabolomics, lipidomics, and transcriptomics. SLOPE offers several omics-specific features, including methods for handling missingness (imputation, conversion, removal), normalization tests, ranking of models based on the structure of a user’s data and user input, and optimal hyperparameter selections using cross-validation splits. By streamlining ML workflows for omics analysis, SLOPE address critical gaps in existing online web tools, facilitating a broader adoption of these models for omics research. Here, SLOPE is applied to data from a lignin exposure study to highlight the workflow for fitting both supervised and unsupervised models to data.

lipidomics↗

Examples L-Band Interference will be Presented and Discussed, as well as the Importance of L-Band Soil Moisture Observations

Examples of L-band interference will be presented and discussed, as well as the importance of L-band soil moisture observations, as part of this one-day GEOSS workshop XXXVII on "Data Quality and Radio Spectrum Allocation Impact on Earth Observations" will address the broad challenges of data quality and the impact of generating reliable information for decision makers who are Earth data users but not necessarily experts in the Earth observation field. GEO has initiated a data quality assessment task (DA-09-01a) and workshop users will review and debate the directions and challenges of this effort. Radio spectrum allocation is an element of data availability and data quality, and is also associated with a GEO task (AR-06-11). A recent U.S. National Research Council report on spectrum management will be addressed as part of the workshop. Key representatives from industry, academia, and government will provide invited talks on these and related issues that impact GEOSS implementation.

Kim, Edward↗

In-Staging Data Placement for Asynchronous Coupling of Task-Based Scientific Workflows, In: 2016 Second International Workshop on Extreme Scale Programming Models and Middlewar (ESPM2)

Coupled application workflows composed of applications implemented using task-based models present new coupling and data exchange challenges, due to the asynchronous interaction and coupling behaviors between tasks of the component applications. In this paper, we present an adaptive data placement approach that addresses these challenges by dynamically adjusting to the asynchronous coupling patterns. Specifically, it places data across a set of staging cores/nodes with an awareness of the application-specific data locality requirements and the runtime task executions at these staging cores/nodes, with the goal of reducing end-to-end execution time and data movement overhead of the workflow. We experimentally demonstrate the effectiveness of our approach on the Titan Cray XK7 system using representative data coupling patterns derived from current scientific workflows. The evaluation demonstrates that our approach efficiently improves performance by reducing the time-to-solution and increasing the quality of insights for scientific discovery.

Computer Science↗

Using Virtual Reality to Envision Deployment of Spacesuit-Compatible Augmented Reality Displays for Lunar Surface Operations

The National Aeronautics and Space Administration (NASA) aims to land crew on the lunar surface to establish a sustainable presence and develop operational concepts for future long-duration missions. New technologies will be necessary to extend planning and execution capabilities for lunar surface activities. NASA’s Joint Augmented Reality Visual Informatics System (Joint AR) is one such technology. Joint AR is a suit-mounted augmented reality (AR) display and computes system which facilitates unprecedented information exchange and data visualization capabilities between mission support operators and suited crew. This paper describes challenges associated with developing AR technology for an envisioned work domain by applying a sociotechnical lens to the iterative testing and development of novel AR technology through virtual reality (VR). A VR testbed was established to simulate a representative lunar surface environment, enabling a series of three human-in-the-loop (HITL) tests evaluating AR navigation interfaces for exploration extravehicular activity (xEVA). Our findings identify several considerations for future Joint AR design and testing efforts, including challenges with data overload, attentional demands, and environment-related perceptual challenges. Trade-offs and potential approaches are discussed to mitigate these challenges and improve future Joint AR testing fidelity.

Jacob Keller↗

Using Virtual Reality to Envision Deployment of Spacesuit-Compatible Augmented Reality Displays for Lunar Surface Operations

The National Aeronautics and Space Administration (NASA) aims to land crew on the lunar surface to establish a sustainable presence and develop operational concepts for future long-duration missions. New technologies will be necessary to extend planning and execution capabilities for lunar surface activities. NASA’s Joint Augmented Reality Visual Informatics System (Joint AR) is one such technology. Joint AR is a suit-mounted augmented reality (AR) display and computes system which facilitates unprecedented information exchange and data visualization capabilities between mission support operators and suited crew. This paper describes challenges associated with developing AR technology for an envisioned work domain by applying a sociotechnical lens to the iterative testing and development of novel AR technology through virtual reality (VR). A VR testbed was established to simulate a representative lunar surface environment, enabling a series of three human-in-the-loop (HITL) tests evaluating AR navigation interfaces for exploration extravehicular activity (xEVA). Our findings identify several considerations for future Joint AR design and testing efforts, including challenges with data overload, attentional demands, and environment-related perceptual challenges. Trade-offs and potential approaches are discussed to mitigate these challenges and improve future Joint AR testing fidelity.

Matthew Miller↗

A-Train Data Depot: Integrating and Exploring Data Along the A-Train Tracks

The immense potential for new science findings as a result of inter-instrument data analysis has led to the development of a new data portal at GSFC: the A-train Data Depot. The power and utility of this new service to the general public is amplified immensely when the archived data are used in conjunction with online data analysis services like Giovanni. This presentation details some of the challenges of data usage from multiple distinct missions and how the tool sets we have developed can help to overcome these challenges, considerably cut down on analysis overhead and promote science exploration in an otherwise very challenging arena.

Leptoukh, G.↗

A Software Developer’s Guide to Informal Evaluation of Visual Analytics Environments Using VAST Challenge Information

The VAST Challenge has been a popular venue for academic and industry participants for over ten years. Many participants comment that the majority of their time in preparing VAST Challenge entries is discovering elements in their software environments that need to be redesigned in order to solve the given task. Fortunately, there is no need to wait until the VAST Challenge is announced to test out software systems. The Visual Analytics Benchmark Repository contains all past VAST Challenge tasks, data, solutions and submissions. This paper details the various types of evaluations that may be conducted using the Repository information. In this paper we describe how developers can do informal evaluations of various aspects of their visual analytics environments using VAST Challenge information. Aspects that can be evaluated include the appropriateness of the software for various tasks, the various data types and formats that can be accommodated, the effectiveness and efficiency of the process supported by the software, and the intuitiveness of the visualizations and interactions. Researchers can compare their visualizations and interactions to those submitted to determine novelty. In addition, the paper provides pointers to various guidelines that software teams can use to evaluate the usability of their software. While these evaluations are not a replacement for formal evaluation methods, this information can be extremely useful during the development of visual analytics environments.

evaluation↗

Uncertainty Management in Remote Sensing of Climate Data. Summary of A Workshop

Great advances have been made in our understanding of the climate system over the past few decades, and remotely sensed data have played a key role in supporting many of these advances. Improvements in satellites and in computational and data-handling techniques have yielded high quality, readily accessible data. However, rapid increases in data volume have also led to large and complex datasets that pose significant challenges in data analysis (NRC, 2007). Uncertainty characterization is needed for every satellite mission and scientists continue to be challenged by the need to reduce the uncertainty in remotely sensed climate records and projections. The approaches currently used to quantify the uncertainty in remotely sensed data, including statistical methods used to calibrate and validate satellite instruments, lack an overall mathematically based framework.

Remote sensing↗

Unsupervised learning of representative local atomic arrangements in molecular dynamics data

Molecular dynamics (MD) simulations present a data-mining challenge, given that they can generate a considerable amount of data but often rely on limited or biased human interpretation to examine their information content. By not asking the right questions of MD data we may miss critical information hidden within it. Here we combine dimensionality reduction (UMAP) and unsupervised hierarchical clustering (HDBSCAN) to quantitatively characterize prevalent coordination environments of chemical species within MD data. By focusing on local coordination, we significantly reduce the amount of data to be analyzed by extracting all distinct molecular formulas within a given coordination sphere. We then efficiently combine UMAP and HDBSCAN with alignment or shape-matching algorithms to partition these formulas into structural isomer families indicating their relative populations. The method was employed to reveal details of cation coordination in electrolytes based on molecular liquids.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

FNETVision: A WAMS Big Data Knowledge Discovery System

With the growing awareness of the power system stability, wide area measurement system (WAMS) has been widely studied and deployed. In WAMS, a phasor data concentrator (PDC) collects and persists the GPS-time-synchronized phasor data from hundreds of phasor measurement units (PMU). Due to the size and complexity of the modern power systems, the number of deployed PMUs has been increasing and their data size becomes increasingly large. The large data volume challenges the analysis, and the knowledge discovery of the measurement data. This paper takes the pioneering distribution level WAMS- the frequency monitoring network FNET/GridEye as an example. It develops an adaptive visualization system- FNETVision to visualize the large-volume WAMS data by interconnection. The case studies suggest, based on the FNETVision system, the system administrator can quickly dig out issues of the FNET system and accordingly pinpoint the cause of these issues.

Wang, Weikang↗

Improving the Transition of Earth Satellite Observations from Research to Operations

There are significant gaps between the observations, models, and decision support tools that make use of new data. These challenges include: 1) Decreasing the time to incorporate new satellite data into operational forecast assimilation systems, 2) Blending in-situ and satellite observing systems to produce the most accurate and comprehensive data products and assessments, 3) Accelerating the transition from research to applications through national test beds, field campaigns, and pilot demonstrations, and 4) Developing the partnerships and organizational structures to effectively transition new technology into operations. At the Short-term Prediction Research and Transition (SPORT) Center in Huntsville, Alabama, a NASA-NOAA-University collaboration has been developed to accelerate the infusion of NASA Earth science observations, data assimilation and modeling research into NWS forecast operations and decision-making. The SPoRT Center research focus is to improve forecasts through new observation capability and the regional prediction objectives of the US Weather Research Program dealing with 0-1 day forecast issues such as convective initiation and 24-hr quantitative precipitation forecasting. The near real-time availability of high-resolution experimental products of the atmosphere, land, and ocean from the Moderate Resolution Imaging Spectroradiometer (MODIS), the Advanced Infrared Spectroradiometer (AIRS), and lightning mapping systems provide an opportunity for science and algorithm risk reduction, and for application assessment prior to planned observations from the next generation of operational low Earth orbiting and geostationary Earth orbiting satellites. This paper describes the process for the transition of experimental products into forecast operations, current products undergoing assessment by forecasters, and plans for the future. The SPoRT Web page is at (http://www.ghcc.msfc.nasa.gov/sport).

Goodman, Steven J.↗