SEARCH · Search NASA

Results for “data challenges”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 451 records · Page 25

All sources NASA NTRS DOE OSTI

NASA NTRS2013

Closed-Loop Analysis of Soft Decisions for Serial Links

We describe the benefit of using closed-loop measurements for a radio receiver paired with a counterpart transmitter. We show that real-time analysis of the soft decision output of a receiver can provide rich and relevant insight far beyond the traditional hard-decision bit error rate (BER) test statistic. We describe a Soft Decision Analyzer (SDA) implementation for closed-loop measurements on single- or dual- (orthogonal) channel serial data communication links. The analyzer has been used to identify, quantify, and prioritize contributors to implementation loss in live-time during the development of software defined radios. This test technique gains importance as modern receivers are providing soft decision symbol synchronization as radio links are challenged to push more data and more protocol overhead through noisier channels, and software-defined radios (SDRs) use error-correction codes that approach Shannon's theoretical limit of performance.

Lansdowne, Chatwin A.↗

NASA NTRS2013

Soft Decision Analyzer

Lansdowne, Chatwin↗

NASA NTRS2005

Large Scale Data Mining to Improve Usability of Data: An Intelligent Archive Testbed

Research in certain scientific disciplines - including Earth science, particle physics, and astrophysics - continually faces the challenge that the volume of data needed to perform valid scientific research can at times overwhelm even a sizable research community. The desire to improve utilization of this data gave rise to the Intelligent Archives project, which seeks to make data archives active participants in a knowledge building system capable of discovering events or patterns that represent new information or knowledge. Data mining can automatically discover patterns and events, but it is generally viewed as unsuited for large-scale use in disciplines like Earth science that routinely involve very high data volumes. Dozens of research projects have shown promising uses of data mining in Earth science, but all of these are based on experiments with data subsets of a few gigabytes or less, rather than the terabytes or petabytes typically encountered in operational systems. To bridge this gap, the Intelligent Archives project is establishing a testbed with the goal of demonstrating the use of data mining techniques in an operationally-relevant environment. This paper discusses the goals of the testbed and the design choices surrounding critical issues that arose during testbed implementation.

Ramapriyan, Hampapuram↗

NASA NTRS2012

The Development and Hover Test Application of a Projection Moire Interferometry Blade Displacement Measurement System

Projection moir interferometry (PMI) was employed to measure blade deflections during a hover test of a generic model-scale rotor in the NASA Langley 14x22 subsonic wind tunnel s hover facility. PMI was one of several optical measurement techniques tasked to acquire deflection and flow visualization data for a rotor at several distinct heights above a ground plane. Two of the main objectives of this test were to demonstrate that multiple optical measurement techniques can be used simultaneously to acquire data and to identify and address deficiencies in the techniques. Several PMI-specific technical challenges needed to be addressed during the test and in post-processing of the data. These challenges included developing an efficient and accurate calibration method for an extremely large (65 inch) height range; automating the analysis of the large amount of data acquired during the test; and developing a method to determinate the absolute displacement of rotor blades without a required anchor point measurement. The results indicate that the use of a single-camera/single-projector approach for the large height range reduced the accuracy of the PMI system compared to PMI systems designed for smaller height ranges. The lack of the anchor point measurement (due to a technical issue with one of the other measurement techniques) limited the ability of the PMI system to correctly measure blade displacements to only one of the three rotor heights tested. The new calibration technique reduced the data required by 80 percent while new post-processing algorithms successfully automated the process of locating rotor blades in images, determining the blade quarter chord location, and calculating the blade root and blade tip heights above the ground plane.

Sekula, Martin K.↗

NASA NTRS1993

Space Shuttle Main Engine instrumented High Pressure Oxidizer Turbopump technology test bed testing results summary

This paper presents the test results from the Space Shuttle Main Engine (SSME) instrumented High Pressure Oxidizer Turbopump (HPOTP). The turbopump was tested on Engine 3001, a highly instrumented engine, in an effort to characterize the turbopump and the engine system. Seven tests, for a total duration of 766 seconds, were performed over a five month time period. The testing was performed at a wide variety of engine conditions. Changes in engine mixture ratio, power level, engine inlet oxidizer pressure, engine inlet fuel pressure, and engine start sequence were made. A discussion of all the HPOTP pressure and temperature data obtained are presented with comparisons to supporting analyses made where applicable. The effect of the various engine conditions on the measured data is addressed. This paper also discusses the challenges that were overcome to obtain the data. The significant instrumentation related problems encountered during the design, fabrication, and testing of this turbopump are summarized. Only those issues that affected the data obtained or the instrumentation itself are discussed. The relevance of the data to other noninstrumented turbomachinery is outlined. Conclusions and recommendations resulting from the test series will be presented.

Koelbl, Mary E.↗

NASA NTRS1998

A Conceptual Approach to Assimilating Remote Sensing Data to Improve Soil Moisture Profile Estimates in a Surface Flux/Hydrology Model: Overview - Part 1

Knowledge of the amount of water in the soil is of great importance to many earth science disciplines. Soil moisture is a key variable in controlling the exchange of water and energy between the land surface and the atmosphere. Thus, soil moisture information is valuable in a wide range of applications including weather and climate, runoff potential and flood control, early warning of droughts, irrigation, crop yield forecasting, soil erosion, reservoir management, geotechnical engineering, and water quality. Despite the importance of soil moisture information, widespread and continuous measurements of soil moisture are not possible today. Although many earth surface conditions can be measured from satellites, we still cannot adequately measure soil moisture from space. Research in soil moisture remote sensing began in the mid 1970s shortly after the surge in satellite development. Recent advances in remote sensing have shown that soil moisture can be measured, at least qualitatively, by several methods. Quantitative measurements of moisture in the soil surface layer have been most successful using both passive and active microwave remote sensing, although complications arise from surface roughness and vegetation type and density. Early attempts to measure soil moisture from space-borne microwave instruments were hindered by what is now considered sub-optimal wavelengths (shorter than 5 cm) and the coarse spatial resolution of the measurements. L-band frequencies between 1 and 3 GHz (10-30 cm) have been deemed optimal for detection of soil moisture in the upper few centimeters of soil. The Electronically Steered Thinned Array Radiometer (ESTAR), an aircraft-based instrument operating a 1,4 GHz, has shown great promise for soil moisture determination. Initiatives are underway to develop a similar instrument for space. Existing space-borne synthetic aperture radars (SARS) operating at C- and L-band have also shown some potential to detect surface wetness. The advantage of radar is its much higher resolution than passive microwave systems, but it is currently hampered by surface roughness effects and the lack of a good algorithm based on a single frequency and single polarization. In addition, its repeat frequency is generally low (about 40 days). In the meantime, two new radiometers offer some hope for remote sensing of soil moisture from space. The Tropical Rainfall Measuring Mission (TRMM) Microwave Imager (TMI), launched in November 1997, possesses a 10.65 GHz channel and the Advanced Microwave Scanning Radiometer (AMSR) on both the ADEOS-11 and Earth Observing System AM-1 platforms to be launched in 1999 possesses a 6.9 GHz channel. Aside from issues about interference from vegetation, the coarse resolution of these data will provide considerable challenges pertaining to their application. The resolution of TMI is about 45 km and that of AMSR is about 70 km. These resolutions are grossly inconsistent with the scale of soil moisture processes and the spatial variability of factors that control soil moisture. Scale disparities such as these are forcing us to rethink how we assimilate data of various scales in hydrologic models. Of particular interest is how to assimilate soil moisture data by reconciling the scale disparity between what we can expect from present and future remote sensing measurements of soil moisture and modeling soil moisture processes. It is because of this disparity between the resolution of space-based sensors and the scale of data needed for capturing the spatial variability of soil moisture and related properties that remote sensing of soil moisture has not met with more widespread success. Within a single footprint of current sensors at the wavelengths optimal for this application, in most cases there is enormous heterogeneity in soil moisture created by differences in landcover, soils and topography, as well as variability in antecedent precipitation. It is difficult to interpret the meaning of 'mean' soil moisture under such conditions and even more difficult to apply such a value. Because of the non-linear relationships between near-surface soil moisture and other variables of interest, such as surface energy fluxes and runoff, mean soil moisture has little applicability at such large scales. It is for these reasons that the use of remote sensing in conjunction with a hydrologic model appears to be of benefit in capturing the complete spatial and temporal structure of soil moisture. This paper is Part I of a four-part series describing a method for intermittently assimilating remotely-sensed soil moisture information to improve performance of a distributed land surface hydrology model. The method, summarized in section II, involves the following components, each of which is detailed in the indicated section of the paper or subsequent papers in this series: Forward radiative transfer model methods (section II and Part IV); Use of a Kalman filter to assimilate remotely-sensed soil moisture estimates with the model profile (section II and Part IV); Application of a soil hydrology model to capture the continuous evolution of the soil moisture profile within and below the root zone (section III); Statistical aggregation techniques (section IV and Part II); Disaggregation techniques using a neural network approach (section IV and Part III); and Maximum likelihood and Bayesian algorithms for inversely solving for the soil moisture profile in the upper few cm (Part IV).

Crosson, William L.↗

NASA NTRS2010

Design of Control Software for a High-Speed Coherent Doppler Lidar System for CO2 Measurement

The design of the software for a 2-micron coherent high-speed Doppler lidar system for CO2 measurement at NASA Langley Research Center is discussed in this paper. The specific strategy and design topology to meet the requirements of the system are reviewed. In order to attain the high-speed digitization of the different types of signals to be sampled on multiple channels, a carefully planned design of the control software is imperative. Samples of digitized data from each channel and their roles in data analysis post processing are also presented. Several challenges of extremely-fast, high volume data acquisition are discussed. The software must check the validity of each lidar return as well as other monitoring channel data in real-time. For such high-speed data acquisition systems, the software is a key component that enables the entire scope of CO2 measurement studies using commercially available system components.

Vanvalkenburg, Randal L.↗

NASA NTRS2019

Bringing Analysis Closer to Data: Developing a Visualization Tool for L2 Earth Science Satellite Data

Earth Science satellite missions provide a unique opportunity for scientists to visualize complex and multifaceted observations projected geospatially across maps of the Earth. While visualization tools can help scientists comprehend, analyze, and share data, visualizing Level-2 Earth Sciences data poses its own specific set of challenges. Since the geospatial information in Level-2 data files is stored as independent variables, the plotting process involves matching dimensional information from latitude and longitude with a desired variable. Variables are stored in different ways across various Earth Science data file formats, which complicates the process of extracting data and plotting variables from a given file without requiring extensive user input and prerequisite familiarity with the file type variable structure. In coordination with NASA’s Goddard Earth Sciences Data Information Services Center (GES DISC), the team developed a Level-2 Earth Science data visualization tool that aims to address some of the complexities associated with plotting Level-2 data. This tool offers command-line and user interface support for file and variable selection to accommodate varying use cases and degrees of user familiarity with the structure of a given file. The visualization tool is written in Python 3 and utilizes a modular approach to facilitate continued expansion and reuse. In addressing some common complications involved in plotting Level-2 Earth Sciences data, the tool aims to help to link the process of analysis more directly with data acquisition and visualization, bringing analysis closer to data across levels of processing.

Li, Angela W.↗

NASA NTRSDate not supplied

Entwine Point Tiles for 3D Visualization and Querying of ICESat-2

Point Cloud data from non-optical sensors present challenges in scientific computing in both volume of data and files, even for cloud services environments. As part of the Multi-Mission Algorithm and Analysis Platform (MAAP), a joint open science platform for global biomass modelling, we’ve developed a cloud optimized workflow for using ATL08 (ICESat-2) data as a point cloud. For MAAP, the ATL08 data product is published as Entwine Point Tiles (EPT), allowing users to visualize and query the full extent of this collection interactively without pre-downloading, or preprocessing. The EPT format is a cloud-optimized point cloud data format which re-organizes points into a cloud friendly spatially indexed data structure. MAAP uses AWS S3 to store these point clouds and serves them over OGC specified APIs, 3DTiles for visualization, and WFS for querying. This workflow allows for interactive 3D visualizations in a web browser, including notebook environments and facilitates on the fly subsetting for interactive data exploration, all of which can be applied to other similar sensors.

Alex Mandel↗

NASA NTRSDate not supplied

Data-Driven Turbulence Modeling: Summary and Outcomes of the 2022 NASA Symposium

A three-day symposium sponsored by NASA was held in July 2022 in Suffolk, Virginia on the subject of Turbulence Modeling: Roadblocks, and the Potential for Machine Learning. This meeting brought together over 80 experts from academia, government, and industry to discuss critical issues for Reynolds-averaged Navier-Stokes (RANS) turbulence and transition models, as well as to evaluate the results from a collaborative testing challenge based on data-driven methods and machine learning (ML) technology. The symposium represents a continuation of an earlier symposium sponsored by the University of Michigan and NASA, held in Ann Arbor, MI in 2017. The 2022 symposium included a wide variety of talks on the subjects of RANS and ML, five invited talks, and three panel discussions. This talk summarizes the main outcomes of the symposium, and includes suggested recommendations and future directions.

machine learning↗

NASA NTRSDate not supplied

Evaluation of Aerosol Data Assimilation and Forecasts in the NASA GEOS Model during the ASIA-AQ Campaign

Fine particulate matter (PM2.5) poses significant risks to human health and the environment by penetrating the lungs and causing respiratory and cardiovascular diseases, making it crucial to understand its sources and behavior for effective air quality management. The Goddard Earth Observing System (GEOS) Forward Processing (FP) system model, operated by the Global Modeling and Assimilation Office (GMAO) at NASA's Goddard Space Flight Center, provides real-time weather and aerosol analyses and forecasts. In addition to meteorological data assimilation, the GEOS-FP system also assimilates aerosol using Moderate Resolution Imaging Spectroradiometer (MODIS) Aerosol Optical Depth (AOD) and Aerosol Robotic Network (AERONET) AOD data. In this study, the aerosol data assimilation and forecasts performance of the GEOS-FP model were evaluated for predicting PM2.5 in Korea using observations from the Airborne and Satellite Investigation of Asian Air Quality (ASIA-AQ) campaign. The ASIA-AQ campaign, an international collaborative field study initiative, aims to enhance understanding of local air quality issues and address common challenges in interpreting satellite data and air quality modeling. Conducted in South Korea from February 15 to March 13, 2024, during the high PM2.5 concentration winter season, this campaign provided extensive airborne and ground observations for intensive analysis of PM2.5 model simulations. We demonstrate how the assimilation runs and the forecasting performance of PM2.5 at 24-hour and 48-hour intervals vary. Additionally, we analyzed the differences and characteristics of PM2.5 composition in cases of long-range transport and local emissions. Using ASIA-AQ airborne data, we also examined the vertical profile of fine particulate matter. Through the intensive observations of this campaign, the GEOS model was assessed over South Korea using both in situ and airborne measurements to establish a baseline and identify priorities for future development.

Seunghee Lee↗

NASA NTRS2021

Autonomous Off-road Navigation over Extreme Terrains with Perceptually-challenging Conditions

We propose a framework for resilient autonomous navigation in perceptuallychallenging unknown environments with mobility-stressing elements such asuneven surfaces with rocks and boulders, steep slopes, negative obstacles like cliffsand holes, and narrow passages. Environments are GPS-denied and perceptuallydegradedwith variable lighting from dark to lit and obscurants (dust, fog, smoke).Lack of prior maps and degraded communication eliminates the possibility of prioror off-board computation or operator intervention. This necessitates real-time onboardcomputation using noisy sensor data. To address these challenges, we proposea resilient architecture that exploits redundancy and heterogeneity in sensing modalities.Further resilience is achieved by triggering recovery behaviors upon failure.We propose a fast settling algorithm to generate robust multi-fidelity traversabilityestimates in real-time. The proposed approach was deployed on multiple physicalsystems including skid-steer and tracked robots, high-speed RC car and legged robotsand as a part of Team CoSTAR’s effort to theDARPASubterranean Challenge, wherethe team won 2nd and 1st place in the Tunnel and Urban Circuit, respectively.

Agha-mohammadi, Ali-akbar↗

NASA NTRS2001

'The surface management system' (SuMS) database: a surface-based database to aid cortical surface reconstruction, visualization and analysis

Surface reconstructions of the cerebral cortex are increasingly widely used in the analysis and visualization of cortical structure, function and connectivity. From a neuroinformatics perspective, dealing with surface-related data poses a number of challenges. These include the multiplicity of configurations in which surfaces are routinely viewed (e.g. inflated maps, spheres and flat maps), plus the diversity of experimental data that can be represented on any given surface. To address these challenges, we have developed a surface management system (SuMS) that allows automated storage and retrieval of complex surface-related datasets. SuMS provides a systematic framework for the classification, storage and retrieval of many types of surface-related data and associated volume data. Within this classification framework, it serves as a version-control system capable of handling large numbers of surface and volume datasets. With built-in database management system support, SuMS provides rapid search and retrieval capabilities across all the datasets, while also incorporating multiple security levels to regulate access. SuMS is implemented in Java and can be accessed via a Web interface (WebSuMS) or using downloaded client software. Thus, SuMS is well positioned to act as a multiplatform, multi-user 'surface request broker' for the neuroscience community.

NASA Discipline Neuroscience↗

NASA NTRS2021

Generating Flood Probability Map Based on Combined Use of Synthetic Aperture Radar and Optical Imagery

Despite a lot of efforts to respond flood hazards with remote sensing data, it is still difficult to generate an accurate flood map using solely optical or radar imagery. While optical data is relatively high-resolution and does not suffer from speckle noise compared to radar data, it is very likely to be impacted by cloud and shadow. On the other hand, radar imagery can be used in all weather conditions due to its capability of penetrating clouds. Although a significant improvement of flood monitoring capability is achieved by using radar data, it is still challenging to map urban floods because of strong backscattering by man-made structures. Therefore, complementary use of optical and radar imagery in flood response is required, particularly in urban areas. In this study, we have adopted the Bayesian Joint Probability function to combine two different flood products generated from SAR and optical imagery. Flood detection with SAR data relies on the difference of backscatter signals between standing water and rougher land surface, while a Normalized Difference Water Index (NDWI) approach is used for optical data. Specifically, Planet Dove data with its 3m spatial resolution is used with higher weight values to detect flood extent in urban areas.

floods↗

NASA NTRSDate not supplied

High Performance Access to Archival Data Stored in HDF4 and HDF5 on Cloud Object Stores Without Reformatting the Files

Cloud computing offers numerous advantages for users of extensive Earth science data collections. These benefits encompass direct online access to data files and granules from any location, scalable access supporting parallel computing workflows, and flexible computing tools enabling innovative experimentation with processing techniques. However, older archival file formats designed for distinct computing systems hinder efficient access to decade-long time-series data when compared to data stored in modern cloud-optimized formats like Web Object Stores (WOS), exemplified by Amazon Web Services’ Simple Storage Service (S3). We describe DMR++ (Dataset Metadata Response plus plus), a technology facilitating efficient access to HDF5 (Hierarchical Data Format, version 5) and HDF4 files stored on WOS systems without requiring data reformatting. DMR++ achieves performance comparable to technologies like Zarr while preserving the original file structure, a substantial benefit considering the vast quantity of archival files held by organizations such as NASA. Moreover, DMR++ typically outperforms cloud-optimized versions of HDF5. Essentially an XML (Extensible Markup Language) document usually stored alongside the described data, DMR++ can also be generated on-the-fly but is generally created during data staging to the WOS. Archival files that use HDF4/5 often store large arrays of numerical data. The data in these files is often compressed, typically reducing their size by a factor of four or more. To achieve efficient access to portions of those arrays, they are 'chunked' into smaller sub-arrays, each individually compressed. The chunk size is a compromise, where spinning disks can efficiently access data in smaller chunks while S3 favors larger chunks. A simple optimization of aggregating smaller chunks that are stored adjacently, transferring them in a single access and then individually decompressing them will improve performance. NASA data pose an additional challenge: special Application Programmer Interface (API) libraries are often needed to compute some variables. These libraries are incompatible with WOS environments. Our solution involves storing computed values in the DMR++ document or a companion file, making them accessible like other variables and eliminating the need for specialized APIs. We outline specific optimizations for both satellite grid and swath data stored in HDF4-EOS2 (Earth Observing System).

James Gallagher↗

NASA NTRS2010

Tools and Methods for the Registration and Fusion of Remotely Sensed Data

Tools and methods for image registration were reviewed. Methods for the registration of remotely sensed data at NASA were discussed. Image fusion techniques were reviewed. Challenges in registration of remotely sensed data were discussed. Examples of image registration and image fusion were given.

Goshtasby, Arthur Ardeshir↗

NASA NTRS2013

Spatial Statistical Data Fusion (SSDF)

As remote sensing for scientific purposes has transitioned from an experimental technology to an operational one, the selection of instruments has become more coordinated, so that the scientific community can exploit complementary measurements. However, tech nological and scientific heterogeneity across devices means that the statistical characteristics of the data they collect are different. The challenge addressed here is how to combine heterogeneous remote sensing data sets in a way that yields optimal statistical estimates of the underlying geophysical field, and provides rigorous uncertainty measures for those estimates. Different remote sensing data sets may have different spatial resolutions, different measurement error biases and variances, and other disparate characteristics. A state-of-the-art spatial statistical model was used to relate the true, but not directly observed, geophysical field to noisy, spatial aggregates observed by remote sensing instruments. The spatial covariances of the true field and the covariances of the true field with the observations were modeled. The observations are spatial averages of the true field values, over pixels, with different measurement noise superimposed. A kriging framework is used to infer optimal (minimum mean squared error and unbiased) estimates of the true field at point locations from pixel-level, noisy observations. A key feature of the spatial statistical model is the spatial mixed effects model that underlies it. The approach models the spatial covariance function of the underlying field using linear combinations of basis functions of fixed size. Approaches based on kriging require the inversion of very large spatial covariance matrices, and this is usually done by making simplifying assumptions about spatial covariance structure that simply do not hold for geophysical variables. In contrast, this method does not require these assumptions, and is also computationally much faster. This method is fundamentally different than other approaches to data fusion for remote sensing data because it is inferential rather than merely descriptive. All approaches combine data in a way that minimizes some specified loss function. Most of these are more or less ad hoc criteria based on what looks good to the eye, or some criteria that relate only to the data at hand.

Braverman, Amy J.↗

NASA NTRSDate not supplied

Design of a Subscale, Inert Gas Test for Plume-Surface Interactions in a Reduced Pressure Environment

Rocket plume-surface interaction is a multi-phase, complex discipline characterized by plume flow physics, erosion physics, and ejecta dynamics. To provide critical data for the validation of predictive modeling capabilities, an inert-gas, subscale ground test has been designed, with one test phase focused on cratering and ejecta and another test phase focused on plume structure and behavior. Using a supersonic, heated N2 jet impinging onto six different mechanical regolith simulants, visual data on the temporal evolution of cratering and ejecta are collected in a vacuum chamber at ambient pressures ranging from 600 Pa down to approximately 7 Pa. Using the same N2 jet impinging onto an instrumented plate, NO planar laser-induced fluorescence data are obtained at lower ambient pressures for the validation of both continuum and rarefied gas-phase physics. This paper discusses the design of this test and data products, with emphasis on the cratering and ejecta experiments, along with the physical scaling, challenges, and relevance of these data to the maturation of the plume-surface interaction discipline.

Ashley M Korzun↗