Search NASA⌕ Search

SEARCH · Search NASA

Results for “cloud computing; data analysis”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 235 records · Page 13
All sourcesNASA NTRSDOE OSTI
NASA NTRS1989

Cloud cover determination in polar regions from satellite imagery

The objectives are to develop a suitable validation data set for evaluating the effectiveness of the International Satellite Cloud Climatology Project (ISCCP) algorithm for cloud retrieval in polar regions, to identify limitations of current procedures and to explore potential means to remedy them using textural classifiers, and to compare synoptic cloud data from model runs with observations. Toward the first goal, a polar data set consisting of visible, thermal, and passive microwave data was developed. The AVHRR and SMMR data were digitally merged to a polar stereographic projection with an effective pixel size of 5 sq km. With this data set, two unconventional methods of classifying the imagery for the analysis of polar clouds and surfaces were examined: one based on fuzzy sets theory and another based on a trained neural network. An algorithm for cloud detection was developed from an early test version of the ISCCP algorithm. This algorithm includes the identification of surface types with passive microwave, then temporal tests at each pixel location in the cloud detection phase. Cloud maps and clear sky radiance composites for 5 day periods are produced. Algorithm testing and validation was done with both actural AVHRR/SMMR data, and simulated imagery. From this point in the algorithm, groups of cloud pixels are examined for their spectral and textural characteristics, and a procedure is developed for the analysis of cloud patterns utilizing albedo, IR temperature, and texture. In a completion of earlier work, empirical analyses of arctic cloud cover were explored through manual interpretations of DMSP imagery and compared to U.S. Air Force 3D-nephanalysis. Comparisons of observed cloudiness from existing climatologies to patterns computed by the GISS climate model were also made.

Barry, R. G.↗
NASA NTRS2016

Polarbrdf: A General Purpose Python Package for Visualization Quantitative Analysis of Multi-Angular Remote Sensing Measurements

The Bidirectional Reflectance Distribution Function (BRDF) is a fundamental concept for characterizing the reflectance property of a surface, and helps in the analysis of remote sensing data from satellite, airborne and surface platforms. Multi-angular remote sensing measurements are required for the development and evaluation of BRDF models for improved characterization of surface properties. However, multi-angular data and the associated BRDF models are typically multidimensional involving multi-angular and multi-wavelength information. Effective visualization of such complex multidimensional measurements for different wavelength combinations is presently somewhat lacking in the literature, and could serve as a potentially useful research and teaching tool in aiding both interpretation and analysis of BRDF measurements. This article describes a newly developed software package in Python (PolarBRDF) to help visualize and analyze multi-angular data in polar and False Color Composite (FCC) forms. PolarBRDF also includes functionalities for computing important multi-angular reflectance/albedo parameters including spectral albedo, principal plane reflectance and spectral reflectance slope. Application of PolarBRDF is demonstrated using various case studies obtained from airborne multi-angular remote sensing measurements using NASA's Cloud Absorption Radiometer (CAR). Our visualization program also provides functionalities for untangling complex surface/atmosphere features embedded in pixel-based remote sensing measurements, such as the FCC imagery generation of BRDF measurements of grasslands in the presence of wild fire smoke and clouds. Furthermore, PolarBRDF also provides quantitative information of the angular distribution of scattered surface/atmosphere radiation, in the form of relevant BRDF variables such as sunglint, hotspot and scattering statistics.

CAR↗
NASA NTRS1980

Evaluation of SLAR and thematic mapper MSS data for forest cover mapping using computer-aided analysis techniques

Several possibilities were considered for defining the data set in which the same test areas could be used for each of the four different spatial resolutions being evaluated. The LARSYS CLUSTER was used to sort the vectors into spectral classes to reduce the within-spectral class variability in an effort to develop training statistics. A data quality test was written to determine the basic signal to noise characteristics within the data set being used. Because preliminary analysis of the LANDSAT MSS data revealed the presence of high cirrus clouds, other data sets are being sought.

Hoffer, R. M.↗
NASA NTRS1985

Interactive information processing for NASA's mesoscale analysis and space sensor program

The Atmospheric Sciences Division (ASD) of the Systems Dynamics Laboratory at NASA's Marshall Space Flight Center (MSFC) is currently involved in interactive information processing for the Mesoscale Analysis and Space Sensor (MASS) program. Specifically, the ASD is engaged in the development and implementation of new space-borne remote sensing technology to observe and measure mesoscale atmospheric processes. These space measurements and conventional observational data are being processed together to gain an improved understanding of the mesoscale structure and the dynamical evolution of the atmosphere relative to cloud development and precipitation processes. To satisfy its vast data processing requirements, the ASD has developed a Researcher Computer System consiting of three primary computer systems which provides over 20 scientists with a wide range of capabilities for processing and displaying a large volumes of remote sensing data. Each of the computers performs a specific function according to its unique capabilities.

Parker, K. G.↗
NASA NTRS1973

Analysis of aircraft microwave measurements of the ocean surface

A data system was developed to process, from calibrated brightness temperature to computation of estimated parameters, the microwave measurements obtained by the NASA CV-990 aircraft during the 1972 Meteorological Expedition. A primary objective of the study was the implementation of an integrated software system at the computing facility of NASA/GSFC, and its application to the 1972 data. A single test case involving measurements away from and over a heavy rain cell was chosen to examine the effect of clouds upon the ability to infer ocean surface parameters. The results indicate substantial agreement with those of the theoretical study; namely, that the values obtained for the surface properties are consistent with available ground-truth information, and are reproducible except within the heaviest portions of the rain cell, at which nonlinear (or saturation) effects become apparent. Finally, it is seen that uncorrected instrumental effects introduce systematic errors which may limit the accuracy of the method.

Willand, J. H.↗
NASA NTRS2018

Data, Meet Compute: NASA's Cumulus Ingest Architecture

NASA's Earth Observing System Data and Information System (EOSDIS) houses nearly 30PBs of critical Earth Science data and with upcoming missions is expected to balloon to between 200PBs-300PBs over the next seven years. In addition to the massive increase in data collected, researchers and application developers want more and faster access - enabling complex visualizations, long time-series analysis, and cross dataset research without needing to copy and manage massive amounts of data locally. NASA has looked to the cloud to address these needs, building its Cumulus system to manage the ingest of diverse data in a wide variety of formats into the cloud. In this talk, we look at what Cumulus is from a high level and then take a deep dive into how it manages complexity and versioning associated with multiple AWS Lambda and ECS microservices communicating through AWS Step Functions across several disparate installations

EOSDIS↗
NASA NTRS2008

Atmospheric Correction at AERONET Locations: A New Science and Validation Data Set

This paper describes an AERONET-based Surface Reflectance Validation Network (ASRVN) and its dataset of spectral surface bidirectional reflectance and albedo based on MODIS TERRA and AQUA data. The ASRVN is an operational data collection and processing system. It receives 50x50 square kilometer subsets of MODIS L1B data from MODAPS and AERONET aerosol and water vapor information. Then it performs an accurate atmospheric correction for about 100 AERONET sites based on accurate radiative transfer theory with high quality control of the input data. The ASRVN processing software consists of L1B data gridding algorithm, a new cloud mask algorithm based on a time series analysis, and an atmospheric correction algorithm. The atmospheric correction is achieved by fitting the MODIS top of atmosphere measurements, accumulated for 16-day interval, with theoretical reflectance parameterized in terms of coefficients of the LSRT BRF model. The ASRVN takes several steps to ensure high quality of results: 1) cloud mask algorithm filters opaque clouds; 2) an aerosol filter has been developed to filter residual semi-transparent and sub-pixel clouds, as well as cases with high inhomogeneity of aerosols in the processing area; 3) imposing requirement of consistency of the new solution with previously retrieved BRF and albedo; 4) rapid adjustment of the 16-day retrieval to the surface changes using the last day of measurements; and 5) development of seasonal back-up spectral BRF database to increase data coverage. The ASRVN provides a gapless or near-gapless coverage for the processing area. The gaps, caused by clouds, are filled most naturally with the latest solution for a given pixels. The ASRVN products include three parameters of LSRT model (k(sup L), k(sup G), k(sup V)), surface albedo, NBRF (a normalized BRF computed for a standard viewing geometry, VZA=0 deg., SZA=45 deg.), and IBRF (instantaneous, or one angle, BRF value derived from the last day of MODIS measurement for specific viewing geometry) for MODIS 500m bands 1-7. The results are produced daily at resolution of 1 km in gridded format. We also provide cloud mask, quality flag and a browse bitmap image. The new dataset can be used for a wide range of applications including validation analysis and science research.

Wang, Yujie↗
DOE OSTI2020

The ARM Data-oriented Metrics and Diagnostics Package for Climate Models - A New Tool for Evaluating Climate Models with Field Data

The U.S. Department of Energy (DOE) Atmospheric Radiation Measurement (ARM) program User Facility produces ground-based long-term continuous unique measurements for atmospheric state, precipitation, turbulent fluxes, radiation, aerosol, cloud and the land surface, which are collected at multiple sites. These comprehensive datasets have been widely used to calibrate climate models and are proven to be invaluable for climate model development and improvement. This article introduces an evaluation package to facilitate the use of ground-based ARM measurements in climate model evaluation. The ARM data-oriented metrics and diagnostics package (ARM-DIAGS) includes both ARM observational datasets and a Python-based analysis toolkit for computation and visualization. The observational datasets are compiled from multiple ARM data products and specifically tailored for use in climate model evaluation. In addition, ARM-DIAGS also includes simulation data from models participating the Coupled Model Inter-comparison Project (CMIP), which will allow climate-modeling groups to compare a new, candidate version of their model to existing CMIP models. The analysis toolkit is designed to make the metrics and diagnostics quickly available to the model developers.

54 ENVIRONMENTAL SCIENCES↗
NASA NTRS1990

The analysis of polar clouds from AVHRR satellite data using pattern recognition techniques

The cloud cover in a set of summertime and wintertime AVHRR data from the Arctic and Antarctic regions was analyzed using a pattern recognition algorithm. The data were collected by the NOAA-7 satellite on 6 to 13 Jan. and 1 to 7 Jul. 1984 between 60 deg and 90 deg north and south latitude in 5 spectral channels, at the Global Area Coverage (GAC) resolution of approximately 4 km. This data embodied a Polar Cloud Pilot Data Set which was analyzed by a number of research groups as part of a polar cloud algorithm intercomparison study. This study was intended to determine whether the additional information contained in the AVHRR channels (beyond the standard visible and infrared bands on geostationary satellites) could be effectively utilized in cloud algorithms to resolve some of the cloud detection problems caused by low visible and thermal contrasts in the polar regions. The analysis described makes use of a pattern recognition algorithm which estimates the surface and cloud classification, cloud fraction, and surface and cloudy visible (channel 1) albedo and infrared (channel 4) brightness temperatures on a 2.5 x 2.5 deg latitude-longitude grid. In each grid box several spectral and textural features were computed from the calibrated pixel values in the multispectral imagery, then used to classify the region into one of eighteen surface and/or cloud types using the maximum likelihood decision rule. A slightly different version of the algorithm was used for each season and hemisphere because of differences in categories and because of the lack of visible imagery during winter. The classification of the scene is used to specify the optimal AVHRR channel for separating clear and cloudy pixels using a hybrid histogram-spatial coherence method. This method estimates values for cloud fraction, clear and cloudy albedos and brightness temperatures in each grid box. The choice of a class-dependent AVHRR channel allows for better separation of clear and cloudy pixels than does a global choice of a visible and/or infrared threshold. The classification also prevents erroneous estimates of large fractional cloudiness in areas of cloudfree snow and sea ice. The hybrid histogram-spatial coherence technique and the advantages of first classifying a scene in the polar regions are detailed. The complete Polar Cloud Pilot Data Set was analyzed and the results are presented and discussed.

Smith, William L.↗
NASA NTRS1990

Diabatic forcing and initialization with assimilation of cloud and rain water in a forecast model: Methodology

The focus of this part of the investigation is to find one or more general modeling techniques that will help reduce the time taken by numerical forecast models to initiate or spin-up precipitation processes and enhance storm intensity. If the conventional data base could explain the atmospheric mesoscale flow in detail, then much of our problem would be eliminated. But the data base is primarily synoptic scale, requiring that a solution must be sought either in nonconventional data, in methods to initialize mesoscale circulations, or in ways of retaining between forecasts the model generated mesoscale dynamics and precipitation fields. All three methods are investigated. The initialization and assimilation of explicit cloud and rainwater quantities computed from conservation equations in a mesoscale regional model are examined. The physical processes include condensation, evaporation, autoconversion, accretion, and the removal of rainwater by fallout. The question of how to initialize the explicit liquid water calculations in numerical models and how to retain information about precipitation processes during the 4-D assimilation cycle are important issues that are addressed. The explicit cloud calculations were purposely kept simple so that different initialization techniques can be easily and economically tested. Precipitation spin-up processes associated with three different types of weather phenomena are examined. Our findings show that diabatic initialization, or diabatic initialization in combination with a new diabatic forcing procedure, work effectively to enhance the spin-up of precipitation in a mesoscale numerical weather prediction forecast. Also, the retention of cloud and rain water during the analysis phase of the 4-D data assimilation procedure is shown to be valuable. Without detailed observations, the vertical placement of the diabatic heating remains a critical problem.

Raymond, William H.↗
NASA NTRS1990

A comparison of ISCCP and FIRE satellite cloud parameters

One of the goals of the First ISCCP Regional Experiment (FIRE) is the quantification of the uncertainties in the cloud parameter products derived by the International Satellite Cloud Climatology Project (ISCCP). This validation effort has many facets including sensitivity analyses and comparisons to similar data or theoretical results with known accuracies. The FIRE provides cloud-truth data at particular points or along particular lines from surface and aircraft measurement systems. Relating these data to the larger, area-averaged ISCCP results requires intermediate steps using higher resolution satellite data analyses. Errors in the cloud products derived with a particular method can be determined by performing analyses of high resolution satellite data over the area surrounding the point or line measurement. This same analysis technique may then be used to derive cloud parameters over a larger area containing similar cloud fields. It is assumed that the uncertainties found for the small scale analyses are the same for the large scale so that the method has been calibrated for the particular cloud type; i.e., its accuracy is known. Differences between the large scale results using the ISCCP technique and the calibrated method can be computed and used to determine if any significant biases or rms errors occur in the ISCCP results. Selected ISCCP results are compared to cloud parameters derived using the hybrid bispectral threshold method over the FIRE IFO and extended observation areas.

Gibson, Gary G.↗
NASA NTRS1994

Theory, Image Simulation, and Data Analysis of Chemical Release Experiments

The final phase of Grant NAG6-1 involved analysis of physics of chemical releases in the upper atmosphere and analysis of data obtained on previous NASA sponsored chemical release rocket experiments. Several lines of investigation of past chemical release experiments and computer simulations have been proceeding in parallel. This report summarizes the work performed and the resulting publications. The following topics are addressed: analysis of the 1987 Greenland rocket experiments; calculation of emission rates for barium, strontium, and calcium; the CRIT 1 and 2 experiments (Collisional Ionization Cross Section experiments); image calibration using background stars; rapid ray motions in ionospheric plasma clouds; and the NOONCUSP rocket experiments.

Wescott, Eugene M.↗
DOE OSTI2021

Surrogate Model of Flexible Research Platform EnergyPlus Models to Enable Sensitivity Analysis

This letter report describes the surrogate models developed from the EnergyPlus model of Oak Ridge National Laboratory’s Flexible Research Platform. Two data-driven black-box models were developed, and the outputs of the surrogate models were compared with the EnergyPlus model. The two models developed are a multilayer perceptron deep learning model, and a long short-term memory (LSTM) neural network model. The three factors for selecting the black-box models are scalability, computation time, and accuracy. A total of 107 input variables were the dominant variables in determining the outputs of building energy consumptions and thermal comfort. A total of 54 output variables were identified as the prediction targets, including the system- and zone-level outputs. The large set of the simulation cases were generated by integrating sensor errors into an emulator based on EnergyPlus and Python EMS, which includes advanced control sequences from ASHRAE Guideline 36-2018: High-Performance Sequences of Operation. The surrogate models were developed based on a set of large-scale simulation runs (i.e., 4,000 runs) on a cloud platform. The comparison analysis shows that the two black-box models had good accuracy for predicting new outputs for sensitivity analysis using the root mean square error metric. As a next step, the developed surrogate models will be used to perform sensitivity analysis for different sensor impacts (e.g., sensor types, sensor locations).

42 ENGINEERING↗
NASA NTRS2011

Polarimetric Decomposition Analysis of the Deepwater Horizon Oil Slick Using L-Band UAVSAR Data

We report here an analysis of the polarization dependence of L-band radar backscatter from the main slick of the Deepwater Horizon oil spill, with specific attention to the utility of polarimetric decomposition analysis for discrimination of oil from clean water and identification of variations in the oil characteristics. For this study we used data collected with the UAVSAR instrument from opposing look directions directly over the main oil slick. We find that both the Cloude-Pottier and Shannon entropy polarimetric decomposition methods offer promise for oil discrimination, with the Shannon entropy method yielding the same information as contained in the Cloude-Pottier entropy and averaged in tensity parameters, but with significantly less computational complexity

Deepwater Horizon oil slick↗
NASA NTRS2020

Observations on the Application of Machine Learning Techniques to Aviation Operations

There is an increasing interest in applying methods based on Machine Learning Techniques (MLT) to problems in aviation operations. The current interest is based on developments in Cloud Computing, the availability of open software and the success of MLT in automation, consumer behavior and finance involving large database. Historically aviation operations have been analyzed using physics-based models and provide information for making operational decisions. This talk describes issues to be addressed in applying either model-driven or data-driven methods. Aviation operations involving many decision makers, multiple objectives, poor or unavailable physics-based models and a rich historical database are prime candidates for analysis using data-driven methods. The issues are illustrated by a detailed example and summary of current research in the area. The application of MLT to aviation operations falls into two categories: (a) based on the lack of a physics-based model, MLT is the favored approach and (b) marginal difference between regression methods using physics-based models and MLT. Further research is needed in the selection of MLT to critical aviation operations. As always, the best approach depends on the task, the physical understanding of the problem and the quality and quantity of the available data.

Sridhar, Banavar↗
NASA NTRS2020

Application of Machine Learning Techniques to Aviation Operations: A Case Study

There is an increasing interest in applying methods based on Machine Learning Techniques (MLT) to problems in aviation operations. The current interest is based on developments in Cloud Computing, the availability of open software and the success of MLT in automation, consumer behavior and finance involving large database. Historically aviation operations have been analyzed using physics-based models and provide information for making operational decisions. This talk describes issues to be addressed in applying either model-driven or data-driven methods. Aviation operations involving many decision makers, multiple objectives, poor or unavailable physics-based models and a rich historical database are prime candidates for analysis using data-driven methods. The issues are illustrated by a detailed example and summary of current research in the area. The application of MLT to aviation operations falls into two categories 58; (a) based on the lack of a physics-based model, MLT is the favored approach and (b) marginal difference between regression methods using physics-based models and MLT. Further research is needed in the selection of MLT to critical aviation operations. As always, the best approach depends on the task, the physical understanding of the problem and the quality and quantity of the available data.

Sridhar, Banavar↗
NASA NTRS2020

Application of Machine Learning Techniques to Aviation Operations: NASA Case Studies

There is an increasing interest in applying methods based on Machine Learning Techniques(MLT) to problems in aviation operations. The current interest is based on developments in Cloud Computing, the availability of open software and the success of MLT in automation, consumer behavior and finance involving large database. Historically aviation operations have been analyzed using physics-based models and provide information for making operational decisions. This talk describes issues to be addressed in applying either model-driven or data-driven methods. Aviation operations involving many decision makers, multiple objectives, poor or unavailable physics-based models and a rich historical database are prime candidates for analysis using data-driven methods. The issues relating to data, feature selection and validation of the models are illustrated by examining case studies of the application of MLT to problems in air traffic management at NASA. Further research is needed in the application of MLT to critical aviation operations. As always, the best approach depends on the task, the physical understanding of the problem and the quality and quantity of the available data.

Sridhar, Banavar↗
DOE OSTI2024

The integration of heterogeneous resources in the CMS Submission Infrastructure for the LHC Run 3 and beyond

While the computing landscape supporting LHC experiments is currently dominated by x86 processors at WLCG sites, this configuration will evolve in the coming years. LHC collaborations will be increasingly employing HPC and Cloud facilities to process the vast amounts of data expected during the LHC Run 3 and the future HL-LHC phase. These facilities often feature diverse compute resources, including alternative CPU architectures like ARM and IBM Power, as well as a variety of GPU specifications. Using these heterogeneous resources efficiently is thus essential for the LHC collaborations reaching their future scientific goals. The Submission Infrastructure (SI) is a central element in CMS Computing, enabling resource acquisition and exploitation by CMS data processing, simulation and analysis tasks. The SI must therefore be adapted to ensure access and optimal utilization of this heterogeneous compute capacity. Some steps in this evolution have been already taken, as CMS is currently using opportunistically a small pool of GPU slots provided mainly at the CMS WLCG sites. Additionally, Power9 processors have been validated for CMS production at the Marconi-100 cluster at CINECA. This note will describe the updated capabilities of the SI to continue ensuring the efficient allocation and use of computing resources by CMS, despite their increasing diversity. The next steps towards a full integration and support of heterogeneous resources according to CMS needs will also be reported.

Pérez-Calero Yzquierdo, Antonio↗