Search NASA⌕ Search

SEARCH · Search NASA

Results for “cloud computing, data cloud”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,477 records · Page 82
All sourcesNASA NTRSDOE OSTI
NASA NTRS1993

Physics of debris clouds from hypervelocity impacts

The protection scheme developed for long duration space platforms relies primarily upon placing thin metal plates or 'bumpers' around flight critical components. The effectiveness of this system is highly dependent upon its ability to break up and redistribute the momentum of any particle which might otherwise strike the outer surface of the spacecraft. Therefore it is of critical importance to design the bumpers such that maximum dispersion of momentum is achieved. This report is devoted to an in-depth study into the design and development of a laboratory instrument which would permit the in-situ monitoring of the momentum distribution as the impact event occurs. A series of four designs were developed, constructed and tested culminating with the working instrument which is currently in use. Each design was individually tested using the Space Environmental Effects Facility (SEEF) at the Marshall Space Flight Center in Huntsville, Alabama. Along with the development of the device, an experimental procedure was developed to assist in the investigation of various bumper materials and designs at the SEEF. Preliminary results were used to compute data which otherwise were not experimentally obtainable. These results were shown to be in relative agreement with previously obtained values derived through other methods. The results of this investigation indicated that momentum distribution could in fact be measured in-situ as the impact event occurred thus giving a more accurate determination of the effects of experimental parameters on the momentum spread. Data produced by the instrument indicated a Gaussian-type momentum distribution. A second apparatus was developed and it was placed before the shield in the line of travel utilized a plate to collect impact debris scattered backwards. This plate had a passage hole in the center to allow the particle to travel through it and impact the proposed shield material. Applying the law of conservation of angular momentum a backward momentum vector was determined from the angular velocity of the plate. The forward scattered and backward scattered momentum values were then analyzed to judge the distribution of debris. Loss of momentum was attributed to the inaccuracies of the means of measurement. Assumptions of symmetrical debris for the forward and backward scattered directions also contributed to this loss.

Zee, Ralph↗
DOE OSTI2025

Point cloud-based diffusion models for the Electron-Ion Collider

At high-energy collider experiments, generative models can be used for a wide range of tasks, including fast detector simulations, unfolding, searches of physics beyond the Standard Model, and inference tasks. In particular, it has been demonstrated that score-based diffusion models can generate high-fidelity and accurate samples of jets or collider events. This work expands on previous generative models in three distinct ways. First, our model is trained to generate entire collider events, including all particle species with complete kinematic information. We quantify how well the model learns event-wide constraints such as the conservation of momentum and discrete quantum numbers. We focus on the events at the future Electron-Ion Collider, but we expect that our results can be extended to proton-proton and heavy-ion collisions. Second, previous generative models often relied on image-based techniques. The sparsity of the data can negatively affect the fidelity and sampling time of the model. We address these issues using point clouds and a novel architecture combining edge creation with transformer modules called Point Edge Transformers. Third, we adapt the foundation model OmniLearn, to generate full collider events. This approach may indicate a transition toward adapting and fine-tuning foundation models for downstream tasks instead of training new models from scratch.

Araz, Jack Y. [Stony Brook Univ., NY (United State↗
DOE OSTI2024

Unveiling Temporal Performance Deviation: Leveraging Clustering in Microservices Performance Analysis

As the market for cloud computing continues to grow, an increasing number of users are deploying applications as microservices. The shift introduces unique challenges in identifying and addressing performance issues, particularly within large and complex infrastructures. To address this challenge, we propose a methodology that unveils temporal performance deviations in microservices by clustering containers based on their performance characteristics at different time intervals. Showcasing our methodology on the Alibaba dataset, we found both stable and dynamic performance patterns, providing a valuable tool for enhancing overall performance and reliability in modern application landscapes.

Clustering↗
DOE OSTI2023

PyFLEXTRKR: a flexible feature tracking Python software for convective cloud analysis

Abstract. This paper describes the new open-source framework PyFLEXTRKR (Python FLEXible object TRacKeR), a flexible atmospheric feature tracking software package with specific capabilities to track convective clouds from a variety of observations and model simulations. This software can track any atmospheric 2D objects and handle merging and splitting explicitly. The package has a collection of multi-object identification algorithms, scalable parallelization options, and has been optimized for large datasets including global high-resolution data. We demonstrate applications of PyFLEXTRKR on tracking individual deep convective cells and mesoscale convective systems from observations and model simulations ranging from large-eddy resolving (∼100s m) to mesoscale (∼10s km) resolutions. Visualization, post-processing, and statistical analysis tools are included in the package. New Lagrangian analyses of convective clouds produced by PyFLEXTRKR applicable to a wide range of datasets and scales facilitate advanced model evaluation and development efforts as well as scientific discovery.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2016

Evaluation of two Vaisala RS92 radiosonde solar radiative dry bias correction algorithms

Solar heating of the relative humidity (RH) probe on Vaisala RS92 radiosondes results in a large dry bias in the upper troposphere. Two different algorithms (Miloshevich et al., 2009, MILO hereafter; and Wang et al., 2013, WANG hereafter) have been designed to account for this solar radiative dry bias (SRDB). These corrections are markedly different with MILO adding up to 40 % more moisture to the original radiosonde profile than WANG; however, the impact of the two algorithms varies with height. The accuracy of these two algorithms is evaluated using three different approaches: a comparison of precipitable water vapor (PWV), downwelling radiative closure with a surface-based microwave radiometer at a high-altitude site (5.3 km m.s.l.), and upwelling radiative closure with the space-based Atmospheric Infrared Sounder (AIRS). The PWV computed from the uncorrected and corrected RH data is compared against PWV retrieved from ground-based microwave radiometers at tropical, midlatitude, and arctic sites. Although MILO generally adds more moisture to the original radiosonde profile in the upper troposphere compared to WANG, both corrections yield similar changes to the PWV, and the corrected data agree well with the ground-based retrievals. The two closure activities – done for clear-sky scenes – use the radiative transfer models MonoRTM and LBLRTM to compute radiance from the radiosonde profiles to compare against spectral observations. Both WANG- and MILO-corrected RHs are statistically better than original RH in all cases except for the driest 30 % of cases in the downwelling experiment, where both algorithms add too much water vapor to the original profile. In the upwelling experiment, the RH correction applied by the WANG vs. MILO algorithm is statistically different above 10 km for the driest 30 % of cases and above 8 km for the moistest 30 % of cases, suggesting that the MILO correction performs better than the WANG in clear-sky scenes. Lastly, the cause of this statistical significance is likely explained by the fact the WANG correction also accounts for cloud cover – a condition not accounted for in the radiance closure experiments.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
NASA NTRS1995

Modeling Ka-band low elevation angle propagation statistics

The statistical variability of the secondary atmospheric propagation effects on satellite communications cannot be ignored at frequencies of 20 GHz or higher, particularly if the propagation margin allocation is such that link availability falls below 99 percent. The secondary effects considered in this paper are gaseous absorption, cloud absorption, and tropospheric scintillation; rain attenuation is the primary effect. Techniques and example results are presented for estimation of the overall combined impact of the atmosphere on satellite communications reliability. Statistical methods are employed throughout and the most widely accepted models for the individual effects are used wherever possible. The degree of correlation between the effects is addressed and some bounds on the expected variability in the combined effects statistics are derived from the expected variability in correlation. Example estimates are presented of combined effects statistics in the Washington D.C. area of 20 GHz and 5 deg elevation angle. The statistics of water vapor are shown to be sufficient for estimation of the statistics of gaseous absorption at 20 GHz. A computer model based on monthly surface weather is described and tested. Significant improvement in prediction of absorption extremes is demonstrated with the use of path weather data instead of surface data.

Russell, Thomas A.↗
NASA NTRS2007

Potential of VIIRS Time Series Data for Aiding the USDA Forest Service Early Warning System for Forest Health Threats: A Gypsy Moth Defoliation Case Study

This report details one of three experiments performed during FY 2007 for the NASA RPC (Rapid Prototyping Capability) at Stennis Space Center. This RPC experiment assesses the potential of VIIRS (Visible/Infrared Imager/Radiometer Suite) and MODIS (Moderate Resolution Imaging Spectroradiometer) data for detecting and monitoring forest defoliation from the non-native Eurasian gypsy moth (Lymantria dispar). The intent of the RPC experiment was to assess the degree to which VIIRS data can provide forest disturbance monitoring information as an input to a forest threat EWS (Early Warning System) as compared to the level of information that can be obtained from MODIS data. The USDA Forest Service (USFS) plans to use MODIS products for generating broad-scaled, regional monitoring products as input to an EWS for forest health threat assessment. NASA SSC is helping the USFS to evaluate and integrate currently available satellite remote sensing technologies and data products for the EWS, including the use of MODIS products for regional monitoring of forest disturbance. Gypsy moth defoliation of the mid-Appalachian highland region was selected as a case study. Gypsy moth is one of eight major forest insect threats listed in the Healthy Forest Restoration Act (HFRA) of 2003; the gypsy moth threatens eastern U.S. hardwood forests, which are also a concern highlighted in the HFRA of 2003. This region was selected for the project because extensive gypsy moth defoliation occurred there over multiple years during the MODIS operational period. This RPC experiment is relevant to several nationally important mapping applications, including agricultural efficiency, coastal management, ecological forecasting, disaster management, and carbon management. In this experiment, MODIS data and VIIRS data simulated from MODIS were assessed for their ability to contribute broad, regional geospatial information on gypsy moth defoliation. Landsat and ASTER (Advanced Spaceborne Thermal Emission and Reflection Radiometer) data were used to assess the quality of gypsy moth defoliation mapping products derived from MODIS data and from simulated VIIRS data. The project focused on use of data from MODIS Terra as opposed to MODIS Aqua mainly because only MODIS Terra data was collected during 2000 and 2001-years with comparatively high amounts of gypsy moth defoliation within the study area. The project assessed the quality of VIIRS data simulation products. Hyperion data was employed to assess the quality of MODIS-based VIIRS simulation datasets using image correlation analysis techniques. The ART (Application Research Toolbox) software was used for data simulation. Correlation analysis between MODIS-simulated VIIRS data and Hyperion-simulated VIIRS data for red, NIR (near-infrared), and NDVI (Normalized Difference Vegetation Index) image data products collectively indicate that useful, effective VIIRS simulations can be produced using Hyperion and MODIS data sources. The r(exp 2) for red, NIR, and NDVI products were 0.56, 0.63, and 0.62, respectively, indicating a moderately high correlation between the 2 data sources. Temporal decorrelation from different data acquisition times and image misregistration may have lowered correlation results. The RPC experiment also generated MODIS-based time series data products using the TSPT (Time Series Product Tool) software. Time series of simulated VIIRS NDVI products were produced at approximately 400-meter resolution GSD (Ground Sampling Distance) at nadir for comparison to MODIS NDVI products at either 250- or 500-meter GSD. The project also computed MODIS (MOD02) NDMI (Normalized Difference Moisture Index) products at 500-meter GSD for comparison to NDVI-based products. For each year during 2000-2006, MODIS and VIIRS (simulated from MOD02) time series were computed during the peak gypsy moth defoliation time frame in the study area (approximately June 10 through July 27). Gypsy moth defoliation mapping products from simated VIIRS and MOD02 time series were produced using multiple methods, including image classification and change detection via image differencing. The latter enabled an automated defoliation detection product computed using percent change in maximum NDVI for a peak defoliation period during 2001 compared to maximum NDVI across the entire 2000-2006 time frame. Final gypsy moth defoliation mapping products were assessed for accuracy using randomly sampled locations found on available geospatial reference data (Landsat and ASTER data in conjunction with defoliation map data from the USFS). Extensive gypsy moth defoliation patches were evident on screen displays of multitemporal color composites derived from MODIS data and from simulated VIIRS vegetation index data. Such defoliation was particularly evident for 2001, although widespread denuded forests were also seen for 2000 and 2003. These visualizations were validated using aforementioned reference data. Defoliation patches were visible on displays of MODIS-based NDVI and NDMI data. The viewing of apparent defoliation patches on all of these products necessitated adoption of a specialized temporal data processing method (e.g., maximum NDVI during the peak defoliation time frame). The frequency of cloud cover necessitated this approach. Multitemporal simulated VIIRS and MODIS Terra data both produced effective general classifications of defoliated forest versus other land cover. For 2001, the MOD02-simulated VIIRS 400-meter NDVI classification produced a similar yet slightly lower overall accuracy (87.28 percent with 0.72 Kappa) than the MOD02 250-meter NDVI classification (88.44 percent with 0.75 Kappa). The MOD13 250-meter NDVI classification had a lower overall accuracy (79.13 percent) and a much lower Kappa (0.46). The report discusses accuracy assessment results in much more detail, comparing overall classification and individual class accuracy statistics for simulated VIIRS 400-meter NDVI, MOD02 250-meter NDVI, MOD02-500 meter NDVI, MOD13 250-meter NDVI, and MOD02 500-meter NDMI classifications. Automated defoliation detection products from simulated VIIRS and MOD02 data for 2001 also yielded similar, relatively high overall classification accuracy (85.55 percent for the VIIRS 400-meter NDVI versus 87.28 percent for the MOD02 250-meter NDVI). In contrast, the USFS aerial sketch map of gypsy moth defoliation showed a lower overall classification accuracy at 73.64 percent. The overall classification Kappa values were also similar for the VIIRS (approximately 0.67 Kappa) versus the MOD02 (approximately 0.72 Kappa) automated defoliation detection product, which were much higher than the values exhibited by the USFS sketch map product (overall Kappa of approximately 0.47). The report provides additional details on the accuracy of automated gypsy moth defoliation detection products compared with USFS sketch maps. The results suggest that VIIRS data can be effectively simulated from MODIS data and that VIIRS data will produce gypsy moth defoliation mapping products that are similar to MODIS-based products. The results of the RPC experiment indicate that VIIRS and MODIS data products have good potential for integration into the forest threat EWS. The accuracy assessment was performed only for 2001 because of time constraints and a relative scarcity of cloud-free Landsat and ASTER data for the peak defoliation period of the other years in the 2000-2006 time series. Additional work should be performed to assess the accuracy of gypsy moth defoliation detection products for additional years.The study area (mid-Appalachian highlands) and application (gypsy moth forest defoliation) are not necessarily representative of all forested regions and of all forest threat disturbance agents. Additional work should be performed on other inland and coastal regions as well as for other major forest threats.

Spruce, Joseph P.↗
NASA NTRS2002

Anomalous Heat Budgets in the Interior Pacific Ocean on Seasonal- to -Timescales and Gyre Spacescales

This study quantifies uncertainties in closing the seasonal cycle of diabatic heat storage over the Pacific Ocean from 20 degrees S to 60 degrees N through the synthesis of World Ocean Circulation Experiment (WOCE) products over 7 years from 1993-1999. We utilize WOCE reanalysis products from the following sources: diabatic heat storage (DHS) from the Scripps Institution of Oceanography (SIO); near-surface geostrophic and Ekman currents from the Earth and Space Research (ESR); and air-sea heat fluxes from Comprehensive Ocean-Atmosphere Data Set (COADS), National Centers for Environmental Prediction (NCEP), and European Center for Mid-Range Weather Forecasts (ECMWF). We interpolate these products onto a common grid, allowing the seasonal cycle of DHS to be modeled for comparison with that observed. Everywhere latent heat flux residuals dominate sensible heat flux residuals and shortwave heat flux residuals dominate longwave heat flux residuals, both comparable in magnitude to the residual horizontal heat advection. We find the root-mean-square (RMS) of the differences between observed and model residual DHS tendencies to be less than 15 W per square meters everywhere except in the Kuroshio extension. Comparable COADS and NCEP products perform better than ECMWF products in the extra-tropics, while the NCEP product performs best in the tropics. Radiative and turbulent air-sea heat flux residuals computed from ship-born measurements perform better than those computed from satellite cloud and wind measurements. Since the RMS differences derive largely from biases in measured wind speed and cloud fraction, least-squares minimization is used to correct the residual Ekman heat advection and air-sea heat flux. Minimization reduces RMS differences less than 5 W per square meters except in the Kuroshio extension, suggesting how winds, clouds, and exchange coefficients in the NCEP, ECMWF, and ESR products can be improved.

White, Warren↗
NASA NTRS2019

Accelerated Simulation of Air Pollution Using NVIDIA RAPIDS

Atmospheric chemistry models are a central tool to study and forecast the impact of air pollution on the environment, vegetation, and human health. However, the numerical simulation of chemical kinetics is computationally expensive due to the stiffness of the system of ordinary differential equations that describes atmospheric chemistry. Here we present an alternative approach to the computation of atmospheric chemistry based on machine learning. Our training data set is produced using the NASA Goddard Earth Observing System (GEOS) model with GEOS-Chem chemistry, run on the NASA Center for Climate Simulation (NCCS) Discover supercomputing cluster on 384 Intel Xeon Haswell cores. This model spends more than 50% of total run time on solving atmospheric chemistry. The data set contains as input features the air pollution concentrations before solving the differential equations, together with some key physical parameters such as temperature and sun intensity. As target variables we define the air pollution concentrations after solving the differential equations. Using Dask-cuDF and Dask-XGBoost on the NVIDIA RAPIDS platform on 8 Tesla V100 GPUs, we generate from this training set gradient boosted decision tree models that can reproduce the simulation of chemical kinetics. We do this on the NCCS Advanced Data Analytics Platform (ADAPT) science cloud environment. Our application takes full advantage of recent advances in Dask-XGBoost, such as multi-node and multi-GPU scaling for distributed training with large data sets. The increase in training data size enabled by this is critical to capture the full range of chemical environments encountered across the globe and all annual seasons.The boosted tree models offer good predictability and show many of the features of the full chemistry reference simulation. Further improvements can be achieved through mass balance considerations and by accounting for error correlations. We incorporate the boosted tree models into the GEOS reference model using XGBoost's C API. This enables a seamless integration of the GPU trained models into GEOS-Chem, which is written in Fortran and optimized for use in a massively parallel CPU environment. We show the benefits of this approach and discuss the potential speedup of this machine learning accelerated atmospheric chemistry model.

Keller, Christoph A.↗
DOE OSTI2016

A path towards uncertainty assignment in an operational cloud-phase algorithm from ARM vertically pointing active sensors

Abstract. Knowledge of cloud phase (liquid, ice, mixed, etc.) is necessary to describe the radiative impact of clouds and their lifetimes, but is a property that is difficult to simulate correctly in climate models. One step towards improving those simulations is to make observations of cloud phase with sufficient accuracy to help constrain model representations of cloud processes. In this study, we outline a methodology using a basic Bayesian classifier to estimate the probabilities of cloud-phase class from Atmospheric Radiation Measurement (ARM) vertically pointing active remote sensors. The advantage of this method over previous ones is that it provides uncertainty information on the phase classification. We also test the value of including higher moments of the cloud radar Doppler spectrum than are traditionally used operationally. Using training data of known phase from the Mixed-Phase Arctic Cloud Experiment (M-PACE) field campaign, we demonstrate a proof of concept for how the method can be used to train an algorithm that identifies ice, liquid, mixed phase, and snow. Over 95 % of data are identified correctly for pure ice and liquid cases used in this study. Mixed-phase and snow cases are more problematic to identify correctly. When lidar data are not available, including additional information from the Doppler spectrum provides substantial improvement to the algorithm. This is a first step towards an operational algorithm and can be expanded to include additional categories such as drizzle with additional training data.

54 ENVIRONMENTAL SCIENCES↗
NASA NTRS1997

Molecular Hydrogen in Diffuse Interstellar Clouds of Arbitrary Three-Dimensional Geometry

We have constructed three-dimensional models for the equilibrium abundance of molecular hydrogen in diffuse interstellar clouds of arbitrary geometry that are illuminated by ultraviolet radiation. The position-dependent photodissociation rate of H2 in such clouds was computed with a 26 ray approximation to model the attenuation of the incident ultraviolet radiation field by dust and by H2 line absorption. We have applied our modeling technique to the isolated diffuse cloud G236+39, assuming that the cloud has a constant density and that the thickness of the cloud along the line of sight is at every point proportional to the 100 micron continuum intensity measured by IRAS. We find that our model can successfully account for observed variations in the ratio of 100 micron continuum intensity to H I column density, with larger values of that ratio occurring along lines of sight in which the molecular hydrogen fraction is expected to be the largest. Using a standard chi square analysis to assess the goodness of fit of our models, we find (at the 60 a level) that a three-dimensional model is more successful in matching the observational data than a one-dimensional model in which the geometrical extent of the cloud along the line of sight is assumed to be much smaller than its extent in the plane of the sky. If D is the distance to G236 + 39, and given standard assumptions about the rate of grain-catalyzed H2 formation, we find that the cloud has an extent along the line of sight that is 0.9 +/- 0.1 times its mean extent projected onto the plane of the sky and a gas density of 53 +/- 8 (100 pc/D) H nuclei/cc and is illuminated by a radiation field of 1.1 +/- 0.2 (100 pc/D) times the mean interstellar radiation field. The derived 100 micron emissivity per nucleon is 1.13 +/- 0.06 x 10(exp -20) MJy/sr sq cm.

Spaans, Marco↗
NASA NTRS2012

Resolution Enhancement of MODIS-Derived Water Indices for Studying Persistent Flooding

Monitoring coastal marshes for persistent flooding and salinity stress is a high priority issue in Louisiana. Remote sensing can identify environmental variables that can be indicators of marsh habitat conditions, and offer timely and relatively accurate information for aiding wetland vegetation management. Monitoring activity accuracy is often limited by mixed pixels which occur when areas represented by the pixel encompasses more than one cover type. Mixtures of marsh grasses and open water in 250m Moderate Resolution Imaging Spectroradiometer (MODIS) data can impede flood area estimation. Flood mapping of such mixtures requires finer spatial resolution data to better represent the cover type composition within 250m MODIS pixel. Fusion of MODIS and Landsat can improve both spectral and temporal resolution of time series products to resolve rapid changes from forcing mechanisms like hurricane winds and storm surge. For this study, using a method for estimating sub-pixel values from a MODIS time series of a Normalized Difference Water Index (NDWI), using temporal weighting, was implemented to map persistent flooding in Louisiana coastal marshes. Ordinarily NDWI computed from daily 250m MODIS pixels represents a mixture of fragmented marshes and water. Here, sub-pixel NDWI values were derived for MODIS data using Landsat 30-m data. Each MODIS pixel was disaggregated into a mixture of the eight cover types according to the classified image pixels falling inside the MODIS pixel. The Landsat pixel means for each cover type inside a MODIS pixel were computed for the Landsat data preceding the MODIS image in time and for the Landsat data succeeding the MODIS image. The Landsat data were then weighted exponentially according to closeness in date to the MODIS data. The reconstructed MODIS data were produced by summing the product of fractional cover type with estimated NDWI values within each cover type. A new daily time series was produced using both the reconstructed 250-m MODIS, with enhanced features, and the approximated daily 30-m high-resolution image based on Landsat data. The algorithm was developed and tested over the Calcasieu-Sabine Basin, which was heavily inundated by storm surge from Hurricane Ike to study the extent and duration of flooding following the storm. Time series for 2000-2009, covering flooding events by Hurricane Rita in 2005 and Hurricane Ike in 2008, were derived. High resolution images were formed for all days in 2008 between the first cloud free Landsat scene and the last cloud-free Landsat scene. To refine and validate flooding maps, each time series was compared to Louisiana Coastwide Reference Monitoring System (CRMS) station water levels adjusted to marsh to optimize thresholds for MODIS-derived time series of NDWI. Seasonal fluctuations were adjusted by subtracting ten year average NDWI for marshes, excluding the hurricane events. Results from different NDWI indices and a combination of indices were compared. Flooding persistence that was mapped with higher-resolution data showed some improvement over the original MODIS time series estimates. The advantage of this novel technique is that improved mapping of extent and duration of inundation can be provided.

Underwood, L. W.↗
DOE OSTI2019

R and DE Robotic Sensor Intern Team: Advanced Sensor Suite Prototype

The Department of Energy (DOE) nuclear complex includes many hazardous and challenging environments that must be inspected remotely and routinely for structural integrity evaluation and long-term planning purposes. The H-Canyon Air Exhaust (HCAEX) Tunnel is an example of such an environment where a biannual inspection is currently performed. Presently inspections are performed by teleoperated robotic crawlers which provide low-res video feedback to the structural engineers. Advanced sensors: including Lidar and panospheric hi-res cameras have the potential to provide additional valuable data in future inspections. The primary goal of the sensor suite is to provide an internal 3D map of the HCAEX tunnel. Possible sensors were a stereo camera, a 3D Lidar, or a 2D Lidar that would need to be spun around. The issues with the stereo camera were that it does not have a large field-of-view and that its depth information was not nearly as accurate as even the baseline Lidars. The Hokuyo UTM-30LX-EW shown below provides a 270 deg. FOV with high accuracy and resolution while the largest FOV on a 3D Lidar is 40 deg. This was the reason a 2D Lidar was chosen over the 3D Lidar. Even though the 2D Lidar requires additional components and software to generate a 3D map, its FOV advantage is too great given that the HCAEX tunnel is a relatively confined space. Along with the Lidar, a camera sensor, IMU sensor, pan-tit unit, and on board computer needed to be identified. The two panospheric cameras allow for full 360 deg. FOV. The IMU will give data on the state of the robot inside the tunnel. The computer is used to gather and process data on board and to control the pan-tilt system. Mounts were designed and 3D printed to build the prototype of the sensor suite. It was designed to have the Lidar on top With its full FOV free while having the camera below and as close to together as possible while still leaving much of their FOV visible. The design also managed the cables such that they were unlikely to be damaged if in tension and so that they did not intrude much on the cameras vision. The sensor tree developed for the H-Canyon tunnel inspection project produces a large quantity of camera and Lidar data. a postprocessing pipeline was created to facilitate the analysis of these data. The pipeline follows the following routine: 1) Registration - Salient local features in subsequent scans are identified and used to align the scans With one another. This allows production of a single cohesive map of the tunnel. 2) Sensor Fusion - RGB camera data is 'painted' onto the depth data created by the Lidar. 3) Segmentation - Individual surfaces from within the scene are segmented out separately for individual analysis. 4) Degradation Analysis - Automatic mapping of local deviation from the plane in position or surface normal is performed, as is detection of exposed rebar using Lidar return intensity. 2) Obstacle Detection - Lidar data is also used in real time to detect obstacles in the environment. This could give feedback to the operator to successfully drive in the tunnel. 6) Graphic User Interface - a custom GUI was created to facilitate customer use of the system. The system provides real-time feedback from both the cameras and Lidar. The Lidar point-cloud data is also used for obstacle detection. As the Lidar spins, various algorithms are run to analyze the current 'slice' and obstacles in the environment are determined and registered on a cost-map. A Graphical User Interface (GUI) was developed to allow the user to easily command the crawler to set the rotational speed of the Lidar, perform a scan, and register and save the 3D generated cloud after a scan is done. For the good of the crawler, a critical aspect of the GUI design is to allow the user to cancel a scan any time after it is commanded and before it is saved. The sensor trees depth accuracy was evaluated in comparison to a high end commercial FARO Terrestrial Laser Scanner (TLS) with specifications for millimeter accuracy at distances above 10 m. For the purposes of these tests, the depth measurements produced by the TLS were taken as ground truth. Example test wall panels were scanned with the TLS and postprocessed with its proprietary software, and subsequently scanned with the sensor tree system. The two scans were manually aligned and for each point in the output cloud from the sensor tree, the distance to the nearest ground truth point in the FARO cloud was determined. 98.7% of all produced points were within the specified target range of 6 mm, with an average deviation of 1.75 mm. The test walls used have an overall in-wall depth variation of around 40 mm.

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗
DOE OSTI2019

Regional Moisture Budget and Land-Atmosphere Coupling Over the U.S. Southern Great Plains Inferred From the ARM Long-Term Observations

Ten–year warm–season (May–August) observations at the Atmospheric Radiation Measurement (ARM) Southern Great Plains (SGP) site are used to examine the large–scale atmospheric moisture budget and the strength of land–atmosphere coupling. Atmospheric moisture budget components, computed from the ARM variational analysis data, are compared between different wet/dry scenarios and convection regimes. Consistent with previous studies, large–scale moisture flux convergence dominates the SGP regional moisture budget on the daily time scale, but surface evaporation does play a more prominent role on late–afternoon deep convection days. Moreover, the relationship between moisture flux convergence and precipitation increases significantly from dry year to wet years. The land–atmosphere coupling strength is quantified for local convective events within the “Local L–A Coupling Process Chain” (Santanello et al., 2018, https://doi.org/10.1175/BAMS–D–17–0001.1). The impact of land surface on the evolution of planetary boundary layer is highly dependent on the vegetation leaf area index. A significantly larger surface sensible heat flux is found over the SGP forest region in midsummer, which is accompanied by a much higher planetary boundary layer development and a large increase in shallow cumulus cloud fraction. Here the significant relationship between afternoon precipitation and surface turbulent flux only exists in the northeast part of the ARM SGP site, which is mainly covered by grassland/pasture.

58 GEOSCIENCES↗
DOE OSTI2014

Globus platform-as-a-service for collaborative science applications

Globus, developed as Software-as-a-Service (SaaS) for research data management, also provides APIs that constitute a flexible and powerful Platform-as-a-Service (PaaS) to which developers can outsource data management activities such as transfer and sharing, as well as identity, profile and group management. By providing these frequently important but always challenging capabilities as a service, accessible over the network, Globus PaaS streamlines web application development and makes it easy for individuals, teams, and institutions to create collaborative applications such as science gateways for science communities. We introduce the capabilities of this platform and review representative applications.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗
DOE OSTI2020

AI-Science for Performance Optimization and Diagnosis of Science Instrument Federations

Next generation of science workflows are expected to be executed over complex federations composed of supercomputers, science instruments, storage systems and networks, with new additions of the edge and cloud systems and services. The sheer complexity of these multi-domain federations makes it hard to manage them and optimize their performance, as small impedance mismatches (that can dynamically develop between systems) could drastically degrade the entire federation performance. Recent proliferation of Software Defined Everything (SDX) technologies combined with containerization frameworks provide custom instruments that can monitor and collect critical measurements at various levels to support diagnoses and performance optimization; but their data too enormous for human operators and analysts to process and generate decisions. Machine Learning (ML) methods that extract critical parameters, relationships and trends from the data offer general solutions. Artificial Intelligence (AI) and ML methods must be custom-developed for these problems based on solid, rigorous foundations, since black-box approaches are often ineffective and unsound.We propose to develop comprehensive AI-Science for the performance of science federations to (i) monitor and control storage, networks, experiments, and computing systems across multiple domains via softwarization layers, at speeds and scales orders of magnitude superior to current practice, (ii) optimally realize and orchestrate complex workflows with high performance by using dynamic state and performance estimation methods, and (iii) aggregate measurements across sites and time to develop infrastructure-level profiles, optimizations and diagnoses using AI-Science based on foundational principles from ML, game theory, and information fusion areas.

Rao, Nageswara↗
DOE OSTI2018

Deep learning to represent subgrid processes in climate models

The representation of nonlinear subgrid processes, especially clouds, has been a major source of uncertainty in climate models for decades. Cloud-resolving models better represent many of these processes and can now be run globally but only for short-term simulations of at most a few years because of computational limitations. Here we demonstrate that deep learning can be used to capture many advantages of cloud-resolving modeling at a fraction of the computational cost. We train a deep neural network to represent all atmospheric subgrid processes in a climate model by learning from a multiscale model in which convection is treated explicitly. The trained neural network then replaces the traditional subgrid parameterizations in a global general circulation model in which it freely interacts with the resolved dynamics and the surface-flux scheme. The prognostic multiyear simulations are stable and closely reproduce not only the mean climate of the cloud-resolving simulation but also key aspects of variability, including precipitation extremes and the equatorial wave spectrum. Furthermore, the neural network approximately conserves energy despite not being explicitly instructed to. Finally, we show that the neural network parameterization generalizes to new surface forcing patterns but struggles to cope with temperatures far outside its training manifold. Our results show the feasibility of using deep learning for climate model parameterization. In a broader context, we anticipate that data-driven Earth system model development could play a key role in reducing climate prediction uncertainty in the coming decade.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2025

HPC Campaign Management: Remote data access with user-defined error bound using ADIOS and ZFP

Remote access to large-scale scientific datasets, like those generated by combustion simulations or other high-performance computing (HPC) applications, presents a significant challenge. Downloading entire datasets is often impractical due to their size and the bandwidth limitations of typical networks. To address this challenge, we propose a novel approach that enables efficient remote access to large datasets distributed across multiple facilities. Our method enables technologies to download only the data values of a select variable, in a select region of interest, to a user-defined accuracy. For this purpose, we extended the ADIOS IO library to provide read functions with user-defined accuracy, a remote data server that understands multidimensional selections of specific variables, steps and accuracy from an ADIOS dataset, and which uses lossy compression on the remote site to reduce the data to be transferred back to the client. In addition, our extension of the ADIOS library collects metadata from multiple datasets in small files called Campaign Archives, which can be shared among project participants on any HPC, cloud or laptop, and which can easily facilitate the discovery of content and pointers to the data location as well as remote access to the data by local tools as if data was local. This feature called Campaign Management, enables a group of scientists to manage related datasets stored in multiple files, across multiple facilities as if it was in a single file/database. We demonstrate the effectiveness of our approach using a 1.5 TB dataset from the S3D combustion simulation on Frontier at the Oak Ridge Leadership Facility. Even a single variable from this dataset, at 64 GB, is too large to be processed on a standard laptop. We show two different reading patterns for 2D plots and 3D visualization, with careful settings that a scientist studying combustion data would do and show that running the same Python scripts on Frontier directly takes comparable time than running them on the local laptop with remote access to the data on Frontier.

Podhorszki, Norbert [ORNL] (ORCID:000000019647542X↗