Search NASA⌕ Search

SEARCH · Search NASA

Results for “cloud computing, data cloud”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 469 records · Page 26
All sourcesNASA NTRSDOE OSTI
DOE OSTI2021

Workflows Community Summit: Tightening the Integration between Computing Facilities and Scientific Workflows

Scientific workflows are used almost universally across science domains for solving complex and largescale computing and data analysis problems. The importance of workflows is highlighted by the fact that they have underpinned some of the most significant discoveries of the past decades. Many of these workflows have significant computational, storage, and communication demands, and thus must execute on a range of large-scale computer systems, from local clusters to public clouds and upcoming exascale HPC platforms. Managing these executions is often a significant undertaking, requiring a sophisticated and versatile software infrastructure. Historically, infrastructures for workflow execution consisted of complex, integrated systems, developed in-house by workflow practitioners with strong dependencies on a range of legacy technologies—even including sets of ad hoc scripts. Due to the increasing need to support workflows, dedicated workflow systems were developed to provide abstractions for creating, executing, and adapting workflows conveniently and efficiently while ensuring portability. While these efforts are all worthwhile individually, there are now hundreds of independent workflow systems. These workflow systems are created and used by thousands of researchers and developers, leading to a rapidly growing corpus of workflows research publications. The resulting workflow system technology landscape is fragmented, which may present significant barriers for future workflow users due to many seemingly comparable, yet usually mutually incompatible, systems that exist. In order to tackle some of the challenges described above, the DOE-funded ExaWorks and NSF-funded WorkflowsRI projects have organized in 2021 a series of events entitled the “Workflows Community Summit”. The third edition of the “Workflows Community Summit” explored workflows challenges and opportunities from the perspective of computing centers and facilities. This third summit builds on two prior summits (https://workflowsri.org/summits) that (i) established a high level vision for workflows research; and (ii) explored technical approaches for realizing that vision. The third summit brought together a small group of facilities representatives with the aim to understand how workflows are currently being used at each facility, how facilities would like to interact with workflow developers and users, how workflows fit with facility roadmaps, and what opportunities there are for tighter integration between facilities and workflows. This report documents and organizes the wealth of information provided by the participants before, during, and after the summit.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Detection of Outliers in LiDAR Data Acquired by Multiple Platforms over Sorghum and Maize

High-resolution point cloud data acquired with a laser scanner from any platform contain random noise and outliers. Therefore, outlier detection in LiDAR data is often necessary prior to analysis. Applications in agriculture are particularly challenging, as there is typically no prior knowledge of the statistical distribution of points, plant complexity, and local point densities, which are crop-dependent. The goals of this study were first to investigate approaches to minimize the impact of outliers on LiDAR acquired over agricultural row crops, and specifically for sorghum and maize breeding experiments, by an unmanned aerial vehicle (UAV) and a wheel-based ground platform; second, to evaluate the impact of existing outliers in the datasets on leaf area index (LAI) prediction using LiDAR data. Two methods were investigated to detect and remove the outliers from the plant datasets. The first was based on surface fitting to noisy point cloud data via normal and curvature estimation in a local neighborhood. The second utilized the PointCleanNet deep learning framework. Both methods were applied to individual plants and field-based datasets. To evaluate the method, an F-score was calculated for synthetic data in the controlled conditions, and LAI, the variable being predicted, was computed both before and after outlier removal for both scenarios. Results indicate that the deep learning method for outlier detection is more robust than the geometric approach to changes in point densities, level of noise, and shapes. The prediction of LAI was also improved for the wheel-based vehicle data based on the coefficient of determination (R2) and the root mean squared error (RMSE) of the residuals before and after the removal of outliers.

36 MATERIALS SCIENCE↗
DOE OSTI2017

Tracing the Magnetic Field of IRDC G028.23-00.19 Using NIR Polarimetry

The importance of the magnetic ( B ) field in the formation of infrared dark clouds (IRDCs) and massive stars is an ongoing topic of investigation. We studied the plane-of-sky B field for one IRDC, G028.23-00.19, to understand the interaction between the field and the cloud. We used near-IR background starlight polarimetry to probe the B field and performed several observational tests to assess the field importance. The polarimetric data, taken with the Mimir instrument, consisted of H -band and K -band observations, totaling 17,160 stellar measurements. We traced the plane-of-sky B -field morphology with respect to the sky-projected cloud elongation. We also found the relationship between the estimated B -field strength and gas volume density, and we computed estimates of the normalized mass-to-magnetic flux ratio. The B -field orientation with respect to the cloud did not show a preferred alignment, but it did exhibit a large-scale pattern. The plane-of-sky B -field strengths ranged from 10 to 165 μ G, and the B -field strength dependence on density followed a power law with an index consistent with 2/3. The mass-to-magnetic flux ratio also increased as a function of density. The relative orientations and relationship between the B field and density imply that the B field was not dynamically important in the formation of the IRDC. The increase in mass-to-flux ratio as a function of density, though, indicates a dynamically important B field. Therefore, it is unclear whether the B field influenced the formation of G28.23. However, it is likely that the presence of the IRDC changed the local B -field morphology.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2023

Advanced Computing Annual Report 2022

National Renewable Energy Laboratory's (NREL) high-performance computing (HPC), efficient data center operation, and state-of-the-art data visualization powered over 300 clean energy projects in Fiscal Year 2022 (FY22). 555 scientists and engineers utilized HPC resources to assess and integrate data into their research. The report highlights the arrival of new HPC Kestrel in 2023, the contributions of the Insight Center, and researchers looking into bias in artificial intelligence, alongside key research collaborations using advanced computing.

advanced computing↗
DOE OSTI2020

Automated construction of clear-sky dictionary from all-sky imager data

All-sky imagers (ASIs) have significant promise as scalable sensors for short-term solar irradiance forecasting. Many of the current computational techniques that use ASIs for this purpose rely on collections of clear-sky images indexed by time of day, solar angle, or both, called clear-sky dictionaries (CSDs). These CSDs act as baselines against which images can be compared to locate and classify clouds within the image frame. CSDs are often compiled by hand, where individuals visually inspect collections of images one at a time to find clear-sky images. This process is not scalable, and it is prone to error. This paper proposes an automated, nonparametric alternative that uses the principles of digital image processing to find clear-sky images within a set of images taken over several days. We use ground-truth measurements of the clearness index to assess the performance of our method, and we show that the images it selects accurately correspond to clear-sky images. We also compare our proposal, which is nonparametric, with a state-of-the-art parametric method. The numerical results indicate that the performance of the method proposed here is superior.

14 SOLAR ENERGY↗
DOE OSTI2012

Pulse Jet Mixing Tests With Noncohesive Solids (Revised)

This report summarizes results from pulse jet mixing (PJM) tests with noncohesive solids in Newtonian liquid. The tests were conducted during FY 2007 and 2008 to support the design of mixing systems for the Hanford Waste Treatment and Immobilization Plant (WTP). Tests were conducted at three geometric scales using noncohesive simulants, and the test data were used to develop models predicting two measures of mixing performance for full-scale WTP vessels. The models predict the cloud height (the height to which solids will be lifted by the PJM action) and the critical suspension velocity (the minimum velocity needed to ensure all solids are suspended off the floor, though not fully mixed). From the cloud height, the concentration of solids at the pump inlet can be estimated. The predicted critical suspension velocity for lifting all solids is not precisely the same as the mixing requirement for 'disturbing' a sufficient volume of solids, but the values will be similar and closely related. These predictive models were successfully benchmarked against larger scale tests and compared well with results from computational fluid dynamics simulations. The application of the models to assess mixing in WTP vessels is illustrated in examples for 13 distinct designs and selected operational conditions. The values selected for these examples are not final; thus, the estimates of performance should not be interpreted as final conclusions of design adequacy or inadequacy. However, this work does reveal that several vessels may require adjustments to design, operating features, or waste feed properties to ensure confidence in operation. The models described in this report will prove to be valuable engineering tools to evaluate options as designs are finalized for the WTP. Revision 1 refines data sets used for model development and summarizes models developed since the completion of Revision 0.

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗
DOE OSTI2014

Core-halo age gradients and star formation in the Orion Nebula and NGS 2024 young stellar clusters

We analyze age distributions of two nearby rich stellar clusters, the NGC 2024 (Flame Nebula) and Orion Nebula cluster (ONC) in the Orion molecular cloud complex. Our analysis is based on samples from the MYStIX survey and a new estimator of pre-main sequence (PMS) stellar ages, Age{sub JX} , derived from X-ray and near-infrared photometric data. To overcome the problem of uncertain individual ages and large spreads of age distributions for entire clusters, we compute median ages and their confidence intervals of stellar samples within annular subregions of the clusters. We find core-halo age gradients in both the NGC 2024 cluster and ONC: PMS stars in cluster cores appear younger and thus were formed later than PMS stars in cluster peripheries. These findings are further supported by the spatial gradients in the disk fraction and K-band excess frequency. Our age analysis is based on Age{sub JX} estimates for PMS stars and is independent of any consideration of OB stars. The result has important implications for the formation of young stellar clusters. One basic implication is that clusters form slowly and the apparent age spreads in young stellar clusters, which are often controversial, are (at least in part) real. The result further implies that simple models where clusters form inside-out are incorrect and more complex models are needed. We provide several star formation scenarios that alone or in combination may lead to the observed core-halo age gradients.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2024

Foldy: An open-source web application for interactive protein structure analysis

Foldy is a cloud-based application that allows non-computational biologists to easily utilize advanced AI-based structural biology tools, including AlphaFold and DiffDock. With many deployment options, it can be employed by individuals, labs, universities, and companies in the cloud without requiring hardware resources, but it can also be configured to utilize locally available computers. Foldy enables scientists to predict the structure of proteins and complexes up to 6000 amino acids with AlphaFold, visualize Pfam annotations, and dock ligands with AutoDock Vina and DiffDock. In our manuscript, we detail Foldy’s interface design, deployment strategies, and optimization for various user scenarios. We demonstrate its application through case studies including rational enzyme design and analyzing proteins with domains of unknown function. Furthermore, we compare Foldy’s interface and management capabilities with other open and closed source tools in the field, illustrating its practicality in managing complex data and computation tasks. Our manuscript underlines the benefits of Foldy as a day-to-day tool for life science researchers, and shows how Foldy can make modern tools more accessible and efficient.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2012

THE MASS-LOSS RETURN FROM EVOLVED STARS TO THE LARGE MAGELLANIC CLOUD. VI. LUMINOSITIES AND MASS-LOSS RATES ON POPULATION SCALES

We present results from the first application of the Grid of Red Supergiant and Asymptotic Giant Branch ModelS (GRAMS) model grid to the entire evolved stellar population of the Large Magellanic Cloud (LMC). GRAMS is a pre-computed grid of 80,843 radiative transfer models of evolved stars and circumstellar dust shells composed of either silicate or carbonaceous dust. We fit GRAMS models to {approx}30,000 asymptotic giant branch (AGB) and red supergiant (RSG) stars in the LMC, using 12 bands of photometry from the optical to the mid-infrared. Our published data set consists of thousands of evolved stars with individually determined evolutionary parameters such as luminosity and mass-loss rate. The GRAMS grid has a greater than 80% accuracy rate discriminating between oxygen- and carbon-rich chemistry. The global dust injection rate to the interstellar medium (ISM) of the LMC from RSGs and AGB stars is on the order of 2.1 Multiplication-Sign 10{sup -5} M{sub Sun} yr{sup -1}, equivalent to a total mass injection rate (including the gas) into the ISM of {approx}6 Multiplication-Sign 10{sup -3} M{sub Sun} yr{sup -1}. Carbon stars inject two and a half times as much dust into the ISM as do O-rich AGB stars, but the same amount of mass. We determine a bolometric correction factor for C-rich AGB stars in the K{sub s} band as a function of J - K{sub s} color, BC{sub K{sub s}}= -0.40(J-K{sub s}){sup 2} + 1.83(J-K{sub s}) + 1.29. We determine several IR color proxies for the dust mass-loss rate (M-dot{sub d}) from C-rich AGB stars, such as log M-dot{sub d} = (-18.90/((K{sub s}-[8.0])+3.37) - 5.93. We find that a larger fraction of AGB stars exhibiting the 'long-secondary period' phenomenon are more O-rich than stars dominated by radial pulsations, and AGB stars without detectable mass loss do not appear on either the first-overtone or fundamental-mode pulsation sequences.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2024

2024 Atmospheric Radiation Measurement (ARM) Annual Report

This report provides an overview of the Atmospheric Radiation Measurement (ARM) user facility and a sample of achievements for fiscal year 2024 (FY2024). Strong collaborations between nine DOE national laboratories enable ARM to successfully operate in remote locations around the world. This unique partnership supports the DOE mission to provide for the energy security of the nation. Without the support of the following laboratories, ARM would not be the state-of-the-art facility that it is today. ARM is a multi-laboratory, U.S. Department of Energy (DOE) Office of Science user facility and a key contributor to national and international atmospheric and climate research efforts. ARM offers scientists cutting-edge, ground-based observatories, aerial observation capabilities, and high-performance computing. ARM’s capabilities have enabled more than 30 years of continuous measurements of cloud and aerosol properties and their effects on Earth’s energy balance.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2022

Evolution of HEP Processing Frameworks

HEP data-processing software must support the disparate physics needs of many experiments. For both collider and neutrino environments, HEP experiments typically use data-processing frameworks to manage the computational complexities of their large-scale data processing needs. Data-processing frameworks are being faced with new challenges this decade. The computing landscape has changed from the past three decades of homogeneous single-core x86 batch jobs running on grid sites. Frameworks must now work on a heterogeneous mixture of different platforms: multi-core machines, different CPU architectures, and computational accelerators; and different computing sites: grid, cloud, and high-performance computing. We describe these challenges in more detail and how frameworks may confront them. Given their historic success, frameworks will continue to be critical software systems that enable HEP experiments to meet their computing needs. Frameworks have weathered computing revolutions in the past; they will do so again with support from the HEP community

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2012

ARM - Midlatitude Continental Convective Clouds

Convective processes play a critical role in the Earth's energy balance through the redistribution of heat and moisture in the atmosphere and their link to the hydrological cycle. Accurate representation of convective processes in numerical models is vital towards improving current and future simulations of Earths climate system. Despite improvements in computing power, current operational weather and global climate models are unable to resolve the natural temporal and spatial scales important to convective processes and therefore must turn to parameterization schemes to represent these processes. In turn, parameterization schemes in cloud-resolving models need to be evaluated for their generality and application to a variety of atmospheric conditions. Data from field campaigns with appropriate forcing descriptors have been traditionally used by modelers for evaluating and improving parameterization schemes.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2014

GlyQ-IQ: Glycomics Quintavariate-Informed Quantification with High-Performance Computing and GlycoGrid 4D Visualization

Dense LC-MS datasets have convoluted extracted ion chromatograms with multiple chromatographic peaks that cloud the differentiation between intact compounds with their overlapping isotopic distributions, peaks due to insource ion fragmentation, and noise. Making this differentiation is critical in glycomics datasets because chromatographic peaks correspond to different intact glycan structural isomers. The GlyQ-IQ software is targeted chromatography centric software designed for chromatogram and mass spectra data processing and subsequent glycan composition annotation. The targeted analysis approach offers several key advantages to LC-MS data processing and annotation over traditional algorithms. A priori information about the individual target’s elemental composition allows for exact isotope profile modeling for improved feature detection and increased sensitivity by focusing chromatogram generation and peak fitting on the isotopic species in the distribution having the highest intensity and data quality. Glycan target annotation is corroborated by glycan family relationships and in source fragmentation detection. The GlyQ-IQ software is developed in this work (Part 1) and was used to profile N-glycan compositions from human serum LC-MS Datasets. The companion manuscript GlyQ-IQ Part 2 discusses developments in human serum N-glycan sample preparation, glycan isomer separation, and glycan electrospray ionization. A case study is presented to demonstrate how GlyQ-IQ identifies and removes confounding chromatographic peaks from high mannose glycan isomers from human blood serum. In addition, GlyQ-IQ was used to generate a broad N-glycan profile from a high resolution (100K/60K) nESI-LS-MS/MS dataset including CID and HCD fragmentation acquired on a Velos Pro Mass spectrometer. 101 glycan compositions and 353 isomer peaks were detected from a single sample. 99% of the GlyQ-IQ glycan-feature assignments passed manual validation and are backed with high resolution mass spectra and mass accuracies less than 7 ppm.

Environmental Molecular Sciences Laboratory↗
DOE OSTI2014

Modeling the Virtual Machine Launching Overhead under Fermicloud

FermiCloud is a private cloud developed by the Fermi National Accelerator Laboratory for scientific workflows. The Cloud Bursting module of the FermiCloud enables the FermiCloud, when more computational resources are needed, to automatically launch virtual machines to available resources such as public clouds. One of the main challenges in developing the cloud bursting module is to decide when and where to launch a VM so that all resources are most effectively and efficiently utilized and the system performance is optimized. However, based on FermiCloud’s system operational data, the VM launching overhead is not a constant. It varies with physical resource (CPU, memory, I/O device) utilization at the time when a VM is launched. Hence, to make judicious decisions as to when and where a VM should be launched, a VM launch overhead reference model is needed. The paper is to develop a VM launch overhead reference model based on operational data we have obtained on FermiCloud and uses the reference model to guide the cloud bursting process.

Garzoglio, Gabriele↗
DOE OSTI2019

Quantum Associative Memory in Hep Track Pattern Recognition

We have entered the Noisy Intermediate-Scale Quantum Era. A plethora of quantum processor prototypes allow evaluation of potential of the Quantum Computing paradigm in applications to pressing computational problems of the future. Growing data input rates and detector resolution foreseen in High-Energy LHC (2030s) experiments expose the often high time and/or space complexity of classical algorithms. Quantum algorithms can potentially become the lower-complexity alternatives in such cases. In this work we discuss the potential of Quantum Associative Memory (QuAM) in the context of LHC data triggering. We examine the practical limits of storage capacity, as well as store and recall errorless efficiency, from the viewpoints of the state-of-the-art IBM quantum processors and LHC real-time charged track pattern recognition requirements. We present a software prototype implementation of the QuAM protocols and analyze the topological limitations for porting the simplest QuAM instances to the public IBM 5Q and 14Q cloud-based superconducting chips.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2019

Federated User Account Management

Abstract. BNL SDCC (Scientific Data and Computing Center) recently deployed a centralized identity management solution to support Single Sign On (SSO) authentication across multiple IT systems. The system supports federated login access via CILogon and InCommon and multi-factor authentication (MFA) to meet security standards for various application and services such as Jupyterhub / Invenio that are provided to the SDCC user community. CoManage (Cloud) and FreeIPA / Keycloak (local) are utilized to provided complex authorization for authenticated users. This talk will focus on technical overviews and strategies to tackle the challenges/obstacles in our facility.

42 ENGINEERING↗
DOE OSTI2017

The HEPCloud Facility: elastic computing for High Energy Physics – The NOvA Use Case

The need for computing in the HEP community follows cycles of peaks and valleys mainly driven by conference dates, accelerator shutdown, holiday schedules, and other factors. Because of this, the classical method of provisioning these resources at providing facilities has drawbacks such as potential overprovisioning. As the appetite for computing increases, however, so does the need to maximize cost efficiency by developing a model for dynamically provisioning resources only when needed. To address this issue, the HEPCloud project was launched by the Fermilab Scientific Computing Division in June 2015. Its goal is to develop a facility that provides a common interface to a variety of resources, including local clusters, grids, high performance computers, and community and commercial Clouds. Initially targeted experiments include CMS and NOvA, as well as other Fermilab stakeholders. In its first phase, the project has demonstrated the use of the “elastic” provisioning model offered by commercial clouds, such as Amazon Web Services. In this model, resources are rented and provisioned automatically over the Internet upon request. In January 2016, the project demonstrated the ability to increase the total amount of global CMS resources by 58,000 cores from 150,000 cores - a 25 percent increase - in preparation for the Recontres de Moriond. In March 2016, the NOvA experiment has also demonstrated resource burst capabilities with an additional 7,300 cores, achieving a scale almost four times as large as the local allocated resources and utilizing the local AWS s3 storage to optimize data handling operations and costs. NOvA was using the same familiar services used for local computations, such as data handling and job submission, in preparation for the Neutrino 2016 conference. In both cases, the cost was contained by the use of the Amazon Spot Instance Market and the Decision Engine, a HEPCloud component that aims at minimizing cost and job interruption. This paper describes the Fermilab HEPCloud Facility and the challenges overcome for the CMS and NOvA communities.

Fuess, S.↗