Search NASA⌕ Search

SEARCH · Search NASA

Results for “cloud database”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 91 records · Page 5
All sourcesNASA NTRSDOE OSTI
DOE OSTI2015

A Toolkit for ARB to Integrate Custom Databases and Externally Built Phylogenies

Researchers are perpetually amassing biological sequence data. The computational approaches employed by ecologists for organizing this data (e.g. alignment, phylogeny, etc.) typically scale nonlinearly in execution time with the size of the dataset. This often serves as a bottleneck for processing experimental data since many molecular studies are characterized by massive datasets. To keep up with experimental data demands, ecologists are forced to choose between continually upgrading expensive in-house computer hardware or outsourcing the most demanding computations to the cloud. Outsourcing is attractive since it is the least expensive option, but does not necessarily allow direct user interaction with the data for exploratory analysis. Desktop analytical tools such as ARB are indispensable for this purpose, but they do not necessarily offer a convenient solution for the coordination and integration of datasets between local and outsourced destinations. Therefore, researchers are currently left with an undesirable tradeoff between computational throughput and analytical capability. To mitigate this tradeoff we introduce a software package to leverage the utility of the interactive exploratory tools offered by ARB with the computational throughput of cloud-based resources. Our pipeline serves as middleware between the desktop and the cloud allowing researchers to form local custom databases containing sequences and metadata from multiple resources and a method for linking data outsourced for computation back to the local database. Furthermore, a tutorial implementation of the toolkit is provided in the supporting information, S1 Tutorial.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗
DOE OSTI2022

Evaluating WRF-Solar EPS cloud mask forecast using the NSRDB

Improving the accuracy of day-ahead solar forecasts using numerical weather prediction models requires improving the forecasting of cloud occurrence and properties. Validating cloud forecasts is challenging because this requires evaluating different types of clouds over a wide variety of regions. This study analyzes the cloud occurrence (or cloud mask) over the contiguous United States (CONUS), predicted by the Weather Research and Forecasting-Solar Ensemble Prediction System (WRF-Solar EPS), to identify the strengths and limitations of the model in reproducing cloud fields. To enable the in-depth analysis of cloud mask forecasts covering CONUS, we use satellite observations from the National Solar Radiation Database (NSRDB). Two evaluation methods are implemented to consider all clouds and partially removed clouds from the 2-km NSRDB in evaluating the 9-km WRF-Solar EPS cloud mask. Cloud detection metrics as well as the frequency of cloud occurrence are used to quantify the monthly performance of WRF-Solar EPS. Mismatched cloud frequency (MCF) is used to assess the model's capability to predict different types of clouds, which are classified using three levels of cloud optical depth (COD) and cloud top height (CTH). The day-ahead forecasts covering the full year of 2018 demonstrate that WRF-Solar EPS produces MCFs ranging from 27%-46%, 13%-34%, and 8%-19% for thin, mid-thickness, and thick clouds, respectively. For three CTH levels, the model shows MCFs ranging from 19%-46%, 16%-33%, and 8%-27% for low-level, middle-level, and high-level clouds, respectively. This comprehensive characterization of model performance helps identify model weakness and will eventually lead to improvements in cloud and solar radiation forecasting.

14 SOLAR ENERGY↗
NASA NTRS2016

Airborne Spectral BRDF of Various Surface Types (Ocean, Vegetation, Snow, Desert, Wetlands, Cloud Decks, Smoke Layers) for Remote Sensing Applications

In this paper we describe measurements of the bidirectional reflectance-distribution function (BRDF) acquired over a 30-year period (1984-2014) by the National Aeronautics and Space Administration's (NASA's) Cloud Absorption Radiometer (CAR). Our BRDF database encompasses various natural surfaces that are representative of many land cover or ecosystem types found throughout the world. CAR's unique measurement geometry allows a comparison of measurements acquired from different satellite instruments with various geometrical configurations, none of which are capable of obtaining such a complete and nearly instantaneous BRDF. This database is therefore of great value in validating many satellite sensors and assessing corrections of reflectances for angular effects. These data can also be used to evaluate the ability of analytical models to reproduce the observed directional signatures, to develop BRDF models that are suitable for sub-kilometer-scale satellite observations over both homogeneous and heterogeneous landscape types, and to test future spaceborne sensors. All of these BRDF data are publicly available and accessible in hierarchical data format (http:car.gsfc.nasa.gov/).

albedo↗
DOE OSTI2017

Streamlining and Transparently Automating Complex Waste Determinations at Los Alamos National Laboratory - 17286

Environmental programs at Los Alamos National Laboratory (LANL) generate a large amount of sampling activities that can result in waste that needs to be characterized and properly disposed. Since 2011, sampling results are managed in the cloud-based Environmental Information Management (EIM) database, which contains over 16 million current and historical data records. In many cases, the database's analytical records are used to characterize and dispose waste from environmental operations. Three factors contribute to streamlining of the waste disposition operations: (1) if possible, use of the existing analytical data for waste characterization to avoid additional laboratory analysis; (2) reduction in time spent on data evaluation for various waste acceptance criteria; and (3) choice of the proper and least expensive disposal alternative. Use of the existing analytical data, when allowed by law, creates some paradoxical problems: when chemical analysis is requested with the purpose of characterizing waste, it creates only 100 to 400 analytical records. However, use of exiting analytical data at LANL may lead to evaluation of tens of thousands of analytical records, with many records for each single analyte. This occurs because data were collected for environmental characterization and not waste characterization. To streamline data analysis for waste disposition using existing data, LANL has created a Waste Data Summary (WDS) module in the site's EIM database. Recognizing that waste determination is performed by groups outside the environmental program at LANL, the WDS allows for logically guided data selection for a specific waste determination (solids, liquids, radiological, etc.) and reduces potentially thousands of rows of data into simple formatted data reports that are easy to comprehend and process. This allows the waste evaluators to review data using multiple approaches to make their own determination on the most appropriate disposal method. There are four basic principles of WDS operation: (1) The highest detected analytical result is evaluated for the selected samples (or the lowest undetected result if there are no detected results); (2) All approved disposal pathways are evaluated; (3) Any failures of the Waste Acceptance Criteria are visibly highlighted to focus the attention of the evaluator; and (4) The decision of the disposal path is left to the trained waste evaluator (WDS does not choose or suggest any disposal paths). The summary results are presented as the set of short data reports. These reports can be divided into two groups: general comparison reports and disposal pathway specific forms. The general comparison forms show all Resource Conservation and Recovery Act (RCRA) analytes with pass/fail evaluation and with preference for toxicity characteristic leaching procedure (TCLP) results (or 20x rule) for solid samples, the potential RCRA hazard codes assignment for detected analytes, and all rad results. The detected chemical and radioactivity results are compared to the background values and to Soil Screening and Soil Action limits (SSL/SAL limits). Disposal pathway specific forms follow Waste Acceptance Criteria requirements for all disposal pathways available at LANL. The programming of the WDS tries to maximize existing data by analyte substitutions (like using gross alpha for necessary alpha emitting isotopes) or by making calculations when analytical results do not agree with legal requirements (like total chlordane in the RCRA regulations, which is most often analyzed and reported as individual isomers). All forms may be annotated by the waste evaluator to provide additional information to support decision making. Upon final decision of a disposal path, the complete dataset used in the evaluation and the WDS forms with annotations is 'frozen' as an un-editable PDF export to document the data used in the waste disposal determination. The streamlined process results in less work for waste evaluators and reduces overall labor and analytical costs. By comparison, a full review of a 10,000 record parent dataset may take a waste evaluator over a week to review, whereas with the WDS module, the task can be accomplished in less than one hour with clear, reproducible results. These results can be reviewed and checked at any time by quality assurance personnel, thus ensuring transparency in the decision making process. LANL believes this approach has broad applicability to other sites where waste determination is based on review of existing data collected for more generalized environmental purposes. (authors)

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗
DOE OSTI2015

Evolving the Land Information System into a Cloud Computing Service

The Land Information System (LIS) was developed to use advanced flexible land surface modeling and data assimilation frameworks to integrate extremely large satellite- and ground-based observations with advanced land surface models to produce continuous high-resolution fields of land surface states and fluxes. The resulting fields are extremely useful for drought and flood assessment, agricultural planning, disaster management, weather and climate forecasting, water resources assessment, and the like. We envisioned transforming the LIS modeling system into a scientific cloud computing-aware web and data service that would allow clients to easily setup and configure for use in addressing large water management issues. The focus of this Phase 1 project was to determine the scientific, technical, commercial merit and feasibility of the proposed LIS-cloud innovations that are currently barriers to broad LIS applicability. We (a) quantified the barriers to broad LIS utility and commercialization (high performance computing, big data, user interface, and licensing issues); (b) designed the proposed LIS-cloud web service, model-data interface, database services, and user interfaces; (c) constructed a prototype LIS user interface including abstractions for simulation control, visualization, and data interaction, (d) used the prototype to conduct a market analysis and survey to determine potential market size and competition, (e) identified LIS software licensing and copyright limitations and developed solutions, and (f) developed a business plan for development and marketing of the LIS-cloud innovation. While some significant feasibility issues were found in the LIS licensing, overall a high degree of LIS-cloud technical feasibility was found.

54 ENVIRONMENTAL SCIENCES↗
NASA NTRS2006

A Study of Rapidly Developing Low Cloud Ceilings in a Stable Atmosphere at the Florida Spaceport

Forecasters at the Space Meteorology Group (SMG) issue 30 to 90 minute forecasts for low cloud ceilings at the Shuttle Landing Facility (KTTS) in Kennedy Space Center, FL for all Space Shuttle missions. Mission verification statistics have shown cloud ceilings to be the biggest forecast challenge. SMG forecasters are especially concerned with rapidly developing cloud ceilings below 8000 ft. in a stable, capped thermodynamic environment because ceilings below 8000 ft restrict Shuttle landing operations and are the most challenging to predict accurately. This project involves the development of a database of these cases over east-central Florida in order to identify the onset, location, and if possible, dissipation times of rapidly-developing low cloud ceilings. Another goal is to document the atmospheric regimes favoring this type of cloud development to improve forecast skill of such events during Space Shuttle launch and landing operations. A 10-year database of stable, rapid low cloud development days during the daylight hours was compiled for the Florida cool-season months by examining the Cape Canaveral Air Force Station sounding data, and identifying days that had high boundary layer relative humidity associated with a thermally-capped environment below 8000 ft. Archived hourly surface observations from KTTS and Melbourne, Orlando, Sanford, and Ocala, FL were then examined for the onset of cloud ceilings below 8000 ft between 1100 and 2000 UTC. Once the database was supplemented with the hourly surface cloud observations, visible satellite imagery was examined in 30-minute intervals to confirm event occurrences. This paper will present results from some of the rapidly developing cloud ceiling cases and the prevailing meteorological conditions associated with these events, focusing on potential pre-curser information that may help improve their prediction.

FROM↗
NASA NTRSDate not supplied

TPSAS-NF1676L-10888-DND

Hyperspectrally-resolved surface emissivities are derived with an algorithm utilizes a combined fast radiative transfer model (RTM) with a molecular RTM and a cloud RTM accounting for both atmospheric absorption and cloud absorption/scattering. Clouds are automatically detected and cloud microphysical parameters are retrieved; and emissivity is retrieved under clear and optically thin cloud conditions. The retrieval technique separates surface emissivity from skin temperature by representing the emissivity spectrum with eigenvectors derived from a laboratory measured emissivity database. Global land emissivities retrieved under the optically thin clouds are investigated by means of their accuracy in comparison with that under cloud-free conditions. Here we present the emissivity derived under optically thin clouds, their accuracy, and the inter-comparison with that under clear conditions.

Daniel K Zhou↗
DOE OSTI2018

Automated data processing architecture for the Gemini Planet Imager Exoplanet Survey

The Gemini Planet Imager Exoplanet Survey (GPIES) is a multiyear direct imaging survey of 600 stars to discover and characterize young Jovian exoplanets and their environments. We have developed an automated data architecture to process and index all data related to the survey uniformly. An automated and flexible data processing framework, which we term the Data Cruncher, combines multiple data reduction pipelines (DRPs) together to process all spectroscopic, polarimetric, and calibration data taken with GPIES. With no human intervention, fully reduced and calibrated data products are available less than an hour after the data are taken to expedite follow up on potential objects of interest. The Data Cruncher can run on a supercomputer to reprocess all GPIES data in a single day as improvements are made to our DRPs. A backend MySQL database indexes all files, which are synced to the cloud, and a front-end web server allows for easy browsing of all files associated with GPIES. To help observers, quicklook displays show reduced data as they are processed in real time, and chatbots on Slack post observing information as well as reduced data products. Together, the GPIES automated data processing architecture reduces our workload, provides real-time data reduction, optimizes our observing strategy, and maintains a homogeneously reduced dataset to study planet occurrence and instrument performance.

79 ASTRONOMY AND ASTROPHYSICS↗
NASA NTRS2009

The Bolocam Galactic Plane Survey

The Bolocam Galactic Plane Survey (BGPS) is a 1.1 millimeter continuum survey of the northern Galactic Plane made with Bolocam and the Caltech Submillimeter Observatory. The coverage totals 170 square degrees, comprised of a contiguous range from -10.5 deg is less than or equal to 90.5 deg, 0.5 deg is less than or equal to b is less than or equal to 0.5 deg, with extended coverage in b in selected regions, and four targeted regions in the outer Galaxy, including: IC1396, toward the Perseus arm at l is approximately 111 deg, W3/4/5, and Gem OB1. Depths of the maps range from 30 to 60 mJy beam (sup 1). Approximately 8,400 sources were detected and the maps and source catalog have been made publicly available. Millimeter-wave thermal dust emission reveals dense regions within molecular clouds, thus the BGPS serves as a database for studies of the dense interstellar medium and star formation within the Milky Way.

interstellar medium↗
NASA NTRS2019

Overview of the New Version 3 NASA Micro-Pulse Lidar Network (MPLNET) Automatic Precipitation Detection Algorithm

Precipitation modifies atmospheric column thermodynamics through the process of evaporation and serves as a proxy for latent heat modulation. For this reason, a correct precipitation parameterization (especially for low-intensity precipitation) within global scale models is crucial. In addition to improving our modeling of the hydrological cycle, this will reduce the associated uncertainty of global climate models in correctly forecasting future scenarios, and will enable the application of mitigation strategies. In this manuscript we present a proof of concept algorithm to automatically detect precipitation from lidar measurements obtained from the National Aeronautics and Space Administration Micropulse lidar network (MPLNET). The algorithm, once tested and validated against other remote sensing instruments, will be operationally implemented into the network to deliver a near real time (latency <1.5 h) rain masking variable that will be publicly available on MPLNET website as part of the new Version 3 data products. The methodology, based on an image processing technique, detects only light precipitation events (defined by intensity and duration) such as light rain, drizzle, and virga. During heavy rain events, the lidar signal is completely extinguished after a few meters in the precipitation or it is unusable because of water accumulated on the receiver optics. Results from the algorithm, in addition to filling a gap in light rain, drizzle, and virga detection by radars, are of particular interest for the scientific community as they help to fully characterize the aerosol cycle, from emission to deposition, as precipitation is a crucial meteorological phenomenon accelerating atmospheric aerosol removal through the scavenging effect. Algorithm results will also help the understanding of long term aerosol–cloud interactions, exploiting the multi-year database from several MPLNET permanent observational sites across the globe. The algorithm is also applicable to other lidar and/or ceilometer network infrastructures in the framework of the Global Aerosol Watch (GAW) aerosol lidar observation network (GALION).

Simone Lolli↗
DOE OSTI2023

Clear-Sky Probability for the 2023 Annular Solar Eclipse and the 2024 Total Solar Eclipse Using the NREL National Solar Radiation Database

The National Renewable Energy Laboratory (NREL) and collaborators have created a clear-sky probability analysis to help guide viewers to the 2023 annular solar eclipse and the 2024 total solar eclipse. Using cloud and solar data from the NREL National Solar Radiation Database (NSRDB), the analysis provides cloudless-sky probabilities specific to the date, time, and location of each eclipse with a 4-km resolution. Though not intended to be an eclipse weather forecast, the detailed maps can help guide eclipse enthusiasts to likely optimal viewing locations. Additionally, high-resolution data are presented for the centerline of the path of each eclipse, representing the likelihood for cloudless skies and atmospheric clarity. The NSRDB provides industry, academia, and other stakeholders with high-resolution solar irradiance data to support feasibility analyses for photovoltaic and concentrating solar power generation projects.

14 SOLAR ENERGY↗
DOE OSTI2017

Virtual machine provisioning, code management, and data movement design for the Fermilab HEPCloud Facility

The Fermilab HEPCloud Facility Project has as its goal to extend the current Fermilab facility interface to provide transparent access to disparate resources including commercial and community clouds, grid federations, and HPC centers. This facility enables experiments to perform the full spectrum of computing tasks, including data-intensive simulation and reconstruction. We have evaluated the use of the commercial cloud to provide elasticity to respond to peaks of demand without overprovisioning local resources. Full scale data-intensive workflows have been successfully completed on Amazon Web Services for two High Energy Physics Experiments, CMS and NOνA, at the scale of 58000 simultaneous cores. This paper describes the significant improvements that were made to the virtual machine provisioning system, code caching system, and data movement system to accomplish this work. The virtual image provisioning and contextualization service was extended to multiple AWS regions, and to support experiment-specific data configurations. A prototype Decision Engine was written to determine the optimal availability zone and instance type to run on, minimizing cost and job interruptions. We have deployed a scalable on-demand caching service to deliver code and database information to jobs running on the commercial cloud. It uses the frontiersquid server and CERN VM File System (CVMFS) clients on EC2 instances and utilizes various services provided by AWS to build the infrastructure (stack). We discuss the architecture and load testing benchmarks on the squid servers. We also describe various approaches that were evaluated to transport experimental data to and from the cloud, and the optimal solutions that were used for the bulk of the data transport. Finally, we summarize lessons learned from this scale test, and our future plans to expand and improve the Fermilab HEP Cloud Facility.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2017

Clear-Sky Probability for the August 21, 2017, Total Solar Eclipse Using the NREL National Solar Radiation Database

The National Renewable Energy Laboratory (NREL) and collaborators have created a clear-sky probability analysis to help guide viewers of the August 21, 2017, total solar eclipse, the first continent-spanning eclipse in nearly 100 years in the United States. Using cloud and solar data from NREL's National Solar Radiation Database (NSRDB), the analysis provides cloudless sky probabilities specific to the date and time of the eclipse. Although this paper is not intended to be an eclipse weather forecast, the detailed maps can help guide eclipse enthusiasts to likely optimal viewing locations. Additionally, high-resolution data are presented for the centerline of the path of totality, representing the likelihood for cloudless skies and atmospheric clarity. The NSRDB provides industry, academia, and other stakeholders with high-resolution solar irradiance data to support feasibility analyses for photovoltaic and concentrating solar power generation projects.

14 SOLAR ENERGY↗