SEARCH · Search NASA

Results for “Resource provisioning”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 37 records · Page 2

All sources NASA NTRS DOE OSTI

DOE OSTI2020

Characterizing, Modeling, and Accurately Simulating Power and Energy Consumption of I/O-intensive Scientific Workflows

While distributed computing infrastructures can provide infrastructure-level techniques for managing energy consumption, application-level energy consumption models have also been developed to support energy-efficient scheduling and resource provisioning algorithms. In this work, we analyze the accuracy of a widely-used application-level model that has been developed and used in the context of scientific workflow executions. To this end, we profile two production scientific workflows on a distributed platform instrumented with power meters. We then conduct an analysis of power and energy consumption measurements. This analysis shows that power consumption is not linearly related to CPU utilization and that I/O operations significantly impact power, and thus energy, consumption. We then propose a power consumption model that accounts for I/O operations, including the impact of waiting for these operations to complete, and for concurrent task executions on multi-socket, multi-core compute nodes. We implement our proposed model as part of a simulator that allows us to draw direct comparisons between real-world and modeled power and energy consumption. Here, we find that our model has high accuracy when compared to real-world executions. Furthermore, our model improves accuracy by about two orders of magnitude when compared to the traditional models used in the energy-efficient workflow scheduling literature.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2023

Relationships between maternally-transferred mercury and hatchling development, behavior, and survival in the American alligator ( Alligator mississippiensis )

Mercury is a toxic and pervasive environmental contaminant that can be transferred from mother to offspring during development. Consequences of maternally-transferred mercury have been observed in vertebrate taxa, including reduced clutch viability, reduced offspring size, and behavioral alterations. These sublethal effects have been assumed to decrease survivorship, though this is seldom assessed. Here, we examined how maternally-transferred mercury interacts with incubation temperature to influence reproductive success, offspring behavior, and subsequent survival in the American alligator (Alligator mississippiensis). We collected nine clutches of eggs from a mercury contaminated reservoir on the Savannah River Site, South Carolina, and incubated eggs at either female- or male-promoting temperatures. Clutch-averaged mercury in egg yolk was high relative to other studies in crocodilians and ranged from 0.248 to 0.554 ppm compared to 0.018–0.052 ppm at a site with low levels of mercury contamination; mercury levels in hatchling blood ranged from 0.090 to 0.490 ppm ($\overline{x}$= 0.240 ppm, n = 158). We found few, mostly negligible correlations between life history traits and mercury but noted a positive relationship with egg mass, possibly mediated by correlated maternal effects such as resource provisioning. Incubation temperature exerted strong effects on hatchling phenotypes, with warmer, male-promoting temperatures producing larger and bolder hatchlings. Presumptive females, produced from cooler incubation temperatures, spent more time in warm areas during behavior trials. Hatchlings were released 10–15 days post-hatch and surveyed over eight months to assess survival. Survivorship was positively correlated with hatchling size and negatively correlated with proportional time spent in warm areas. Presumptive females had much lower survival, and overall survivorship for the eight-month period was 0.185–0.208, depending on the modelling approach. Finally, our study suggests that, within the range of concentrations we observed, incubation temperature has a stronger effect on offspring behavior and survival than maternally-transferred mercury pollution in American alligators.

59 BASIC BIOLOGICAL SCIENCES↗

DOE OSTI2025

Transforming US agriculture for carbon removal with enhanced weathering

Abstract Enhanced weathering (EW) with agriculture uses crushed silicate rocks to drive carbon dioxide removal (CDR) 1,2 . If widely adopted on farmlands, it could help achieve net-zero emissions by 2050 2–4 . Here we show, with a detailed US state-specific carbon cycle analysis constrained by resource provision, that EW deployed on agricultural land could sequester 0.16–0.30 GtCO 2 yr −1 by 2050, rising to 0.25–0.49 GtCO 2 yr −1 by 2070. Geochemical assessment of rivers and oceans suggests effective transport of dissolved products from EW from soils, offering CDR on intergenerational timescales. Our analysis further indicates that EW may temporarily help lower ground-level ozone and concentrations of secondary aerosols in agricultural regions. Geospatially mapped CDR costs show heterogeneity across the USA, reflecting a combination of cropland distance from basalt source regions, timing of EW deployment and evolving CDR rates. CDR costs are highest in the first two decades before declining to about US$100–150 tCO 2 −1 by 2050, including for states that contribute most to total national CDR. Although EW cannot be a substitute for emission reductions, our assessment strengthens the case for EW as an overlooked practical innovation for helping the USA meet net-zero 2050 goals 5,6 . Public awareness of EW and equity impacts of EW deployment across the USA require further exploration 7,8 and we note that mobilizing an EW industry at the necessary scale could take decades.

Science & Technology - Other Topics↗

DOE OSTI2024

Transitioning GlideinWMS, a multi domain distributed workload manager, from GSI proxies to tokens and other granular credentials

GlideinWMS is a distributed workload manager that has been used in production for many years to provision resources for experiments like CERN’s CMS, many Neutrino experiments, and the OSG. Its security model was based mainly on GSI (Grid Security Infrastructure), using X.509 certificate proxies and VOMS (Virtual Organization Membership Service) extensions. Even when other credentials, like SSH keys, were used to authenticate with resources, proxies were also added all the time, to establish the identity of the requestor and the associated memberships or privileges. This single credential was used for everything and was, often implicitly, forwarded wherever needed. The addition of identity and access tokens and the phase-out of GSI forced us to reconsider the security model of GlideinWMS, to handle multiple credentials which can differ in type, technology, and functionality. Both identity tokens and access tokens are supported. GSI proxies even if no more mandatory, are still used, together with various JWT (JSON Web Token) based tokens and other certificates. The functionality of the credentials, defined by issuer, audience, and scope, also differ: a credential can allow access to a computing resource, or can protect the GlideinWMS framework from tampering, or can grant read or write access to storage, can provide an identity for accounting or auditing, or can provide a combination of any the formers. Furthermore, the tools in use do not include automatic forwarding and renewal of the new credentials so credential lifetime and renewal requirements became part of the discussion as well. In this paper, we will present how GlideinWMS was able to change its design and code to respond to all these changes.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2024

Utilizing Distributed Heterogeneous Computing with PanDA in ATLAS

In recent years, advanced and complex analysis workflows have gained increasing importance in the ATLAS experiment at CERN, one of the large scientific experiments at LHC. Support for such workflows has allowed users to exploit remote computing resources and service providers distributed worldwide, overcoming limitations on local resources and services. The spectrum of computing options keeps increasing across the Worldwide LHC Computing Grid (WLCG), volunteer computing, high-performance computing, commercial clouds, and emerging service levels like Platform-as-a-Service (PaaS), Container-as-a-Service (CaaS) and Function-as-a-Service (FaaS), each one providing new advantages and constraints. Users can significantly benefit from these providers, but at the same time, it is cumbersome to deal with multiple providers, even in a single analysis workflow with fine-grained requirements coming from their applications’ nature and characteristics. In this paper, we will first highlight issues in geographically-distributed heterogeneous computing, such as the insulation of users from the complexities of dealing with remote providers, smart workload routing, complex resource provisioning, seamless execution of advanced workflows, workflow description, pseudointeractive analysis, and integration of PaaS, CaaS, and FaaS providers. We will also outline solutions developed in ATLAS with the Production and Distributed Analysis (PanDA) system and future challenges for LHC Run4.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2023

Maternal provisioning interacts with incubation temperature to affect hatchling mercury exposure in an oviparous reptile

The thermal environment experienced by developing embryos can influence the utilization of maternally provisioned resources. Despite being particularly consequential for oviparous ectotherms, these dynamics are largely unexplored within ecotoxicological frameworks. Here, we test if incubation temperature interacts with maternally transferred mercury to affect subsequent body burdens and tissue distributions of mercury in hatchling American alligators (Alligator mississippiensis). Nine clutches of alligator eggs were collected from a mercury-contaminated reservoir and incubated at either female- or male-promoting temperatures. Total mercury (THg) concentration was measured in egg yolk collected during incubation and in a suite of tissues collected from hatchlings. THg concentrations in residual yolk and blood were higher in hatchlings incubated at cooler, female-promoting temperatures compared to the warmer, male-promoting temperatures. THg concentrations in most tissues were positively correlated with THg concentrations in blood and dermis, and egg yolk THg concentration was the best predictor of THg concentration in many resultant tissues. Furthermore, our results highlight a hereto unknown role of the developmental environment in mediating tissue specific uptake of contaminants in an oviparous reptile.

59 BASIC BIOLOGICAL SCIENCES↗

DOE OSTI2023

Effectiveness and predictability of in-network storage cache for Scientific Workflows

Large scientific collaborations often have multiple scientists accessing the same set of files while doing different analyses, which create repeated accesses to the large amounts of shared data located far away. These data accesses have long latency due to distance and occupy the limited bandwidth available over the wide-area network. To reduce the wide-area network traffic and the data access latency, regional data storage caches have been installed as a new networking service. To study the effectiveness of such a cache system in scientific applications, we examine the Southern California Petabyte Scale Cache for a high-energy physics experiment. By examining about 3TB of operational logs, we show that this cache removed 67.6% of file requests from the wide-area network and reduced the traffic volume on wide-area network by 12. 3TB (or 35.4%) an average day. The reduction in the traffic volume (35.4%) is less than the reduction in file counts (67.6%) because the larger files are less likely to be reused. Due to this difference in data access patterns, the cache system has implemented a policy to avoid evicting smaller files when processing larger files. We also build a machine learning model to study the predictability of the cache behavior. Tests show that this model is able to accurately predict the cache accesses, cache misses, and network throughput, making the model useful for future studies on resource provisioning and planning.

Sim, Caitlin↗

DOE OSTI2023

ESnet/JLab FPGA Accelerated Transport

To increase the science rate for high data rates/volumes, Thomas Jefferson National Accelerator Facility (JLab) has partnered with Energy Sciences Network (ESnet) to define an edge to data center traffic shaping / steering transport capability featuring data event aware network shaping and forwarding. The keystone of this ESnet+JLab FPGA Accelerated Transport (EJFAT) is the joint development of an AI/ML directed dynamic compute work Load Balancer (LB) of UDP streamed data. The LB is a suite consisting of a Field Programmable Gate Array (FPGA) executing the dynamically configurable, low fixed latency LB data plane featuring real-time packet redirection and high throughput, and a control plane running on the FPGA host computer that monitors network and compute farm telemetry in order to make dynamic AI/ML guided decisions for destination compute host redirection/load balancing and destination resource provisioning. The LB provides for three-tier horizontal scaling across LB suites, core compute hosts, and CPUs within a host. The LB effectively provides seamless integration of edge/core computing to support direct experimental data processing for immediate use by JLab science programs and others such as the EIC as well as data centers of the future requiring high throughput and low latency for both hot and cooled data for both running experiment data acquisition systems and data center use cases.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2024

Operational experience and R&D results using the Google Cloud for High-Energy Physics in the ATLAS experiment

The ATLAS experiment at CERN relies on a Worldwide Distributed Computing Grid infrastructure to support its physics program at the Large Hadron Collider. ATLAS has integrated cloud computing resources to complement its Grid infrastructure and conducted an R&D program on Google Cloud Platform. These initiatives leverage key features of commercial cloud providers: lightweight configuration and operation, elasticity and availability of diverse infrastructures. Here this paper examines the seamless integration of cloud computing services as a conventional Grid site within the ATLAS workflow management and data management systems, while also offering new setups for interactive, parallel analysis. It underscores pivotal results that enhance the on-site computing model and outlines several R&D projects that have benefited from large-scale, elastic resource provisioning models. Furthermore, this study discusses the impact of cloud-enabled R&D projects in three domains: accelerators and AI/ML, ARM CPUs and columnar data analysis techniques.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

DOE OSTI2023

Design and implementation of I/O performance prediction scheme on HPC systems through large-scale log analysis

Abstract Large-scale high performance computing (HPC) systems typically consist of many thousands of CPUs and storage units used by hundreds to thousands of users simultaneously. Applications from large numbers of users have diverse characteristics, such as varying computation, communication, memory, and I/O intensity. A good understanding of the performance characteristics of each user application is important for job scheduling and resource provisioning. Among these performance characteristics, I/O performance is becoming increasingly important as data sizes rapidly increase and large-scale applications, such as simulation and model training, are widely adopted. However, predicting I/O performance is difficult because I/O systems are shared among all users and involve many layers of software and hardware stack, including the application, network interconnect, operating system, file system, and storage devices. Furthermore, updates to these layers and changes in system management policy can significantly alter the I/O behavior of applications and the entire system. To improve the prediction of the I/O performance on HPC systems, we propose integrating information from several different system logs and developing a regression-based approach to predict the I/O performance. Our proposed scheme can dynamically select the most relevant features from the log entries using various feature selection algorithms and scoring functions, and can automatically select the regression algorithm with the best accuracy for the prediction task. The evaluation results show that our proposed scheme can predict the write performance with up to 90% prediction accuracy and the read performance with up to 99% prediction accuracy using the real logs from the Cori supercomputer system at NERSC.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2021

Adaptive bandwidth management for entanglement distribution in quantum networks

Flexible grid wavelength division multiplexing is a powerful tool in lightwave communications to maximize spectral efficiency. In the emerging field of quantum networking, the need for effective resource provisioning is particularly acute, given the generally lower power levels, higher sensitivity to loss, and inapplicability of optical detection and retransmission. In this letter, we leverage flex grid technology to demonstrate reconfigurable distribution of quantum entanglement in a four-user tabletop network. By adaptively partitioning bandwidth with a single wavelength-selective switch, we successfully equalize two-party coincidence rates that initially differ by over two orders of magnitude. Our scalable approach introduces loss that is fixed with the number of users, offering a practical path for the establishment and management of quality-of-service guarantees in large quantum networks.

42 ENGINEERING↗

NASA NTRS2007

Earth Observations to Assess Impact of Hurricane Katrina on John C. Stennis Space Center

The peril from hurricanes to Space Operations Centers is real and is forecast to continue; Katrina, Rita, and Wilma of 2005 and Charley, Frances, Ivan, and Jeanne of 2004 are sufficient motivation for NASA to develop a multi-Center plan for preparedness and response. As was demonstrated at SSC (Stennis Space Center) in response to Hurricane Katrina, NASA Centers are efficiently activated as local command centers, playing host to Federal and State agencies and first responders to coordinate and provide evacuation, relocation, response, and recovery activities. Remote sensing decision support provides critical insight for managing NASA infrastructure and for assisting Center decision makers. Managers require geospatial information to manage the federal city. Immediately following Katrina, SSC s power and network connections were disabled, hardware was inoperative, technical staff was displaced and/or out of contact, and graphical decision support tools were non-existent or less than fully effective. Despite this circumstance, SSC EOC (Emergency Operations Center) implemented response operations to assess damage and to activate recovery plans. To assist Center Managers, the NASA ASP (Applied Sciences Program) made its archive of high-resolution data over the site available. In the weeks and months after the immediate crisis, NASA supplemented this data with high-resolution, post-Katrina imagery over SSC and much of the affected coastal areas. Much of the high-resolution imagery was made available through the Department of Defense Clear View contract and was distributed through U.S. Geological Survey Center for Earth Resources Observation and Science "Hurricane Katrina Disaster Response" Web site. By integrating multiple image data types with other information sources, ASP applied an all-source solutions approach to develop decision support tools that enabled managers to respond to critical issues, such as expedient access to infrastructure and deployment of resources, provision of temporary shelter, logistical control of critical supplies, and the mobilization and coordination of assets from ground crews to aircraft/airspace management. Furthermore, ASP developed information products that illustrate risks to SSC's infrastructure from surge, inundation, and flood. Current plans include developing wind-risk prototype products for refinement and adoption into EOC plans.

Graham, William D.↗

NASA NTRS2006

Towards Designing an Integrated Earth Observation System for the Provision of Solar Energy Resource and Assessment

The GEOSS strategic plan specifically targets the area of improved energy resource management due to the importance of these to the economic and social viability of every nation of the world. With the world s increasing demand for energy resources, the need for new alternative energy resources grows. This paper overviews a new initiative within the International Energy Agency that addresses needs to better manage and develop solar energy resources worldwide. The goal is to provide the solar energy industry, the electricity sector, governments, and renewable energy organizations and institutions with the most suitable and accurate information of the solar radiation resources at the Earth's surface in easily-accessible formats and understandable quality metrics. The scope of solar resource assessment information includes historic data sets and currently derived data products using satellite imagery and other means. Thus, this new task will address the needs of the solar energy sector while at the same time will serve as a model that satisfies GEOSS objectives and goals.

Stackouse, Paul W., Jr.↗

DOE OSTI2020

Performance Prediction of Big Data Transfer Through Experimental Analysis and Machine Learning

Big data transfer in next-generation scientific applications is now commonly carried out over connections with guaranteed bandwidth provisioned in High-performance Networks (HPNs) through advance bandwidth reservation. To use HPN resources efficiently, provisioning agents need to carefully schedule data transfer requests and allocate appropriate bandwidths. Such reserved bandwidths, if not fully utilized by the requesting user, could be simply wasted or cause extra overhead and complexity in management due to exclusive access. This calls for the capability of performance prediction to reserve bandwidth resources that match actual needs. Towards this goal, we employ machine learning algorithms to predict big data transfer performance based on extensive performance measurements, which are collected over a span of several years from a large number of data transfer tests using different protocols and toolkits between various end sites on several real-life physical or emulated HPN testbeds. We first identify a comprehensive list of attributes involved in a typical big data transfer process, including end host system configurations, network connection properties, and control parameters of data transfer methods. We then conduct an in-depth exploratory analysis of their impacts on application-level throughput, which provides insights into big data transfer performance and motivates the use of machine learning. We also investigate the applicability of machine learning algorithms and derive their general performance bounds for performance prediction of big data transfer in HPNs. Experimental results show that, with appropriate data preprocessing, the proposed machine learning-based approach achieves 95% or higher prediction accuracy in up to 90% of the cases with very noisy real-life performance measurements.

Yun, Daqing↗

NASA NTRSDate not supplied

TPSAS-NF1676L-33992-DND

The CERES Science Team integrates and fuses observations from 6 CERES instruments aboard the Terra, Aqua, S-NPP, and NOAA-20 missions with data from more than 20 other unique data sources. Following the November 2017 launch of CERES Flight Model 6 (FM6) onboard NOAA20, CERES has now amassed over 80 instrument-years of valuable Earth radiation budget data. The rapidly growing volume of CERES data coupled with the introduction of new data products alongside improvements to existing science algorithms fosters the requirement for faster, more flexible, and scalable data production and orchestration. New virtualized, cloud-centric compute hardware hosted by the NASA Langley Research Center’s (LaRC) Atmospheric Sciences Data Center (ASDC) provides an ideal environment for these ever-increasing data production demands for CERES. This poster discusses updates to the implementation of the CERES Data Management Team’s (DMT) CERES AuTomAted job Loading sYSTem (CATALYST), a custom data processing workflow engine for CERES, to use on-demand computing resources to perform automated CERES data production processing in a Linux-based container environment. Linux containers provide CERES the flexibility to build multiple production environments in containers tailored for specific workloads and allow effortless provisioning of resources based on the CERES Science Team’s data production requirements.

Thomas N. Hillyer↗

DOE OSTI2021

Managing Solar Photovoltaic Integration in the Western United States: Resource Adequacy Considerations

This study examines the impact of reserve margin-based reliability assessment, as commonly used in capacity expansion models, on planning resource-adequate power systems under high penetrations of solar photovoltaics (PV). As a generation resource, PV is operationally different from the conventional dispatchable resources for which most capacity expansion models were designed. The question this study attempts to answer is whether large amounts of PV on a system (in this case, the Western Interconnection of North America) would bias the results of conventional reserve margin-based capacity expansion modeling towards an over- or under-provisioning of resource adequacy. This analysis used NREL’s Resource Planning Model (RPM) for capacity expansion modeling and NREL’s Probabilistic Resource Adequacy Suite (PRAS) for resource adequacy assessment. RPM uses a reserve margin requirement to enforce resource adequacy. PRAS, a collection of tools for studying the resource adequacy of power systems and the adequacy contributions of individual resources on a probabilistic basis, was used to compute multiple resource adequacy metrics across a number of simulated scenarios and system representations with differing regional detail. In all cases, including high PV penetrations (up to 33% annual generation from PV, interconnection-wide), RPM was able to produce resource-adequate systems as measured by normalized expected unserved energy and loss-of-load expectation results from PRAS. The accuracy of reserve margin approaches depends heavily on the underlying assumptions informing the capacity credit assigned to variable and energy-limited resources, particularly when such resources are abundant in the modeled system. RPM’s standard methodology for estimating variable and flexible resources’ capacity contributions, which is based on the top 100 hours of net load, did not appear to systematically undervalue or overvalue variable generation relative to a more rigorous equivalent firm capacity assessment using PRAS, although both over- and under-valuations were observed in specific scenarios. In the worst cases, the top 100 hour method underestimated the equivalent firm capacity of PV by two percentage points, and overestimated the equivalent firm capacity of PV by five percentage points. Calculating capacity contributions based on the top 10 hours of net load systematically underestimated equivalent firm capacities at more modest PV penetrations, but was often a better approximation of equivalent firm capacity than the existing 100-hour approach in scenarios with higher PV penetrations.

14 SOLAR ENERGY↗

NASA NTRS2001

Designing Extraterrestrial Plant Growth Habitats With Low Pressure Atmospheres

In-situ resource utilization, provision of human life support requirements by bioregenerative methods, and engineering constraints for construction and deployment of plant growth structures on the surface of Mars all suggest the need for plant growth studies at hypobaric pressures. Past work demonstrated that plants will likely tolerate and grow at pressures at or below 10 kPa. Based upon this premise, concepts are developed for the design of reduced pressure atmospheres in lightweight, inflatable structures for plant growth systems on Mars with the goals of maximizing design simplicity and the use of local resources. A modular pod design is proposed as it could be integrated with large-scale production systems. Atmospheric modification of pod clusters would be based upon a pulse and scrub system using mass flow methods for atmospheric transport. A specific modification and control scenario is developed for a lettuce pod to illustrate the dynamics of carbon dioxide and oxygen exchange within a pod. Considerations of minimal atmospheric crop requirements will aid in the development of engineering designs and strategies for extraterrestrial plant growth structures that employ rarefied atmospheres.

Corey, Kenneth A.↗

NASA NTRS2002

Designing Extraterrestrial Plant Growth Habitats with Low Pressure Atmospheres

Corey, Kenneth A.↗