Results for “data imbalance” — Page 12

DOE OSTI2024

Quantum/AI Topology-Aware Latency-Adaptive HPC Workflow Scheduling Optimization

The growing demand for more powerful high-performance computing (HPC) systems has led to a steady rise in energy consumption by supercomputing worldwide. This study is focused on comparing our Application-Topology Mapper (ATMapper) to the popular Simple Linux Utility for Resource Management (SLURM) for the purpose of exploring methods that can further optimize job-scheduling within HPC systems. ATMapper is an Artificial-Intelligence based approach to job-scheduling that is currently being enhanced with quantum annealing (QA) to generate optimal schedules faster. We are applying QA to speedup our ATMapper process to achieve higher computing efficiency, thereby reducing HPC energy consumption. Here, we examine how four job-scheduling approaches perform in processor node assignment when using an example network architecture of 4 interconnected nodes. Using a specialized script, we are assessing the schedule of a computation flow with 11 interdependent tasks. The data movements among nodes were tracked to count for the number of interactions (network hops) between nodes needed to complete the tasks. The total number of hops and the job completion time were then used to quantify the efficiency of the different mapping approaches. In addition to SLURM, we also compare our ATMapper to the QA-enabled LBNL TIGER and the D-Wave Distributed Computing processor assignment approaches. The preliminary results showed that our topology-aware, latency-adaptive ATMapper is significantly more efficient when compared to the other scheduling approaches due to its load-imbalance network allocation. The scheduler displayed a computing efficiency of 53% by performing significantly fewer network hops than its alternatives. By reducing the number of hops, ATMapper was able to perform all 11 tasks by using only 3 nodes out of given 4. This research indicates the potential to use QA/AI for HPC job-scheduling. Later, we will test a SLURM simulator program to draw further comparisons on the effectiveness of ATMapper's scheduling approach. The results of this comparison will serve as a baseline for later improving SLURM's performance using a QA-enhanced ATMapper approach.

Caraveo, Braulio [University of Huston - Clear Lak↗

NASA NTRS1998

PLUM: Parallel Load Balancing for Unstructured Adaptive Meshes

Dynamic mesh adaption on unstructured grids is a powerful tool for computing large-scale problems that require grid modifications to efficiently resolve solution features. Unfortunately, an efficient parallel implementation is difficult to achieve, primarily due to the load imbalance created by the dynamically-changing nonuniform grid. To address this problem, we have developed PLUM, an automatic portable framework for performing adaptive large-scale numerical computations in a message-passing environment. First, we present an efficient parallel implementation of a tetrahedral mesh adaption scheme. Extremely promising parallel performance is achieved for various refinement and coarsening strategies on a realistic-sized domain. Next we describe PLUM, a novel method for dynamically balancing the processor workloads in adaptive grid computations. This research includes interfacing the parallel mesh adaption procedure based on actual flow solutions to a data remapping module, and incorporating an efficient parallel mesh repartitioner. A significant runtime improvement is achieved by observing that data movement for a refinement step should be performed after the edge-marking phase but before the actual subdivision. We also present optimal and heuristic remapping cost metrics that can accurately predict the total overhead for data redistribution. Several experiments are performed to verify the effectiveness of PLUM on sequences of dynamically adapted unstructured grids. Portability is demonstrated by presenting results on the two vastly different architectures of the SP2 and the Origin2OOO. Additionally, we evaluate the performance of five state-of-the-art partitioning algorithms that can be used within PLUM. It is shown that for certain classes of unsteady adaption, globally repartitioning the computational mesh produces higher quality results than diffusive repartitioning schemes. We also demonstrate that a coarse starting mesh produces high quality load balancing, at a fraction of the cost required a fine initial mesh. Results indicate that our parallel load balancing strategy will remain viable on large numbers of processors.

Oliker, Leonid↗

NASA NTRS2021

Analysis of Impacts of Terminal Restrictions on Departures in D10 TRACON

The D10 – Dallas-Fort Worth Terminal Radar Approach CONtrol (TRACON) is an air traffic facility that controls and manages aircraft and airspace that consists of multiple airports located in the North Texas area. From an air traffic management standpoint, one of the main challenges in the D10 TRACON is the departure fix capacity as multiple airports compete for resources. This problem is magnified when en route demand/capacity imbalance and inclement weather around the TRACON reduce the capacity at the terminal fixes. This leads to multiple, dynamic Traffic Management Initiative (TMI) restrictions being issued by the Air Traffic Control (ATC) on departing flights. This, in turn, propagates delay to the surface of each airport within the metroplex. The NASA Airspace Technology Demonstration 2 (ATD-2) Phase 3 is deployed in D10 to demonstrate new technologies developed to manage the Integrated Arrival, Departure, and Surface (IADS) traffic in a metroplex environment where multiple airports are interacting and sharing resources at the terminal boundary. This paper uses the ATD-2 terminal restriction data collected in the D10 TRACON to quantify the impact of restrictions on the demand, analyze the relation between terminal restrictions and departure taxi time on airport surface, and establish relationships between restrictions and surface delay. We found that the restrictions on departure flights have a direct adverse effect on departure excess taxi time on the airport surface.

ATD-2↗

NASA NTRS2021

Analysis of Impacts of Terminal Restrictions on Departures in D10 TRACON

The D10 – Dallas-Fort Worth Terminal Radar Approach CONtrol (TRACON) is an air traffic facility that controls and manages aircraft and airspace that consists of multiple airports located in the North Texas area. From an air traffic management standpoint, one of the main challenges in the D10 TRACON is the departure fix capacity as multiple airports compete for resources. This problem is magnified when en route demand/capacity imbalance and inclement weather around the TRACON reduce the capacity at the terminal fixes. This leads to multiple, dynamic Traffic Management Initiative (TMI) restrictions being issued by the Air Traffic Control (ATC) on departing flights. This, in turn, propagates delay to the surface of each airport within the metroplex. The NASA Airspace Technology Demonstration 2 (ATD-2) Phase 3 is deployed in D10 to demonstrate new technologies developed to manage the Integrated Arrival, Departure, and Surface (IADS) traffic in a metroplex environment where multiple airports are interacting and sharing resources at the terminal boundary. This paper uses the ATD-2 terminal restriction data collected in the D10 TRACON to quantify the impact of restrictions on the demand, analyze the relation between terminal restrictions and departure taxi time on airport surface, and establish relationships between restrictions and surface delay. We found that the restrictions on departure flights have a direct adverse effect on departure excess taxi time on the airport surface.

ATD-2↗

NASA NTRS2007

Comprehensive Software Eases Air Traffic Management

To help air traffic control centers improve the safety and the efficiency of the National Airspace System, Ames Research Center developed the Future Air Traffic Management Concepts Evaluation Tool (FACET) software, which won NASA's 2006 "Software of the Year" competition. In 2005, Ames licensed FACET to Flight Explorer Inc., for integration into its Flight Explorer (version 6.0) software. The primary FACET features incorporated in the Flight Explorer software system alert airspace users to forecasted demand and capacity imbalances. Advance access to this information helps dispatchers anticipate congested sectors (airspace) and delays at airports, and decide if they need to reroute flights. FACET is now a fully integrated feature in the Flight Explorer Professional Edition (version 7.0). Flight Explorer Professional offers end-users other benefits, including ease of operation; automatic alerts to inform users of important events such as weather conditions and potential airport delays; and international, real-time flight coverage over Canada, the United Kingdom, New Zealand, and sections of the Atlantic and Pacific Oceans. Flight Explorer Inc. recently broadened coverage by partnering with Honeywell International Inc.'s Global Data Center, Blue Sky Network, Sky Connect LLC, SITA, ARINC Incorporated, Latitude Technologies Corporation, and Wingspeed Corporation, to track their aircraft anywhere in the world.

Source record↗

NASA NTRS1988

Ocean alkalinity and the Cretaceous/Tertiary boundary

A biogeochemical cycle model resolving ocean carbon and alkalinity content is applied to the Maestrichtian and Danian. The model computes oceanic concentrations and distributions of Ca(2+), Mg(2+), and Sigma-CO2. From these values an atmospheric pCO2 value is calculated, which is used to estimate rates of terrestrial weathering of calcite, dolomite, and calcium and magnesium silicates. Metamorphism of carbonate rocks and the subsequent outgassing of CO2 to the atmosphere are parameterized in terms of carbonate rock reservoir sizes, total land area, and a measure of overall tectonic activity, the sea-floor generation rate. The ocean carbon reservoir computed by the model is used with Deep Sea Drilling Project (DSDP) C-13 data to estimate organic detrital fluxes under a variety of ocean mixing rate assumptions. Using Redfield ratios, the biogenic detrital flux estimate is used to partition the ocean carbon and alkalinity reservoirs between the mixed layer and deep ocean. The calcite flux estimate and carbonate ion concentrations are used to determine the rate of biologically mediated CaCO3 titration. Oceanic productivity was severely limited for approximately 500 kyr following the K/T boundary resulting in significant increases in total ocean alkalinity. As productivity returned to the ocean, excess carbon and alkalinity was removed from the ocean as CaCO3. Model runs indicate that this resulted in a transient imbalance in the other direction. Ocean chemistry returned to near-equilibrium by about 64 mybp.

Caldeira, K. G.↗

NASA NTRS2008

Identifying Bearing Rotordynamic Coefficients using an Extended Kalman Filter

An Extended Kalman Filter is developed to estimate the linearized direct and indirect stiffness and damping force coefficients for bearings in rotor-dynamic applications from noisy measurements of the shaft displacement in response to imbalance and impact excitation. The bearing properties are modeled as stochastic random variables using a Gauss-Markov model. Noise terms are introduced into the system model to account for all of the estimation error, including modeling errors and uncertainties and the propagation of measurement errors into the parameter estimates. The system model contains two user-defined parameters that can be tuned to improve the filter s performance; these parameters correspond to the covariance of the system and measurement noise variables. The filter is also strongly influenced by the initial values of the states and the error covariance matrix. The filter is demonstrated using numerically simulated data for a rotor-bearing system with two identical bearings, which reduces the number of unknown linear dynamic coefficients to eight. The filter estimates for the direct damping coefficients and all four stiffness coefficients correlated well with actual values, whereas the estimates for the cross-coupled damping coefficients were the least accurate.

Miller, Bard A.↗

NASA NTRS2008

Identifying Bearing Rotordynamic Coefficients using an Extended Kalman Filter

An Extended Kalman Filter is developed to estimate the linearized direct and indirect stiffness and damping force coefficients for bearings in rotor-dynamic applications from noisy measurements of the shaft displacement in response to imbalance and impact excitation. The bearing properties are modeled as stochastic random variables using a Gauss-Markov model. Noise terms are introduced into the system model to account for all of the estimation error, including modeling errors and uncertainties and the propagation of measurement errors into the parameter estimates. The system model contains two user-defined parameters that can be tuned to improve the filter s performance; these parameters correspond to the covariance of the system and measurement noise variables. The filter is also strongly influenced by the initial values of the states and the error covariance matrix. The filter is demonstrated using numerically simulated data for a rotor-bearing system with two identical bearings, which reduces the number of unknown linear dynamic coefficients to eight. The filter estimates for the direct damping coefficients and all four stiffness coefficients correlated well with actual values, whereas the estimates for the cross-coupled damping coefficients were the least accurate.

Miller, Brad A.↗

NASA NTRS2008

Identifying Bearing Rotodynamic Coefficients Using an Extended Kalman Filter

An Extended Kalman Filter is developed to estimate the linearized direct and indirect stiffness and damping force coefficients for bearings in rotor dynamic applications from noisy measurements of the shaft displacement in response to imbalance and impact excitation. The bearing properties are modeled as stochastic random variables using a Gauss-Markov model. Noise terms are introduced into the system model to account for all of the estimation error, including modeling errors and uncertainties and the propagation of measurement errors into the parameter estimates. The system model contains two user-defined parameters that can be tuned to improve the filter's performance; these parameters correspond to the covariance of the system and measurement noise variables. The filter is also strongly influenced by the initial values of the states and the error covariance matrix. The filter is demonstrated using numerically simulated data for a rotor bearing system with two identical bearings, which reduces the number of unknown linear dynamic coefficients to eight. The filter estimates for the direct damping coefficients and all four stiffness coefficients correlated well with actual values, whereas the estimates for the cross-coupled damping coefficients were the least accurate.

Miller, Brad A.↗

NASA NTRSDate not supplied

Exploring Radiation, Water and Energy Closure in Key Deep Tropical Regions Over the Last 3 Decades

The closure of global energy and water budgets at various time scales depends heavily upon the on the processes of water and energy exchange in the deep tropics. A key process observed is the change in convection regimes from disorganized convection to organized deep convection. During these periods an imbalance in key tropical regions is observed of water budget terms between observations and atmospheric reanalysis systems, implying that processes are not simulated well. For instance, periods of imbalance in the water budget may be at least partly attributed to precipitation estimates due to the transition between cloud and precipitation systems in which ice cloud affects microwave radar estimates (Lietmann-Niimi et al. 2024, submitted). A recent paper by Hsaio et al (2023, pre-published version) found that the transition process is characterized by wind shear and longwave cloud radiative feedback signatures as well. This is just one example how these processes also affect the radiative fluxes and thus the energy budget at the TOA. In this presentation, we assess the net radiative budget and components in the deep tropics by comparing multiple data products (GEWEX SRB Rel4IP, CERES EBAF/SYN1deg Ed4.2, ISSCP FH, etc.) and describing the temporal variability of energy balance terms across the deep tropics for the period from 1988 to near present in six key oceanic regions. These six key tropical oceanic regions are West Indian Ocean, the south Indian Ocean east, tropical western, central and eastern Pacific Ocean, and Tropical Atlantic Ocean. In these regions we assess the sensitivity of the fluxes as a function of fluctuations in cloud types in response to various larger scale atmospheric processes (e.g. El Niño, Indian Ocean Dipole). We assess this variability of the radiative flux anomalies (including the net TOA, surface and the atmospheric divergence fluxes) against water vapor divergence, cloud properties (include ISCCP “Weather States, Tselioudis et al. 2021) and larger scale wind shear to assess the uncertainties of energy closure in these regions. These are contrasted against the modeled radiative fluxes from ERA-5 and MERRA-2. Ocean buoy measurements of radiative fluxes are utilized to help assess data radiative flux uncertainty over the nearly 40 year period. In general, the variability for overlap periods of these various data products agrees well, but there are significant differences in the net fluxes that vary according to the rendering of surface, atmospheric, aerosol and cloud properties. These differences also affect closure terms and inferences on systematic uncertainties within the observational estimates. We conclude with recommendations for continuing work

Radiation budget↗

NASA NTRS2009

Modern Era Retrospective-analysis for Research and Applications (MERRA) Global Water and Energy Budgets

In the Summer of 2009, NASA's Modern Era Retrospective-analysis for Research and Applications (MERRA) will have completed 28 years of global satellite data analyses. Here, we characterize the global water and energy budgets of MERRA, compared with available observations and the latest reanalyses. In this analysis, the climatology of the global average components are studied as well as the separate land and ocean averages. In addition, the time series of the global averages are evaluated. For example, the global difference of precipitation and evaporation generally shows the influence of water vapor observations on the system. Since the observing systems change in time, especially remotely sensed observations of water, significant temporal variations can occur across the 28 year record. These then are also closely connected to changes in the atmospheric energy and water budgets. The net imbalance of the energy budget at the surface can be large and different signs for different reanalyses. In MERRA, the imbalance of energy at the surface tends to improve with time being the smallest during the most recent and abundant satellite observations.

Bosilovich, Michael G.↗

NASA NTRS2016

Novel Radiomitigator for Radiation-Induced Bone Loss

Radiation-induced bone loss can occur with radiotherapy patients, accidental radiation exposure and during long-term spaceflight. Bone loss due to radiation is due to an early increase in oxidative stress, inflammation and bone resorption, resulting in an imbalance in bone remodeling. Furthermore, exposure to high-Linear Energy Transfer (LET) radiation will impair the bone forming progenitors and reduce bone formation. Radiation can be classified as high-LET or low-LET based on the amount of energy released. Dried Plum (DP) diet prevents bone loss in mice exposed to total body irradiation with both low-LET and high-LET radiation. DP prevents the early radiation-induced bone resorption, but furthermore, we show that DP protects the bone forming osteoblast progenitors from high-LET radiation. These results provide insight that DP re-balances the bone remodeling by preventing resorption and protecting the bone formation capacity. This data is important considering that most of the current osteoporosis treatments only block the bone resorption but do not protect bone formation. In addition, DP seems to act on both the oxidative stress and inflammation pathways. Finally, we have preliminary data showing the potential of DP to be radio-protective at a systemic effect and could possible protect other tissues at risk of total body-irradiation such as skin, brain and heart.

countermeasure↗

NASA NTRSDate not supplied

Inferred Sea Level Prediction in the NASA GMAO Seasonal Forecasting System

Reliable predictions of sea level anomalies on seasonal timescales with lead times of 1 to 9 months may have relevance to stakeholders – for example, in the advance deployment of resources for coastal flood mitigation. Routine prediction and analysis may also highlight physical processes associated with sea level change and modeling capabilities on seasonal and other timescales. These forecasts may represent interannual changes in the seasonal slope of the ocean surface, teleconnection effects such as the El Niño/Southern Oscillation phenomenon, and variations in seasonal hydrology including precipitation and coastal runoff. Coupled atmosphere/ocean models are routinely used in the seasonal prediction of temperature anomalies, precipitation anomalies, sea ice cover, and climate indices such as the Niño3.4 predictions under the North American Multi-Model Ensemble (NMME) protocol. Within the limits of their configuration, these complex Earth-system models have a potential for depicting regional changes in oceanic column properties, including the sea surface height. Seasonal prediction models generally have no representation of long-term mass contributions from melting land ice, or changes in vertical land motion; their output may be more specifically characterized as predictions of the ocean dynamic sea level. In practice however, the sea surface height prognostic variable is substantially compromised by the forecast model response to initial conditions. Imbalances between the initial, observed hydrologic cycle and the forecast model state produce abrupt adjustments in the model sea surface height. As a result, most seasonal prediction systems employ a constraint on the globally-averaged sea surface height that is applied at each time step. This essentially renders the prognostic sea surface height variable as unserviceable. Several approaches have previously been used to retrieve sea level information from seasonal forecasts beyond the use of the sea surface height variable. Here, we extend a method of relating other prognostic values, including ocean circulation and climate indices, to observed sea level variations. We use the merged altimetry record of the NASA MEaSUREs Gridded Sea Surface Height Anomalies data set and monthly revised local reference gauge observations from the National Oceanography Centre Permanent Service for Mean Sea Level (PSMSL) to evaluate derived prognostic variables from the NASA Global Modeling and Assimilation Office subseasonal-to-seasonal system version 2.1 (GMAO S2S v2.1). We focus on results for the midlatitudes with particular emphasis on US gauge locations. As shown in previous studies, prognostic ENSO-related indices in boreal winter are well correlated with gauge observations for the US west coast, but also for other locations in the southeastern US. Other forecast climate indices such as the North Atlantic Oscillation have relations to sea level that are limited both seasonally and spatially. As expected, surface atmospheric pressure (e.g., inverse barometer effect) is found to be particularly well correlated with observed sea level. We provide a characterization of forecast skill for seasonal sea level with this method.

Richard I Cullather↗

NASA NTRS1997

Load Balancing Sequences of Unstructured Adaptive Grids

Mesh adaption is a powerful tool for efficient unstructured grid computations but causes load imbalance on multiprocessor systems. To address this problem, we have developed PLUM, an automatic portable framework for performing adaptive large-scale numerical computations in a message-passing environment. This paper makes several important additions to our previous work. First, a new remapping cost model is presented and empirically validated on an SP2. Next, our load balancing strategy is applied to sequences of dynamically adapted unstructured grids. Results indicate that our framework is effective on many processors for both steady and unsteady problems with several levels of adaption. Additionally, we demonstrate that a coarse starting mesh produces high quality load balancing, at a fraction of the cost required for a fine initial mesh. Finally, we show that the data remapping overhead can be significantly reduced by applying our heuristic processor reassignment algorithm.

Biswas, Rupak↗

DOE OSTI2024

Exploring Wholesale Energy Price Trends: The Renewables and Wholesale Electricity Prices (ReWEP) tool (Ver. 2024.1)

The Renewables and Wholesale Electricity Prices (ReWEP) visualization tool from Berkeley Lab has been updated with nodal electricity pricing and wind and solar generation data through the end of 2023. ReWEP users can explore trends in wholesale electricity prices and their relationship to wind and solar generation. ReWEP includes nodal pricing trends across locations, regions, and different timeframes. The tool consists of maps, time series, and other interactive figures that provide: (1) a general overview of how average pricing, negative price frequency, and extreme high prices vary over time, and (2) a summary of how pricing patterns are related to wind and solar generation. Interactive functionality allows investigation by year, season, time of day, and region, where region is defined as the Independent System Operators (ISO) or Regional Transmission Organizations (RTO) region. ReWEP also contains prices throughout much of the western United States from the Western Energy Imbalance Market and the Western Energy Imbalance Service Market.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗

DOE OSTI2024

Exploring Wholesale Energy Price Trends: The Renewables and Wholesale Electricity Prices (ReWEP) tool (Ver. 2024.1)

The Renewables and Wholesale Electricity Prices (ReWEP) visualization tool from Berkeley Lab has been updated with nodal electricity pricing and wind and solar generation data through the end of 2023. ReWEP users can explore trends in wholesale electricity prices and their relationship to wind and solar generation. ReWEP includes nodal pricing trends across locations, regions, and different timeframes. The tool consists of maps, time series, and other interactive figures that provide: (1) a general overview of how average pricing, negative price frequency, and extreme high prices vary over time, and (2) a summary of how pricing patterns are related to wind and solar generation. Interactive functionality allows investigation by year, season, time of day, and region, where region is defined as the Independent System Operators (ISO) or Regional Transmission Organizations (RTO) region. ReWEP also contains prices throughout much of the western United States from the Western Energy Imbalance Market and the Western Energy Imbalance Service Market.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗

NASA NTRS2003

Limits to the Indirect Aerosol Forcing in Stratocumulus

The indirect radiative forcing of aerosols is poorly constrained by the observational data underlying the simple cloud parameterizations in GCMs. signal of cloud response to increased aerosol concentrations from meteorological noise. Recent satellite observations indicate a significant decrease of cloud water in ship tracks, in contrast to an ensemble of in situ measurements showing no average change in cloud water relative to the surrounding clouds. Both results contradict the expectation of cloud water increasing in polluted clouds. We find through large-eddy simulations of stratocumulus that the trend in the satellite data is likely an artifact of sampling only overcast clouds. The simulations instead show cloud cover increasing with droplet concentrations. The simulations also show that increases in cloud water from suppressing drizzle by increased droplet concentrations are favored at night or at extremely low droplet concentrations. At typical droplet concentrations we find that the Twomey effect on cloud albedo is amplified very little by the secondary indirect effect of drizzle suppression, largely because the absorption of solar radiation by cloud water reduces boundary-layer mixing in the daytime and thereby restricts any possible increase in cloud water from drizzle suppression. The cloud and boundary layer respond to radiative heating variations on a time scale of hours, and on longer time scales respond to imbalances between large-scale horizontal advection and the entrainment of inversion air. We analyze the co-varying response of cloud water, cloud thickness, width of droplet size distributions, and dispersion of the optical depth, as well as the overall response of cloud albedo, to changes in droplet concentrations. We also dissect the underlying physical mechanisms through sensitivity studies. Ship tracks represent an ideal natural laboratory to extricate the

Ackerman, Andrew↗

NASA NTRS1987

A VAS-numerical model impact study using the Gal-Chen variational approach

Numerical experiments to assess the impact of incorporating temperature data from the VISSR Atmospheric Sounder (VAS) using the assimilation technique developed by Gal-Chen (1986) modified for use in the Mesoscale Atmospheric Simulation System (MASS) model were conducted. The scheme is designed to utilize the high temporal and horizontal resolution of satellite retrievals while maintaining the fine vertical structure generated by the model. This is accomplished by adjusting the model lapse rates to reflect thicknesses retrieved from VAS and applying a three-dimensional variational that preserves the distribution of the geopotential fields in the model. A nudging technique whereby the model temperature fields are gradually adjusted toward the updated temperature fields during model integration is also tested. An adiabatic version of MASS is used in all experiments to better isolate mass-momentum imbalances. The method has a sustained impact over an 18 hr model simulation.

Aune, Robert M.↗