Search NASA⌕ Search

SEARCH · Search NASA

Results for “Distributed Computing Resources”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 541 records · Page 30

Collaborative: in situ visual analytics technologies for extreme scale combustion simulations

This project aims to drastically enhance the usability of in situ analysis and visualization for extreme-scale scientific simulations. Current exascale computing capabilities promise to offer greater predictive ability of simulations and to further push the frontiers of science and technology. However, to validate the simulation output at extreme scale, examine the modeled phenomena, and discover previously unknowns from the output data, the output must be reduced or transformed in situ as it is being generated during the simulation such that the amount of data to examine and store is kept to a minimum. Such in situ approaches allow us to process and analyze the data and any embedded geometry to an extent that would be prohibitively expensive, if not impossible, to perform as a post hoc task. While in situ processing has been demonstrated to be a feasible and promising approach, its full potential has not yet been leveraged. In this project, we have developed comprehensive enhancements to in situ technology based on probability distributions in data. Our research focuses on jointly developing new ways of interacting with massive statistical samples while creatively utilizing new state-of-the-art computational resources to push the boundaries of in situ exploration. Moreover, we have developed new time-dependent techniques to enable previously unattainable capabilities in areas such as intelligent simulation steering and precise feature identification. We have experimentally studied our design and implementation at NERSC and OLCF, and are able to leverage existing in situ infrastructures whenever possible. While the exemplar in this project is combustion, many other fields for which turbulent transport is important, e.g., fusion, climate, astrophysics among others, encounter similar issues as simulations scale up to the exascale. This project shows its potential to generate high impact on DOE missions since the resulting technology promises to improve scientists’ ability to rapidly and correctly interpret and tune extreme-scale simulations, leading to new scientific understanding and advancements.

97 MATHEMATICS AND COMPUTING↗

Batching System for Superior Service

Veridian's Portable Batch System (PBS) was the recipient of the 1997 NASA Space Act Award for outstanding software. A batch system is a set of processes for managing queues and jobs. Without a batch system, it is difficult to manage the workload of a computer system. By bundling the enterprise's computing resources, the PBS technology offers users a single coherent interface, resulting in efficient management of the batch services. Users choose which information to package into "containers" for system-wide use. PBS also provides detailed system usage data, a procedure not easily executed without this software. PBS operates on networked, multi-platform UNIX environments. Veridian's new version, PBS Pro,TM has additional features and enhancements, including support for additional operating systems. Veridian distributes the original version of PBS as Open Source software via the PBS website. Customers can register and download the software at no cost. PBS Pro is also available via the web and offers additional features such as increased stability, reliability, and fault tolerance.A company using PBS can expect a significant increase in the effective management of its computing resources. Tangible benefits include increased utilization of costly resources and enhanced understanding of computational requirements and user needs.

Source record↗

Transformation of OODT CAS to Perform Larger Tasks

A computer program denoted OODT CAS has been transformed to enable performance of larger tasks that involve greatly increased data volumes and increasingly intensive processing of data on heterogeneous, geographically dispersed computers. Prior to the transformation, OODT CAS (also alternatively denoted, simply, 'CAS') [wherein 'OODT' signifies 'Object-Oriented Data Technology' and 'CAS' signifies 'Catalog and Archive Service'] was a proven software component used to manage scientific data from spaceflight missions. In the transformation, CAS was split into two separate components representing its canonical capabilities: file management and workflow management. In addition, CAS was augmented by addition of a resource-management component. This third component enables CAS to manage heterogeneous computing by use of diverse resources, including high-performance clusters of computers, commodity computing hardware, and grid computing infrastructures. CAS is now more easily maintainable, evolvable, and reusable. These components can be used separately or, taking advantage of synergies, can be used together. Other elements of the transformation included addition of a separate Web presentation layer that supports distribution of data products via Really Simple Syndication (RSS) feeds, and provision for full Resource Description Framework (RDF) exports of metadata.

Mattmann, Chris↗

Remote sensing by ERTS satellite of vegetational resources believed to be under possible threat of environmental stress

The distribution of natural vegetation types on North Merritt Island, Florida, was studied by analysis of ERTS multispectral scanner data on the image-100 computer system. The boundaries of six distinct plant associations were located on photos made on the image analyzer, with an insignificant mean error of -24.38 meters. The six plant associations are described; each had a characteristic spectral signature. The difference in average reflectance grey level between the lowest of the four spectral scanning bands and the highest spectral scanning band for the six vegetation types was determined. The decreasing trend of the differences is strongly negatively correlated with height of land.

Poonai, P.↗

High-Fidelity Large-Signal Order Reduction Approach for Composite Load Model

With the increasing penetration of electronic loads and distributed energy resources, conventional load models cannot capture their dynamics. Therefore, a new comprehensive composite load model is developed by Western Electricity Coordinating Council (WECC). However, this model is a complex high-order non-linear system with multi-time-scale property, which poses challenges on power system studies with heavy computational burden. In order to reduce the model complexity, the authors firstly develop a large-signal order reduction (LSOR) method using singular perturbation theory. In this method, the fast dynamics are integrated into the slow ones to preserve transient characteristics of the former. Then, accuracy assessment conditions are proposed and embedded into the LSOR to improve and guarantee the accuracy of reduced-order model. Finally, the reduced-order WECC composite load model is derived by using the proposed algorithm. Overall, simulation results show that the reduced-order large-signal model significantly alleviates the computational burden while maintaining similar dynamic responses as the original composite load model.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Dominant Wave Energy Systems and Conditional Wave Resource Characterization for Coastal Waters of the United States

Opportunities and constraints for wave energy conversion technologies and projects are evaluated by identifying and characterizing the dominant wave energy systems for United States (US) coastal waters using marginal and joint distributions of the wave energy in terms of the peak period, wave direction, and month. These distributions are computed using partitioned wave parameters generated from a 30 year WaveWatch III model hindcast, and regionally averaged to identify the dominant wave systems contributing to the total annual available energy ( A A E ) for eleven distinct US wave energy climate regions. These dominant wave systems are linked to the wind systems driving their generation and propagation. In addition, conditional resource parameters characterizing peak period spread, directional spread, and seasonal variability, which consider dependencies of the peak period, direction, and month, are introduced to augment characterization methods recommended by international standards. These conditional resource parameters reveal information that supports project planning, conceptual design, and operation and maintenance. The present study shows that wave energy resources for the United States are dominated by long-period North Pacific swells (Alaska, West Coast, Hawaii), short-period trade winds and nor’easter swells (East Coast, Puerto Rico), and wind seas (Gulf of Mexico). Seasonality, peak period spread, and directional spread of these dominant wave systems are characterized to assess regional opportunities and constraints for wave energy conversion technologies targeting the dominant wave systems.

16 TIDAL AND WAVE POWER↗

X-composer: enabling cross-environments in-situ workflows between HPC and cloud

As large-scale scientific simulations and big data analyses become more popular, it is increasingly more expensive to store huge amounts of raw simulation results to perform post-analysis. To minimize the expensive data I/O, "in-situ" analysis is a promising approach, where data analysis applications analyze the simulation generated data on the fly without storing it first. However, it is challenging to organize, transform, and transport data at scales between two semantically different ecosystems due to the distinct software and hardware difference. To tackle these challenges, we design and implement the X-Composer framework. X-Composer connects cross-ecosystem applications to form an "in-situ" scientific workflow, and provides a unified approach and recipe for supporting such hybrid in-situ workflows on distributed heterogeneous resources. X-Composer reorganizes simulation data as continuous data streams and feeds them seamlessly into the Cloud-based stream processing services to minimize I/O overheads. For evaluation, we use X-Composer to set up and execute a cross-ecosystem workflow, which consists of a parallel Computational Fluid Dynamics simulation running on HPC, and a distributed Dynamic Mode Decomposition analysis application running on Cloud. Our experimental results show that X-Composer can seamlessly couple HPC and Big Data jobs in their own native environments, achieve good scalability, and provide high-fidelity analytics for ongoing simulations in real-time.

Wang, Dali↗

Comparison of numerical weather prediction based deterministic and probabilistic wind resource assessment methods

Numerical weather prediction (NWP) models have been widely used for wind resource assessment. Model runs with higher spatial resolution are generally more accurate, yet extremely computational expensive. An alternative approach is to use data generated by a low resolution NWP model, in conjunction with statistical methods. In order to analyze the accuracy and computational efficiency of different types of NWP-based wind resource assessment methods, this paper performs a comparison of three deterministic and probabilistic NWP-based wind resource assessment methodologies: (i) a coarse resolution (0.5 degrees x 0.67 degrees) global reanalysis data set, the Modern-Era Retrospective Analysis for Research and Applications (MERRA); (ii) an analog ensemble methodology based on the MERRA, which provides both deterministic and probabilistic predictions; and (iii) a fine resolution (2-km) NWP data set, the Wind Integration National Dataset (WIND) Toolkit, based on the Weather Research and Forecasting model. Results show that: (i) as expected, the analog ensemble and WIND Toolkit perform significantly better than MERRA confirming their ability to downscale coarse estimates; (ii) the analog ensemble provides the best estimate of the multi-year wind distribution at seven of the nine sites, while the WIND Toolkit is the best at one site; (iii) the WIND Toolkit is more accurate in estimating the distribution of hourly wind speed differences, which characterizes the wind variability, at five of the available sites, with the analog ensemble being best at the remaining four locations; and (iv) the analog ensemble computational cost is negligible, whereas the WIND Toolkit requires large computational resources. Future efforts could focus on the combination of the analog ensemble with intermediate resolution (e.g., 10-15 km) NWP estimates, to considerably reduce the computational burden, while providing accurate deterministic estimates and reliable probabilistic assessments.

17 WIND ENERGY↗

A Blueprint for Demonstrating Quantum Supremacy with Superconducting Qubits

Long coherence times and high fidelity control recently achieved in scalable superconducting circuits paved the way for the growing number of experimental studies of many-qubit quantum coherent phenomena in these devices. Albeit full implementation of quantum error correction and fault tolerant quantum computation remains a challenge the near term pre-error correction devices could allow new fundamental experiments despite inevitable accumulation of errors. One such open question foundational for quantum computing is achieving the so called quantum supremacy, an experimental demonstration of a computational task that takes polynomial time on the quantum computer whereas the best classical algorithm would require exponential time and/or resources. It is possible to formulate such a task for a quantum computer consisting of less than a 100 qubits. The computational task we consider is to provide approximate samples from a non-trivial quantum distribution. This is a generalization for the case of superconducting circuits of ideas behind boson sampling protocol for quantum optics introduced by Arkhipov and Aaronson. In this presentation we discuss a proof-of-principle demonstration of such a sampling task on a 9-qubit chain of superconducting gmon qubits developed by Google. We discuss theoretical analysis of the driven evolution of the device resulting in output approximating samples from a uniform distribution in the Hilbert space, a quantum chaotic state. We analyze quantum chaotic characteristics of the output of the circuit and the time required to generate a sufficiently complex quantum distribution. We demonstrate that the classical simulation of the sampling output requires exponential resources by connecting the task of calculating the output amplitudes to the sign problem of the Quantum Monte Carlo method. We also discuss the detailed theoretical modeling required to achieve high fidelity control and calibration of the multi-qubit unitary evolution in the device. We use a novel cross-entropy statistical metric as a figure of merit to verify the output and calibrate the device controls. Finally, we demonstrate the statistics of the wave function amplitudes generated on the 9-gmon chain and verify the quantum chaotic nature of the generated quantum distribution. This verifies the implementation of the quantum supremacy protocol.

Kechedzhi, Kostyantyn↗

Quantum Technologies for UAS (QTech)

Harness the power of quantum technologies to assure the availability of UAS communications against disruptions. Make use of quantum computing (e.g. quantum optimization) and quantum communication (e.g. quantum key distribution) to address the availability cybersecurity challenge. Our approach is three-fold: (1) Utilize quantum optimization algorithms to design robust network with routing redundancy that can respond adaptively to dynamically changing real-time environment and disruptions, (2) Utilize quantum optimization algorithms resource allocation for detection, localization, and tracking of mobile communication disruption agents, (3) Utilize quantum key distribution (QKD) to execute secure key sharing in high data rate optical communication and/or anti-jamming protocols for secure RF communication.

Quantum Computing↗

Modeling distributed energy resource aggregations in security constrained unit commitment and economic dispatch

The Federal Energy Regulatory Commission (FERC) recently issued Order 2222, which requires all wholesale electricity markets in the US to allow distributed energy resources (DERs) to participate in the market as aggregated resources. These DER aggregations may be composed of many individual resources that are offered and dispatched by the market as a single entity. We present here a model of a distributed energy resource aggregator (DERA) that is scheduled by a market operator’s security constrained unit commitment (SCUC) and security constrained economic dispatch (SCED). The DERA model includes constraints for battery energy storage systems (BESSs), demand response resources (DRRs), and a simple distributed energy resource (DER). This paper describes a model for each resource type and presents two methods for the DERA to generate market offer curves: a profit-maximizing optimization to compute cost curves and a direct cost algorithm to determine dispatch costs for each resource and combine into cost curves. Once all participating DERAs are scheduled in SCUC/SCED, the model is then modified to dispatch individual DERs to maximize profit or minimize schedule deviation of the DERAs. A simulation of a representative day illustrates the DERA offers, the scheduled generation, and the DERA dispatch. Findings show the potential for unavoidable schedule deviations due to internal DER constraints and due to economic incentives to deviate from the SCUC/SCED schedules. This highlights the importance of DERA offer construction on market efficiency and system reliability. Novel aspects of our approach include: (1) We consider the asymmetry of price incentives impacting DERAs from the wholesale market compared to those impacting consumers from the retail market, as imposed by current regulations and laws. (2) We model aggregate consumer response through statistically parameterizable utility functions rather than a potentially impractical approach of modeling each individual consumer. (3) We show how to use the DERA operational dispatch model to create offers into the wholesale electricity market. (4) We show how DERAs may fail to meet their scheduled dispatch because the market offer format may not permit them to fully express their operational features such as intertemporal costs and constraints to the market.

aggregations↗

Roadmap and Benchmarking: Privacy in Federated Load Forecasting

Data-driven techniques for energy demand forecasting continue to emerge with promising impacts on distribution grid planning. However, the development of robust and generalizable machine learning models requires that representative high quality training data are available. Distributed energy resources have begun to embed intelligence, gathering large amounts of data on customer demand, behavior, and household devices that are connected to the grid. Though utilities aggregate meter-level demand data for load shaping, demand response, outage management, reliability planning, and billing applications, there lies an inherent privacy concern in sharing consumption data that may identify individual consumer behavioral patterns. Hence, while sharing the data is crucial, the private sensitive customer data must be safeguarded from being exposed or manipulated. In this study, we propose a roadmap for implementing a based privacy preserving framework to support the advancement of data-driven analytics in data-sensitive distributed energy resources environments. The roadmap incorporates federated learning–a distributed training framework, differential privacy–a statistical framework that provides guarantees to safeguard the leakage of sensitive data, secure multiparty computation and homomorphic encryption– techniques for encrypting model gradients and applying secure aggregation on the server. Moreover, we perform baseline experiments on the federated short-term load forecasting (STLF) task using open-source residential load profile datasets, offering insights into the challenges of integrating differential privacy into federated learning.

Abebe, Waqwoya [Oak Ridge National Laboratory (ORN↗

On the reliability of HELIOSAT method: A comparison with experimental data

This study deals with comparison of Cameroonian ground based measurements with estimations done using satellite images, by applying the second version of the HELIOSAT method, as computed by the web tool HELIOCLIM of the SoDa project. Measurements and satellite resources were chosen for the same period of the year 1985, on ten Cameroonian locations, distributed with mean spatial resolution of 200 km between two neighbouring stations. Correlations curves are constructed between daily irradiation estimates from satellite data and from measurements. The consequent regression equations are established, using the least square method. Finally, a brief discussion is carried out on the spatial distribution of errors observed on results. Similarly to previous studies this study confirmed the capability of the method to produce good results for the Garoua location, and established its extension on other sahelian sites, comparatively to the poor agreement observed on southern sites. Furthermore, even if the study was not done on a long period, it brought out the necessity to correct the HELIOSAT results for many Cameroonian locations, to march the experimental results. (author)

14 SOLAR ENERGY↗

Design and early experience with promoting user-created data in CMS

The Computing Model of the CMS experiment [1] does not address transfering user-created data between different Grid sites. Due to the limited resources of a single site, distribution of individual user-created datasets between sites is crucial to ensure accessibility. In contrast to official datasets, there are no special requirements for user datasets (e.g. concerning data quality). The StoreResults service provides a mechanism to elevate user-created datasets to central bookkeeping ensuring the data quality is the same as an official dataset. This is a prerequisite for further distribution within the CMS dataset infrastructure.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

Structured Adaptive Mesh Refinement Adaptations to Retain Performance Portability With Increasing Heterogeneity

Adaptive mesh refinement (AMR) is an important method that enables many mesh-based applications to run at effectively higher resolution within limited computing resources by allowing high resolution only where really needed. This advantage comes at a cost, however: greater complexity in the mesh management machinery and challenges with load distribution. With the current trend of increasing heterogeneity in hardware architecture, AMR presents an orthogonal axis of complexity. Additionally, the usual techniques, such as asynchronous communication and hierarchy management for parallelism and memory that are necessary to obtain reasonable performance are very challenging to reason about with AMR. Different groups working with AMR are bringing different approaches to this challenge. Here, we examine the design choices of several AMR codes and also the degree to which demands placed on them by their users influence these choices.

42 ENGINEERING↗

ArborX

ArborX library tackles a problem of efficiently finding geometric objects that are close in space. Variations of this problem, such as finding the nearest neighbors of a point, or finding all objects within a certain distance, are inherent components of applications in many fields. The data may be large so that solving the problem efficiently may require significant computational resources, such as multiple processors or accelerators such as general purpose GPUs. ArborX' main advantage in its ability to solve large problems efficiently utilizing a combination of distributed and on-node parallelism. ArborX can be run efficiently on a wide variety of hardware, including GPUs from different vendors, which distinguishes it from other available libraries which typically choose only few of these. The other advantage is that it supports both types of user problems: spatial problems (useful for intersections and finding objects within certain distance), and nearest neighbor problems. ArborX also supports flexible interface in its interaction with a user. Particularly, it allows a user to call user's own function on a positive match, a functionality not rarely available in other libraries. ArborX implements construction and traversal algorithms using efficient tree structures, such as bounding volume hierarchy (BVH). At its core, it uses linear BVH for its low construction cost and sufficient quality. ArborX is written using C++, and is parallelized using the message passing interface (MPI) for the distributed communication, and the Kokkos library for on-node parallelism. This approach allows ArborX to be run on a wide variety of hardware, from common laptops and desktops to supercomputers while using the same codebase. ArborX also implements several advanced algorithms using geometric search, such as density-based clustering algorithm DBSCAN.

ECP↗

ArborX 2.0

ArborX library tackles a problem of efficiently finding geometric objects that are close in space. Variations of this problem, such as finding the nearest neighbors of a point, or finding all objects within a certain distance, are inherent components of applications in many fields. The data may be large so that solving the problem efficiently may require significant computational resources, such as multiple processors or accelerators such as general purpose GPUs. ArborX' main advantage in its ability to solve large problems efficiently utilizing a combination of distributed and on-node parallelism. ArborX can be run efficiently on a wide variety of hardware, including GPUs from different vendors, which distinguishes it from other available libraries which typically choose only few of these. The other advantage is that it supports both types of user problems: spatial problems (useful for intersections and finding objects within certain distance), and nearest neighbor problems. ArborX also supports flexible interface in its interaction with a user. Particularly, it allows a user to call user's own function on a positive match, a functionality not rarely available in other libraries. ArborX implements construction and traversal algorithms using efficient tree structures, such as bounding volume hierarchy (BVH). At its core, ArborX uses linear BVH for its low construction cost and sufficient quality. ArborX implements both spatial and nearest-neighbor traversal algorithms. ArborX also provides several clustering algorithms (minimum spanning tree, DBSCAN, HDBSCAN*), interpolation using minimum least squares and ray tracing. ArborX is written using C++, and is parallelized using the message passing interface (MPI) for the distributed communication, and the Kokkos library for on-node parallelism. This approach allows ArborX to be run on a wide variety of hardware, from common laptops and desktops to supercomputers while using the same codebase.

Prokopenko, Andrey [Oak Ridge National Laboratory ↗

Evaluating the Efficacy of the Cloud for Cluster Computation

Computing requirements vary by industry, and it follows that NASA and other research organizations have computing demands that fall outside the mainstream. While cloud computing made rapid inroads for tasks such as powering web applications, performance issues on highly distributed tasks hindered early adoption for scientific computation. One venture to address this problem is Nebula, NASA's homegrown cloud project tasked with delivering science-quality cloud computing resources. However, another industry development is Amazon's high-performance computing (HPC) instances on Elastic Cloud Compute (EC2) that promises improved performance for cluster computation. This paper presents results from a series of benchmarks run on Amazon EC2 and discusses the efficacy of current commercial cloud technology for running scientific applications across a cluster. In particular, a 240-core cluster of cloud instances achieved 2 TFLOPS on High-Performance Linpack (HPL) at 70% of theoretical computational performance. The cluster's local network also demonstrated sub-100 ?s inter-process latency with sustained inter-node throughput in excess of 8 Gbps. Beyond HPL, a real-world Hadoop image processing task from NASA's Lunar Mapping and Modeling Project (LMMP) was run on a 29 instance cluster to process lunar and Martian surface images with sizes on the order of tens of gigapixels. These results demonstrate that while not a rival of dedicated supercomputing clusters, commercial cloud technology is now a feasible option for moderately demanding scientific workloads.

Cloud Computing↗