SEARCH · Search NASA

Results for “Runtime Systems”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 595 records · Page 33

All sources NASA NTRS DOE OSTI

DOE OSTI2016

Many-Core Acceleration of a Discrete Ordinates Transport Mini-App at Extreme Scale, In: ISC High Performance 2016: High Performance Computing

Time-dependent deterministic discrete ordinates transport codes are an important class of application which provide significant challenges for large, many-core systems. One such challenge is the large memory capacity needed by the solve step, which requires us to have a scalable solution in order to have enough node-level memory to store all the data. In our previous work, we demonstrated the first implementation which showed a significant performance benefit for single node solves using GPUs. In this paper we extend our work to large problems and demonstrate the scalability of our solution on two Petascale GPU-based supercomputers: Titan at Oak Ridge and Piz Daint at CSCS. Our results show that our improved node-level parallelism scheme scales just as well across large systems as previous approaches when using the tried and tested KBA domain decomposition technique. We validate our results against an improved performance model which predicts the runtime of the main ‘sweep’ routine when running on different hardware, including CPUs or GPUs.

Computer Science↗

DOE OSTI2022

Search of strong lens systems in the Dark Energy Survey using convolutional neural networks

We present our search for strong lens, galaxy-scale systems in the first data release of the Dark Energy Survey (DES), based on a color-selected parent sample of 18 745 029 luminous red galaxies (LRGs). We used a convolutional neural network (CNN) to grade this LRG sample with values between 0 (non-lens) and 1 (lens). Our training set of mock lenses is data-driven, that is, it uses lensed sources taken from HST-COSMOS images and lensing galaxies from DES images of our LRG sample. A total of 76 582 cutouts were obtained with a score above 0.9, which were then visually inspected and classified into two catalogs. The first one contains 405 lens candidates, of which 90 present clear lensing features and counterparts, while the other 315 require more evidence, such as higher resolution imaging or spectra, to be conclusive. A total of 186 candidates are newly identified by our search, of which 16 are among the 90 most promising (best) candidates. The second catalog includes 539 ring galaxy candidates. This catalog will be a useful false positive sample for training future CNNs. For the 90 best lens candidates we carry out color-based deblending of the lens and source light without fitting any analytical profile to the data. This method is shown to be very efficient in the deblending, even for very compact objects and for objects with a complex morphology. Finally, from the 90 best lens candidates, we selected 52 systems with one single deflector to test an automated modeling pipeline that has the capacity to successfully model 79% of the sample within an acceptable computing runtime.

79 ASTRONOMY AND ASTROPHYSICS↗

DOE OSTI2015

Adding Data Management Services to Parallel File Systems

The objective of this project, called DAMASC for “Data Management in Scientific Computing”, is to coalesce data management with parallel file system management to present a declarative interface to scientists for managing, querying, and analyzing extremely large data sets efficiently and predictably. Managing extremely large data sets is a key challenge of exascale computing. The overhead, energy, and cost of moving massive volumes of data demand designs where computation is close to storage. In current architectures, compute/analysis clusters access data in a physically separate parallel file system and largely leave it scientist to reduce data movement. Over the past decades the high-end computing community has adopted middleware with multiple layers of abstractions and specialized file formats such as NetCDF-4 and HDF5. These abstractions provide a limited set of high-level data processing functions, but have inherent functionality and performance limitations: middleware that provides access to the highly structured contents of scientific data files stored in the (unstructured) file systems can only optimize to the extent that file system interfaces permit; the highly structured formats of these files often impedes native file system performance optimizations. We are developing Damasc, an enhanced high-performance file system with native rich data management services. Damasc will enable efficient queries and updates over files stored in their native byte-stream format while retaining the inherent performance of file system data storage via declarative queries and updates over views of underlying files. Damasc has four key benefits for the development of data-intensive scientific code: (1) applications can use important data-management services, such as declarative queries, views, and provenance tracking, that are currently available only within database systems; (2) the use of these services becomes easier, as they are provided within a familiar file-based ecosystem; (3) common optimizations, e.g., indexing and caching, are readily supported across several file formats, avoiding effort duplication; and (4) performance improves significantly, as data processing is integrated more tightly with data storage. Our key contributions are: SciHadoop which explores changes to MapReduce assumption by taking advantage of semantics of structured data while preserving MapReduce’s failure and resource management; DataMods which extends common abstractions of parallel file systems so they become programmable such that they can be extended to natively support a variety of data models and can be hooked into emerging distributed runtimes such as Stanford’s Legion; and Miso which combines Hadoop and relational data warehousing to minimize time to insight, taking into account the overhead of ingesting data into data warehousing.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗

NASA NTRS2009

Analog Input Data Acquisition Software

DAQ Master Software allows users to easily set up a system to monitor up to five analog input channels and save the data after acquisition. This program was written in LabVIEW 8.0, and requires the LabVIEW runtime engine 8.0 to run the executable.

Arens, Ellen↗

DOE OSTI2017

High Performance Computing Based Parallel HIearchical Modal Association Clustering (HPAR HMAC)

For many applications, clustering is a crucial step in order to gain insight into the makeup of a dataset. The best approach to a given problem often depends on a variety of factors, such as the size of the dataset, time restrictions, and soft clustering requirements. The HMAC algorithm seeks to combine the strengths of 2 particular clustering approaches: model-based and linkage-based clustering. One particular weakness of HMAC is its computational complexity. HMAC is not practical for mega-scale data clustering. For high-definition imagery, a user would have to wait months or years for a result; for a 16-megapixel image, the estimated runtime skyrockets to over a decade! To improve the execution time of HMAC, it is reasonable to consider an multi-core implementation that utilizes available system resources. An existing imple-mentation (Ray and Cheng 2014) divides the dataset into N partitions - one for each thread prior to executing the HMAC algorithm. This implementation benefits from 2 types of optimization: parallelization and divide-and-conquer. By running each partition in parallel, the program is able to accelerate computation by utilizing more system resources. Although the parallel implementation provides considerable improvement over the serial HMAC, it still suffers from poor computational complexity, O(N2). Once the maximum number of cores on a system is exhausted, the program exhibits slower behavior. We now consider a modification to HMAC that involves a recursive partitioning scheme. Our modification aims to exploit divide-and-conquer benefits seen by the parallel HMAC implementation. At each level in the recursion tree, partitions are divided into 2 sub-partitions until a threshold size is reached. When the partition can no longer be divided without falling below threshold size, the base HMAC algorithm is applied. This results in a significant speedup over the parallel HMAC.

Patlolla, Dilip R↗

DOE OSTI2016

Building America Case Study: Control Retrofits for Multifamily Domestic Hot Water Recirculation Systems, Brooklyn, New York

Domestic hot water (DHW) heating is the second largest energy end use in U.S. buildings, exceeded only by space conditioning. Recirculation systems consisting of a pump and piping loop(s) are commonly used in multifamily buildings to reduce wait time for hot water at faucets; however, constant pumping increases energy consumption by exposing supply and return line piping to continuous heat loss, even during periods when there is no demand for hot water. In this study, ARIES installed and tested two types of recirculation controls in a pair of buildings in order to evaluate their energy savings potential. Demand control, temperature modulation controls, and the simultaneous operation of both were compared to the baseline case of constant recirculation. Additionally, interactive effects between DHW control fuel reductions and space conditioning (heating and cooling) were estimated in order to make more realistic predictions of the payback and financial viability of retrofitting DHW systems with these controls. Results showed that DHW fuel consumption reduced by 7% after implementing the demand control technique, 2% after implementing temperature modulation, and 15% after implementing demand control and temperature modulation techniques simultaneously; recirculation pump runtime was reduced to 14 minutes or less per day. With space heating and cooling interactions included, the estimated annual cost savings were 8%, 1%, and 14% for the respective control techniques. Possible complications in the installation, commissioning and operation of the controls were identified and solutions offered.

ARIES↗

DOE OSTI2014

Tool coupling for the design and operation of building energy and control systems based on the Functional Mock-up Interface standard

This paper describes software tools developed at the Lawrence Berkeley National Laboratory (LBNL) that can be coupled through the Functional Mock-up Interface standard in support of the design and operation of building energy and control systems. These tools have been developed to address the gaps and limitations encountered in legacy simulation tools. These tools were originally designed for the analysis of individual domains of buildings, and have been difficult to integrate with other tools for runtime data exchange. The coupling has been realized by use of the Functional Mock-up Interface for co-simulation, which standardizes an application programming interface for simulator interoperability that has been adopted in a variety of industrial domains. As a variety of coupling scenarios are possible, this paper provides users with guidance on what coupling may be best suited for their application. Furthermore, the paper illustrates how tools can be integrated into a building management system to support the operation of buildings. These tools may be a design model that is used for real-time performance monitoring, a fault detection and diagnostics algorithm, or a control sequence, each of which may be exported as a Functional Mock-up Unit and made available in a building management system as an input/output block. We anticipate that this capability can contribute to bridging the observed performance gap between design and operational energy use of buildings.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

DOE OSTI2025

Limitations of Fault-Tolerant Quantum Linear System Solvers for Quantum Power Flow

Quantum computers hold promise for solving problems intractable for classical computers, especially those with high time or space complexity. Practical quantum advantage can be said to exist for such problems when the end-to-end time for solving such a problem using a classical algorithm exceeds that required by a quantum algorithm. Reducing the power flow (PF) problem into a linear system of equations allows for the formulation of quantum PF (QPF) algorithms, which are based on solving methods for quantum linear systems such as the Harrow-Hassidim-Lloyd (HHL) algorithm. Speedup from using QPF algorithms is often claimed to be exponential when compared to classical PF solved by state-of-the-art algorithms. Here, we investigate the potential for practical quantum advantage in solving QPF compared to classical methods on gate-based quantum computers. Notably, this paper does not present a new QPF solving algorithm but scrutinizes the end-to-end complexity of the QPF approach, providing a nuanced evaluation of the purported quantum speedup in this problem. Our analysis establishes a best-case bound for the HHL-based quantum power flow complexity, conclusively demonstrating that the HHL-based method has higher runtime complexity compared to the classical algorithm for solving the direct current power flow (DCPF) and fast decoupled load flow (FDLF) problem. Notably, our analysis and conclusions can be extended to any quantum linear system solver with rigorous performance guarantees, based on the known complexity lower bounds for this problem. Additionally, we establish that for potential practical quantum advantage (PQA) to exist it is necessary to consider DCPF-type problems with a very narrow range of condition number values and readout requirements.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗

NASA NTRS2008

Orchestrator Telemetry Processing Pipeline

Orchestrator is a software application infrastructure for telemetry monitoring, logging, processing, and distribution. The architecture has been applied to support operations of a variety of planetary rovers. Built in Java with the Eclipse Rich Client Platform, Orchestrator can run on most commonly used operating systems. The pipeline supports configurable parallel processing that can significantly reduce the time needed to process a large volume of data products. Processors in the pipeline implement a simple Java interface and declare their required input from upstream processors. Orchestrator is programmatically constructed by specifying a list of Java processor classes that are initiated at runtime to form the pipeline. Input dependencies are checked at runtime. Fault tolerance can be configured to attempt continuation of processing in the event of an error or failed input dependency if possible, or to abort further processing when an error is detected. This innovation also provides support for Java Message Service broadcasts of telemetry objects to clients and provides a file system and relational database logging of telemetry. Orchestrator supports remote monitoring and control of the pipeline using browser-based JMX controls and provides several integration paths for pre-compiled legacy data processors. At the time of this reporting, the Orchestrator architecture has been used by four NASA customers to build telemetry pipelines to support field operations. Example applications include high-volume stereo image capture and processing, simultaneous data monitoring and logging from multiple vehicles. Example telemetry processors used in field test operations support include vehicle position, attitude, articulation, GPS location, power, and stereo images.

Powell, Mark↗

DOE OSTI2021

Concurrent Optimization of Capital Cost and Expected O&M

Concentrating solar power (CSP) technologies can utilize heat from concentrated sunlight from a field of tracking mirrors to generate electricity, reform fuel, provide process heat, or augment fossil plant heat sources. Electricity-generating power tower systems focus light from thousands of independent heliostats onto a thermal receiver, which uses the focused light to warm a heat transfer fluid (HTF), typically, a molten nitrate salt. The HTF is then sent to a power generation cycle or diverted into thermal energy storage (TES) for later use. Thermal storage is – in principle – a straightforward proposition. However, optimal utilization of a TES resource is complex and multi-faceted: thermal energy may be dispatched to produce electricity immediately upon first availability, or thermal energy may be reserved for next-day peak periods at risk of filling storage and dumping energy, or a portion of the thermal energy can be reserved to maintain equipment temperatures, reducing power cycle startup time, etc. Many possible dispatch permutations variously emphasize producing peak power, operating through transients, expediting daily startup, etc. The best operation strategy can change day-to-day throughout the year, depending on the weather and market pricing forecasts. The project we describe in this report develops a software package that allows users to explore design optimization, operations decisions, and performance characterization of concentrating solar power tower plants. Users interface with the tool through a scripting language, and results are reported in time series tables, plots, runtime logs, and design outputs. Users choose from a list of variables such as tower height, solar multiple, design-point irradiance, thermal storage size, etc., and specify information about the system using a list of parameters. The software can then optimize the specified variables to reduce the cost of energy produced by the system while meeting certain production requirements, accounting for uncertain weather and electricity price forecasts, and correcting for equipment failures or repair time. The software we develop is the first comprehensive design tool of its kind to incorporate all of these aspects while being deployed as open source.

14 SOLAR ENERGY↗

DOE OSTI2015

Local search to improve coordinate-based task mapping

We present a local search strategy to improve the coordinate-based mapping of a parallel job’s tasks to the MPI ranks of its parallel allocation in order to reduce network congestion and the job’s communication time. The goal is to reduce the number of network hops between communicating pairs of ranks. Our target is applications with a nearest-neighbor stencil communication pattern running on mesh systems with non-contiguous processor allocation, such as Cray XE and XK Systems. Utilizing the miniGhost mini-app, which models the shock physics application CTH, we demonstrate that our strategy reduces application running time while also reducing the runtime variability. Furthermore, we further show that mapping quality can vary based on the selected allocation algorithm, even between allocation algorithms of similar apparent quality.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2015

Fault-Tolerant and Elastic Streaming MapReduce with Decentralized Coordination

The MapReduce programming model, due to its simplicity and scalability, has become an essential tool for processing large data volumes in distributed environments. Recent Stream Processing Systems (SPS) extend this model to provide low-latency analysis of high-velocity continuous data streams. However, integrating MapReduce with streaming poses challenges: first, the runtime variations in data characteristics such as data-rates and key-distribution cause resource overload, that inturn leads to fluctuations in the Quality of the Service (QoS); and second, the stateful reducers, whose state depends on the complete tuple history, necessitates efficient fault-recovery mechanisms to maintain the desired QoS in the presence of resource failures. We propose an integrated streaming MapReduce architecture leveraging the concept of consistent hashing to support runtime elasticity along with locality-aware data and state replication to provide efficient load-balancing with low-overhead fault-tolerance and parallel fault-recovery from multiple simultaneous failures. Our evaluation on a private cloud shows up to 2:8 improvement in peak throughput compared to Apache Storm SPS, and a low recovery latency of 700 -1500 ms from multiple failures.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2023

libEnsemble: A complete Python toolkit for dynamic ensembles of calculations

Almost all science and engineering applications eventually stop scaling: their runtime no longer decreases as available computational resources increase. Therefore, many applications will struggle to efficiently use emerging extreme-scale high-performance, parallel, and distributed systems. libEnsemble is a complete Python toolkit and workflow system for intelligently driving ensembles of experiments or simulations at massive scales. It enables and encourages multidisciplinary design, decision, and inference studies portably running on laptops, clusters, and supercomputers.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2021

Position Papers for the ASCR Workshop on Cybersecurity and Privacy for Scientific Computing Ecosystems

At the request of the Department of Energy's (DOE) Office of Advanced Scientific Computing Research (ASCR), this program committee has been tasked with organizing a workshop to identify basic research needs in cybersecurity and privacy to better support DOE's science and energy mission. As part of the process, the program committee is soliciting community input in the form of position papers to help identify significant use cases, facility issues, and other barriers to enabling verifiably trustworthy computational science while preserving data confidentiality as appropriate for scientific workflows of interest to DOE. The program committee will review these position papers and based on the fit of their area of expertise and interest, selected contributors will have the opportunity to participate in the workshop currently planned as a virtual event November 3-5th, 2021. The thrust areas that will be explored by this workshop are the following: (1) Algorithms for secure, scalable, privacy-enhancing technologies and frameworks, including: Federated AI/ML, Differential privacy, Randomized algorithms, Adversarial modeling & simulation, Graph algorithms, and Formal methods; (2) Platforms to support the entire scientific-computing ecosystem, including edge computing for large-scale experiments, focusing on heterogeneous systems and distributed systems, including: Heterogeneous computing systems, Distributed computing systems, and Secure data architectures; and (3) Data workflows to allow agile use of data while preserving integrity and privacy, making the important properties verifiable either at runtime or post-computation, including: Integrity and provenance and Data management infrastructure. Topics that are out-of-scope for the workshop include discussing specific proposed solutions or areas that are clearly out of DOE's fundamental and applied-sciences mission scope, e.g., cryptography, enterprise security, and general-operations technology.

97 MATHEMATICS AND COMPUTING↗

NASA NTRS2018

BDDs on the Run

Runtime veriﬁcation (RV) of ﬁrst-order temporal logic must handle a potentially large amount of data, accumulated during the monitoring of an execution. The DEJAVU RV system represents data elements and relations using BDDs. This achieves a compact representation, which allows monitoring long executions. However, the potentially unbounded, and frequently very large amounts of data value scan,ultimately, limit the executions that can be monitored. We present an automatic method for “forgetting” data values when they no longer affect the RV verdict on an observed execution.We describe the algorithm and illustrate its operation through an example.

Peled, Doron↗

DOE OSTI2024

ECP libraries and tools: An overview

The Exascale Computing Project (ECP) Software Technology and Co-Design teams addressed the growing complexities in high-performance computing (HPC) by developing scalable software libraries and tools that leverage exascale system capabilities. As we enter the exascale era, the need for reusable, optimized software solutions that can handle the unique challenges posed by these systems becomes increasingly important. The primary challenges the ECP teams faced were to create software libraries and tools that are performant on exascale architectures and portable and usable across diverse hardware platforms. Efforts addressed issues related to concurrent execution, memory management, and the integration of heterogeneous computing resources, such as GPUs from multiple vendors. The ECP’s strategy involved a structured development process encompassing the creation, optimization, and deployment of software in collaboration with industry, academia, and national laboratories. The project was organized into several technical areas: co-design of domain-specific suites with target applications, programming models and runtimes, development tools, mathematical libraries, data and visualization tools, and software ecosystem and delivery mechanisms. ECP has successfully developed a large portfolio of software libraries and tools that demonstrate significant improvements in performance and scalability on exascale systems. These products have been integrated into the Department of Energy’s computing facilities, supporting various scientific applications and ensuring robust performance across different hardware setups. ECP advancements in software development for exascale computing highlight the importance of a collaborative and adaptive approach to handling next-generation HPC systems complexities. The lessons learned emphasize the need for continuous engagement with end-users and vendors, and the importance of maintaining a balance between innovation and practical implementation. Future efforts will focus on ensuring scalability, keeping pace with rapid hardware advancements, and further enhancing the interoperability and usability of the software ecosystem. In conclusion, subsequent articles in this special issue provide in-depth discussions and case studies into specific library and tool efforts.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2025

IRIS-MEMFLOW: Data Flow-Enabled Portable Memory Orchestration in IRIS Runtime for Diverse Heterogeneity

Task-based programming models and execution paradigms provide a means to decompose a computation by expressing it as a graph in which each node represents a specific computation operating on memory objects and the edges define the dependencies in the execution flow. In this execution model, independent nodes in the graph can be executed concurrently in different computing devices, making it suitable for heterogeneous systems in which computing devices with different architectures coexist. However, careful memory orchestration across heterogeneous devices is needed because copies of the same memory object may reside in multiple devices during execution. Manually ensuring such an orchestration is quite challenging. Not only must an application developer guard against race conditions, but they must also optimize data movement between the host and devices because unnecessary data movement significantly impacts performance. To mitigate these challenges, we enhance the IRIS heterogeneous runtime and introduce IRIS-MEMFLOW–a data flow–enabled portable memory abstraction for seamlessly orchestrating memory in diverse heterogeneous computing environments. By using data-flow analysis, IRIS-MEMFLOW guards against race conditions while multiple heterogeneous devices access memory objects. IRIS-MEMFLOW also optimizes data movement between the host and devices without manual intervention. As a result, IRIS provides improved programming productivity, performance, and portability for multidevice heterogeneous executions in high-performance computing and cloud systems that run diverse architectures from different vendors. The efficacy of IRIS-MEMFLOW is evaluated through experiments that show its capability in terms of programming productivity, multidevice heterogeneity, portability, and low overhead versus the state of the art.

Monil, M. A. H. [ORNL] (ORCID:0000000334194037)↗

DOE OSTI2016

Control Strategies to Reduce the Energy Consumption of Central Domestic Hot Water Systems

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗