Search NASA⌕ Search

SEARCH · Search NASA

Results for “DISTRIBUTED DATA PROCESSING”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 451 records · Page 25

Introduction to the JPSS-2 Advanced Technology Microwave Sounder (ATMS) Government Calibration Data Book (GCDB)

The third Advanced Technology Microwave Sounder (ATMS) is an instrument onboard the Joint Polar Satellite System (JPSS), JPSS-2 (renamed NOAA-21 in orbit) mission. This report is to introduce the JPSS-2 Government Calibration Data Book (J2 GCDB) for ATMS, SN 304. This J2 GCDB document contains key information generated during the calibration testing campaign that is driving parameters for radiometric performance. This document also contains supporting data that augments the calibration results. The values in this document are utilized by ATMS’s calibration packet which is, in turn, an integral component in the interpretation of science data. The calibration data in this report was collected from tests such as shelf-level testing, antenna testing, instrument thermal vacuum (TVAC) testing; satellite TVAC testing; and JPSS-2 post-launch tests. JPSS-2 was launched on November 10, 2022. In the subsequent years, the Government will release an ATMS GCDB for each JPSS mission. We expect that all public users can download these ATMS GCDBs from the NOAA operational Integrated Calibration and Validation System (ICVS) website, see more discussions below. The goal of this GCDB is to demonstrate how to characterize ATMS measurements using JPSS-2 ATMS on-orbit operational data and to provide relevant explanations. This document serves as a primary public domain reference for calibrating operational ATMS Raw Data Records (RDR) science data, as used in the current operational Interface Data Processing Segment (IDPS) system. This same RDR science data is distributed through direct broadcast (DB) to DB users for use in their ground processing systems. This J2 GCDB provides the results of the ATMS system radiometric calibration, the antenna flat reflector emissivity [1], the antenna pattern measurements, the antenna pattern corrected brightness temperature [2], the brightness temperature of the lunar disk [3], Lunar Intrusion (LI) correction algorithm [4], receiver spectral parameters, and mechanical alignment on-orbit pointing results, and the striping effect appeared significantly in S-NPP on-orbit radiance data when the data are compared to the Radiative Transfer Model (RTM) simulation in numerical weather prediction (NWP) system [5]. It also provides the parameters required for conversion of telemetry counts to engineering units, for radiometric calibration, and for antenna beam geo-location. Moreover, it provides JPSS-2 ATMS Spectral Response Functions data, some additional information related to ATMS on-orbit performance, on-orbit lunar intrusion correction parameters and Earth contamination bias, and on how to derive ATMS RDR, antenna Temperature Data Records (TDR), and Sensor Data Records (SDR). Furthermore, an introduction of NOAA operational Integrated Calibration and Validation System (ICVS) website and services is added in this J2 GCDB. This ICVS hosts a long-term monitoring system which allows to visualization and comparison of data from JPSS missions, NOAA legacy Polar Operational Environmental Satellites (POES), and Geostationary Operational Environmental Satellites (GOES). From NOAA Comprehensive Large Array-data Stewardship System (CLASS), the public users can download all JPSS ATMS data products for all JPSS missions.

Microwave Sounder↗

Data Albums: An Event Driven Search, Aggregation and Curation Tool for Earth Science

One of the largest continuing challenges in any Earth science investigation is the discovery and access of useful science content from the increasingly large volumes of Earth science data and related information available. Approaches used in Earth science research such as case study analysis and climatology studies involve gathering discovering and gathering diverse data sets and information to support the research goals. Research based on case studies involves a detailed description of specific weather events using data from different sources, to characterize physical processes in play for a specific event. Climatology-based research tends to focus on the representativeness of a given event, by studying the characteristics and distribution of a large number of events. This allows researchers to generalize characteristics such as spatio-temporal distribution, intensity, annual cycle, duration, etc. To gather relevant data and information for case studies and climatology analysis is both tedious and time consuming. Current Earth science data systems are designed with the assumption that researchers access data primarily by instrument or geophysical parameter. Those who know exactly the datasets of interest can obtain the specific files they need using these systems. However, in cases where researchers are interested in studying a significant event, they have to manually assemble a variety of datasets relevant to it by searching the different distributed data systems. In these cases, a search process needs to be organized around the event rather than observing instruments. In addition, the existing data systems assume users have sufficient knowledge regarding the domain vocabulary to be able to effectively utilize their catalogs. These systems do not support new or interdisciplinary researchers who may be unfamiliar with the domain terminology. This paper presents a specialized search, aggregation and curation tool for Earth science to address these existing challenges. The search tool automatically creates curated "Data Albums", aggregated collections of information related to a specific science topic or event, containing links to relevant data files (granules) from different instruments; tools and services for visualization and analysis; and information about the event contained in news reports, images or videos to supplement research analysis. Curation in the tool is driven via an ontology based relevancy ranking algorithm to filter out non-relevant information and data.

Ramachandran, Rahul↗

A PARALLEL MONTE CARLO CODE FOR SIMULATING COLLISIONAL N-BODY SYSTEMS

We present a new parallel code for computing the dynamical evolution of collisional N-body systems with up to N {approx} 10{sup 7} particles. Our code is based on the Henon Monte Carlo method for solving the Fokker-Planck equation, and makes assumptions of spherical symmetry and dynamical equilibrium. The principal algorithmic developments involve optimizing data structures and the introduction of a parallel random number generation scheme as well as a parallel sorting algorithm required to find nearest neighbors for interactions and to compute the gravitational potential. The new algorithms we introduce along with our choice of decomposition scheme minimize communication costs and ensure optimal distribution of data and workload among the processing units. Our implementation uses the Message Passing Interface library for communication, which makes it portable to many different supercomputing architectures. We validate the code by calculating the evolution of clusters with initial Plummer distribution functions up to core collapse with the number of stars, N, spanning three orders of magnitude from 10{sup 5} to 10{sup 7}. We find that our results are in good agreement with self-similar core-collapse solutions, and the core-collapse times generally agree with expectations from the literature. Also, we observe good total energy conservation, within {approx}< 0.04% throughout all simulations. We analyze the performance of the code, and demonstrate near-linear scaling of the runtime with the number of processors up to 64 processors for N = 10{sup 5}, 128 for N = 10{sup 6} and 256 for N = 10{sup 7}. The runtime reaches saturation with the addition of processors beyond these limits, which is a characteristic of the parallel sorting algorithm. The resulting maximum speedups we achieve are approximately 60 Multiplication-Sign , 100 Multiplication-Sign , and 220 Multiplication-Sign , respectively.

79 ASTRONOMY AND ASTROPHYSICS↗

Optimization of distributed compute resources utilization in the CMS Global Pool

The CMS Submission Infrastructure is the primary system for managing computing resources for CMS workflows, including data processing, simulation, and analysis. It integrates geographically distributed resources from Grid, HPC, and cloud providers into federated pools managed by HTCondor and Glidein- WMS, for a total of around 500k CPU cores. This system dynamically manages workloads based on priorities defined by the collaboration. Additionally, CMS scheduling strategies must be flexible to handle multiple concurrent workloads while considering changing processing demands and resource availability from various providers.Efficient utilization of vast amounts of distributed compute resources is a key element for the success of the scientific programs of the LHC experiments. Optimizing the system is essential to maximize resource efficiency and fully utilize the distributed computing power. The CMS Submission Infrastructure team thus systematically investigates sources of inefficiency in workload scheduling to reduce their impact. In addition, a strategy of pilot overloading has been introduced to compensate for other inefficiency sources, thereby optimizing resource utilization and enhancing computational throughput.

Mascheroni, Marco [UC, San Diego (main)]↗

Network-Capable Application Process and Wireless Intelligent Sensors for ISHM

Intelligent sensor technology and systems are increasingly becoming attractive means to serve as frameworks for intelligent rocket test facilities with embedded intelligent sensor elements, distributed data acquisition elements, and onboard data acquisition elements. Networked intelligent processors enable users and systems integrators to automatically configure their measurement automation systems for analog sensors. NASA and leading sensor vendors are working together to apply the IEEE 1451 standard for adding plug-and-play capabilities for wireless analog transducers through the use of a Transducer Electronic Data Sheet (TEDS) in order to simplify sensor setup, use, and maintenance, to automatically obtain calibration data, and to eliminate manual data entry and error. A TEDS contains the critical information needed by an instrument or measurement system to identify, characterize, interface, and properly use the signal from an analog sensor. A TEDS is deployed for a sensor in one of two ways. First, the TEDS can reside in embedded, nonvolatile memory (typically flash memory) within the intelligent processor. Second, a virtual TEDS can exist as a separate file, downloadable from the Internet. This concept of virtual TEDS extends the benefits of the standardized TEDS to legacy sensors and applications where the embedded memory is not available. An HTML-based user interface provides a visual tool to interface with those distributed sensors that a TEDS is associated with, to automate the sensor management process. Implementing and deploying the IEEE 1451.1-based Network-Capable Application Process (NCAP) can achieve support for intelligent process in Integrated Systems Health Management (ISHM) for the purpose of monitoring, detection of anomalies, diagnosis of causes of anomalies, prediction of future anomalies, mitigation to maintain operability, and integrated awareness of system health by the operator. It can also support local data collection and storage. This invention enables wide-area sensing and employs numerous globally distributed sensing devices that observe the physical world through the existing sensor network. This innovation enables distributed storage, distributed processing, distributed intelligence, and the availability of DiaK (Data, Information, and Knowledge) to any element as needed. It also enables the simultaneous execution of multiple processes, and represents models that contribute to the determination of the condition and health of each element in the system. The NCAP (intelligent process) can configure data-collection and filtering processes in reaction to sensed data, allowing it to decide when and how to adapt collection and processing with regard to sophisticated analysis of data derived from multiple sensors. The user will be able to view the sensing device network as a single unit that supports a high-level query language. Each query would be able to operate over data collected from across the global sensor network just as a search query encompasses millions of Web pages. The sensor web can preserve ubiquitous information access between the querier and the queried data. Pervasive monitoring of the physical world raises significant data and privacy concerns. This innovation enables different authorities to control portions of the sensing infrastructure, and sensor service authors may wish to compose services across authority boundaries.

Figueroa, Fernando↗

WetNet operations

WetNet is an interdisciplinary Earth science data analysis and research project with an emphasis on the study of the global hydrological cycle. The project goals are to facilitate scientific discussion, collaboration, and interaction among a selected group of investigators by providing data access and data analysis software on a personal computer. The WetNet system fulfills some of the functionality of a prototype Product Generation System (PGS), Data Archive and Distribution System (DADS), and Information Management System for the Distributed Active Archive Center. The PGS functionality is satisfied in WetNet by processing the Special Sensor Microwave/Imager (SSM/I) data into a standard format (McIDAS) data sets and generating geophysical parameter Level II browse data sets. The DADS functionality is fulfilled when the data sets are archived on magneto optical cartridges and distributed to the WetNet investigators. The WetNet data sets on the magneto optical cartridges contain the complete WetNet processing, catalogue, and menu software in addition to SSM/I orbit data for the respective two week time period.

Goodman, H. Michael↗

Application of ESE Data and Tools to Air Quality Management: Services for Helping the Air Quality Community use ESE Data (SHAirED)

The goal of this REASoN applications and technology project is to deliver and use Earth Science Enterprise (ESE) data and tools in support of air quality management. Its scope falls within the domain of air quality management and aims to develop a federated air quality information sharing network that includes data from NASA, EPA, US States and others. Project goals were achieved through a access of satellite and ground observation data, web services information technology, interoperability standards, and air quality community collaboration. In contributing to a network of NASA ESE data in support of particulate air quality management, the project will develop access to distributed data, build Web infrastructure, and create tools for data processing and analysis. The key technologies used in the project include emerging web services for developing self describing and modular data access and processing tools, and service oriented architecture for chaining web services together to assemble customized air quality management applications. The technology and tools required for this project were developed within DataFed.net, a shared infrastructure that supports collaborative atmospheric data sharing and processing web services. Much of the collaboration was facilitated through community interactions through the Federation of Earth Science Information Partners (ESIP) Air Quality Workgroup. The main activities during the project that successfully advanced DataFed, enabled air quality applications and established community-oriented infrastructures were: develop access to distributed data (surface and satellite), build Web infrastructure to support data access, processing and analysis create tools for data processing and analysis foster air quality community collaboration and interoperability.

Falke, Stefan↗

DER Cybersecurity Detection and Response Suite

SAND2024-08475O The Distributed Energy Resource (DER) Cybersecurity Detection and Response Suite is a solution for distributed energy resource (DER) systems. The DER Security Orchestration, Automation, and Response (SOAR) solution that uses alerts from signature- and behavior-based Intrusion Detection Systems are intended to be deployed as bump-in-the-wire (BITW) devices in front of DER equipment. The fielded application would use multiple intrusion detection systems that report data to SOAR to respond to cyberattacks. The suite consists of two software components: • The proactive intrusion detection and mitigation system (PIDMS) secures grid-edge photovoltaic smart inverters and other equipment in distributed energy resource systems. It is a distributed BITW solution; cyber and physical data are automatically processed using network inspection tools and custom machine learning algorithms to detect abnormal events and correlate cyber-physical events. • The Security Orchestration, Automation, and Response for Distributed Energy Resources (SOAR4DER) application ingests data from several intrusion detection systems to quickly block attacks and revert DER systems to good states. Using a collection of intrusion detection system technologies on a BITW device, it incorporates physical and cyber data to detect abnormal and potential malicious behaviors. Multiple SOAR playbooks then use the intrusion detection system data streams to automatically defend the system. SOAR4DER system testing showed detection and response times under 30 seconds for all adversary reconnaissance, denial-of-service attacks, malicious Modbus commands, brute-force logins, and machine-in-the-middle attacks. Sandia National Laboratories is a multimission laboratory managed and operated by National Technology & Engineering Solutions of Sandia, LLC, a wholly owned subsidiary of Honeywell International Inc., for the U.S. Department of Energy’s National Nuclear Security Administration under contract DE-NA0003525.

Johnson, Jay↗

Distributing Power Grid State Estimation on HPC Clusters A System Architecture Prototype

The future power grid is expected to further expand with highly distributed energy sources and smart loads. The increased size and complexity lead to increased burden on existing computational resources in energy control centers. Thus the need to perform real-time assessment on such systems entails efficient means to distribute centralized functions such as state estimation in the power system. In this paper, we present our early prototype of a system architecture that connects distributed state estimators individually running parallel programs to solve non-linear estimation procedure. The prototype consists of a middleware and data processing toolkits that allows data exchange in the distributed state estimation. We build a test case based on the IEEE 118 bus system and partition the state estimation of the whole system model to available HPC clusters. The measurement from the testbed demonstrates the low overhead of our solution.

Distributed architecture↗

The problem analysis of data processing performance in nuclear power plant computerized information and control system - 288

This paper is based on the data processing performance problems of operating command delay and system display not updating real-time for nuclear power plant computerized information and control system. The plant computerized information and control system is responsible for monitoring and controlling the plant operation, and it is a key part in the plant DCS system. Its data processing performance will affect the operation stability of nuclear power plant. The architecture, hardware configuration and data communication for plant computerized information and control system among the different levels in DCS are described. The main reasons affecting data processing performance in different signal channels are analyzed according to the plant load test results. The CPU load curves for real time server and central processing server are displayed, although the detailed cause for operating command delay and system display not updating realtime are not found, central processing server load is heavier than real time server which remind us to analyze the data processing of software resource scheduling deeply. So the resource distribution problem of software execution task is also analyzed according to CPU load test results. This paper provides the optimization analysis for data processing performance, and it is available and effective according to the latest test results. (authors)

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

Aerial Lidar and Geomorphic Change Detection Software- Tools for Monitoring Canyon Sediments at LANL - 18668

The Los Alamos National Laboratory Environmental Remediation program has conducted extensive geomorphic surveys of sediment deposits in the canyon bottoms within and surrounding the facility. The goal of the geomorphic surveys was to provide basic information needed to characterize chemical inventories of the sediments downstream of historical release sites. Sediment units were mapped based on depositional setting and age. Representative characterization samples were collected for each unit and submitted for chemical analysis. Chemical inventories were estimated by calculating the total volume of each depositional unit and multiplying by average chemical concentration. This method of characterization provides a spatial distribution of chemical data based on fluvial processes. In general, for the canyon bottom sediments, the higher concentrations of industrial chemicals were associated with fine-grained floodplain deposits closest to the source. Since initial characterization, many of the reaches have undergone changes in response to post-wildfire floods and severe rain events. These events impacted the channel and floodplain deposits. Much of the elevated concentrations are confined to the over-bank deposits distal from the primary channel. In order to ensure that the sediment units containing elevated chemical concentrations are stable, repeat aerial light detection and ranging (LiDAR) surveys are conducted and changes are assessed using the Geomorphic Change Detection (GCD) plug-in for ArcGIS. These tools provide high confidence uncertainty estimates and extensive areal coverage. This is a valuable method for long-term monitoring of surface sediments with low-level but widespread contaminant distributions. All of the monitored canyon bottom reaches pass risk assessment and present technically infeasible sediment removal options. Following the 2014 monsoon season, aerial LiDAR data has been collected annually and the resulting digital elevation models (DEMs) processed using the GCD method. Changes are identified using GCD and are verified by field walk downs. If erosion is identified in areas which are vulnerable to mobilizing contaminated sediment, enhanced controls can be installed to stabilize banks or other areas with high erosion potential. Annual sediment monitoring samples are collected where there is evidence for flow and deposition within the past year. Since 2014, no severe floods have been catalogued using the LiDAR surveys and GCD software. The recent rainfall/runoff events have not caused major changes to the canyon bottom channels and floodplains. Geomorphic processes identified in the monitoring results include channel aggradation and incision that over the course of a monsoon season results in minor changes to the system. Other active processes contributing to observed changes are minor slides, flows, slumps, and falls of unconsolidated sediment on steep bedrock or soil surfaces. A comprehensive baseline has been established by geomorphic mapping and characterization sampling. Periodic monitoring consists of LiDAR surveys and GCD analysis which offers an effective and defensible methodology for documenting stability of surface sediments. (authors)

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗

Multi-Fidelity Gaussian Process for Distribution System Voltage Probabilistic Estimation with PVs

The increasing penetration of behind-the-meter PVs causes challenges to maintain voltage security due to the lack of distribution system visibility. This paper proposes a nonlinear autoregressive Gaussian process (NARGP) approach to fuse limited number of SCADA/AMI data together with historical pseudo measurements for distribution node voltage probabilistic estimation. The high-fidelity SCADA data are fused with the low-fidelity AMI and pseudo measurements by the autoregressive algorithm embedded in the Gaussian process. This allows us to use multi-fidelity data to achieve entire distribution system voltage visibility. Numerical results carried out on the IEEE 123node system demonstrate that the NARGP method is able to obtain high accuracy in estimating bus voltage and quantifying estimation uncertainties as compared to other approaches.

distribution system estimation↗

Spatio-Temporal Signal Twice-Whitening Algorithms on the hx3100 Ultra-Low Power Multicore Processor

While modern signal detection theory fully accounts for spatially distributed sensors, exploiting these techniques for real-time sensing using large, underwater acoustic arrays requires advances in the spatio-temporal signal processing algorithms. In particular, the computational complexity of many spatio-temporal processing techniques is so large that conventional computer processors lack sufficient throughput to provide real-time processing of large spatio-temporal data sets. These limits are exacerbated when constraints, such as power consumption or footprint, reduce the available computational resources. In this report, we demonstrate an implementation of a signal twice-whitening algorithm that is better suited for processing spatio-temporal data in real time. We emphasize these advances by implementing data whitening on the Coherent Logix hx3100 processor, a programmable multicore processor intended for low-power and high-throughput signal processing. These results serve as an example of how the novel capabilities available from emerging multicore processor platforms can provide real-time, software-defined processing of large data sets acquired by spatially distributed sensing.

99 GENERAL AND MISCELLANEOUS↗

Neural network for processing both spatial and temporal data with time based back-propagation

Neural networks are computing systems modeled after the paradigm of the biological brain. For years, researchers using various forms of neural networks have attempted to model the brain's information processing and decision-making capabilities. Neural network algorithms have impressively demonstrated the capability of modeling spatial information. On the other hand, the application of parallel distributed models to the processing of temporal data has been severely restricted. The invention introduces a novel technique which adds the dimension of time to the well known back-propagation neural network algorithm. In the space-time neural network disclosed herein, the synaptic weights between two artificial neurons (processing elements) are replaced with an adaptable-adjustable filter. Instead of a single synaptic weight, the invention provides a plurality of weights representing not only association, but also temporal dependencies. In this case, the synaptic weights are the coefficients to the adaptable digital filters. Novelty is believed to lie in the disclosure of a processing element and a network of the processing elements which are capable of processing temporal as well as spacial data.

Villarreal, James A.↗

The Solar Dynamics Observatory: Your Eye on the Sun

The Solar Dynamics Observatory (SDO) was launched on February 11,2010 into partly cloudy skies over Cape Canaveral, Florida. SDO moved into a 28 degree inclined geosynchronous orbit over the longitude of the ground station in New Mexico. SDO is the first Space Weather Mission in NASA's Living With a Star Program. SDO's main goal is to understand and predict those solar variations that influence life on Earth and our technological systems. The SDO science investigations will determine how the Sun's magnetic field is generated and structured, how this stored magnetic energy is released into the heliosphere as the solar wind, energetic particles, and variations in the solar irradiance. The SDO mission consists of three scientific investigations (AIA, EVE, and HMI), a spacecraft bus, and a dedicated Ka-band ground station to handle the 150 Mbps data flow. SDO continues a long tradition of NASA missions providing calibrated solar spectral irradiance data, in this case using multiple measurements of the irradiance and rocket underflights of the spacecraft. The other instruments on SDO will be used to explain and develop predictive models of the solar spectral irradiance in the extreme ultraviolet. Science teams at LMSAL, LASP, and Stanford are responsible for processing, analyzing, distributing, and archiving the science data. We will talk about the building of SDO, its launch, and the data and science it will provide to NASA.

Pesnell, William Dean↗

The Solar Dynamics Observatory: Your On-Orbit Eye on the Sun

The Solar Dynamics Observatory (SDO) was launched on February 11, 2010 into the partly cloudy skies above Cape Canaveral, Florida. Over the next month SDO moved into a 28 degree inclined geosynchronous orbit at the longitude of the ground station in New Mexico. SDO is the first Space Weather Mission in NASA's Living With a Star Program. SDO's main goal is to understand and predict those solar variations that influence life on Earth and our technological systems. The SDO science investigations will determine how the Sun's magnetic field is generated and structured, how this stored magnetic energy is released into the heliosphere as the solar wind, energetic particles, and variations in the solar irradiance. The SDO mission consists of three scientific investigations (AIA, EVE, and HMI), a spacecraft bus, and a dedicated Ka-band ground station to handle the 150 Mbps data flow. SDO continues a long tradition of NASA missions providing calibrated solar spectral irradiance data, in this case using multiple measurements of the irradiance and rocket underflights of the spacecraft. The other instruments on SDO will be used to explain and develop predictive models of the solar spectral irradiance in the extreme ultraviolet. Science teams at LMSAL, LASP, and Stanford are responsible for processing, analyzing, distributing, and archiving the science data. We will talk about the launch of SDO and describe the data and science it is providing to NASA.

Pesnell, W. Dean↗

E4D_RT

Time-lapse ERT imaging for monitoring both natural and engineered subsurface processes has advanced rapidly over the past 15-20 years. However, imaging results generally required a significant amount of manual and computational effort, and therefore were not available while the process was occurring. Although the value of real-time imaging was recognized, several obstacles prevented it's implementation. E4D_RT addresses these obstacles by 1) providing specialized algorithms that negate the need for user intervention, thereby automating the time-lapse data processing steps, 2) linking field data collection systems with parallel supercomputing systems via wireless data transfer link, and 3) addressing the computational burdens by utilizing distributed memory supercomputing resources, thereby enabling rapid data processing and imaging results.

Source record↗

Using cross-layer adaptations for dynamic data management in large scale coupled scientific workflows, In: SC '13 Proceedings of the International Conference on High Performance Computing, Networking, Storage and Analysis

As system scales and application complexity grow, managing and processing simulation data has become a significant challenge. While recent approaches based on data staging and in-situ/in-transit data processing are promising, dynamic data volumes and distributions, such as those occurring in AMR-based simulations, make the efficient use of these techniques challenging. In this paper we propose cross-layer adaptations that address these challenges and respond at runtime to dynamic data management requirements. Specifically we explore (1) adaptations of the spatial resolution at which the data is processed, (2) dynamic placement and scheduling of data processing kernels, and (3) dynamic allocation of in-transit resources. We also exploit coordinated approaches that dynamically combine these adaptations at the different layers. We evaluate the performance of our adaptive cross-layer management approach on the Intrepid IBM-BlueGene/P and Titan Cray-XK7 systems using Chombo-based AMR applications, and demonstrate its effectiveness in improving overall time-to-solution and increasing resource efficiency.

Computer Science↗