Search NASA⌕ Search

SEARCH · Search NASA

Results for “Best Effort”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records
All sourcesNASA NTRSDOE OSTI
NASA NTRS2018

A Roundtrip Journey off the Career Track: Having a Successful Career Despite Your Best Efforts

Although career tracks in science and engineering are often depicted as straight and smooth, many contain detours, side journeys and interruptions. In the end, these detours not only influence who we are, they can make us better scientists and engineers. In this presentation, Dr. Ladbury tells the tale of how his detours took him to far-flung locations from the Himalayan mountains to the African Savannah and eventually back to a rewarding career in radiation physics.

Space Radiation↗
NASA NTRSDate not supplied

Gateway Command and Data Handling Network Implementation and Validation

As initial Lunar Gateway modules approach design maturity, the Artemis Network Validation and Integration Lab (ANVIL) has begun demonstrations to validate avionics network dataflows. The flight architecture design includes utilizing Time-Triggered Ethernet (TTE) and layer-3 switching capabilities to enable greater automation and flexibility of critical and best effort traffic. Critical traffic is considered as Time-Triggered (TT), Rate Constrained (RC) and prioritized Best Effort (BE) traffic classes. End systems, such as mission computers, power control, and robotics, use three planes and all traffic classes while other devices interface via Best Effort. Typical best effort devices include video, laptops, wireless access points, and payloads. Other devices have a various hybrid approach of interfaces including alarms, telemetry/logging, and communication units. In the paper, we will present an update of the Gateway network architecture and how the system will operate nominally and during a stack topology reconfiguration. We will also discuss the network risks, and trade-offs of performance, flexibility, and redundancy. Finally, we will show the process for validation, demonstration, and verification approaches to the vehicle network.

Gateway↗
NASA NTRS2012

Enhancing TRMM with a Near-realtime System and Special Products

During the development phase of the TRMM mission, the decision was made that no near-realtime (NRT) system would be available during the TRMM 5 year extended mission period. Concern about the costs that might be incurred by requiring such an additional capability In 2000 the TRMM project scientist felt that a best effort should be made to provide such an ability for TRMM. The addition of this best-effort capability has added greatly to the utility of TRMM to the community especially the applications community. This paper will describe the current TRMM NRT products, how they are derived their availability and accessing them. It will provide some examples of how these products have been used.

Stocker, Erich F.↗
NASA NTRS2020

Operations Concept for Responding to Urgent Requests for NASA-ISRO Synthetic Aperture Radar (NISAR)

The NASA-ISRO Synthetic Aperture Radar, or NISAR, mission is an Earth-mapping radar observatory to be launched from Sriharikota (India) in 2022. This mission is a collaboration between the National Aeronautics and Space Administration (NASA) and the Indian Space Research Organization (ISRO). This spacecraft will carry two instruments that will operate at radar wavelengths (L and Sband) and will provide data for understanding changes in the Earth’s land surface. The scientific data from this mission will revolutionize our understanding of the causes and consequences of land surface changes on Earth, ranging from Solid Earth Deformation in the form of natural hazards like earthquakes, volcanic eruptions and landslides, to ecosystem disturbances, to changes in the cryosphere (measurements of polar ice caps, ice sheets and sea ice). A nominal Reference Observation Plan, that repeats roughly every 12-24 days, developed prior to launch by the NISAR Mission Planning team, in consultation with the Science Team, will form the basis of science data collection by the payload instruments onboard the NISAR observatory after launch. Scheduling of science observations for the mission requires accounting for limited spacecraft resources like onboard data storage, downlink capacity, energy/power, thermal limits and instrument duty cycles. In addition to nominal science data collection, the project has a Level 1 requirement to respond to requests for urgent data acquisition over disaster sites (natural or anthropogenic) by scheduling new acquisitions within 24 hrs of notification and delivering science data within 5 hours of data acquisition. This capability is to be exercised on a ‘best-efforts basis’. While the definition of what constitutes an ‘urgent request’, and how such requests would be submitted to the project, is within the domain of the Science Team, the Mission System team is responsible for developing the baseline operations concept and implementation approach for responding to such requests. Given the ‘best-efforts’ nature of this requirement, a few highlevel guidelines have been developed to help guide the formulation of the operations concept, and are presented in this paper. Requests for urgent response data will be accommodated following the guiding principle of minimal to no impact on nominal science and planned engineering activities. No change in satellite orbit or attitude will be made for urgent response. Restricting response approaches to only changing the downlink and/or ground processing priority for existing observations, and adding new observations only in areas where NISAR will not be nominally imaging, allows for minimal impact on the Reference science Observation Plan. No instrument mode changes will be allowed for urgent response (except for high-priority requests), and no new observations that impact either planned science or engineering activities will be scheduled. Additionally, data requests must fit within the available project resource margins (both spacecraft and ground resources are to be evaluated). Both JPL and ISRO will be involved at various steps of the implementation, irrespective of whether the urgent request is for L-SAR (NASA instrument) or S-SAR (ISRO instrument) or a joint dataset.

Sharma, Priyanka↗
DOE OSTI2022

Satellite-Based Solar Forecasting Including Maintenance of Computer and Storage Equipment at CSU/CIRA (CRADA Final Report)

The Cooperative Institute for Research in the Atmosphere (CIRA) at Colorado State University (CSU) proposes to continue its research partnership with the NREL in Golden, CO. The partnership resonates with the goals outlined in a Memorandum of Understanding between the Department of Energy and the Department of Commerce (with the National Oceanic and Atmospheric Administration (NOAA)) signed in 2011. This CRADA will sustain the research partnership and provide an avenue and resources for ongoing collaborations, including the maintenance of NREL-furnished computer and storage equipment at CSU/CIRA. There is no formal exchange of funds in this CRADA; support above and beyond the in-kind items outlined in this document will come on a best-effort basis and/or in association with formal sponsored-research enabled collaborations. The research conducted under this CRADA continues to work toward enabling and improving short-term solar energy forecasting as developed under previous CIRA/CSU and NREL collaborations, with specific focus on point-based (e.g., solar farm) forecasting on the 1-3 hour timeframe. Under the best-effort actions during the CRADA period, technical maintenance of CIRA's existing satellite-based solar forecasting algorithm ("CIRACast"; Miller et al., 2017) was continued to maintain operability with evolving datasets. These efforts benefit the public by maintaining national capability for point solar forecasting using geostationary observation platforms, and laying the groundwork for future technical improvements as opportunities arise.

14 SOLAR ENERGY↗
DOE OSTI2019

Can I/O Variability Be Reduced on QoS-Less HPC Storage Systems?

For a production high-performance computing (HPC) system, where storage devices are shared between multiple applications and managed in a best effort manner, I/O contention is often a major problem. In this paper, we propose a balanced messaging-based re-routing in conjunction with throttling at the middleware level. This work tackles two key challenges that have not been fully resolved in the past: whether I/O variability can be reduced on a QoS-less HPC storage system, and how to design a runtime scheduling system that can scale up to a large amount of cores. The proposed scheme uses a two-level messaging system to re-route I/O requests to a less congested storage location so that write performance is improved, while limiting the impact on read by throttling re-routing. An analytical model is derived to guide the setup of optimal throttling factor. We thoroughly analyze the virtual messaging layer overhead and explore whether the in-transit buffering is effective in managing I/O variability. Contrary to the intuition, in-transit buffer cannot completely solve the problem. It can reduce the absolute variability but not the relative variability. The proposed scheme is verified against a synthetic benchmark as well as being used by production applications.

Huang, Dan↗
NASA NTRS2011

Land and Atmosphere Near-Real-Time Capability for Earth Observing System

The past decade has seen a rapid increase in availability and usage of near-real-time data from satellite sensors. The EOSDIS (Earth Observing System Data and Information System) was not originally designed to provide data with sufficiently low latency to satisfy the requirements for near-real-time users. The EOS (Earth Observing System) instruments aboard the Terra, Aqua and Aura satellites make global measurements daily, which are processed into higher-level 'standard' products within 8-40 hours of observation and then made available to users, primarily earth science researchers. However, applications users, operational agencies, and even researchers desire EOS products in near-real-time to support research and applications, including numerical weather and climate prediction and forecasting, monitoring of natural hazards, ecological/invasive species, agriculture, air quality, disaster relief and homeland security. These users often need data much sooner than routine science processing allows, usually within 3 hours, and are willing to trade science product quality for timely access. While Direct Broadcast provides more timely access to data, it does not provide global coverage. In 2002, a joint initiative between NASA (National Aeronautics and Space Administration), NOAA (National Oceanic and Atmospheric Administration), and the DOD (Department of Defense) was undertaken to provide data from EOS instruments in near-real-time. The NRTPE (Near Real Time Processing Effort) provided products within 3 hours of observation on a best-effort basis. As the popularity of these near-real-time products and applications grew, multiple near-real-time systems began to spring up such as the Rapid Response System. In recognizing the dependence of customers on this data and the need for highly reliable and timely data access, NASA's Earth Science Division sponsored the Earth Science Data and Information System Project (ESDIS)-led development of a new near-real-time system called LANCE (Land, Atmosphere Near-Real-Time Capability for EOS) in 2009. LANCE consists of special processing elements, co-located with selected EOSDIS data centers and processing facilities. A primary goal of LANCE is to bring multiple near-real-time systems under one umbrella, offering commonality in data access, quality control, and latency. LANCE now processes and distributes data from the Moderate Resolution Imaging Spectroradiometer (MODIS), Atmospheric Infrared Sounder (AIRS), Advanced Microwave Scanning Radiometer Earth Observing System (AMSR-E), Microwave Limb Sounder (MLS) and Ozone Monitoring Instrument (OMI) instruments within 3 hours of satellite observation. The Rapid Response System and the Fire Information for Resource Management System (FIRMS) capabilities will be incorporated into LANCE in 2011. LANCE maintains a central website to facilitate easy access to data and user services. LANCE products are extensively tested and compared with science products before being made available to users. Each element also plans to implement redundant network, power and server infrastructure to ensure high availability of data and services. Through the user registration system, users are informed of any data outages and when new products or services will be available for access. Building on a significant investment by NASA in developing science algorithms and products, LANCE creates products that have a demonstrated utility for applications requiring near-real-time data. From lower level data products such as calibrated geolocated radiances to higher-level products such as sea ice extent, snow cover, and cloud cover, users have integrated LANCE data into forecast models and decision support systems. The table above shows the current near-real-time product categories by instrument. The ESDIS Project continues to improve the LANCE system and use the experience gained through practice to seek adjustments to improve the quality and performance of the system. For example, anGC-compliant Web Map Service (WMS) will be added shortly that will allow users to download geo-referenced MODIS images for arbitrary bounding boxes. Further, an OGC-compliant Web Coverage Service (WCS) will be added later this year that will expedite user access to arbitrary data subsets or re-formatted products. AIRS images are now served through WMS and available in multiple formats (PNG, GeoTIFF, KMZ). NASA has established a LANCE User Working Group to steer the development of the system and create a forum for sharing ideas and experiences that are expected to further improve the LANCE capabilities. The LANCE system has proved a success by satisfying the growing needs of the applications and operational communities for land and atmosphere data in near-real-time. NASA's Earth Sciences Division was able to leverage existing science research capabilities to provide the near-real-time community with products and imagery that support monitoring of disasters in a timely manner.

Murphy, Kevin J.↗
NASA NTRS2014

International Space Station Data Collection for Disaster Response

Natural disasters - including such events as tropical storms, earthquakes, floods, volcanic eruptions, and wildfires -effect hundreds of millions of people worldwide, and also cause billions of dollars (USD) in damage to the global economy. Remotely sensed data acquired by orbital sensor systems has emerged as a vital tool to identify the extent of damage resulting from a natural disaster, as well as providing near-real time mapping support to response efforts on the ground and humanitarian aid efforts. The International Space Station (ISS) is a unique terrestrial remote sensing platform for acquiring disaster response imagery. Unlike automated remote-sensing platforms it has a human crew; is equipped with both internal and externally-mounted remote sensing instruments; and has an inclined, low-Earth orbit that provides variable views and lighting (day and night) over 95 percent of the inhabited surface of the Earth. As such, it provides a useful complement to free-flyer based, sun-synchronous sensor systems in higher altitude polar orbits. While several nations have well-developed terrestrial remote sensing programs and assets for data collection, many developing nations do not have ready access to such resources. The International Charter, Space and Major Disasters (also known as the "International Disaster Charter", or IDC; http://www.disasterscharter.org/home) addresses this disparity. It is an agreement between agencies of several countries to provide - on a best-effort basis - remotely sensed data of natural disasters to requesting countries in support of disaster response. The lead US agency for interaction with the IDC is the United States Geological Survey (USGS); when an IDC request or "activation" is received, the USGS notifies the science teams for NASA instruments with targeting information for data collection. In the case of the ISS, the Earth Sciences and Remote Sensing (ESRS) Unit, part of the Astromaterials Research and Exploration Science Directorate and supporting the ISS Program Science Office at NASA's Johnson Space Center, receives notification from the USGS and coordinates targeting and data collection with the NASA ISS sensor teams. If data is collected, it is passed back to the USGS for posting on their Hazards Data Distribution System and made available for download. The ISS International Partners (CSA, ESA, JAXA, Roscosmos/Energia) have their own procedures for independently supporting IDC activations using their assets on ISS, and there is currently no joint coordination with NASA ISS sensor teams. Following completion of ISS assembly, NASA remote sensing assets began collecting IDC response data in May 2012. The initial NASA ISS sensor systems available to respond to IDC activations included the ISS Agricultural Camera (ISSAC), an internal multispectral visible-near infrared wavelength system mounted in the Window Observational Research Facility, or WORF; the Crew Earth Observations (CEO) Facility, where the crew collects imagery through Station windows using off-the-shelf handheld digital visible-wavelength cameras; and the Hyperspectral Imager for the Coastal Oceans (HICO), a visible to near-infrared system mounted externally on the Japan Experiment Module Exposed Facility. The ISSAC completed its primary mission and was removed from the WORF in January 2013. It was replaced by the very high resolution ISS SERVIR Environmental Research and Visualization System (ISERV) Pathfinder, a visible-wavelength digital camera, telescope, and pointing system. Since the start of IDC response by NASA sensors on the ISS in May 2012 and as of this report, there have been eighty IDC activations; NASA sensor systems have collected data for twenty-three of these events. Of the twenty-three successful data collections, five involved 2 or more ISS sensor systems responding to the same event. Data has also been collected by International Partners in response to natural disasters, most notably JAXA and Roscosmos/Energia through the Urugan program. Data collected in response to IDC activations is delivered by the ISS sensor teams to the ESRS for quality review and transfer to the USGS, where it is ingested into the Hazards Data Distribution System, or HDDS (https://hdds.usgs.gov/hdds2/; figure 1). This system allows the local agencies that issued the IDC activation request to review and download data. The data is then used to develop secondary products useful for humanitarian response such as flood maps. As of this report, approximately 1000 images collected by NASA ISS sensor systems have been downloaded from the HDDS, indicating that the ISS has assumed a valuable role in disaster response efforts. The ISS is also a unique platform in that it will have multiple users over its lifetime, and that no single remote sensing system has a permanent internal or external berth. This scheduled turnover provides for development of new remote sensing capabilities relevant to disaster response -as well as both research and applied science-and represents a significant contribution to continuance and enhancement of the NASA mission to investigate changes on our home planet.

Stefanov, William L.↗
DOE OSTI2022

Feasibility of critical infrastructure protection using network functions for programmable and decoupled ICS policy enforcement over WAN

Industrial control systems (ICS) represent a major component of our critical infrastructure. With the increasing need for more control and monitoring of such systems, ICS have seen an increase in connectivity to wide area networks (WAN) exposing aging equipment to rapidly evolving cybersecurity threats. Furthermore, the ICS data requires a reliability measure from the networks for critical functions for infrastructure monitoring and control. Especially when remote plant sites are involved such as pipelines, energy distribution networks, and transportation, WAN transport impairments most often provide a best effort delivery with no strict reliability guarantees. Network functions can provide a vendor agnostic, programmable critical infrastructure protection with a single maintenance, policy determination, and reliability assurance surface. A network function (NF) can be utilized for policy enforcement over the communication between remote entities and the main control office. This paper presents the research on transparent integration with existing ICS without disrupting communications, resulting in minimal downtime while decoupling the fast paced evolution of defensive security measures from the upgrade cycle of expensive long term hardware. We report our measurements on the resource requirements and overhead in the network for successful NF insertion under a wide variety of network impairments (network packet delay, reordering, and loss). Our paired NF implementation provides a policy enforcement platform extensible to cover myriad cybersecurity-related communication goals, including packet signing for verification, encryption for data privacy, packet filtering and data diode operation (i.e. protecting against eavesdropping, packet injection, and denial-of-service). Furthermore, bundling communication specifications into packet flows allows for tunability in applying policies as coarse- or fine-grained as the needs of the operator. We report on network function resource requirements in the form of required queue depth and network utilization overhead to inform the decision making against hardware cost constraints.

42 ENGINEERING↗
DOE OSTI2022

Activity gradient driven mass transport in molten fluoride salt medium

Abstract The molten salt-cooled reactor concept has garnered significant interest and one of the current challenges limiting the deployment of these reactor concepts is the complex corrosion phenomenon observed in molten salt environments. One of these phenomena is activity gradient mass transport, which has been shown to affect dissimilar materials submerged in the same salt medium even when best efforts have been made to electrically isolate dissimilar materials from one another. This mechanism while shown experimentally, has not been predictively studied through a modeling approach. In this study, activity gradients in several 316L-X materials systems have been modeled and the mass transport predicted by the model has been confirmed through static isothermal corrosion testing in a molten fluoride salt medium.

36 MATERIALS SCIENCE↗
DOE OSTI2022

Chapter 3: Ultrathin Oxides for Solar Cells

This chapter discusses the many uses of ultrathin oxide layers across a broad range of photovoltaic (PV) technologies. The chapter is broken down into six sections, working from the solar absorber outwards: passivation layers, selective contact/buffer layers, recombination layers, barrier layers, anti-reflective coatings, and anti-soiling layers. The goal is to introduce the layer function, outline the scientific mechanisms behind each function, and provide illustrative examples from academic literature and industry. One theme in this chapter is the multifunctionality of oxide layers in the PV device stack, that is, a single layer rarely has only one purpose and therefore certain ideas will build upon other sections. The chapter covers a wide range of material and the discussions and examples are by no means exhaustive. The best efforts have been made to capture the central concepts for each topic and direct the reader to additional resources that have been found by the authors to be useful and relevant.

anti-reflection coatings↗
DOE OSTI2024

Migration to WebDAV in Belle II Experiment

The usage of WebDAV protocol has become more and more popular within the physics experiments using grid middleware in the last decade, and today it represents a valid alternative to the GridFTP currently supported at best-effort level after the retirement of Globus Toolkit. Belle II experiment established the adoption of WebDAV protocol as the main protocol for data access and third-party-copy transfers, without relying on Storage Resource Manager interface (SRM). The migration process, carried on with continuous and gradual steps, has required a large effort to guarantee a smooth transition maintaining the production infrastructure fully operational. In this contribution we show the transition process, the tool of support developed to monitor step by step the status of third-party-copy support with WebDAV protocol by storages of the collaboration tested in both case pull and push, the strategy adopted to configure DIRAC and the solutions put in place for the corner cases. Finally, we will present some statistics of utilization and we will analyse the achieved results.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

Indicator-directed Dynamic Power Management for Iterative Workloads on GPU-Accelerated Systems

Modern high-performance and warehouse computing centers show strong interest in minimizing system power consumption while satisfying customers’ quality of service (QoS). Dynamic voltage and frequency scaling (DVFS) is effective for achieving this goal. Nevertheless, automating the process online and making it transparent to users must address three major challenges: (1) Complexity — today’s hardware components (e.g., CPUs, GPUs, memory, network, etc.) can be configured in several or dozens of frequency/voltage states for satisfying divergent system demands. Given their combination and the emergence of heterogeneity, searching the optimal configuration in the design space online can be timing consuming. (2) QoS guarantee — user-defined objectives such as power constraint and performance target must be monitored, predicted and ensured at the best effort. (3) Adaptability — various known and unknown workloads run on systems. Workloads characteristics should be quickly determined and configurations dynamically adjusted in accord with workloads and QoS. In this work, we focus on applications exhibiting an interesting feature – iterative or periodic, which is common among conventional HPC and emerging machine learning workloads. We propose an online dynamic power-performance (ODPP) management framework to dynamically adjust GPU DVFS configurations to meet performance and power objectives and constraints, without any code annotation or intrusion. Particularly, ODPP extracts the performance and power indicators for applications from their resources utilization profiles in a short episode. It further automatically constructs an accurate model that infers from the indicators how the application's performance and power vary with GPU core and memory frequencies. Aided with the model, for both seen and unseen applications, ODPP can quickly determine the most appropriate DVFS configuration for their execution. We evaluate ODPP on an NVIDIA GPU using multiple exascale computing (ECP) and deep learning applications.

Zou, Pengfei↗
DOE OSTI2019

The engagement and performance operations center: Epoc

In 2018, the US National Science Foundation (NSF) funded the Engagement and Performance Operations Center (EPOC), a joint project between Indiana University (IU) and the Department of Energy's Energy Science Network (ESnet), to work with domain scientists to accelerate the ability of distributed collaborations to share data in order to reach broader science goals. The goal of this funding was to create an operations center for engagement-including definition of formal processes, tracking of engagements, and funded staff, not simply best effort by volunteers, with a goal of enabling digital societies to better share scientific data.

Moynihan, E↗
DOE OSTI2024

Dual Channel Dual Staging: Hierarchical and Portable Staging for GPU-Based In-Situ Workflow

In-situ workflows have emerged as an attractive approach for addressing data movement challenges at very large scales. Since GPU-based architectures dominate the HPC landscapes, porting these in-situ workflows, and, specifically, the inter-application data exchange, to GPU-based systems can be challenging. Technologies such as GPUDirect RDMA (GDR), which is typically used for I/O in GPU applications as an optimization that circumvents the CPU overhead, can be leveraged to support bulk data exchanges between GPU applications. However, current GDR design often lacks performance portability across HPC clusters built with different hardware configurations. Furthermore, the local CPU may also be effectively used as an auxiliary communication mechanism to offload data exchanges. In this paper, we present a dual channel dual staging approach for efficient, scalable, and performance-portable inter-application data exchange for in-situ workflows. This approach exploits the data access pattern within in-situ workflows along with the inherent execution asynchrony to accelerate data exchanges and, at the same time, improve performance portability. Specifically, the dual channel dual staging method leverages both the local CPU and the remote data staging server to build a hierarchical joint staging area and uses this staging area to transform blocking inter-application bulk data exchanges into best-effort local data movements between GPU and CPU. The dual channel dual staging is implemented as a portability extension of the Dataspaces-GPU staging framework. We present an experimental evaluation of its performance, portability, and scalability using this implementation on three leadership GPU clusters. The evaluation results demonstrate that the dual channel dual staging method saves up to 75% in data-exchange time compared to host-based, GDR, and alternate portable designs, while maintaining scalability (up to 512 GPUs) and performance portability across the three platforms.

Zhang, Bo [University of Utah]↗
DOE OSTI2024

HPC Network Simulation Tuning via Automatic Extraction of Hardware Parameters

Popular HPC network interconnection simulators such as SST/macro provide a variety of configurable parameters to explore the design space of hardware components such as network interface cards (NIC), switches, and links among them. While such knobs provide flexibility to explore design trade-offs for novel hardware, manually configuring simulations for matching configurations of the existing hardware to focus on topology exploration can be cumbersome and error-prone, leading to widely inaccurate simulations. This challenge is compounded when specifications of various (proprietary) technologies are not readily available or intentionally omitted. In this work, we propose a framework to autotune the multiple network models’ simulation configurations within SST/macro using Tree-structured Parzen Estimator-based Bayesian optimization to observe the effect on simulation accuracy across different message regimes. These regimes consist of small to large message sizes and latency to bandwidth-bound messages. We provide a detailed analysis of the simulation error for four representative HPC systems. Our Bayesian optimization based autotuning framework for network models achieves a maximum of 5x improvement in accuracy over best-effort manual configurations based on available hardware specifications.

Simulation, autotuning↗
DOE OSTI2020

Resource Management for Processing Wide Area Data Streams on Supercomputers

Modern scientific instruments generate enormous amount of data. Typically, the data collected from the instruments are stored in one or more files that are then moved to a distant supercomputer for processing. The final results are sent back to the user. In order to make effective use of the time on expensive instruments, experimenters want to process the data as they are generated. They want to stream the data from instruments' memory directly to a supercomputer's memory for analysis. Since the compute nodes in a supercomputer are not connected directly to the wide area network, the data streams need to be passed through intermediate gateway nodes. As opposed to the best effort file transfers, data streaming applications require resources at a specific time for a specific period. In this paper, we present a system model for enabling data streaming through gateway nodes and an algorithm to efficiently allocate gateway node resources along with compute nodes. We evaluate the algorithm using real-world traces on the Chameleon Cloud. The results show that our system can schedule compute and gateway resources efficiently for streaming analysis.

data streaming↗
DOE OSTI2024

Automatic Extraction of Network Configurations for Realistic Simulation and Validation

Popular HPC network interconnection simulators such as SST Macro provide a variety of configurable parameters to explore the design space of hardware components such as network links and switches. While such knobs provide flexibility to explore design trade-offs for novel hardware, manually configuring simulations for existing hardware to focus on topology exploration can be cumbersome and error-prone, leading to widely inaccurate simulations. This challenge is compounded when specifications of various (proprietary) technologies are not readily available or are intentionally omitted. In this work, we provide a methodology to automatically tune the simulation configuration of the multiple network models running within SST Macro using Bayesian optimization. We perform this optimization in the context of multiple messaging regimes (i.e., small to large and latency to bandwidth-bound messages) and provide a detailed analysis of the simulation error for four systems. With our automated framework, we achieve a 5x improvement in accuracy over best-effort configurations based on available hardware specifications.

Suetterlein, Joshua D.↗