Search NASA⌕ Search

SEARCH · Search NASA

Results for “checkpoint”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 73 records · Page 4
All sourcesNASA NTRSDOE OSTI
NASA NTRS1987

Checkpoint and restart procedures for single and multi-stage structural model analysis in NASTRAN/COSMIC on a CDC 176

The Underwater Explosions Research Division (UERD) of the David Taylor Naval Ship Research and Development Center makes extensive use of NASTRAN/COSMIC on a CDC 176 to evaluate the structural response of ship structures subjected to underwater explosion shock loadings in the time domain. As relatively new users, UERD engineers have experienced difficulties with the checkpoint/restart feature because of the vague instructions in the user manual. Working procedures for the application of the checkpoint/restart feature to the transient analysis using NASTRAN/COSMIC are illustrated.

Camp, George H.↗
NASA NTRS2008

Exhaustive Thresholds and Resistance Checkpoints

Once deployed, all intricate systems that operate for a long time (such as an airplane or chemical processing plant) experience degraded performance during operational lifetime. These can result from losses of integrity in subsystems and parts that generally do not materially impact the operation of the vehicle (e.g., the light behind the button that opens the sliding door of the minivan). Or it can result from loss of more critical parts or subsystems. Such losses need to be handled quickly in order to avoid loss of personnel, mission, or part of the system itself. In order to manage degraded systems, knowledge of its potential problem areas and the means by which these problems are detected should be developed during the initial development of the system. Once determined, a web of sensors is employed and their outputs are monitored with other system parameters while the system is in preparation or operation. Just gathering the data is only part of the story. The interpretation of the data itself and the response of the system must be carefully developed as well to avoid a mishap. Typically, systems use a test-threshold-response paradigm to process potential system faults. However, such processing sub-systems can suffer from errors and oversights of a consistent type, causing system aberrant behavior instead of expected system and recovery operations. In our study, we developed a complete checklist for determining the completeness of a fault system and its robustness to common processing and response difficulties.

Easton, Charles↗
NASA NTRS2015

eCryo SHIIVER Customer/Stakeholder Checkpoint Briefing

Given the wide diversity of cryogenic fluid management technology that had been developed at the research level, there was a need for eCryo to prioritize and focus on a limited subset of the possibilities in order to set a practical scope. As part of the effort to determine that focus, a survey was conducted in May of 2014 to solicit opinions of members of the aerospace industry as to what they considered the most important and beneficial cryogenic technologies to be developed in the near term. The project was also directed to consider the SLS exploration upper stage (EUS) as a potential infusion target, and to focus on technology that would provide the most immediate benefit to a cryogenic system of that type.

Cryogenic cooling↗
DOE OSTI2023

Error containment for enabling local checkpoint and recovery

Various embodiments include a parallel processing computer system that detects memory errors as a memory client loads data from memory and disables the memory client from storing data to memory, thereby reducing the likelihood that the memory error propagates to other memory clients. The memory client initiates a stall sequence, while other memory clients continue to execute instructions and the memory continues to service memory load and store operations. When a memory error is detected, a specific bit pattern is stored in conjunction with the data associated with the memory error. When the data is copied from one memory to another memory, the specific bit pattern is also copied, in order to identify the data as having a memory error.

Cherukuri, Naveen↗
NASA NTRS2000

Homologous recombination as a potential target for caffeine radiosensitization in mammalian cells: reduced caffeine radiosensitization in XRCC2 and XRCC3 mutants

The radiosensitizing effect of caffeine has been associated with the disruption of multiple DNA damage-responsive cell cycle checkpoints, but several lines of evidence also implicate inhibition of DNA repair. The role of DNA repair inhibition in caffeine radiosensitization remains uncharacterized, and it is unknown which repair process, or lesion, is affected. We show that a radiosensitive cell line, mutant for the RAD51 homolog XRCC2 and defective in homologous recombination repair (HRR), displays significantly diminished caffeine radiosensitization that can be restored by expression of XRCC2. Despite the reduced radiosensitization, caffeine effectively abrogates checkpoints in S and G2 phases in XRCC2 mutant cells indicating that checkpoint abrogation is not sufficient for radiosensitization. Another radiosensitive line, mutant for XRCC3 and defective in HRR, similarly shows reduced caffeine radiosensitization. On the other hand, a radiosensitive mutant (irs-20) of DNA-PKcs with a defect in non-homologous end-joining (NHEJ) is radiosensitized by caffeine to an extent comparable to wild-type cells. In addition, rejoining of radiation-induced DNA DSBs, that mainly reflects NHEJ, remains unaffected by caffeine in XRCC2 and XRCC3 mutants, or their wild-type counterparts. These observations suggest that caffeine targets steps in HRR but not in NHEJ and that abrogation of checkpoint response is not sufficient to explain radiosensitization. Indeed, immortalized fibroblasts from AT patients show caffeine radiosensitization despite the checkpoint defects associated with ATM mutation. We propose that caffeine radiosensitization is mediated by inhibition of stages in DNA DSB repair requiring HRR and that checkpoint disruption contributes by allowing these DSBs to transit into irreparable states. Thus, checkpoints may contribute to genomic stability by promoting error-free HRR.

NASA Discipline Radiation Health↗
DOE OSTI2024

Aviation security screening optimizer for risk and throughput (ASSORT)

The increasing number of air travelers each year presents a challenge as many airports are near their capacity in terms of resources and space for passenger screening. Fortunately, advancements in technologies like next-generation millimeter wave scanning offer solutions to ease this strain. The focus remains on managing risk while enhancing the passenger experience for the traveling public. The risk model presented in this paper known as the Aviation Security Screening Optimizer for Risk and Throughput (ASSORT) is designed to assess risk-based approaches for passenger screening and checkpoint operations. Additionally, ASSORT is exploring various traveler categories — general, trusted, and trusted-plus — along with different checkpoint screening Concept of Operations tailored to each traveler type. For instance, travelers with a higher trust level may experience fewer screening technologies, resulting in quicker processing times at the checkpoint. The output of ASSORT provides a risk score for predefined threat scenarios, as well as the overall risk to the checkpoint, aircraft, and airport by traveler type. In conclusion, benefits of using this tool include assessing the trade-offs between the overall risk associated with checkpoints and the throughput rate of passengers screened. We show for example the impact that different passenger volumes at the checkpoint can have on risk.

99 GENERAL AND MISCELLANEOUS↗
NASA NTRS2003

An overactivated ATR/CHK1 pathway is responsible for the prolonged G2 accumulation in irradiated AT cells

Induction of checkpoint responses in G1, S, and G2 phases of the cell cycle after exposure of cells to ionizing radiation (IR) is essential for maintaining genomic integrity. Ataxia telangiectasia mutated (ATM) plays a key role in initiating this response in all three phases of the cell cycle. However, cells lacking functional ATM exhibit a prolonged G2 arrest after IR, suggesting regulation by an ATM-independent checkpoint response. The mechanism for this ataxia telangiectasia (AT)-independent G2-checkpoint response remains unknown. We report here that the G2 checkpoint in irradiated human AT cells derives from an overactivation of the ATR/CHK1 pathway. Chk1 small interfering RNA abolishes the IR-induced prolonged G2 checkpoint and radiosensitizes AT cells to killing. These results link the activation of ATR/CHK1 with the prolonged G2 arrest in AT cells and show that activation of this G2 checkpoint contributes to the survival of AT cells.

NASA Discipline Radiation Health↗
NASA NTRS1989

Recoverable distributed shared virtual memory - Memory coherence and storage structures

This paper examines the problem of implementing rollback recovery in multicomputer distributed shared virtual memory environments, in which the shared memory is implemented in software and exists only virtually. A user-transparent checkpointing recovery scheme and new twin-page disk storage management are presented to implement a recoverable distributed shared virtual memory. The checkpointing scheme is integrated with the shared virtual memory management. The twin-page disk approach allows incremental checkpointing without an explicit undo at the time of recovery. A single consistent checkpoint state is maintained on stable disk storage. The recoverable distributed shared virtual memory allows the system to restart computation from a previous checkpoint due to a processor failure without a global restart.

Wu, Kun-Lung↗
NASA NTRS1993

Relaxing consistency in recoverable distributed shared memory

Relaxed memory consistency models have recently been proposed to tolerate memory access latency in both hardware and software distributed shared memory systems. In recoverable shared memory multiprocessors, relaxing consistency has the added benefit of reducing the number of checkpoints needed to avoid rollback propagation. In this paper, we introduce new checkpointing algorithms that take advantage of relaxed consistency to reduce the performance overhead of checkpointing. We also introduce a scheme based on lazy relaxed consistency, that reduces both checkpointing overhead and the overhead of avoiding error propagation in systems with error latency. Multiprocessor address traces are used to evaluate the relaxed consistency approach to checkpointing with distributed shared memory.

Janssens, Bob↗
NASA NTRS1993

Relaxing consistency in recoverable distributed shared memory

Relaxed memory consistency models tolerate increased memory access latency in both hardware and software distributed shared memory systems. In recoverable systems, relaxing consistency has the added benefit of reducing the number of checkpoints needed to avoid rollback propagation. In this paper, we introduce new checkpointing algorithms that take advantage of relaxed consistency to reduce the performance overhead of checkpointing. We also introduce a scheme based on lazy relaxed consistency, that reduces both checkpointing overhead and the overhead of avoiding error propagation in systems with error latency. We use multiprocessor address traces to evaluate the relaxed consistency approach to checkpointing with distributed shared memory.

Janssens, Bob↗
DOE OSTI2020

Optimizing Asynchronous Multi-Level Checkpoint/Restart Configurations with Machine Learning

With the emergence of versatile storage systems, multi-level checkpointing (MLC) has become a common approach to gain efficiency. However, multi-level checkpoint/restart can cause enormous I/O traffic on HPC systems. To use multilevel checkpointing efficiently, it is important to optimize check-point/restart configurations. Current approaches, namely modeling and simulation, are either inaccurate or slow in determining the optimal configuration for a large scale system. In this paper, we show that machine learning models can be used in combination with accurate simulation to determine the optimal checkpoint configurations. We also demonstrate that more advanced techniques such as neural networks can further improve the performance in optimizing checkpoint configurations.

checkpoint-restart↗
DOE OSTI2023

The Calpain-7 protease functions together with the ESCRT-III protein IST1 within the midbody to regulate the timing and completion of abscission

The Endosomal Sorting Complexes Required for Transport (ESCRT) machinery mediates the membrane fission step that completes cytokinetic abscission and separates dividing cells. Filaments composed of ESCRT-III subunits constrict membranes of the intercellular bridge midbody to the abscission point. These filaments also bind and recruit cofactors whose activities help execute abscission and/or delay abscission timing in response to mitotic errors via the NoCut/Abscission checkpoint. We previously showed that the ESCRT-III subunit IST1 binds the cysteine protease Calpain-7 (CAPN7) and that CAPN7 is required for both efficient abscission and NoCut checkpoint maintenance. Here, we report biochemical and crystallographic studies showing that the tandem microtubule-interacting and trafficking (MIT) domains of CAPN7 bind simultaneously to two distinct IST1 MIT interaction motifs. Structure-guided point mutations in either CAPN7 MIT domain disrupted IST1 binding in vitro and in cells, and depletion/rescue experiments showed that the CAPN7-IST1 interaction is required for (1) CAPN7 recruitment to midbodies, (2) efficient abscission, and (3) NoCut checkpoint arrest. CAPN7 proteolytic activity is also required for abscission and checkpoint maintenance. Hence, IST1 recruits CAPN7 to midbodies, where its proteolytic activity is required to regulate and complete abscission.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2023

An autoencoder compression approach for accelerating large-scale inverse problems

Abstract Partial differential equation (PDE)-constrained inverse problems are some of the most challenging and computationally demanding problems in computational science today. Fine meshes required to accurately compute the PDE solution introduce an enormous number of parameters and require large-scale computing resources such as more processors and more memory to solve such systems in a reasonable time. For inverse problems constrained by time-dependent PDEs, the adjoint method often employed to compute gradients and higher order derivatives efficiently requires solving a time-reversed, so-called adjoint PDE that depends on the forward PDE solution at each timestep. This necessitates the storage of a high-dimensional forward solution vector at every timestep. Such a procedure quickly exhausts the available memory resources. Several approaches that trade additional computation for reduced memory footprint have been proposed to mitigate the memory bottleneck, including checkpointing and compression strategies. In this work, we propose a close-to-ideal scalable compression approach using autoencoders to eliminate the need for checkpointing and substantial memory storage, thereby reducing the time-to-solution and memory requirements. We compare our approach with checkpointing and an off-the-shelf compression approach on an earth-scale ill-posed seismic inverse problem. The results verify the expected close-to-ideal speedup for the gradient and Hessian-vector product using the proposed autoencoder compression approach. To highlight the usefulness of the proposed approach, we combine the autoencoder compression with the data-informed active subspace (DIAS) prior showing how the DIAS method can be affordably extended to large-scale problems without the need for checkpointing and large memory.

Mathematics↗
DOE OSTI2025

Stability-preserving Lossy Compression for Large-scale Partial Differential Equations

Checkpoint/Restart (C/R) strategies are vital for fault tolerance in PDE-based scientific simulations, yet traditional checkpointing incurs significant I/O overhead. Lossy compression offers a scalable solution by reducing checkpoint data size, but conventional methods often lack control over physical invariants (e.g., energy), leading to instability such as oscillations or divergence in Partial Differential Equations (PDE) systems. This paper introduces a stability-preserving compression approach tailored for PDE simulations by explicitly controlling kinetic and potential energy perturbations to ensure stable restarts. Extensive experiments conducted across diverse PDE configurations demonstrate that our method maintains numerical stability with minimal error magnification—even across multiple checkpoint-restart cycles—outperforming state-of-the-art lossy compressors. Parallel evaluations on the Frontier supercomputer show up to 8.4× improvement in checkpoint write performance and 6.3× in read performance, while maintaining relative L2 errors ∼ 2e-6 throughout continued simulation. These results provide practical guidance for balancing compression accuracy, stability, and computational efficiency in large-scale PDE applications.

Gong, Qian [ORNL] (ORCID:0000000235704142)↗
DOE OSTI2020

Hindsight logging for model training

In modern Machine Learning, model training is an iterative, experimental process that can consume enormous computation resources and developer time. To aid in that process, experienced model developers log and visualize program variables during training runs. Exhaustive logging of all variables is infeasible, so developers are left to choose between slowing down training via extensive conservative logging, or letting training run fast via minimalist optimistic logging that may omit key information. As a compromise, optimistic logging can be accompanied by program checkpoints; this allows developers to add log statements post-hoc, and "replay" desired log statements from checkpoint---a process we refer to as hindsight logging. Unfortunately, hindsight logging raises tricky problems in data management and software engineering. Done poorly, hindsight logging can waste resources and generate technical debt embodied in multiple variants of training code. In this paper, we present methodologies for efficient and effective logging practices for model training, with a focus on techniques for hindsight logging. Our goal is for experienced model developers to learn and adopt these practices. To make this easier, we provide an open-source suite of tools for Fast Low-Overhead Recovery (flor) that embodies our design across three tasks: (i) efficient background logging in Python, (ii) adaptive periodic checkpointing, and (iii) an instrumentation library that codifies hindsight logging for efficient and automatic record-replay of model-training. Model developers can use each flor tool separately as they see fit, or they can use flor in hands-free mode, entrusting it to instrument their code end-to-end for efficient record-replay. Our solutions leverage techniques from physiological transaction logs and recovery in database systems. Evaluations on modern ML benchmarks demonstrate that flor can produce fast checkpointing with small user-specifiable overheads (e.g. 7%), and still provide hindsight log replay times orders of magnitude faster than restarting training from scratch.

Computer Science↗
DOE OSTI2020

Implementing Software Resiliency in HPX for Extreme Scale Computing

The DOE Office of Science Exascale Computing Project (ECP) outlines the next milestones in the supercomputing domain. The target computing systems under the project will deliver 10x performance while keeping the power budget under 30 megawatts. With such large machines, the need to make applications resilient has become paramount. The benefits of adding resiliency to mission critical and scientific applications, includes the reduced cost of restarting the failed simulation both in terms of time and power. Most of the current implementation of resiliency at the software level makes use of a Coordinated Checkpoint and Restart (C/R). This technique of resiliency generates a consistent global snapshot, also called a checkpoint. Generating snapshots involves global communication and coordination and is achieved by synchronizing all running processes. The generated checkpoint is then stored in some form of persistent storage. On failure detection, the runtime initiates a global rollback to the most recent previously saved checkpoint. This involves aborting all running processes, rolling them back to the previous state and restarting them.

97 MATHEMATICS AND COMPUTING↗