Search NASA⌕ Search

SEARCH · Search NASA

Results for “data intensive computing”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,225 records · Page 68
All sourcesNASA NTRSDOE OSTI
DOE OSTI2018

Deploy threading in Nalu solver stack

The goal of the ExaWind project is to enable predictive simulations of wind farms composed of many MW-scale turbines situated in complex terrain. Predictive simulations will require computational fluid dynamics (CFD) simulations for which the mesh resolves the geometry of the turbines, and captures the rotation and large deflections of blades. Whereas such simulations for a single turbine are arguably petascale class, multi-turbine wind farm simulations will require exascale-class resources. The primary code in the ExaWind project is Nalu, which is an unstructured-grid solver for the acousticallyincompressible Navier-Stokes equations, and mass continuity is maintained through pressure projection. The model consists of the mass-continuity Poisson-type equation for pressure and a momentum equation for the velocity. For such modeling approaches, simulation times are dominated by linear-system setup and solution for the continuity and momentum systems. For the ExaWind challenge problem, the moving meshes greatly affect overall solver costs as re-initialization of matrices and re-computation of preconditioners is required at every time step In this Milestone, we examine the effect of threading on the solver stack performance against flat-MPI results obtained from previous milestones using Haswell performance data full-turbine simulations. Whereas the momentum equations are solved only with the Trilinos solvers, we investigate two algebraic-multigrid preconditioners for the continuity equations: Trilinos/Muelu and HYPRE/BoomerAMG. These two packages embody smoothed-aggregation and classical Ruge-Stiiben AMG methods, respectively. In our FY18 Q2 report, we described our efforts to improve setup and solve of the continuity equations under flat-MPI parallelism. While significant improvement was demonstrated in the solve phase, setup times remained larger than expected. Starting with the optimized settings described in the Q2 report, we explore here simulation performance where OpenMP threading is employed in the solver stack. For Trilinos, threading is acheived through the Kokkos abstraction where, whereas HYPRE/BoomerAMG employs straight OpenMP. We examined results for our mid-resolution baseline turbine simulation configuration (229M DOF). Simulations on 2048 Haswell cores explored the effect of decreasing the number of MPI ranks while increasing the number of threads. Both HYPRE and Trilinos exhibited similar overal solution times, and both showed dramatic increases in simulation time in the shift from MPI ranks to OpenMP threads. This increase is attributed to the large amount of work per MPI rank starting at the single-thread configuration. Decreasing MPI ranks, while increasing threads, may be increasing simulation time due to thread synchronization and start-up overhead contributing to the latency and serial time in the model. These result showed that an MPI+OpenMP parallel decomposition will be more effective as the amount per MPI rank computation per MPI rank decreases and the communication latency increases. This idea was demonstrated in a strong scaling study of our low-resolution baseline model (29M DOF) with the Trilinos-HYPRE configuration. While MPI-only results showed scaling improvement out to about 1536 cores, engaging threading carried scaling improvements out to 4128 cores — roughly 7000 DOF per core. This is an important result as improved strong scaling is needed for simulations to be executed over sufficiently long simulated durations (i.e., for many timesteps). In addition to threading work described above, the team examined solver-performance improvements by exploring communication-overhead in the HYPRE-GMRES implementation through a communicationoptimal- GMRE algorithm (CO-GMRES), and offloading compute-intensive solver actions to GPUs. To those ends, a HYPRE mini-app was allow us to easily test different solver approaches and HYPRE parameter settings without running the entire Nalu code. With GPU acceleration on the Summitdev supercomputer, a 20x speedup was achieved for the overall preconditioner and solver execution time for the mini-app. A study on Haswell processors showed that CO-GMRES provides benefits as one increases MPI ranks.

17 WIND ENERGY↗
DOE OSTI2022

Report on Year-4 of Water NSTF Matrix Testing: Facility Maintenance and Accident Testing

Under support from the Department of Energy (DOE) and the Office of Advanced Reactor Technologies (ART), a large-scale test facility has been constructed at Argonne National Laboratory to generate NQA-1 qualified validation data for passive decay heat removal systems in advanced reactors. The Natural convection Shutdown heat removal Test Facility (NSTF) reflects key features of a ½ scale, water-based, Reactor Cavity Cooling System (RCCS) and is intended to study the behavior, bound performance, and ultimately guide design decisions for passive decay heat removal systems for advanced reactors. In addition to the experimental activities detailed in this report, a supportive computational modeling effort is on-going which has been demonstrated to significantly strengthen the experimental program while also improving accuracy of the computer models. Together these create a mutually beneficial relationship integral to meeting the overall program objective of examining the heat removal performance of the RCCS concept. This report serves as a summary of maintenance and experimental activities during the program’s fourth year of water-based operation. A planned six-month maintenance period began in August 2021, during which major inspections, repairs, cleaning, and installation of new instrumentation and data acquisition hardware were conducted. Most significantly, two heaters that faulted during Year-3 were repaired, allowing the facility to resume use of the full heated section area and full range of available electric power. The remainder of the year consisted of eight months of test operations, during which the facility logged 211 hours of active heating across one bake-out (following the maintenance period) and seven matrix tests; five classifieds as Accepted per NQA-1, one as Trending, and one as Failed. Testing began by performing two repeat cases to confirm expected facility response and behavior during both single- and two-phase flow conditions, ensuring no changes were introduced during the maintenance period that might have altered the thermal-hydraulic characteristics of the facility. In continuation of the power parametric series initiated in previous years, a high-power test case was then performed examining heat removal performance at a decay heat load equivalent to 2.4 MWt, full-scale, a level exceeding maximum design targets. Additional testing then introduced various blockage scenarios along the network piping, examining the effects of partial and complete blockages of the flow paths on the system behavior and heat removal performance. A study of static boiling tests directed at understanding the geysering two-phase instability was also conducted. The loop was filled only to the bottom of the tank outlet, creating an open loop configuration that prevents any natural circulation flow from occurring, and the heaters were powered on until the facility reached saturation conditions. Following, a series of quasi-steady-state conditions were introduced by adjusting the inventory level in the adiabatic chimney piping at decreasingly lower elevations above the heated region. A strong correlation of geysering characteristics and loop level was observed, with flow and temperature excursions decreasing in intensity, but increasing in frequency, as the fill were reduced to lower elevations along the chimney piping. Once the level fell very low in the chimney, at points near the top of the heated section, the system reached a stable state of continuous boiling without any occurrence of geysering eruptions. A final significant testing accomplishment this year was successful completion of an “accident scenario” test, whose operating conditions were based on a prototypic decay heat curve provided by Framatome and scaled for the NSTF. This test began by establishing steady-state, single-phase “normal operation” conditions, before simulating an accident trip where the availability of active cooling systems was lost. Loop temperatures gradually increased until reaching saturation and subsequent two-phase boiling flow. Over the course of an extended operational period along the defined decay heat curve, steam boil-off caused gradual but continued depletion of liquid inventory until reaching a critically low level causing flow stagnation and cessation of natural circulation heat removal. At this point, after nearly 72 hours of continuous operation, a cold refill was performed to replenish the system inventory and allow the facility to re-establish closed loop natural circulation flow and return to a safe operational state.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2017

First-Principles-Based Computational Study on Nucleation and Growth Mechanisms of U on Mo(110) Surface Solvated in an Eutectic LiCl-KCl Molten Salt

In nuclear engineering and industry electrochemical methods have been intensively employed in pyro-processing technology to recycle used nuclear fuels for substantial alleviation of the concern with the safe treatment of high-level radioactive nuclear wastes. Pyro-processing is based on sequential electrochemical reactions at high temperature (typically above 700 K at ambient pressure) to enhance the efficiency for decoupling nuclear fuels from unfavorable fission fragments. Using a molten salt in electrorefining process, at which actinides can be selectively deposited in solid cathode from ionic states, gives several advantages over an aqueous solution: higher thermal conductivity (it is liquid phase at high temperature, molten salt plays a role as medium at high temperature over 773 K in electrochemical reaction, lower kinetic barrier in ionic diffusion mobility and better stability against undesired side reactions with radioactive materials. Therefore, the volume of high-level nuclear wastes to be, otherwise, disposed into secured sites is dramatically reduced. Most of all it conforms to nuclear non-proliferation treaty. Electrochemical reactions in a molten salt medium are, however, much more complicated than in aqueous solution rendering the accurate understanding of thermodynamic and kinetic properties formidably challenging. It is represented by extensive researches in experiments and computations to identify underlying mechanisms on how U and other fission fragments behave in the electrorefining system. Regardless of the researches there is a general agreement that the initial stage of the nucleation and growth process of the U deposit is critical for determining the properties of final product and the electrorefining efficiency. In spite of the focused studies it still remains unclear. Theoretical and computational researches have been often carried out with various empirical parameter for interatomic potential energy, which are unknown variables until numerical fits to experimental measurements or just assumption is made. Thus, empirical simulations sometimes predicted outcomes are inconsistent depending on materials of interest with experimental measurements. This may be largely because the accuracy of an empirical interatomic potential energy models critically depends on the input parameters and thus, can be limited in transferability of the results. Furthermore, it may be devoid of fundamental physics or chemistry. This situation requires special caution when an electrorefining system is studied to recycle spent nuclear fuels because of the extreme difficulty in reliable acquisition of thermodynamic and kinetic data from direct experimental measurements. In this presentation, we extensively utilized first principles density functional theory (DFT) calculations and ab-initio molecular dynamic (AIMD) simulations to figure out underlying mechanisms of the initial stage of nucleation and growth behaviors of U on Mo(110) surface in a eutectic LiCl-KCl molten salt at T = 773 K. To rigorously investigate the effect of the molten salt electrolyte we compared the results with those calculated under vacuum condition.

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗
DOE OSTI2016

Vision 20/20: Magnetic resonance imaging-guided attenuation correction in PET/MRI: Challenges, solutions, and opportunities

Attenuation correction is an essential component of the long chain of data correction techniques required to achieve the full potential of quantitative positron emission tomography (PET) imaging. The development of combined PET/magnetic resonance imaging (MRI) systems mandated the widespread interest in developing novel strategies for deriving accurate attenuation maps with the aim to improve the quantitative accuracy of these emerging hybrid imaging systems. The attenuation map in PET/MRI should ideally be derived from anatomical MR images; however, MRI intensities reflect proton density and relaxation time properties of biological tissues rather than their electron density and photon attenuation properties. Therefore, in contrast to PET/computed tomography, there is a lack of standardized global mapping between the intensities of MRI signal and linear attenuation coefficients at 511 keV. Moreover, in standard MRI sequences, bones and lung tissues do not produce measurable signals owing to their low proton density and short transverse relaxation times. MR images are also inevitably subject to artifacts that degrade their quality, thus compromising their applicability for the task of attenuation correction in PET/MRI. MRI-guided attenuation correction strategies can be classified in three broad categories: (i) segmentation-based approaches, (ii) atlas-registration and machine learning methods, and (iii) emission/transmission-based approaches. This paper summarizes past and current state-of-the-art developments and latest advances in PET/MRI attenuation correction. The advantages and drawbacks of each approach for addressing the challenges of MR-based attenuation correction are comprehensively described. The opportunities brought by both MRI and PET imaging modalities for deriving accurate attenuation maps and improving PET quantification will be elaborated. Future prospects and potential clinical applications of these techniques and their integration in commercial systems will also be discussed.

61 RADIATION PROTECTION AND DOSIMETRY↗
DOE OSTI2018

Exploiting Internal Parallelism for Address Translation in Solid-State Drives

Solid-state Drives (SSDs) have changed the landscape of storage systems and present a promising storage solution for data-intensive applications due to their low latency, high bandwidth, and low power consumption compared to traditional hard disk drives. SSDs achieve these desirable characteristics using internal parallelism—parallel access to multiple internal flash memory chips—and a Flash Translation Layer (FTL) that determines where data are stored on those chips so that they do not wear out prematurely. However, current state-of-the-art cache-based FTLs like the Demand-based Flash Translation Layer (DFTL) do not allow IO schedulers to take full advantage of internal parallelism, because they impose a tight coupling between the logical-to-physical address translation and the data access. In this study to address this limitation, we introduce a new FTL design called Parallel-DFTL that works with the DFTL to decouple address translation operations from data accesses. Parallel-DFTL separates address translation and data access operations into different queues, allowing the SSD to use concurrent flash accesses for both types of operations. We also present a Parallel-LRU cache replacement algorithm to improve the concurrency of address translation operations. To compare Parallel-DFTL against existing FTL approaches, we present a Parallel-DFTL performance model and compare its predictions against those for DFTL and an ideal page-mapping approach. We also implemented the Parallel-DFTL approach in an SSD simulator using real device parameters, and used trace-driven simulation to evaluate Parallel-DFTL’s efficacy. Our evaluation results show that Parallel-DFTL improved the overall performance by up to 32% for the real IO workloads we tested, and by up to two orders of magnitude with synthetic test workloads. Finally, we also found that Parallel-DFTL is able to achieve reasonable performance with a very small cache size and that it provides the best benefit for those workloads with large request size or with high write ratio.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2017

Automatic and Transparent Resource Contention Mitigation for Improving Large-Scale Parallel File System Performance

Proportional to the scale increases in HPC systems, many scientific applications are becoming increasingly data intensive, and parallel I/O has become one of the dominant factors impacting the large-scale HPC application performance. On a typical large-scale HPC system, we have observed that the lack of a global workload coordination coupled with the shared nature of storage systems cause load imbalance and resource contention over the end-to-end I/O paths resulting in severe performance degradation. I/O load imbalance on HPC systems is generally a self-inflicted wound and mostly occurs between the I/O paths and resources consumed by each individual job. In this paper, we introduce TAPP-IO, a dynamic, shared load balancing framework for mitigating resource contention. TAPP-IO extends our previous work and solves two major limitations: First, it transparently intercepts file creation calls during runtime to balance the workload over all available storage targets. The usage of TAPP-IO requires no application source code modifications and is independent from any I/O middleware. The framework can be applied to almost any HPC platform and is suitable for systems that lack a centralized file system resource manager. Second, the framework proposes a new placement strategy to support not only file-per-process I/O, but also single shared file I/O. This opens the door to a new class of scientific applications that can leverage the placement library for improved performance. We demonstrate the effectiveness of our integration on the Titan system at the Oak Ridge National Laboratory. Our experiments with a synthetic benchmark and real-world HPC workload show that, even in a noisy production environment, TAPP-IO can improve large-scale application performance significantly.

Neuwirth, Sarah↗
DOE OSTI2026

Image Distinguishability Analysis Testing Through Principal Components and Its Application to Hot Spot Scale Invariance

Hot spots are spatial regions of intense energy localization that govern initiation of secondary high explosives. Studies that characterize or compare simulated hot spots are frequently either qualitatively descriptive or resort to quantitative distribution functions that neglect stochastic variations and spatial correlations—effects that are also neglected in common comparison tests like the Kolmogorov–Smirnov test. To this end, we develop an image distinguishability analysis (IDA) test based on principal component (PC) analysis that makes pixel-by-pixel comparisons between small, for example, O(<10), image data sets. The IDA test makes comparisons through a generalized distance metric in the PC space and a test statistic that is derived to calculate mathematical equation-values. Here, we derive a statistical distribution and criticality criterion to determine whether images are distinguishable from established baselines. We apply the IDA test on images generated from molecular dynamics simulations of hot spots from pore collapse in TATB to assess scale invariance in the complex patterns of hot spots that form in a representative high explosive crystal. The IDA test shows that TATB hot spot spatial temperature fields and their derived temperature histograms exhibit scale-invariant features over specific intervals of shock orientation, strength, and initial pore diameter. However, the IDA test also shows that qualitatively different conclusions regarding invariance can be reached depending on whether the hot spot is treated as a spatially correlated field as opposed to a distribution function that lacks spatial information.

organic↗
DOE OSTI2014

SU-E-T-163: Evaluation of Dose Distributions Recalculated with Per-Field Measurement Data Under the Condition of Respiratory Motion During IMRT for Liver Cancer

Purpose: The dose distributions within the real volumes of tumor targets and critical organs during internal target volume-based intensity-modulated radiation therapy (ITV-IMRT) for liver cancer were recalculated by applying the effects of actual respiratory organ motion, and the dosimetric features were analyzed through comparison with gating IMRT (Gate-IMRT) plan results. Methods: The 4DCT data for 10 patients who had been treated with Gate-IMRT for liver cancer were selected to create ITV-IMRT plans. The ITV was created using MIM software, and a moving phantom was used to simulate respiratory motion. The period and range of respiratory motion were recorded in all patients from 4DCT-generated movie data, and the same period and range were applied when operating the dynamic phantom to realize coincident respiratory conditions in each patient. The doses were recalculated with a 3 dose-volume histogram (3DVH) program based on the per-field data measured with a MapCHECK2 2-dimensional diode detector array and compared with the DVHs calculated for the Gate-IMRT plan. Results: Although a sufficient prescription dose covered the PTV during ITV-IMRT delivery, the dose homogeneity in the PTV was inferior to that with the Gate-IMRT plan. We confirmed that there were higher doses to the organs-at-risk (OARs) with ITV-IMRT, as expected when using an enlarged field, but the increased dose to the spinal cord was not significant and the increased doses to the liver and kidney could be considered as minor when the reinforced constraints were applied during IMRT plan optimization. Conclusion: Because Gate-IMRT cannot always be considered an ideal method with which to correct the respiratory motional effect, given the dosimetric variations in the gating system application and the increased treatment time, a prior analysis for optimal IMRT method selection should be performed while considering the patient's respiratory condition and IMRT plan results.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2015

Validation of a deformable image registration technique for cone beam CT-based dose verification

Purpose: As radiation therapy evolves toward more adaptive techniques, image guidance plays an increasingly important role, not only in patient setup but also in monitoring the delivered dose and adapting the treatment to patient changes. This study aimed to validate a method for evaluation of delivered intensity modulated radiotherapy (IMRT) dose based on multimodal deformable image registration (DIR) for prostate treatments. Methods: A pelvic phantom was scanned with CT and cone-beam computed tomography (CBCT). Both images were digitally deformed using two realistic patient-based deformation fields. The original CT was then registered to the deformed CBCT resulting in a secondary deformed CT. The registration quality was assessed as the ability of the DIR method to recover the artificially induced deformations. The primary and secondary deformed CT images as well as vector fields were compared to evaluate the efficacy of the registration method and it’s suitability to be used for dose calculation. PLASTIMATCH, a free and open source software was used for deformable image registration. A B-spline algorithm with optimized parameters was used to achieve the best registration quality. Geometric image evaluation was performed through voxel-based Hounsfield unit (HU) and vector field comparison. For dosimetric evaluation, IMRT treatment plans were created and optimized on the original CT image and recomputed on the two warped images to be compared. The dose volume histograms were compared for the warped structures that were identical in both warped images. This procedure was repeated for the phantom with full, half full, and empty bladder. Results: The results indicated mean HU differences of up to 120 between registered and ground-truth deformed CT images. However, when the CBCT intensities were calibrated using a region of interest (ROI)-based calibration curve, these differences were reduced by up to 60%. Similarly, the mean differences in average vector field lengths decreased from 10.1 to 2.5 mm when CBCT was calibrated prior to registration. The results showed no dependence on the level of bladder filling. In comparison with the dose calculated on the primary deformed CT, differences in mean dose averaged over all organs were 0.2% and 3.9% for dose calculated on the secondary deformed CT with and without CBCT calibration, respectively, and 0.5% for dose calculated directly on the calibrated CBCT, for the full-bladder scenario. Gamma analysis for the distance to agreement of 2 mm and 2% of prescribed dose indicated a pass rate of 100% for both cases involving calibrated CBCT and on average 86% without CBCT calibration. Conclusions: Using deformable registration on the planning CT images to evaluate the IMRT dose based on daily CBCTs was found feasible. The proposed method will provide an accurate dose distribution using planning CT and pretreatment CBCT data, avoiding the additional uncertainties introduced by CBCT inhomogeneity and artifacts. This is a necessary initial step toward future image-guided adaptive radiotherapy of the prostate.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2021

Real-Time Health Monitoring for Gas Turbine Components Using Online Learning and High-Dimensional Data

Capital-intensive turbomachinery, such as gas turbines and combined cycle plants, are constantly being monitored for performance anomalies, faults, and physical degradation. Although these power-generating assets are equipped with hundreds of sensors, existing monitoring tools can only handle moderate-sized data. As a result, only a handful of aggregate metrics are used to monitor machine health. At the same time, developing advanced tools suitable for large datasets have been restricted by the lack of appropriate data. The objective of this proposal was to demonstrate a Big Data analytics framework for fault detection and diagnosis in gas turbine applications. We develop a predictive analytics framework methodology guided by these experimental data, industrial data from our collaborators, and physics-based models with engineering domain knowledge. Our analytics framework consists of four key components (1) a data curation process that addresses data storage, data quality assessments, and integrity checks, (2) a feature engineering component that utilizes statistical methods and transformation algorithms guided by physics-based models to extract high-fidelity fault features that can be leveraged for fault detection and classifying fault severities, (3) a Machine Learning-based fault detection and diagnostics algorithms for detecting operational and hardware faults in the combustion and the turbines section. We utilize two industry-class gas turbine component test rigs to generate first of its kind data for critical gas turbine faults with varying severity levels. Advanced gas turbine test facilities will be interrogated using state-of-the-art instrumentation techniques to build fault signatures and data trends for key combustor and turbine faults. Data generated from a combustor test rig (Georgia Tech) and a turbine test rig (Penn State) during both normal operation and with seeded faults serve as the basis for the Big Data sets. The test conditions in the two test facilities include common, critical events that occur in the operation. Utilizing the combustor test rig, we examine two common combustor faults: lean blowout and centerbody degradation. For the turbine section we develop analytic models for monitoring cooling faults in the gas turbine

03 NATURAL GAS↗
DOE OSTI2013

Characterization of a novel 2D array dosimeter for patient-specific quality assurance with volumetric arc therapy

Purpose: In this study, the authors are evaluating a new, commercially available 2D array that offers 3D dose reconstruction for patient specific intensity modulated radiation therapy quality assurance (IMRT QA).Methods: The OCTAVIUS 4D system and its accompanying software (VERISOFT) by PTW were evaluated for the accuracy of the dose reconstruction for patient specific pretreatment IMRT QA. OCTAVIUS 4D measures the dose plane at the linac isocenter as the phantom rotates synchronously with the gantry, maintaining perpendicularity with the beam, by means of an inclinometer and a motor. The measurements collected during a volumetric modulated arc therapy delivery (VMAT) are reconstructed into a 3D dose volume. The VERISOFT application is used to perform the analysis, by comparing the reconstructed dose against the 3D dose matrix from the treatment planning system (TPS) that is computed for the same geometry and beam arrangement as that of the measurement. In this study, the authors evaluated the 3D dose reconstruction algorithm of this new system using a series of tests. Using the Octavius 4D phantom as the patient, dose distributions for various field sizes, beam orientations, shapes, and combination of fields were calculated using the Pinnacle3, TPS, and the respective DICOMRT dose was exported to the VERISOFT analysis software. Measurements were obtained by delivering the test treatment plans and comparisons were made based on gamma index, dose profiles, and isodose distribution analysis. In addition, output factors were measured and the dose linearity of the array was assessed. Those measurements were compared against measurements in water using a single, calibrated ionization chamber as well as calculations from Pinnacle for the same delivery geometries.Results: The number of voxels that met the 3%/3 mm criteria for the volumetric 3D gamma index analysis ranged from 92.3% to 98.9% for all the patient plans that the authors evaluated. 2D gamma analysis in the axial, sagittal, and coronal planes produced similar results to those in the 3D gamma analysis. The new detector system does not require an angular dependence correction because it rotates in synchrony with the gantry and the detector array maintains a constant SAD while always perpendicular to the beam axis. Output factors were within 2% when compared to ionization chamber measurements and Pinnacle calculations. Similar agreement was observed when testing the MU linearity (for MU values above 2) as well as dose rate effect.Conclusions: The OCTAVIUS 4D system has some unique characteristics that can potentially improve the patient specific pretreatment IMRT QA data collection and analysis. The ability of the software to reconstruct from the measurements the true 3D dose distribution in the phantom, provides a unique perspective for the medical physicist that evaluates a patient's QA plan.

61 RADIATION PROTECTION AND DOSIMETRY↗
DOE OSTI2014

Blackcomb: Hardware-Software Co-design for Non-Volatile Memory in Exascale Systems

Summary of technical results of Blackcomb Memory Devices We explored various different memory technologies (STTRAM, PCRAM, FeRAM, and ReRAM). The progress can be classified into three categories, below. Modeling and Tool Releases Various modeling tools have been developed over the last decade to help in the design of SRAM or DRAM-based memory hierarchies. To explore new design opportunities that NVM technologies can bring to the designers, we have developed similar high-level models for NVM, including PCRAMsim [Dong 2009], NVSim [Dong 2012], and NVMain [Poremba 2012]. NVSim is a circuit-level model for NVM performance, energy, and area estimation, which supports various NVM technologies, including STT-RAM, PCRAM, ReRAM, and legacy NAND Flash. NVSim is successfully validated against industrial NVM prototypes, and it is expected to help boost architecture-level NVM-related studies. On the other side, NVMain is a cycle accurate main memory simulator designed to simulate emerging nonvolatile memories at the architectural level. We have released these models as open source tools and provided contiguous support to them. We also proposed PS3-RAM, which is a fast, portable and scalable statistical STT-RAM reliability analysis model [Wen 2012]. Design Space Exploration and Optimization With the support of these models, we explore different device/circuit optimization techniques. For example, in [Niu 2012a] we studied the power reduction technique for the application of ECC scheme in ReRAM designs and proposed to use ECC code to relax the BER (Bit Error Rate) requirement of a single memory to improve the write energy consumption and latency for both 1T1R and cross-point ReRAM designs. In [Xu 2011], we proposed a methodology to design STT-RAM for different optimization goals such as read performance, write performance and write energy by leveraging the trade-off between write current and write time of MTJ. We also studied the tradeoffs in building a reliable crosspoint ReRAM array [Niu 2012b]. We have conducted an in depth analysis of the circuit and system level design implications of multi-layer cross-point Resistive RAM (MLCReRAM) from performance, power and reliability perspectives [Xu 2013]. The objective of this study is to understand the design trade-offs of this technology with respect to the MLC Phase Change Memory (MLCPCM).Our MLC ReRAM design at the circuit and system levels indicates that different resistance allocation schemes, programming strategies, peripheral designs, and material selections profoundly affect the area, latency, power, and reliability of MLC ReRAM. Based on this analysis, we conduct two case studies: first we compare MLC ReRAM design against MLC phase-change memory (PCM) and multi-layer cross-point ReRAM design, and point out why multi-level ReRAM is appealing; second we further explore the design space for MLC ReRAM. Architecture and Application We explored hybrid checkpointing using phase-change memory for future exascale systems [Dong 2011] and showed that the use of nonvolatile memory for local checkpointing significantly increases the number of faults covered by local checkpoints and reduces the probability of a global failure in the middle of a global checkpoint to less than 1%. We also proposed a technique called i2WAP to mitigate the write variations in NVM-based last-level cache for the improvement of the NVM lifetime [Wang 2013]. Our wear leveling technique attempts to work around the limitations of write endurance by arranging data access so that write operations can be distributed evenly across all the storage cells. During our intensive research on fault-tolerant NVM design, we found that ECC cannot effectively tolerate hard errors from limited write endurance and process imperfection. Therefore, we devised a novel Point and Discard (PAD) architecture in in [ 2012] as a hard-error-tolerant architecture for ReRAM-based Last Level Caches. PAD improves the lifetime of ReRAM caches by 1.6X-440X under different process variations without performance overhead in the system's early life. We have investigated the applicability of NVM for persistent memory design [Zhao 2013]. New byte addressable NVM enables fast persistent memory that allows in-memory persistent data objects to be updated with much higher throughput. Despite the significant improvement, the performance of these designs is only 50% of the native system with no persistence support, due to the logging or copy-on-write mechanisms used to update the persistent memory. A challenge in this approach is therefore how to efficiently enable atomic, consistent, and durable updates to ensure data persistence that survives application and/or system failures. We have designed a persistent memory system, called Klin, that can provide performance as close as that of the native system. The Klin design adopts a non-volatile cache and a non-volatile main memory for constructing a multi-versioned durable memory system, enabling atomic updates without logging or copy-on-write. Our evaluation shows that the proposed Kiln mechanism can achieve up to 2X of performance improvement to NVRAM-based persistent memory employing write-ahead logging. In addition, our design has numerous practical advantages: a simple and intuitive abstract interface, microarchitecture-level optimizations, fast recovery from failures, and no redundant writes to slow non-volatile storage media. The work was published in MICRO 2013 and received Best Paper Honorable Mentioned Award.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2014

SU-E-T-03: 3D GPU-Accelerated Secondary Checks of Radiation Therapy Treatment Plans

Purpose: Redundant treatment verifications in conformal and intensity-modulated radiation therapy techniques are traditionally performed with single point calculations. New solutions can replace these checks with 3D treatment plan verifications. This work describes a software tool (Mobius3D, Mobius Medical Systems) that uses a GPU-accelerated collapsed cone algorithm to perform 3D independent verifications of TPS calculations. Methods: Mobius3D comes with reference beam models for common linear accelerators. The system uses an independently developed collapsed cone algorithm updated with recent enhancements. 144 isotropically-spaced cones are used for each voxel for calculations. These complex calculations can be sped up by using GPUs. Mobius3D calculate dose using DICOM information coming from TPS (CT, RT Struct, RT Plan RT Dose). DVH-metrics and 3D gamma tests can be used to compare both TPS and secondary calculations. 170 patients treated with all common techniques as 3DCFRT (including wedged), static and dynamic IMRT and VMAT have been successfully verified with this solution. Results: Calculation times are between 3–5 minutes for 3DCFRT treatments and 15–20 for most complex dMLC and VMAT plans. For all PTVs mean dose and 90% coverage differences are (1.12±0.97)% and (0.68±1.19)%, respectively. Mean dose discrepancies for all OARs is (0.64±1.00)%. 3D gamma (global, 3%/3 mm) analysis shows a mean passing rate of (97.8 ± 3.0)% for PTVs and (99.0±3.0)% for OARs. 3D gamma pasing rate for all voxels in CT has a mean value of (98.5±1.6)%. Conclusion: Mobius3D is a powerful tool to verify all modalities of radiation therapy treatments. Dose discrepancies calculated by this system are in good agreement with TPS. The use of reference beam data results in time savings and can be used to avoid the propagation of errors in original beam data into our QA system. GPU calculations permit enhanced collapsed cone calculations with reasonable calculation times.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2024

Semi-automatic image annotation using 3D LiDAR projections and depth camera data

Efficient image annotation is necessary to utilize deep learning object recognition neural networks in nuclear safeguards, such as for the detection and localization of target objects like nuclear material containers (NMCs). This capability can help automate the inventory accounting of different types of NMCs within nuclear storage facilities. The conventional manual annotation process is labor-intensive and time-consuming, hindering the rapid deployment of deep learning models for NMC identifications. This paper introduces a novel semi-automatic method for annotating 2D images of nuclear material containers (NMCs) by combining 3D light detection and ranging (LiDAR) data with color and depth camera images collected from a handheld scan system. The annotation pipeline involves an operator manually marking new target objects on a LiDAR-generated map, and projecting these 3D locations to images, thereby automatically creating annotations from the projections. The semi-automatic approach significantly reduces manual efforts and the expertise in image annotation that is required to perform the task, allowing deep learning models to be trained on-site within a few hours. The paper compares the performance of models trained on datasets annotated through various methods, including semi-automatic, manual, and commercial annotation services. The evaluation demonstrates that the semi-automatic annotation method achieves comparable or superior results, with a mean average precision (mAP) above 0.9, showcasing its efficiency in training object recognition models. Additionally, the paper explores the application of the proposed method to instance segmentation, achieving promising results in detecting multiple types of NMCs in various formations.

98 NUCLEAR DISARMAMENT, SAFEGUARDS, AND PHYSICAL P↗
DOE OSTI2016

Experimental and theoretical study of rotationally inelastic diffraction of H{sub 2}(D{sub 2}) from methyl-terminated Si(111)

Fundamental details concerning the interaction between H{sub 2} and CH{sub 3}–Si(111) have been elucidated by the combination of diffractive scattering experiments and electronic structure and scattering calculations. Rotationally inelastic diffraction (RID) of H{sub 2} and D{sub 2} from this model hydrocarbon-decorated semiconductor interface has been confirmed for the first time via both time-of-flight and diffraction measurements, with modest j = 0 → 2 RID intensities for H{sub 2} compared to the strong RID features observed for D{sub 2} over a large range of kinematic scattering conditions along two high-symmetry azimuthal directions. The Debye-Waller model was applied to the thermal attenuation of diffraction peaks, allowing for precise determination of the RID probabilities by accounting for incoherent motion of the CH{sub 3}–Si(111) surface atoms. The probabilities of rotationally inelastic diffraction of H{sub 2} and D{sub 2} have been quantitatively evaluated as a function of beam energy and scattering angle, and have been compared with complementary electronic structure and scattering calculations to provide insight into the interaction potential between H{sub 2} (D{sub 2}) and hence the surface charge density distribution. Specifically, a six-dimensional potential energy surface (PES), describing the electronic structure of the H{sub 2}(D{sub 2})/CH{sub 3}−Si(111) system, has been computed based on interpolation of density functional theory energies. Quantum and classical dynamics simulations have allowed for an assessment of the accuracy of the PES, and subsequently for identification of the features of the PES that serve as classical turning points. A close scrutiny of the PES reveals the highly anisotropic character of the interaction potential at these turning points. This combination of experiment and theory provides new and important details about the interaction of H{sub 2} with a hybrid organic-semiconductor interface, which can be used to further investigate energy flow in technologically relevant systems.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2017

Digital Elevation Model, 0.25 m, Barrow Environmental Observatory, Utqiagvik (Barrow), Alaska, 2013

This 0.25m horizontal resolution digital elevation model (DEM) was developed from Airborne Laser Altimetry flown by Aerometric Inc, now known as Quantum Spatial, Inc. on July 12, 2013. One Mission was flown and the data jointly processed with LANL personnel to produce a 0.25m DEM covering a region approximately 2.8km wide and 12.4km long extending from the coast above North Salt Lagoon to south of Gas Well Road. This DEM encompasses a diverse range of hydrologic, geomorphic, geophysical and biological features typical of the Barrow Peninsula. Vertical accuracy at the 95% confidence interval was computed as 0.143m. The coordinate system, datum, and geoid for this DEM are UTM Zone 4N, NAD83 (2011), NAVD88 (GEOID09). This dataset includes a variety of file types.The Next-Generation Ecosystem Experiments: Arctic (NGEE Arctic), was a research effort to reduce uncertainty in Earth System Models by developing a predictive understanding of carbon-rich Arctic ecosystems and feedbacks to climate. NGEE Arctic was supported by the Department of Energy's Office of Biological and Environmental Research.The NGEE Arctic project had two field research sites: 1) located within the Arctic polygonal tundra coastal region on the Barrow Environmental Observatory (BEO) and the North Slope near Utqiagvik (Barrow), Alaska and 2) multiple areas on the discontinuous permafrost region of the Seward Peninsula north of Nome, Alaska.Through observations, experiments, and synthesis with existing datasets, NGEE Arctic provided an enhanced knowledge base for multi-scale modeling and contributed to improved process representation at global pan-Arctic scales within the Department of Energy's Earth system Model (the Energy Exascale Earth System Model, or E3SM), and specifically within the E3SM Land Model component (ELM).

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2014

Study of antineutrino oscillations using accelerator and atmospheric data in MINOS

The Main Injector Neutrino Oscillation Search (MINOS) is a long baseline experiment that was built for studying the neutrino oscillation phenomena. The MINOS experiment uses high intensity muon neutrino and antineutrino beams created by Neutrinos at the Main Injector facility (NuMI) at the Fermi National Accelerator Laboratory (Fermilab). Neutrino interactions are recorded by two sampling steel-scintillator tracking calorimeters: 0.98\,kton Near Detector at Fermilab, IL and 5.4\,kton Far Detector at the Soudan Underground Laboratory, MN. These two detectors are functionally identical, which helps to reduce the systematic uncertainties in the muon neutrino and antineutrino disappearance measurements. The Near Detector, located 1.04\,km from the neutrino production target, is used to measure the initial beam composition and neutrino energy proximal to the neutrino source. The collected data at the Near Detector is then used to predict energy spectrum in the Far Detector. By comparing this prediction to collected data at the Far Detector, which is 735\,km away from the target, it enables a measurement of a set of parameters that govern the neutrino oscillation phenomenon. \\ \indent The flexibility of the NuMI beam configuration and the magnetization of the MINOS detectors facilitate the identification of $\nu_{\mu}$ and $\bar{\nu}_{\mu}$ charged-current interactions on an event-by-event basis. This enables one to measure neutrino and antineutrino oscillation parameters independently and therefore allows us to test the CPT symmetry in the lepton sector. To enhance the sensitivity of the oscillation parameters measurement, a number of techniques have been implemented. Event classification, shower energy estimation and energy resolution bin fitting, which are described in this dissertation, are three of these techniques. Moreover, the most stringent constraints on oscillation parameters can be achieved by combining multiple data sets. \\ \indent This dissertation reports the measurement of antineutrino oscill! ation parameters using the complete MINOS accelerator and atmospheric data set of charged-current $\bar{\nu}_{\mu}$ events. This set comprises exposures of (i) 3.36\times 10^{20} proton-on-target (POT) in the \bar{\nu}_{\mu}-beam mode, (ii) 10.71\times 10^{20} POT in the \nu_{\mu}-beam mode, and (iii) 37.88 kton yr of atmospheric antineutrinos. The data analysis provides the world's most precise measurement to date on the antineutrino oscillation parameters: $|\Delta \overline{m}^2|=\left( 2.50^{+0.23}_{-0.29}\right) \times 10^{-3} \,eV^2$ and $\sin^2(2\bar{\theta})=0.97^{+0.03}_{-0.08}$. This result is consistent with neutrino oscillation parameters independently measured by MINOS and by others. The difference between antineutrino and neutrino mass-squared splittings is computed to be $|\Delta \overline{m}^2|-|\Delta m^2|=\left(0.13^{+0.23}_{-0.25}\right) \times 10^{-3} \, eV^2$.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2013

Digital Elevation Model, 0.5 m, Barrow Environmental Observatory, Utqiagvik (Barrow), Alaska, 2012

This 0.5m horizontal resolution digital elevation model (DEM) was developed from Airborne Laser Altimetry flown by Aerometric Inc, now know as Quantum Spatial, Inc. on August 12, 2012. One Mission was flown and the data jointly processed with Los Alamos National Laboratory (LANL) personnel to produce a 0.5m DEM covering a region approximately 2.8km wide and 12.5km long extending from the coast above North Salt Lagoon to south of Gas Well Road. This DEM encompasses a diverse range of hyrdrologic, geomorphic, geophysical and biological features typical of the Barrow Peninsula. Vertical accuracy at the 95% confidence interval was computed as 0.145m. The coordinate system, datum, and geoid for this DEM are UTM Zone 4N, NAD83 (2011), NAVD88 (GEOID09). This dataset includes one .tif file and one .pdf file.The Next-Generation Ecosystem Experiments: Arctic (NGEE Arctic), was a research effort to reduce uncertainty in Earth System Models by developing a predictive understanding of carbon-rich Arctic ecosystems and feedbacks to climate. NGEE Arctic was supported by the Department of Energy's Office of Biological and Environmental Research.The NGEE Arctic project had two field research sites: 1) located within the Arctic polygonal tundra coastal region on the Barrow Environmental Observatory (BEO) and the North Slope near Utqiagvik (Barrow), Alaska and 2) multiple areas on the discontinuous permafrost region of the Seward Peninsula north of Nome, Alaska.Through observations, experiments, and synthesis with existing datasets, NGEE Arctic provided an enhanced knowledge base for multi-scale modeling and contributed to improved process representation at global pan-Arctic scales within the Department of Energy's Earth system Model (the Energy Exascale Earth System Model, or E3SM), and specifically within the E3SM Land Model component (ELM).

54 ENVIRONMENTAL SCIENCES↗