Search NASA⌕ Search

SEARCH · Search NASA

Results for “Data Intensive Computing”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 577 records · Page 32
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

Employing artificial intelligence to steer exascale workflows with colmena

Computational workflows are a common class of application on supercomputers, yet the loosely coupled and heterogeneous nature of workflows often fails to take full advantage of their capabilities. We created Colmena to leverage the massive parallelism of a supercomputer by using Artificial Intelligence (AI) to learn from and adapt a workflow as it executes. Colmena allows scientists to define how their application should respond to events (e.g., task completion) as a series of cooperative agents. In this paper, we describe the design of Colmena, the challenges we overcame while deploying applications on exascale systems, and the science workflows we have enhanced through interweaving AI. The scaling challenges we discuss include developing steering strategies that maximize node utilization, introducing data fabrics that reduce communication overhead of data-intensive tasks, and implementing workflow tasks that cache costly operations between invocations. These innovations coupled with a variety of application patterns accessible through our agent-based steering model have enabled science advances in chemistry, biophysics, and materials science using different types of AI. In conclusion, our vision is that Colmena will spur creative solutions that harness AI across many domains of scientific computing.

Workflows↗
DOE OSTI2024

GPU-enabled extreme-scale turbulence simulations: Fourier pseudo-spectral algorithms at the exascale using OpenMP offloading

Fourier pseudo-spectral methods for nonlinear partial differential equations are of wide interest in many areas of advanced computational science, including direct numerical simulation of three-dimensional (3-D) turbulence governed by the Navier-Stokes equations in fluid dynamics. This paper presents a new capability for simulating turbulence at a new record resolution up to 35 trillion grid points, on the world's first exascale computer, Frontier, comprising AMD MI250x GPUs with HPE's Slingshot interconnect and operated by the US Department of Energy's Oak Ridge Leadership Computing Facility (OLCF). Key programming strategies designed to take maximum advantage of the machine architecture involve performing almost all computations on the GPU which has the same memory capacity as the CPU, performing all-to-all communication among sets of parallel processes directly on the GPU, and targeting GPUs efficiently using OpenMP offloading for intensive number-crunching including 1-D Fast Fourier Transforms (FFT) performed using AMD ROCm library calls. With 99% of computing power on Frontier being on the GPU, leaving the CPU idle leads to a net performance gain via avoiding the overhead of data movement between host and device except when needed for some I/O purposes. Memory footprint including the size of communication buffers for MPI_ALLTOALL is managed carefully to maximize the largest problem size possible for a given node count. Detailed performance data including separate contributions from different categories of operations to the elapsed wall time per step are reported for five grid resolutions, from 2048 3 on a single node to 32768 3 on 4096 or 8192 nodes out of 9408 on the system. Both 1D and 2D domain decompositions which divide a 3D periodic domain into slabs and pencils respectively are implemented. The present code suite (labeled by the acronym GESTS, GPUs for Extreme Scale Turbulence Simulations) achieves a figure of merit (in grid points per second) exceeding goals set in the Center for Accelerated Application Readiness (CAAR) program for Frontier. The performance attained is highly favorable in both weak scaling and strong scaling, with notable departures only for 2048 3 where communication is entirely intra-node, and for 32768 3 , where a challenge due to small message sizes does arise. Communication performance is addressed further using a lightweight test code that performs all-to-all communication in a manner matching the full turbulence simulation code. Performance at large problem sizes is affected by both small message size due to high node counts as well as dragonfly network topology features on the machine, but is consistent with official expectations of sustained performance on Frontier. Overall, although not perfect, the scalability achieved at the extreme problem size of 32768 3 (and up to 8192 nodes — which corresponds to hardware rated at just under 1 exaflop/sec of theoretical peak computational performance) is arguably better than the scalability observed using prior state-of-the-art algorithms on Frontier's predecessor machine (Summit) at OLCF. New science results for the study of intermittency in turbulence enabled by this code and its extensions are to be reported separately in the near future.

3D fast Fourier transform↗
DOE OSTI2022

Circular Trajectory Approach for Online Sinusoidal Signal Distortion Monitoring and Visualization

The increasing complexity and uncertainties of modern power systems are placing significant demands on signal monitoring techniques. This work proposes the Circular Trajectory Approach (CTA) for online sinusoidal signal distortion monitoring and visualization. CTA can detect distortions of a sinusoidal signal. Compared with existing waveform anomaly detection techniques, CTA is faster in detection and less computation intensive. It thus supports edge devices and online applications. CTA also offers a new means of sinusoidal signal distortion visualization. It can reveal the distorted sections in a sinusoidal cycle and clearly display the distortions. The proposed approach is tested on real data from an open source EPRI dataset.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2017

Template Interfaces for Agile Parallel Data-Intensive Science

Tigres provides a programming library to compose and execute large-scale data-intensive scientific workflows from desktops to supercomputers. DOE User Facilities and large science collaborations are increasingly generating large enough data sets that it is no longer practical to download them to a desktop to operate on them. They are instead stored at centralized compute and storage resources such as high performance computing (HPC) centers. Analysis of this data requires an ability to run on these facilities, but with current technologies, scaling an analysis to an HPC center and to a large data set is difficult even for experts. Tigres is addressing the challenge of enabling collaborative analysis of DOE Science data through a new concept of reusable "templates" that enable scientists to easily compose, run and manage collaborative computational tasks. These templates define common computation patterns used in analyzing a data set.

Ramakrishnan, Lavanya↗
DOE OSTI2019

Preservation of Alcator C-Mod Data and Support of ITER Research Through ITPA Participation (Final Technical Report)

This grant supported two primary activities at the MIT Plasma Science and Fusion Center. First, personnel support, hardware upgrades, and software development were used to maintain the C-Mod data archive set. This highly valuable repository contains all of the data generated from operation of the Alcator C-Mod FES User Facility, covering first operation in 1992 through completion of the final experimental campaign on Sept 30, 2016. The archive, stored using the MDSplus architecture, also includes all of the data analysis, which is ongoing, and most of the results of computer modeling of the data. Second, this grant supported activities related to participation of the C-Mod team in the International Tokamak Physics Activites (ITPA), which includes analysis of C-Mod data, in conjunction with data from tokamaks operated around the world, with intense coordination of joint experiments.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2021

Scalable, In-situ Data Clustering Data Analysis for Extreme Scale Scientific Computing (Final Report)

The objective of this project is to address challenges in the design and development of scalable in-situ data clustering and analytics algorithms and software. Our goal is to develop parallel software consisting of a set of spatio-temporal data clustering and anomaly detection functions, both of which are very important for large-scale analysis and have wide applicability for in-situ runs as well as post-processing analysis. Our design principles for in-situ analysis consider the following: (1) identify parts of the computation can be done close to the data within the nodes, while it is still in memory; (2) extract analysis components can (and should) be performed in remote staging and analysis nodes; (3) develop error-bound approximation methods for applications tolerable for small errors; (4) identify the type of derived distributions and statistics, for spatio-temporal data, that can be kept locally in order to both accelerate computations and meet energy constraints in subsequent iterations and phases; (5) use a self-describing data format so that data can be consistent and understood among local storage (memory and SSDs) and at staging and analysis nodes, thereby providing portability and flexibility; (6) develop service-oriented functions that can schedule in-situ and post-hoc analysis tasks based on the dynamic requirements of applications. Our development focus is to produce the parallel data analysis software/library that will be scalable, reusable, extensible, and generic for applications in different disciplines. The software will be able to run in-situ with the simulations as well as post-hoc analysis. This approach will satisfy many synergistic requirements for data intensive applications executed on data coming from instruments and experiments. In particular, the proposed multilevel approach is directly applicable to perform design tradeoffs for running part of the algorithms near the instruments and the rest on remote (analysis) systems.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2015

TWO-DIMENSIONAL CELLULAR AUTOMATON MODEL FOR THE EVOLUTION OF ACTIVE REGION CORONAL PLASMAS

We study a two-dimensional cellular automaton (CA) model for the evolution of coronal loop plasmas. The model is based on the idea that coronal loops are made of elementary magnetic strands that are tangled and stressed by the displacement of their footpoints by photospheric motions. The magnetic stress accumulated between neighbor strands is released in sudden reconnection events or nanoflares that heat the plasma. We combine the CA model with the Enthalpy Based Thermal Evolution of Loops model to compute the response of the plasma to the heating events. Using the known response of the X-Ray Telescope on board Hinode, we also obtain synthetic data. The model obeys easy-to-understand scaling laws relating the output (nanoflare energy, temperature, density, intensity) to the input parameters (field strength, strand length, critical misalignment angle). The nanoflares have a power-law distribution with a universal slope of –2.5, independent of the input parameters. The repetition frequency of nanoflares, expressed in terms of the plasma cooling time, increases with strand length. We discuss the implications of our results for the problem of heating and evolution of active region coronal plasmas.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2014

SU-E-T-491: Importance of Energy Dependent Protons Per MU Calibration Factors in IMPT Dose Calculations Using Monte Carlo Technique

Purpose: In intensity modulated proton therapy (IMPT), energy dependent, protons per monitor unit (MU) calibration factors are important parameters that determine absolute dose values from energy deposition data obtained from Monte Carlo (MC) simulations. Purpose of this study was to assess the sensitivity of MC-computed absolute dose distributions to the protons/MU calibration factors in IMPT. Methods: A “verification plan” (i.e., treatment beams applied individually to water phantom) of a head and neck patient plan was calculated using MC technique. The patient plan had three beams; one posterior-anterior (PA); two anterior oblique. Dose prescription was 66 Gy in 30 fractions. Of the total MUs, 58% was delivered in PA beam, 25% and 17% in other two. Energy deposition data obtained from the MC simulation were converted to Gy using energy dependent protons/MU calibrations factors obtained from two methods. First method is based on experimental measurements and MC simulations. Second is based on hand calculations, based on how many ion pairs were produced per proton in the dose monitor and how many ion pairs is equal to 1 MU (vendor recommended method). Dose distributions obtained from method one was compared with those from method two. Results: Average difference of 8% in protons/MU calibration factors between method one and two converted into 27 % difference in absolute dose values for PA beam; although dose distributions preserved the shape of 3D dose distribution qualitatively, they were different quantitatively. For two oblique beams, significant difference in absolute dose was not observed. Conclusion: Results demonstrate that protons/MU calibration factors can have a significant impact on absolute dose values in IMPT depending on the fraction of MUs delivered. When number of MUs increases the effect due to the calibration factors amplify. In determining protons/MU calibration factors, experimental method should be preferred in MC dose calculations. Research supported by National Cancer Institute grant P01CA021239.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2015

SU-E-E-11: Novel Matching Module for Respiration-Gated Motion Tumor of Cone-Beam Computed Tomography (CBCT) to 4DCT

Purpose: Four dimensional computed tomography (4DCT) scans reliably record whole respiratory phase and generate internal target volumes (ITV) for radiotherapy planning. However, image guiding with cone-beam computed tomography (CBCT) cannot acquire all or specific respiratory phases. This study was designed to investigate the correlation between average CT and Maximum Intensity Projection (MIP) from 4DCT and CBCT. Methods: Retrospective respiratory gating were performed by GE Discovery CT590 RT. 4DCT and CBCT data from CRIS Dynamic Thorax Phantom with simulated breathing mode were analyzed. The lung tissue equivalent material encompassed 3 cm sphere tissue equivalent material. Simulated breathing cycle period was set as 4 seconds, 5 seconds and 6 seconds for representing variation of patient breathing cycle time, and the sphere material moved toward inferior and superior direction with 1 cm amplitude simulating lung tumor motion during respiration. Results: Under lung window, the volume ratio of CBCT scans to ITVs derived from 10 phases average scans was 1.00 ± 0.02, and 1.03 ± 0.03 for ratio of CBCT scans to MIP scans. Under abdomen window, the ratio of CBCT scans to ITVs derived from 10 phases average scans was 0.39 ± 0.06, and 0.06 ± 0.00 for ratio of CBCT scans to MIP scans. There was a significant difference between lung window Result and abdomen window Result. For reducing image guiding uncertainty, CBCT window was set with width 500 and level-250. The ratio of CBCT scans to ITVs derived from 4 phases average scans with abdomen window was 1.19 ± 0.02, and 1.06 ± 0.01 for ratio of CBCT to MIP scans. Conclusion: CBCT images with suitable window width and level can efficiently reduce image guiding uncertainty for patient with mobile tumor. By our setting, we can match motion tumor to gating tumor location on planning CT more accurately neglecting other motion artifacts during CBCT scans.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2026

Accelerating lattice gauge theory studies with Agentic AI

Lattice gauge theory research, with its computationally intensive simulations and complex multi‑stage workflows, is well positioned to benefit from agentic AI systems. We demonstrate how such tools can support key components of lattice gauge theory research, including novel simulation code development using standard LQCD frameworks, HPC job orchestration, simulation data analysis, and expert‑guided tuning of algorithmic parameters such as Hasenbusch mass preconditioning and multigrid solvers. Our results show that agentic AI can reduce manual effort, improve productivity, and accelerate the research cycle while maintaining essential human oversight.

Ayyar, Venkitesh [Fermilab]↗
DOE OSTI2022

Modern Trends in Neutron Scattering Instrument Technologies

This article reviews some current trends that can be observed in the development of neutron scattering instrument technologies. While the number of neutron scattering facilities worldwide and the number of beam days they offer are largely stable, their scientific impact is increasing through improving instrumental capabilities, new and more versatile instruments, and more efficient data collection protocols. Neutron beams are becoming smaller but more intense, and instruments are being designed to utilize more ‘useful’ neutrons in unit time. This article picks and discusses a few recent developments in the areas of integrated source and instrument design, use of computational tools, new detectors, and experiment automation.

47 OTHER INSTRUMENTATION↗
DOE OSTI2015

BAYESIAN CALIBRATION OF SAFETY CODES USING DATA FROM SEPARATE-AND INTEGRAL EFFECTS TESTS

Large-scale system codes for simulation of safety performance of nuclear plants may contain parameters whose values are not known very accurately. In order to be able to use the results of these simulation codes with confidence, it is important to learn how the uncertainty on the values of these parameters affects the output of the codes. New information from tests or operating experience is incorporated into safety codes by a process known as calibration, which reduces uncertainty in the output of the safety code, and thereby improves its support for decision-making. Modern analysis capabilities afford very significant improvements on classical ways of doing calibration, and the work reported here implements some of those improvements. The key innovation has come from development of safety code surrogate model (code emulator) construction and prediction algorithms. A surrogate is needed for calibration of plant-scale simulation codes because the multivariate nature of the problem (i.e., the need to adjust multiple uncertain parameters at once to fit multiple pieces of new information) calls for multiple evaluations of performance, which, for a computation-intensive model, makes calibration very computation-intensive. Use of a fast surrogate makes the calibration processes used here with Markov Chain Monte Carlo (MCMC) sampling feasible. Moreover, most traditional surrogates do not provide uncertainty information along with their predictions, but the Gaussian Process (GP) based code surrogates used here do. This improves the soundness of the code calibration process. Results are demonstrated on a simplified scenario with data from Separate and Integral Effect Tests.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2018

Creation of a Robust and Generalizable Machine Learning Classifier for Patient Ventilator Asynchrony

Background: As healthcare increasingly digitizes, streaming waveform data is being made available from an variety of sources, but there still remains a paucity of performant clinical decision support systems. For example, in the intensive care unit (ICU) existing automated alarm systems typically rely on simple thresholding that result in frequent false positives. Recurrent false positive alerts create distrust of alarm mechanisms that can be directly detrimental to patient health. To improve patient care in the ICU, we need alert systems that are both pervasive, and accurate so as to be informative and trusted by providers. Objective: We aimed to develop a machine learning-based classifier to detect abnormal waveform events using the use case of mechanical ventilation waveform analysis, and the detection of harmful forms of ventilation delivery to patients. We specifically focused on detecting injurious subtypes of patient-ventilator asynchrony (PVA).Methods: Using a dataset of breaths recorded from 35 different patients, we used machine learning to create computational models to automatically detect, and classify two types of injurious PVA, double trigger asynchrony (DTA), breath stacking asynchrony (BSA). We examined the use of synthetic minority over-sampling technique (SMOTE) to overcome class imbalance problems, varied methods for feature selection, and use of ensemble methods to optimize the performance of our model. Results: We created an ensemble classifier that is able to accurately detect DTA at a sensitivity/specificity of 0.960/0.975, BSA at sensitivity/specificity of 0.944/0.987, and non-PVA events at sensitivity/specificity of .967/.980. Conclusions: Our results suggest that it is possible to create a high-performing machine learning-based model for detecting PVA in mechanical ventilator waveform data in spite of both intra-patient, and inter-patient variability in waveform patterns, and the presence of clinical artifacts like cough and suction procedures. Our work highlights the importance of addressing class imbalance in clinical data sets, and the combined use of statistical methods and expert knowledge in feature selection.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2011

Scenario driven data modelling: a method for integrating diverse sources of data and data streams

Background: Biology is rapidly becoming a data intensive, data-driven science. It is essential that data is represented and connected in ways that best represent its full conceptual content and allows both automated integration and data driven decision-making. Recent advancements in distributed multi-relational directed graphs, implemented in the form of the Semantic Web make it possible to deal with complicated heterogeneous data in new and interesting ways. Results: This paper presents a new approach, scenario driven data modelling (SDDM), that integrates multirelational directed graphs with data streams. SDDM can be applied to virtually any data integration challenge with widely divergent types of data and data streams. In this work, we explored integrating genetics data with reports from traditional media. SDDM was applied to the New Delhi metallo-beta-lactamase gene (NDM-1), an emerging global health threat. The SDDM process constructed a scenario, created a RDF multi-relational directed graph that linked diverse types of data to the Semantic Web, implemented RDF conversion tools (RDFizers) to bring content into the Sematic Web, identified data streams and analytical routines to analyse those streams, and identified user requirements and graph traversals to meet end-user requirements. Conclusions: We provided an example where SDDM was applied to a complex data integration challenge. The process created a model of the emerging NDM-1 health threat, identified and filled gaps in that model, and constructed reliable software that monitored data streams based on the scenario derived multi-relational directed graph. The SDDM process significantly reduced the software requirements phase by letting the scenario and resulting multi-relational directed graph define what is possible and then set the scope of the user requirements. Approaches like SDDM will be critical to the future of data intensive, data-driven science because they automate the process of converting massive data streams into usable knowledge.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2019

Mathematical modelling the pathway of genomic instability in lung cancer

Genomic instability plays a significant role in lung cancer. Although substantial research has been conducted using both clinical and theoretical studies, it is still a hotly debated issue to whether genomic instability is necessary or whether genomic instability precedes oncogenes activation and tumor suppressor genes inactivation for lung cancer. In response to this issue, we come up with a mathematical model incorporating effects of genomic instability to investigate the genomic instability pathway of human lung cancer. The presented model are applied to match the incidence rate data of lung cancer from the Life Span Study cohort of the atomic bomb survivors in Nagasaki and Hiroshima and the Surveillance Epidemiology and End Results registry in the United States. Model results suggest that genomic instability is necessary in the tumorigenesis of lung cancer, and genomic instability has no significant impact on the net proliferation rate of cells by statistical criteria. By comparing the results of the LSS data to those of the SEER data, we conclude that the genomic instability pathway exhibits a sensitivity to radiation exposure, more intensive in male patients.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2021

Pooling Data Improves Multimodel IDF Estimates over Median-Based IDF Estimates: Analysis over the Susquehanna and Florida

Traditional multimodel methods for estimating future changes in precipitation intensity, duration, and frequency (IDF) curves rely on mean or median of models’ IDF estimates. Such multimodel estimates are impaired by large estimation uncertainty, shadowing their efficacy in planning efforts. Here, assuming that each climate model is one representation of the underlying data generating process, i.e., the Earth system, we propose a novel extension of current methods through pooling model data: (i) evaluate performance of climate models in simulating the spatial and temporal variability of the observed annual maximum precipitation (AMP), (ii) bias-correct and pool historical and future AMP data of reasonably performing models, and (iii) compute IDF estimates in a nonstationary framework from pooled historical and future model data. Pooling enhances fitting of the extreme value distribution to the data and assumes that data from reasonably performing models represent samples from the “true” underlying data generating distribution. Through Monte Carlo simulations with synthetic data, we show that return periods derived from pooled data have smaller biases and lesser uncertainty than those derived from ensembles of individual model data. We apply this method to NA-CORDEX models to estimate changes in 24-h precipitation intensity–frequency (PIF) estimates over the Susquehanna watershed and Florida peninsula. Our approach identifies significant future changes at more stations compared to median-based PIF estimates. The analysis suggests that almost all stations over the Susquehanna and at least two-thirds of the stations over the Florida peninsula will observe significant increases in 24-h precipitation for 2–100-yr return periods.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗