Search NASA⌕ Search

SEARCH · Search NASA

Results for “Computed diagnosis”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 73 records · Page 4
All sourcesNASA NTRSDOE OSTI
DOE OSTI2021

Deep learning for Alzheimer's disease: Mapping large-scale histological tau protein for neuroimaging biomarker validation

Abnormal tau inclusions are hallmarks of Alzheimer's disease and predictors of clinical decline. Several tau PET tracers are available for neurodegenerative disease research, opening avenues for molecular diagnosis in vivo. However, few have been approved for clinical use. Understanding the neurobiological basis of PET signal validation remains problematic because it requires a large-scale, voxel-to-voxel correlation between PET and (immuno) histological signals. Large dimensionality of whole human brains, tissue deformation impacting co-registration, and computing requirements to process terabytes of information preclude proper validation. We developed a computational pipeline to identify and segment particles of interest in billion-pixel digital pathology images to generate quantitative, 3D density maps. The proposed convolutional neural network for immunohistochemistry samples, IHCNet, is at the pipeline's core. We have successfully processed and immunostained over 500 slides from two whole human brains with three phospho-tau antibodies (AT100, AT8, and MC1), spanning several terabytes of images. Our artificial neural network estimated tau inclusion from brain images, which performs with ROC AUC of 0.87, 0.85, and 0.91 for AT100, AT8, and MC1, respectively. Introspection studies further assessed the ability of our trained model to learn tau-related features. We present an end-to-end pipeline to create terabytes-large 3D tau inclusion density maps co-registered to MRI as a means to facilitate validation of PET tracers.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2025

Nondestructive Evaluation of Concrete: Elastic Property Imaging Through Full Waveform Inversion

Concrete is a major construction material worldwide and plays a crucial role in the nuclear industry. The elastic properties of concrete are prone to change and degrade while in service, as it is often subjected to extreme operational and environmental conditions. An accurate evaluation of concrete's elastic properties is thus essential to ensure structural integrity and safety. This is especially true for concrete in nuclear power plants, where irradiation effects significantly impact concrete mechanical properties. There are various methods to assess these properties, with ultrasound-based techniques showing high potential due to their nondestructive nature, cost-effectiveness, and safety. While several nondestructive evaluation methods exist, most rely on idealizations such as assuming homogeneous material and plane wavefronts. In this work, we address these issues by introducing an ultrasound-based nondestructive method aimed at reconstructing spatially varying images of concrete mechanical properties. By accurately modeling wave physics, including scattering and reflection, we overcome several of the aforementioned idealizations and aim to utilize the full waveform for imaging material properties through depth, resulting in more reliable images. Full waveform inversion (FWI) was first introduced by geophysicists to reconstruct subsurface elastic property images. The goal is to minimize the difference between simulated and recorded wavefield signals, often through gradient-based optimization algorithms. While FWI is primarily conducted using the acoustic approximation of the wave equation, few works focus on elastic FWI, where the goal is to reconstruct images of not only the pressure wave speed but also the shear wave speed and density (or their equivalents). This work explores the potential of using elastic FWI to predict concrete mechanical properties as an initial effort for a more accurate monitoring of concrete conditions in service. Reconstructing images of different elastic parameters enables more specificity and accurate condition diagnosis. This paper will detail this approach and provide examples demonstrating the effectiveness of elastic FWI in reconstructing comprehensive maps of concrete mechanical properties.

42 - ENGINEERING↗
DOE OSTI2021

An Algebra of Machine Learners with Applications

Machine learning (ML) methods are increasingly being applied to solve complex, data-driven problems in diverse areas, by exploiting the physical laws derived from first principles such as thermal hydraulics and the abstract laws developed recently for data and computing infrastructures. These physical and abstract laws encapsulate, typically in compact algebraic forms, the critical knowledge that complements data-driven ML models. We present a unified perspective of these laws and ML methods using an abstract algebra (A;⊕,Ⓧ), wherein the performance estimation and classification tasks are characterized by the additive ⊕ operations, and the diagnosis, reconstruction, and optimization tasks are characterized by the difference Ⓧ operations. This abstraction provides ML codes and their performance characterizations that are transferable across different areas. We describe practical applications of these abstract operations using examples of throughput profile estimation tasks in data transport infrastructures, and power-level and sensor error estimation tasks in nuclear reactor systems.

Rao, Nageswara↗
DOE OSTI2023

FrESCO: Framework for Exploring Scalable Computational Oncology

The National Cancer Institute (NCI) monitors population level cancer trends as part of its Surveillance, Epidemiology, and End Results (SEER) program. This program consists of state or regional level cancer registries which collect, analyze, and annotate cancer pathology reports. From these annotated pathology reports, each individual registry aggregates cancer phenotype information from electronic health records. This data is then used to create summary statistics about cancer incidence and mortality to facilitate population health monitoring. Extracting phenotypic information from these reports is a labor intensive task, requiring specialized knowledge about the reports and cancer. Automating the information extraction process from cancer pathology reports has the potential to improve data quality by extracting information in a consistent manner across registries. It can also improve patient outcomes by reducing the time from diagnosis, enabling rapid case ascertainment for clinical trials. Here we present FrESCO, a modular deep-learning natural language processing (NLP) library initially designed for extracting pathology information from clinical text documents. This repository is not solely limited to clinical medical text, but may also be used by researchers just getting started with NLP methods and those looking for a robust solution for their classification problems.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2022

Image transformers for classifying acute lymphoblastic leukemia

Cancer is the leading cause of death by disease in American children. Each year, nearly 16,000 children in the United States and over 300,000 children globally are diagnosed with cancer. Leukemia is a form of blood cancer that originates in the bone marrow and accounts for one-third of pediatric cancers. This disease occurs when the bone marrow contains 20% or more immature white blood cell blasts. Acute lymphoblastic leukemia is the most prevalent leukemia type found in children, with half of all annual cases in the U.S. diagnosed for subjects under 20 years of age. To diagnose acute lymphoblastic leukemia, pathologists often conduct a morphological bone marrow assessment. This assessment determines whether the immature white blood cell blasts in bone marrow display the correct morphological characteristics, such as size and appearance of nuclei. Pathologists also use immunophenotyping via multi-channel flow cytometry to test whether certain antigens are present on the surface of blast cells; the antigens are used to identify the cell lineage of acute lymphoblastic leukemia. These manual processes require well-trained personnel and medical professionals, thus being costly in time and expenses. Computerized decision support via machine learning can accelerate the diagnosis process and reduce the cost. Training a reliable classification model to distinguish between mature and immature white blood cells is essential to the decision support system. Here, we adopted the Vision Transformer model to classify white blood cells. The Vision Transformer achieved superb classification performance compared to state-of-the-art convolutional neural networks while requiring less computational resources for training. Additionally, the latent self-attention architecture provided attention maps for a given image, providing clues as to which portion(s) of the image were significant in decision-making. We applied the Vision Transformer model and a convolutional neural network model to an acute lymphoblastic leukemia classification dataset of 12,528 samples and achieved accuracies of 88.4% and 86.2%.

Cho, Priscilla↗
DOE OSTI2021

Diagnosing nuclear power plant pipe wall thinning due to flow accelerated corrosion using a passive, thermal non-destructive evaluation method: Feasibility assessment via numerical experiments

Flow accelerated corrosion (FAC) in nuclear power plant pipes is one of the leading causes of accidents, fatalities, damage and outages. Current FAC identification methods employ expensive sensing technology and are “active” methods, where the response of the piping system to an externally-generated thermal, mechanical or optical excitation must be measured. As a result, these techniques require a disruptive and time-consuming setup. Here we propose a method that utilizes pipe surface temperature measurements to passively monitor for FAC-induced pipe wall thinning without the need for expensive equipment or post-installation setup time. This diagnostic method utilizes a simulation data-driven diagnostic model to estimate the amount of thickness reduction in a pipe based on changes in measured steady-state pipe temperatures. In order to reduce the computational burden of generating large, simulation-based datasets, the behavior of the insulation of the pipe was modeled using a suitably calibrated heat transfer boundary parameter. Additionally, global sensitivity analysis was performed to determine system parameter(s), such as the temperature of water flowing inside the pipe, which significantly affect the steady state pipe wall temperature and could cause errors in diagnosis. Two diagnostic models, one using only the change in steady-state temperature as an indicator for FAC-induced pipe wall thinning and the other using water temperature as an additional diagnostic model input were evaluated for their ability to estimate thickness reductions in a pipe using simulated pipe wall temperature data. For the numerical experiments conducted in this work, both models estimated wall thickness with errors within 0.5 mm, indicating that the proposed technique can potentially be used as a low-cost, first-pass method for FAC monitoring.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2024

Path-BigBird: An AI-Driven Transformer Approach to Classification of Cancer Pathology Reports

PURPOSE Surgical pathology reports are critical for cancer diagnosis and management. To accurately extract information about tumor characteristics from pathology reports in near real time, we explore the impact of using domain-specific transformer models that understand cancer pathology reports. METHODS We built a pathology transformer model, Path-BigBird, by using 2.7 million pathology reports from six SEER cancer registries. We then compare different variations of Path-BigBird with two less computationally intensive methods: Hierarchical Self-Attention Network (HiSAN) classification model and an offthe-shelf clinical transformer model (Clinical BigBird). We use five pathology information extraction tasks for evaluation: site, subsite, laterality, histology, and behavior. Model performance is evaluated by using macro and micro F 1 scores. RESULTS We found that Path-BigBird and Clinical BigBird outperformed the HiSAN in all tasks. Clinical BigBird performed better on the site and laterality tasks. Versions of the Path-BigBird model performed best on the two most difficult tasks: subsite (micro F 1 score of 72.53, macro F 1 score of 35.76) and histology (micro F 1 score of 80.96, macro F 1 score of 37.94). The largest performance gains over the HiSAN model were for histology, for which a Path-BigBird model increased the micro F 1 score by 1.44 points and the macro F 1 score by 3.55 points. Overall, the results suggest that a Path-BigBird model with a vocabulary derived from wellcurated and deidentified data is the best-performing model. CONCLUSION The Path-BigBird pathology transformer model improves automated information extraction from pathology reports. Although Path-BigBird outperforms Clinical BigBird and HiSAN, these less computationally expensive models still have utility when resources are constrained.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2020

State-of-the-art on research and applications of machine learning in the building life cycle

Fueled by big data, powerful and affordable computing resources, and advanced algorithms, machine learning has been explored and applied to buildings research for the past decades and has demonstrated its potential to enhance building performance. This study systematically surveyed how machine learning has been applied at different stages of building life cycle. By conducting a literature search on the Web of Knowledge platform, we found 9579 papers in this field and selected 153 papers for an in-depth review. The number of published papers is increasing year by year, with a focus on building design, operation, and control. However, no study was found using machine learning in building commissioning. There are successful pilot studies on fault detection and diagnosis of HVAC equipment and systems, load prediction, energy baseline estimate, load shape clustering, occupancy prediction, and learning occupant behaviors and energy use patterns. None of the existing studies were adopted broadly by the building industry, due to common challenges including (1) lack of large scale labeled data to train and validate the model, (2) lack of model transferability, which limits a model trained with one data-rich building to be used in another building with limited data, (3) lack of strong justification of costs and benefits of deploying machine learning, and (4) the performance might not be reliable and robust for the stated goals, as the method might work for some buildings but could not be generalized to others. Finally, findings from the study can inform future machine learning research to improve occupant comfort, energy efficiency, demand flexibility, and resilience of buildings, as well as to inspire young researchers in the field to explore multidisciplinary approaches that integrate building science, computing science, data science, and social science.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2022

AICCA: AI-Driven Cloud Classification Atlas

Clouds play an important role in the Earth’s energy budget, and their behavior is one of the largest uncertainties in future climate projections. Satellite observations should help in understanding cloud responses, but decades and petabytes of multispectral cloud imagery have to date received only limited use. This study describes a new analysis approach that reduces the dimensionality of satellite cloud observations by grouping them via a novel automated, unsupervised cloud classification technique based on a convolutional autoencoder, an artificial intelligence (AI) method good at identifying patterns in spatial data. Our technique combines a rotation-invariant autoencoder and hierarchical agglomerative clustering to generate cloud clusters that capture meaningful distinctions among cloud textures, using only raw multispectral imagery as input. Cloud classes are therefore defined based on spectral properties and spatial textures without reliance on location, time/season, derived physical properties, or pre-designated class definitions. We use this approach to generate a unique new cloud dataset, the AI-driven cloud classification atlas (AICCA), which clusters 22 years of ocean images from the Moderate Resolution Imaging Spectroradiometer (MODIS) on NASA’s Aqua and Terra instruments—198 million patches, each roughly 100 km × 100 km (128 × 128 pixels)—into 42 AI-generated cloud classes, a number determined via a newly-developed stability protocol that we use to maximize richness of information while ensuring stable groupings of patches. AICCA thereby translates 801 TB of satellite images into 54.2 GB of class labels and cloud top and optical properties, a reduction by a factor of 15,000. The 42 AICCA classes produce meaningful spatio-temporal and physical distinctions and capture a greater variety of cloud types than do the nine International Satellite Cloud Climatology Project (ISCCP) categories—for example, multiple textures in the stratocumulus decks along the West coasts of North and South America. We conclude that our methodology has explanatory power, capturing regionally unique cloud classes and providing rich but tractable information for global analysis. AICCA delivers the information from multi-spectral images in a compact form, enables data-driven diagnosis of patterns of cloud organization, provides insight into cloud evolution on timescales of hours to decades, and helps democratize climate research by facilitating access to core data.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

High-throughput terahertz imaging: progress and challenges

Abstract Many exciting terahertz imaging applications, such as non-destructive evaluation, biomedical diagnosis, and security screening, have been historically limited in practical usage due to the raster-scanning requirement of imaging systems, which impose very low imaging speeds. However, recent advancements in terahertz imaging systems have greatly increased the imaging throughput and brought the promising potential of terahertz radiation from research laboratories closer to real-world applications. Here, we review the development of terahertz imaging technologies from both hardware and computational imaging perspectives. We introduce and compare different types of hardware enabling frequency-domain and time-domain imaging using various thermal, photon, and field image sensor arrays. We discuss how different imaging hardware and computational imaging algorithms provide opportunities for capturing time-of-flight, spectroscopic, phase, and intensity image data at high throughputs. Furthermore, the new prospects and challenges for the development of future high-throughput terahertz imaging systems are briefly introduced.

36 MATERIALS SCIENCE↗
DOE OSTI2025

Mondo: integrating disease terminology across communities

Precision medicine aims to enhance diagnosis, treatment, and prognosis by integrating multimodal data at the point of care. However, challenges arise due to the vast number of diseases, differing methods of classification, and conflicting terminological coding systems and practices used to represent molecular definitions of disease. This lack of interoperability artificially constrains the potential for diagnosis, clinical decision support, care outcome analysis, as well as data linkage across research domains to support the development or repurposing of therapeutics. There is a clear and pressing need for a unified system for managing disease entities⁠—including identifiers, synonyms, and definitions. To address these issues, we created the Mondo disease ontology—a community-driven, open-source, unified disease classification system that harmonizes diverse terminologies into a consistent, computable framework. Mondo integrates key medical and biomedical terminologies, including Online Mendelian Inheritance in Man (OMIM), Orphanet, Medical Subject Headings (MeSH), National Cancer Institute Thesaurus (NCIt), and more, to provide a comprehensive and accurate representation of disease concepts with fully provenanced and attributed links back to the sources. Mondo can be used as the handle for curation of gene–disease associations utilized in diagnostic applications, research applications such as computational phenotyping, and in clinical coding systems in clinical decision support by pointing the clinician to the numerous knowledge resources linked to the Mondo identifier. Mondo's community-centric approach, stewarded by the Monarch Initiative's expertise in ontologies, ensures that the ontology remains adaptable to the evolving needs of biomedical research and clinical communities, as well as the knowledge providers.

biomedical informatics↗
DOE OSTI2025

A Mass Conservation Relaxed (MCR) LSTM Model for Streamflow Simulation Across CONUS

The recent development of the physics-aware Mass-Conserving Long Short-Term Memory network (MC-LSTM) provides an alternative to other data-driven Deep Learning (DL) models in hydrology. Mass-Conserving Long Short-Term Memory incorporates mass conservation directly into the LSTM architecture. Despite the theoretical advancements, studies have reported a surprisingly limited performance of the MC-LSTM in streamflow simulation. We hypothesize that such a limitation is due to the unrealistic mass conservation scheme in MC-LSTM, which overlooks unobserved incoming water fluxes beyond precipitation. As an attempt to verify this hypothesis, we propose a Mass Conservation Relaxed LSTM (MCR-LSTM), which incorporates a bi-directional mass relaxation (MR) component to account for potential incoming water fluxes beyond precipitation. We train and test the proposed MCR-LSTM model across 531 watersheds in the contiguous United States (CONUS) against three baseline models: the Sacramento Soil Moisture Accounting, LSTM, and MC-LSTM. Our results show that MCR-LSTM outperforms MC-LSTM despite its underperformance compared to LSTM. Specifically, MCR-LSTM's advantage over MC-LSTM is mainly seen in the Plains and Western U.S., where the newly incorporated MR component better simulates water loss and suggests the likely existence of additional incoming water fluxes beyond precipitation, respectively. The novelty and contribution of this study are twofold: firstly, it introduces an alternative physics-aware DL tool (i.e., MCR-LSTM) in hydrology with higher accuracy in specific regions compared to MC-LSTM. Secondly, it provides a diagnosis of regions where strict, precipitation-based mass conservation constraints may be unrealistic in streamflow simulation.

deep learning↗
DOE OSTI2021

A hierarchical evaluation framework for assessing climate simulations relevant to the energy-water-land nexus (Final Report)

The overarching goal was to construct a hierarchy of new and well-tested metrics and analysis tools that support both fundamental and use-inspired research. Motivating this goal was a convergence of needs from climate scientists and stakeholders alike for a systematic, robust framework of model evaluation and diagnosis to provide scientific insights, inform model development, support best practices for the use of climate model outputs, and facilitate communication of climate information in the evolving landscapes of multi-model, multi-resolution, and large ensemble simulations that generate terabytes of data for any single climate run. Steps toward attaining these goals benefited from expertise and established capability of the project team, which included leadership of the North American Regional Climate Change Assessment Program (NARCCAP) and the Coordinated Regional Downscaling Experiment (CORDEX), development of hierarchical model evaluation approaches, and successful research in the analysis and diagnosis of climate model skill, as well as the understanding and modeling of regional climate processes in North America. As part of the overarching goal, the project worked to disseminate a suite of methodologies, algorithms, and software components that the wider community can employ to advance climate science and applications. With rigorous demonstration, the evaluation framework and the mix of standard and high risk / high reward approaches helped form the basis for future development of a computationally enabled user-friendly system for community use.

17 WIND ENERGY↗
DOE OSTI2021

Subnetwork representation learning for discovering network biomarkers in predicting lymph node metastasis in early oral cancer

Cervical lymph node metastasis is the leading cause of poor prognosis in oral tongue squamous cell carcinoma and also occurs in the early stages. The current clinical diagnosis depends on a physical examination that is not enough to determine whether micrometastasis remains. The transcriptome profiling technique has shown great potential for predicting micrometastasis by capturing the dynamic activation state of genes. However, there are several technical challenges in using transcriptome data to model patient conditions: (1) An Insufficient number of samples compared to the number of genes, (2) Complex dependence between genes that govern the cancer phenotype, and (3) Heterogeneity between patients between cohorts that differ geographically and racially. We developed a computational framework to learn the subnetwork representation of the transcriptome to discover network biomarkers and determine the potential of metastasis in early oral tongue squamous cell carcinoma. Our method achieved high accuracy in predicting the potential of metastasis in two geographically and racially different groups of patients. The robustness of the model and the reproducibility of the discovered network biomarkers show great potential as a tool to diagnose lymph node metastasis in early oral cancer.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2024

Multi-Level Structural Damage Characterization Using Sparse Acoustic Sensor Networks and Knowledge Transferred Deep Learning

Standard structural health monitoring techniques face well-known difficulties for comprehensive defect diagnosis in real-world structures that have structural, material, or geometric complexity. This motivates the exploration of machine-learning-based structural health monitoring methods in complex structures. However, creating sufficient training data sets with various defects is an ongoing challenge for data-driven machine (deep) learning algorithms. The ability to transfer the knowledge of a trained neural network from one component to another or to other sections of the same component would drastically reduce the required training data set. Also, it would facilitate computationally inexpensive machine learning based inspection systems. In this work, a machine-learning-based multi-level damage characterization is demonstrated with the ability to transfer trained knowledge within the sparse sensor network. A novel network spatial assistance and an adaptive convolution technique are proposed for efficient knowledge transfer within the deep learning algorithm. Proposed structural health monitoring method is experimentally evaluated on an aluminum plate with artificially induced defects. It was observed that the method improves the performance of knowledge transferred damage characterization by 50% during localization and 24% during severity assessment. Further, experiments using time windows with and without multiple edge reflections are studied. Results reveal that multiply scattered waves contain rich and deterministic defect signatures that can be mined using deep learning neural networks, improving the accuracy of both identification and quantification. In the case of a fixed sensor network, using multiply scattered waves shows 100% prediction accuracy at all levels of damage characterization.

36 MATERIALS SCIENCE↗
DOE OSTI2026

A Physics-Aligned Multi-Domain Machine Learning Framework for Time-Localised Diagnosis of Power Electronics Faults

This paper presents a physics-aligned framework for fault diagnosis in multi-phase power-electronic systems using cycle-synchronous windowing and multi-domain features derived from Fourier, wavelet, and Hilbert–Huang representations. While both logistic regression and multilayer perceptron (MLP) models achieve perfect performance under standard evaluation, blind unseen testing reveals a critical failure in a baseline MLP. This is shown to arise from model selection based on validation accuracy. Using validation-loss-based selection restores correct unseen performance and improves confidence. Feature ablation shows that Fourier and wavelet features dominate, while computational analysis indicates that feature extraction, particularly HHT, governs runtime.

Kumar, Praveen [ORNL] (ORCID:0000000291877857)↗
DOE OSTI2022

Super-resolution and segmentation deep learning for breast cancer histopathology image analysis

Traditionally, a high-performance microscope with a large numerical aperture is required to acquire high-resolution images. However, the images’ size is typically tremendous. Therefore, they are not conveniently managed and transferred across a computer network or stored in a limited computer storage system. As a result, image compression is commonly used to reduce image size resulting in poor image resolution. Here, we demonstrate custom convolution neural networks (CNNs) for both super-resolution image enhancement from low-resolution images and characterization of both cells and nuclei from hematoxylin and eosin (H&E) stained breast cancer histopathological images by using a combination of generator and discriminator networks so-called super-resolution generative adversarial network-based on aggregated residual transformation (SRGAN-ResNeXt) to facilitate cancer diagnosis in low resource settings. The results provide high enhancement in image quality where the peak signal-to-noise ratio and structural similarity of our network results are over 30 dB and 0.93, respectively. The derived performance is superior to the results obtained from both the bicubic interpolation and the well-known SRGAN deep-learning methods. In addition, another custom CNN is used to perform image segmentation from the generated high-resolution breast cancer images derived with our model with an average Intersection over Union of 0.869 and an average dice similarity coefficient of 0.893 for the H&E image segmentation results. Finally, we propose the jointly trained SRGAN-ResNeXt and Inception U-net Models, which applied the weights from the individually trained SRGAN-ResNeXt and inception U-net models as the pre-trained weights for transfer learning. The jointly trained model’s results are progressively improved and promising. We anticipate these custom CNNs can help resolve the inaccessibility of advanced microscopes or whole slide imaging (WSI) systems to acquire high-resolution images from low-performance microscopes located in remote-constraint settings.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2023

Integrated End-to-end Performance Prediction and Diagnosis for Extreme Scientific Workflows

This report details recent progress for the ASCR funded project “Integrated End-to-end Performance Prediction and Diagnosis for Extreme Scientific Workflows”. We refer to the project as IPPD/2, reflecting the 2017 renewal under expanded scope and partners In IPPD/2, we increased our research scope to include data motion. We are focusing on three major aspects: a) observe how data is generated, distributed, and used; b) analyze how data is (repeatedly) consumed with a focus both on repeated patterns and anomalies; and c) explore how to optimize data motion. This new work on data motion will augment and complement IPPD/2’s research that focused on the computational aspects of tasks. We leverage and extend our existing tools and demonstrate our work on the Belle II workflow suite as well as on workflows from NSLS-II. The highlights of our work are as follows: Provenance for Workflows: Provenance is used to provide information enabling quality control, re-run computational workflows, and reproduce results. IPPD/2 has been building a scalable provenance management system that enables the capture of provenance from the high-level workflow through all relevant system levels in one integrated environment. Leveraging this work, our recent efforts have included using provenance as an enabling technique. Workload characterization: Leveraging provenance and analysis, we characterize data movement within network, storage, and memory over a variety of workloads. This characterization enables an understanding by performance analysts and application developers of the range of behaviors that could be expected. Performance Prediction for Workflows: The goal of modeling distributed workflows is to understand performance bottlenecks and enable more intelligent task scheduling to optimize selected metrics of interest (e.g., task throughput or output data rate). IPPD/2 has utilized both analytical and AI/ML modeling methodologies for performance modeling. Advanced Scheduling and Fault Modeling for Workflows: Scheduling of large-scale scientific workflows on geographically distributed resources is a challenging problem. To improve workflow throughput, we combined novel scheduling algorithms with task predictions from performance modeling and fault modeling. Dynamically Alleviating Bottlenecks in Workflows: Exploiting our provenance, analysis, and modeling efforts, we have explored and developed several techniques for dynamically detecting and alleviating bottlenecks in data movement. In particular, we have spent considerable effort demonstrating our techniques on production-like workflow configurations.

97 MATHEMATICS AND COMPUTING↗