Search NASA⌕ Search

SEARCH · Search NASA

Results for “analysis workflow”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 307 records · Page 17

Building a Cyberinfrastructure for Environmental System Science: Modeling Frameworks, Data Management, and Scientific Workflows, Workshop Report, Potomac, Maryland, April 30-May 1, 2015

The Environmental System Science (ESS) activity within the Department of Energy’s (DOE) Office of Biological and Environmental Research (BER) seeks to advance a robust, predictive understanding of terrestrial environments, extending from bedrock to the top of the vegetative canopy and from molecular to global scales, through an iterative cycle of model-driven experimentation and observation dubbed MODEX. Considerable progress has been made toward achieving this overarching goal, but widely recognized is the fragmentation across projects and disciplines of the relevant modeling and simulation capabilities, observational and experimental data, analysis algorithms, and workflow tools. This fragmentation creates significant challenges for studying impacts and feedbacks in these complex multiscale systems. These challenges are further exacerbated by ongoing disruptive changes in high-performance computational architectures and the exponential growth in the types and volume of data that render obsolete the conventional approaches to software development and data management. Overcoming these challenges will require the development of a BER Climate and Environmental Sciences Division (CESD)–wide enabling cyberinfrastructure to support data management, cross-domain modeling, data analysis, and collaborative research. To explore the potential for working groups to initiate and guide a more integrated and community-based cyberinfrastructure, BER held the ESS Workshop on Model-Data Integration: Modeling Frameworks, Data Management, and Scientific Workflows on April 30-May 1, 2015, following the ESS Principal Investigator Meeting in Potomac, Maryland. Participants included model developers, software engineers, and data management specialists from eight national laboratories, which represented a wide range of projects and programs from CESD as well as cross-cutting projects from DOE’s Office of Advanced Scientific Computing Research (ASCR; both BER and ASCR are operated from within DOE’s Office of Science). A series of plenary talks provided background information and clarified the workshop’s three objectives: (1) develop requirements for this community-based cyberinfrastructure to ensure enhanced scientific productivity of the community as a whole; (2) identify challenges associated with developing this new cyberinfrastructure using a phased approach guided by project-driven use cases; and, given these requirements and challenges, (3) chart a path forward for ESS working groups to lead the phased development of the new cyberinfrastructure.

54 ENVIRONMENTAL SCIENCES↗

Profiling of Histone Post-Translational Modifications in Mouse Brain with High-Resolution Top-Down Mass Spectrometry

Histones play central roles in most chromosomal functions and both their basic biology and roles in disease have been the subject of intense study. Since multiple PTMs along the entire protein sequence are potential regulators of histones, a top-down approach, where intact proteins are analyzed, is ultimately required for complete characterization of proteoforms. However, significant challenges remain for top-down histone analysis primarily because of deficiencies in separation/resolving power and effective identification algorithms. Here, we used state of the art mass spectrometry and a bioinformatics workflow for targeted data analysis and visualization. The workflow uses ProMex for intact mass deconvolution, MSPathFinder as search engine, and LcMsSpectator as a data visualization tool. ProMex sums across retention time to maximize sensitivity and accuracy for low abundance species in MS1deconvolution. MSPathFinder searches the MS2 data against protein sequence databases with user-defined modifications. LcMsSpectator presents the results from ProMex and MSPathFinder in a format that allows quick manual evaluation of critical attributes for high-confidence identifications. When complemented with the open-modification tool TopPIC, this workflow enabled identification of novel histone PTMs including tyrosine bromination on histone H4 and H2A, H3 glutathionylation, and mapping of conventional PTMs along the entire protein for many histone subunits.

Environmental Molecular Sciences Laboratory↗

Discovering invariant spatial features in electron energy loss spectroscopy images on the mesoscopic and atomic levels

Over the last two decades, Electron Energy Loss Spectroscopy (EELS) imaging with a scanning transmission electron microscope has emerged as a technique of choice for visualizing complex chemical, electronic, plasmonic, and phononic phenomena in complex materials and structures. The availability of the EELS data necessitates the development of methods to analyze multidimensional data sets with complex spatial and energy structures. Traditionally, the analysis of these data sets has been based on analysis of individual spectra, one at a time, whereas the spatial structure and correlations between individual spatial pixels containing the relevant information of the physics of underpinning processes have generally been ignored and analyzed only via the visualization as 2D maps. Here, we develop a machine learning-based approach and workflows for the analysis of spatial structures in 3D EELS data sets using a combination of dimensionality reduction and multichannel rotationally invariant variational autoencoders. This approach is illustrated for the analysis of both the plasmonic phenomena in a system of nanowires and in the core excitations in functional oxides using low loss and core-loss EELS, respectively. The code developed in this manuscript is open sourced and freely available and provided as a Jupyter notebook for the interested reader.

36 MATERIALS SCIENCE↗

Quantifying the Dynamics of Protein Self-Organization Using Deep Learning Analysis of Atomic Force Microscopy Data

The dynamics of protein self-assembly on the inorganic surface and the resultant geometric patterns are visualized using high-speed atomic force microscopy. The time dynamics of the classical macroscopic descriptors such as 2D fast Fourier transforms, correlation, and pair distribution functions are explored using the unsupervised linear unmixing, demonstrating the presence of static ordered and dynamic disordered phases and establishing their time dynamics. Here, the deep learning (DL)-based workflow is developed to analyze detailed particle dynamics and explore the evolution of local geometries. Finally, we use a combination of DL feature extraction and mixture modeling to define particle neighborhoods free of physics constraints, allowing for a separation of possible classes of particle behavior and identification of the associated transitions. Overall, this work establishes the workflow for the analysis of the self-organization processes in complex systems from observational data and provides insight into the fundamental mechanisms.

36 MATERIALS SCIENCE↗

GRDC. A Collaborative Framework for Radiological Background and Contextual Data Analysis

The Radiation Mobile Analysis Platform (RadMAP) is unique in its capability to collect both high quality radiological data from both gamma-ray detectors and fast neutron detectors and a broad array of contextual data that includes positioning and stance data, high-resolution 3D radiological data from weather sensors, LiDAR, and visual and hyperspectral cameras. The datasets obtained from RadMAP are both voluminous and complex and require analyses from highly diverse communities within both the national laboratory and academic communities. Maintaining a high level of transparency will enable analysis products to further enrich the RadMAP dataset. It is in this spirit of open and collaborative data that the RadMAP team proposed to collect, calibrate, and make available online data from the RadMAP system. The Berkeley Data Cloud (BDC) is a cloud-based data management framework that enables web-based data browsing visualization, and connects curated datasets to custom workflows such that analysis products can be managed and disseminated while maintaining user access rights. BDC enables cloud-based analyses of large datasets in a manner that simulates real-time data collection, such that BDC can be used to test algorithm performance on real and source-injected datasets. Using the BDC framework, a subset of the RadMAP datasets have been disseminated via the Gamma Ray Data Cloud (GRDC) that is hosted through the National Energy Research Science Computing (NERSC) Center, enabling data access to over 40 users at 10 institutions.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗

OmicsMLMentor: A Web Application for Guided Machine Learning Analysis of Omics Data

Expression-based omics technologies (e.g. proteomics, metabolomics, transcriptomics, etc.) increasingly rely on supervised and unsupervised machine learning (ML) models to find key biomolecules distinguishing conditions, identify natural groupings in biological data, or generate predictions for outcomes of interest. Fitting ML models to omics data presents several challenges, including handling missing data, selecting a normalization method, choosing a valid model, and optimizing hyperparameters, all requiring statistical programming skills to address these challenges. Thus, the open-source web application SLOPE was designed to lower the barrier to ML modeling for omics data. SLOPE supports the fitting of 15 ML models (10 supervised and 5 unsupervised) tailored to omics datasets, such as proteomics, metabolomics, lipidomics, and transcriptomics. SLOPE offers several omics-specific features, including methods for handling missingness (imputation, conversion, removal), normalization tests, ranking of models based on the structure of a user’s data and user input, and optimal hyperparameter selections using cross-validation splits. By streamlining ML workflows for omics analysis, SLOPE address critical gaps in existing online web tools, facilitating a broader adoption of these models for omics research. Here, SLOPE is applied to data from a lignin exposure study to highlight the workflow for fitting both supervised and unsupervised models to data.

lipidomics↗

Automated Quantitative Analysis of Extended Irradiation Defects - Dislocations, Voids and Precipitates in Neutron Irradiated HT-9 Steel

Structural alloys that make up the core components of nuclear reactors are exposed to radiation that can substantially alter the microstructure and mechanical properties of the material. Irradiation can cause the formation of extended defects including dislocation line and loops, voids and various types of precipitates. To understand the irradiation effects, it is critical to characterize these defects individually and collectively to quantify how they evolve throughout irradiation stages. Transmission electron microscopy (TEM) is a standard tool for providing extensive size, number density and spatial distribution of extended irradiation defects, however, manual measurement of defect size and number density requires time-intensive effort to analyze enough defects that provide a statistically meaningful result and are prone to errors of subjectivity. In this work, we developed a set of MATLAB algorithms offering robust and automated identification and quantification of extended irradiation defects like dislocation lines, voids and precipitates commonly observed in neutron irradiated cladding alloys [1]. Using a neutron-irradiated HT-9 martensitic alloy irradiated at 412°C to 111.8 dpa, we demonstrate the capability of these algorithms for quantitative defect analysis. Figure 1 presents defect images of the HT-9 using an advanced diffraction contrast imaging scanning transmission electron microscopy (DCI STEM) that suppresses bend contours and provides better defect contrast compared to conventional TEM [2]. The first step of defect analysis is defect identification, a process that can be digitally achieved by contrast thresholding (for simple defect contrast) and by machine learning computer vision for defect semantic segmentation [3]. Here, labeled binary defect images (Figures 2a and 2c) were used. For dislocation density quantification, we employed an optimized circular-grid intersection method first introduced by Dr. John Steeds [4]. This method is shown in Figures 2a and 2b, where the number of intersections of dislocation lines with circular traces was measured. This produces an unbiased sampling of dislocation lines, regardless of their orientation. This process was optimized with automated digital intersection number and grid length registration. Interestingly, it was found that a relatively large number of circles in a set circular gird is was needed to produce an invariant result (Figure 2b), with a low systematic measurement error. This demonstrates the main advantage of computer-based defect analysis, in performing repetitive operations quickly with a reduced systematic error. For the quantitative analysis of voids and precipitates a challenge encountered with automated defect analysis is feature overlap, particularly when the density of voids/precipitates is high. To mitigate this problem, as shown in Figure 2d, we modified the popular circular Hough transformation and treated ‘circular shape’ as the particle descriptor for voids/precipitates, efficiently resolving the overlapping particles. In summary, this automated digital quantification together with machine learning enabled defect identification [3] opens new opportunities for standardizing the characterization workflow for quantitative analysis of extended irradiation defects.

Sainju, Rajat↗

Data Analyst - A Device for Continuous Repeating Real-Time Assay of HPGe, CZT, or Scintillation Gamma Spectra - 18574

There are many situations where a continuous repeating series of quantitative gamma spectral assays is desirable. These include measurements of fluid going through a pipe, of items on a conveyor passing under a detector, a detector aimed at the ground on a moving platform, or a detector in free air as a plume of radioactivity drifts by. While Canberra and many others have built and deployed many of these devices, they have all been custom built for that particular application, and rather expensive. The Data Analyst [DA] is a general purpose device that is designed to fulfill many of these applications without modification, but can be fairly-easily modified if needed. The DA is a small [12 x 18 x 5 cm], low-power [3 W] box that accepts the input from external MCAs, controls the acquisition cycle, analyzes the results, and transmits them to the outside world when needed. The detector-MCAs supported are HPGe with Lynx MCA, NaI or CeBr or LaBr with Osprey MCA, and CZT or CsI with internal MCAs. The DA internal software used for analysis is the full standard Genie analysis software, which is very widely used and well accepted in the industry. This allows exported spectral files to be reviewed and confirmed by Experts, and reanalyzed if needed. The DA hardware is totally autonomous. As soon as power is applied, it starts the pre-programmed sequences of acquisition and analysis. Multiple analysis sequences [called Workflows] can be in operation simultaneously, e.g. with different count times and different nuclide libraries. Only the detector with MCA, and the DA are required in the field. A PC can be connected to the DA locally or remotely via WiFi, USB, or Ethernet. The PC is needed to setup the DA, and to readout the results. The DA also supports [with application-specific programming] external Inputs/Outputs either for recording with the analysis results or for use in the computation of the result, e.g. temperature, flow rate, pressure, ... ]. The DA also has an internal GPS receiver with external antenna. Applications of the DA that are discussed include stack gas effluent activity where a portion of the effluent stream was extracted to flow through a shield with a HPGe detector and Lynx MCA, and measurement of reactor coolant activity where a shielded CZT detector and MCA were aimed at various sections of reactor primary coolant during a maintenance outage. Potential other applications are also described. (author)

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗

Near-Single-Cell Proteomics Profiling of the Proximal Tubular and Glomerulus of the Normal Human Kidney

Molecular assessments at the single-cell level can accelerate biological research by providing detailed assessments of cellular organization and tissue heterogeneity in both disease and health. The human kidney has complex multi-cellular states with varying functionality, much of which can now be completely harnessed with recent technological advances in tissue proteomics at a near single-cell level. We discuss the foundational steps in the first application of this mass spectrometry (MS) based proteomics method for analysis of sub-sections of the normal human kidney, as part of the Kidney Precision Medicine Project (KPMP). Using approximately 30-40 laser captured micro-dissected kidney cells, we identified more than 2500 human proteins, with specificity to the proximal tubular (PT; n=25 proteins) and glomerular (Glom; n=67 proteins) regions of the kidney and their unique metabolic functions. This pilot study provides the roadmap for application of our near-single-cell proteomics workflow for the analysis of other renal micro-compartments, on a larger scale, to unravel perturbations of renal sub-cellular function in the normal kidney as well as different etiologies of acute and chronic kidney disease.

60 APPLIED LIFE SCIENCES↗

Correlating and Simulating Socio-Demographically Driven Residential End-Use Activity Schedules

Incorporating socio-demographic and behavioral considerations into decision-support tools is crucial for identifying gaps and addressing consumer needs to ensure reliable and affordable energy solutions. In energy simulation models, the correlation between socio-demographics and time-use behavior is not well-captured. Thus, we developed a large-scale simulation workflow to generate schedules for 10 residential activities across 24 population segments defined by age, income, and employment status. Using pre-pandemic 2015-2019 American Time Use Survey (ATUS) data, we used ANOVA to confirm the correlation between demographic factors and time use. We explored three k-modes clustering methods-backward, forward, and a new hybrid approach-to delineate the occupancy patterns based on demographics. Using the probability of cluster membership for each population segment and a time inhomogeneous Markov chain to generate activity transition probabilities for each cluster, we simulated 50,000 schedules per segment and validated them against the ATUS data. The hybrid method produced the most socio-demographically differentiated clusters while demonstrating comparable performance to other approaches, with an overall root mean square error of 0.12 for both weekday and weekend schedules. Thus, the hybrid method, where each cluster is dominated by certain demographic segments and occupancy patterns, offers more modeling versatility in terms of scenario analysis. The new workflow improves the socio demographic differentiation of energy consumption by considering differences in time use. This approach enables future research on demographically segmented time of use (TOU) energy consumption, including impacts of TOU utility bills and rate analysis, long-run marginal emissions, and energy retrofits.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

Simulation model analysis of the most promising geological sequestration formation candidates in the Rocky Mountain region, USA, with focus on uncertainty assessment

The purpose of this report is to report results of reservoir model simulation analyses for forecasting subsurface CO 2 storage capacity estimation for the most promising formations in the Rocky Mountain region of the USA. A particular emphasis of this project was to assess uncertainty of the simulation-based forecasts. Results illustrate how local-scale data, including well information, number of wells, and location of wells, affect storage capacity estimates and what degree of well density (number of wells over a fixed area) may be required to estimate capacity within a specified degree of confidence. A major outcome of this work was development of a new workflow of simulation analysis, accommodating the addition of “random pseudo wells” to represent virtual characterization wells.

58 GEOSCIENCES↗

Boosting RDataFrame performance with transparent bulk event processing

RDataFrame is ROOT’s high-level interface for Python and C++ data analysis. Since it first became available, RDataFrame adoption has grown steadily and it is now poised to be a major component of analysis software pipelines for LHC Run 3 and beyond. Thanks to its design inspired by declarative programming principles, RDataFrame enables the development of highperformance, highly parallel analyses without requiring expert knowledge of multi-threading and I/O: user logic is expressed in terms of self-contained, small computation kernels tied together by a high-level API. This design completely decouples analysis logic from its actual execution, and opens several interesting avenues for workflow optimization. In particular, in this work we explore the benefits of moving internal data processing from an event-by-event to a bulkby-bulk loop. This refactoring dramatically reduces the framework’s runtime overheads; in collaboration with the I/O layer it improves data access patterns; it exposes information that optimizing compilers might use to auto-vectorize the invocation of user-defined computations; finally, while existing user-facing interfaces remain unaffected, it becomes possible to additionally offer interfaces that explicitly expose bulks of events, useful e.g. for the injection of GPU kernels into the analysis workflow. In order to inform similar future R&D, design challenges will be presented, as well as an investigation of the relevant timememory trade-off backed by novel performance benchmarks.

Guiraud, Enrico↗

The ATLAS Workflow Management System Evolution in the LHC Run3 and towards the High-Luminosity LHC era

The ATLAS experiment has 18+ years of experience using workload management systems to deploy and develop workflows to process and to simulate data on the distributed computing infrastructure. Simulation, processing and analysis of LHC experiment data require the coordinated work of heterogeneous computing resources. In particular, the ATLAS experiment utilizes the resources of 250 computing centers worldwide, the power of supercomputing centres, and national, academic and commercial cloud computing resources. In this contribution, we present new techniques for cost-effectively improving efficiency introduced in workflow management system software. The evolution from a mesh framework to new types of computing facilities such as cloud and HPCs is described, as well as new types of production and analysis workflows.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

Veridical data science

Building and expanding on principles of statistics, machine learning, and scientific inquiry, we propose the predictability, computability, and stability (PCS) framework for veridical data science. Our framework, composed of both a workflow and documentation, aims to provide responsible, reliable, reproducible, and transparent results across the data science life cycle. The PCS workflow uses predictability as a reality check and considers the importance of computation in data collection/storage and algorithm design. It augments predictability and computability with an overarching stability principle. Stability expands on statistical uncertainty considerations to assess how human judgment calls impact data results through data and model/algorithm perturbations. As part of the PCS workflow, we develop PCS inference procedures, namely PCS perturbation intervals and PCS hypothesis testing, to investigate the stability of data results relative to problem formulation, data cleaning, modeling decisions, and interpretations. We illustrate PCS inference through neuroscience and genomics projects of our own and others. Moreover, we demonstrate its favorable performance over existing methods in terms of receiver operating characteristic (ROC) curves in high-dimensional, sparse linear model simulations, including a wide range of misspecified models. Finally, we propose PCS documentation based on R Markdown or Jupyter Notebook, with publicly available, reproducible codes and narratives to back up human choices made throughout an analysis. The PCS workflow and documentation are demonstrated in a genomics case study available on Zenodo.

97 MATHEMATICS AND COMPUTING↗

Separability of Mesh Bias and Parametric Uncertainty for a Full System Thermal Analysis

When making computational simulation predictions of multi-physics engineering systems, sources of uncertainty in the prediction need to be acknowledged and included in the analysis within the current paradigm of striving for simulation credibility. A thermal analysis of an aerospace geometry was performed at Sandia National Laboratories. For this analysis a verification, validation and uncertainty quantification workflow provided structure for the analysis, resulting in the quantification of significant uncertainty sources including spatial numerical error and material property parametric uncertainty. It was hypothesized that the parametric uncertainty and numerical errors were independent and separable for this application. This hypothesis was supported by performing uncertainty quantification simulations at multiple mesh resolutions, while being limited by resources to minimize the number of medium and high resolution simulations. Based on this supported hypothesis, a prediction including parametric uncertainty and a systematic mesh bias are used to make a margin assessment that avoids unnecessary uncertainty obscuring the results and optimizes computing resources.

42 ENGINEERING↗

Highly interactive, steered scientific workflows on HPC systems: optimizing design solutions

Scientific workflows are becoming increasingly important in high performance computing (HPC) settings, as the feasibility and appeal of many simultaneous heterogeneous tasks increases with increasing hardware capabilities. Currently no HPC-based workflow platform supports a dynamically adaptable workflow with interactive steering and analysis at run-time. Furthermore, for most workflow programs, compute resources are fixed for a given instance, resulting in a possible waste of expensive allocation resources when tasks are spawned and killed. Here we describe the design and testing of a run-time-interactive, adaptable, steered workflow tool capable of executing thousands of parallel tasks without an MPI programming model, using a database management system to facilitate task management through multiple live connections. We find that on the Oak Ridge Leadership Computing Facility pre-exascale Summit supercomputer it is possible to launch and interactively steer workflows with thousands of simultaneous tasks with negligible latency. For the case of particle simulation and analysis tasks that run for minutes to hours, this paradigm offers the prospect of a robust and efficient means to perform simulation-space exploration with on-the-fly analysis and adaptation.

Ossyra, John R↗

A New Workflow of X-ray CT Image Processing and Data Analysis of Structural Features in Rock Using Open-Source Software

X-ray computed tomography (CT) images of rock specimens often contain artifacts which must be corrected before scientific analyses are performed. Here, we present a new workflow of automated image processing to utilize poor-quality X-ray CT scan images. The workflow runs on the open-source image analysis software and efficiently separates desired features from low-contrast scanned images. The new workflow is a two-step technique using contrast enhancement and automated feature segmentation to generate noise-free binary images. The results of binary images using the proposed workflow and using a conventional thresholding technique are analyzed to show the quality of the proposed method. The paper also presents a workflow of estimating the structural geometries of features in two and three dimensions. The results of the structural feature analyses and computational time were compared between the open-source (ImageJ) and commercial image analysis software (Bruker Computed Tomography Analyzer). The commercial software was more computationally efficient, but the task-specific macros in open-source software enabled the user-desired automation in image processing and data extraction of desired structural features of comparable quality.

47 OTHER INSTRUMENTATION↗