Search NASA⌕ Search

SEARCH · Search NASA

Results for “analysis workflow”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 55 records · Page 3

Cultivating an Emergent Earth Observation Analytics Ecosystem in the Cloud

A diverse set of data analytics systems for Earth Observations are sprouting up in the Earth Science community, with a wealth of processing algorithms and analysis methods. There is a similar wealth of data resources available via myriad data providers and clearinghouses, including large institutional systems like the Earth Observing System Data and Information System, Comprehensive Large Scale Array-data Stewardship System, and Federated Earth Observation Missions gateway. With Earth system science driving a need to work with more datasets together, and the community developing more analysis tools (some of them dataset-specific), how can we develop analysis workflows that incorporate far-flung datasets and leverage analysis resources from multiple organizations? Cloud computing points the way toward a solution in two different respects. Firstly, the access to and abstraction of virtually unlimited storage and computing power provides an environment that enables more straightforward means of pulling datasets and analysis resources together. Just as importantly, however, cloud computing serves as an example of an "ecosystem" of interoperating services, since the essence of cloud computing is the presentation of all resources as a service, from hardware to infrastructure to platform to software. This enables the combination of off-the-shelf, diverse services to construct entire systems that emerge out of an equally diverse community of architects and developers. This approach can be similarly applied to the data and analysis resources in the Earth Observation community. By exposing these resources via well understood services, and consuming resources in the same way, different organizations can construct bespoke analysis workflows and systems for their own purposes. The key leap the community needs to make is to develop analysis systems in components that interact with other components via services. The result would be a rich ecosystem of analytics components that can be combined to analyze datasets at scale and in conjunction with other datasets from other sources.

chaos↗

Utilizing Distributed Heterogeneous Computing with PanDA in ATLAS

In recent years, advanced and complex analysis workflows have gained increasing importance in the ATLAS experiment at CERN, one of the large scientific experiments at LHC. Support for such workflows has allowed users to exploit remote computing resources and service providers distributed worldwide, overcoming limitations on local resources and services. The spectrum of computing options keeps increasing across the Worldwide LHC Computing Grid (WLCG), volunteer computing, high-performance computing, commercial clouds, and emerging service levels like Platform-as-a-Service (PaaS), Container-as-a-Service (CaaS) and Function-as-a-Service (FaaS), each one providing new advantages and constraints. Users can significantly benefit from these providers, but at the same time, it is cumbersome to deal with multiple providers, even in a single analysis workflow with fine-grained requirements coming from their applications’ nature and characteristics. In this paper, we will first highlight issues in geographically-distributed heterogeneous computing, such as the insulation of users from the complexities of dealing with remote providers, smart workload routing, complex resource provisioning, seamless execution of advanced workflows, workflow description, pseudointeractive analysis, and integration of PaaS, CaaS, and FaaS providers. We will also outline solutions developed in ATLAS with the Production and Distributed Analysis (PanDA) system and future challenges for LHC Run4.

97 MATHEMATICS AND COMPUTING↗

Discovery of unknown posttranslational modifications by top-down mass spectrometry

Post-translational modifications can be important biomarkers of function within biological systems. Top-down mass spectrometry is a valuable tool for the characterization of combinatorial modifications on proteins such as histones that have a range of complex modifications. In this chapter, we present a top-down liquid chromatography-mass spectrometry experimental and data analysis workflow for the identification of novel modifications on proteins. The data analysis workflow presented here is reliant on the combination of proteoform searching using TopPIC and data visualization and fragment ion spectral confirmation using LcMsSpectator, part of the Informed-Proteomics package of software.

Wilson, Jesse W.↗

Two-step neutronics calculations with Shift and Griffin for advanced reactor systems

This research develops the initial coupling of the Shift Monte Carlo (MC) code and the Griffin reactor physics code for reactor analysis of non–light-water reactor systems. The novelty of this work is twofold. It is the first application of Shift to produce the multigroup cross sections needed for Griffin as applied to a non–light-water reactor system; and, the first investigation and analysis of characteristics of the Empire microreactor benchmark that should be considered for steady state and transient reactor physics calculations. This application uses the previously developed two-step neutronics analysis workflow to demonstrate this initial coupling. Here, we outline the two-step neutronics analysis workflow in which the Shift MC code is used to generate the multigroup cross sections and fluxes needed by the Griffin deterministic solver. Details on how these multigroup cross sections are generated using MC tallies are given, as well as the practicalities and limitations of the two-step neutronics workflow. The Empire microreactor benchmark was used to investigate and validate this coupling. Results using this benchmark show good agreement between Griffin calculations using Serpent-generated cross sections and Shift-generated cross sections. Analysis of the characteristics of this Empire benchmark show larger eigenvalue differences between heterogeneous and pin–homogenized solutions compared to those of traditional light-water reactor (LWR) designs, thus requiring super homogenization factor corrections for accurate eigenvalue and power distribution predictions.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗

A General Framework for Monitoring Image Acquisition Workflow in the Radiology Environment: Timeliness for Acute Stroke CT Imaging

Many facets of an image acquisition workflow leave a digital footprint, making workflow analysis amenable to an informatics-based solution. This paper describes a detailed framework for analyzing workflow and uses acute stroke response timeliness in CT as a practical demonstration. We review methods for accessing the digital footprints resulting from common technologist/device interactions. This overview lays a foundation for obtaining data for workflow analysis. We demonstrate the method by analyzing CT imaging efficiency in the setting of acute stroke. We successfully used digital footprints of CT technologists to analyze their workflow. We presented an overview of other digital footprints including but not limited to contrast administration, patient positioning, billing, reformat creation, and scheduling. A framework for analyzing image acquisition workflow was presented. This framework is transferable to any modality, as the key steps of image acquisition, image reconstruction, image post processing, and image transfer to PACS are common to any imaging modality in diagnostic radiology.

62 RADIOLOGY AND NUCLEAR MEDICINE↗

CMS distributed analysis infrastructure and operations: Experience with the first LHC data

The CMS distributed analysis infrastructure represents a heterogeneous pool of resources distributed across several continents. The resources are harnessed using glite and glidein-based work load management systems (WMS). We provide the operational experience of the analysis workflows using CRAB-based servers interfaced with the underlying WMS. The automatized interaction of the server with the WMS provides a successful analysis workflow. We present the operational experience as well as methods used in CMS to analyze the LHC data. The interaction with CMS Run-registry for Run and luminosity block selections via CRAB is discussed. The variations of different workflows during the LHC data-taking period and the lessons drawn from this experience are also outlined.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

FLIMJ: An open-source ImageJ toolkit for fluorescence lifetime image data analysis

In the field of fluorescence microscopy, there is continued demand for dynamic technologies that can exploit the complete information from every pixel of an image. One imaging technique with proven ability for yielding additional information from fluorescence imaging is Fluorescence Lifetime Imaging Microscopy (FLIM). FLIM allows for the measurement of how long a fluorophore stays in an excited energy state, and this measurement is affected by changes in its chemical microenvironment, such as proximity to other fluorophores, pH, and hydrophobic regions. This ability to provide information about the microenvironment has made FLIM a powerful tool for cellular imaging studies ranging from metabolic measurement to measuring distances between proteins. The increased use of FLIM has necessitated the development of computational tools for integrating FLIM analysis with image and data processing. To address this need, we have created FLIMJ, an ImageJ plugin and toolkit that allows for easy use and development of extensible image analysis workflows with FLIM data. Built on the FLIMLib decay curve fitting library and the ImageJ Ops framework, FLIMJ offers FLIM fitting routines with seamless integration with many other ImageJ components, and the ability to be extended to create complex FLIM analysis workflows. Building on ImageJ Ops also enables FLIMJ’s routines to be used with Jupyter notebooks and integrate naturally with science-friendly programming in, e.g., Python and Groovy. We show the extensibility of FLIMJ in two analysis scenarios: lifetime-based image segmentation and image colocalization. We also validate the fitting routines by comparing them against industry FLIM analysis standards.

47 OTHER INSTRUMENTATION↗

Prescriptive provenance for streaming analysis of workflows at scale

Abstract- We extend our approach capturing and relating the provenance and performance metrics of computational workflows as a diagnostic tool for runtime optimization and placement. One important challenge is the volume of extracted data, both for performance metrics and provenance, even when specifying filters and focusing on quantities of interest in a simulation. We reduce this data by performing anomaly detection on streaming data and store provenance for the detected anomalies, an approach we call prescriptive provenance. This paper discusses the Chimbuko architecture enabling the approach. We present the use of a protein structure propagation workflow based on NWChemEx. We are testing algorithms for anomaly detection and present preliminary results here obtained with Local Outlier Factor. While scaling remains a challenge, these results show that our robust Chimbuko architecture for streaming analysis with prescriptive provenance is a promising approach.

97 MATHEMATICS AND COMPUTING↗

PIXLISE-C: Exploring The Data Analysis Needs of NASA Scientists for Mineral Identification

NASA JPL scientists working on the micro x-ray fluorescence (microXRF) spectroscopy data collected from Mars surface perform data analysis to look for signs of past microbial life on Mars. Their data analysis workflow mainly involves identifying mineral com- pounds through the element abundance in spatially distributed data points. Working with the NASA JPL team, we identified pain points and needs to further develop their existing data visualization and analysis tool. Specifically, the team desired improvements for the process of creating and interpreting mineral composition groups. To address this problem, we developed an interactive tool that enables scientists to (1) cluster the data using either manual lasso-tool selection or through various machine learning clustering algorithms, and (2) compare the clusters and individual data points to make informed decisions about mineral compositions. Our preliminary tool supports a hybrid data analysis workflow where the user can manually refine the machine-generated clusters.

Davidoff, Scott↗

Algorithm sensitivity analysis and parameter tuning for tissue image segmentation pipelines

Motivation: Sensitivity analysis and parameter tuning are important processes in large-scale image analysis. They are very costly because the image analysis workflows are required to be executed several times to systematically correlate output variations with parameter changes or to tune parameters. An integrated solution with minimum user interaction that uses effective methodologies and high performance computing is required to scale these studies to large imaging datasets and expensive analysis workflows. Results: The experiments with two segmentation workflows show that the proposed approach can (i) quickly identify and prune parameters that are non-influential; (ii) search a small fraction (about 100 points) of the parameter search space with billions to trillions of points and improve the quality of segmentation results (Dice and Jaccard metrics) by as much as 1.42 x compared to the results from the default parameters; (iii) attain good scalability on a high performance cluster with several effective optimizations. Conclusions: Our work demonstrates the feasibility of performing sensitivity analyses, parameter studies and auto-tuning with large datasets. The proposed framework can enable the quantification of error estimations and output variations in image segmentation pipelines.

Biochemistry & Molecular Biology↗

Efficient Workflows for Curation of Heterogeneous Data Supporting Modeling of U-Nb Alloy Aging

These are slides from a presentation summarizing a graduate research associate's summer project. The following topics are covered in these slides: data challenges in materials, aging in U-Nb Alloys, Building an Aging Model, Different Phase Trans. in U-Nb, the Challenge, Storing Materials Data, Example Data Source, Organizing Data: What is a Schema?, What does a "XML Schema" look like?, Our Data Schema: Nice and Simple, Storing Data: Materials Data Curation System (MDCS), Problem with MDCS: Slow Data Entry, Getting Literature into MDCS, Staging Data in Excel Document, Final Result: MDCS Records, Analyzing Image Data, Process for Making TTT Diagram, Bottleneck Number 1: Image Analysis, Fitting a TTP Boundary, Fitting a TTP Curve: Comparable Results, How Does it Compare to Our Data?, Image Analysis Workflow, Curating Hardness Records, Hardness Data: Two Key Decisions, Before Peak Age? - Automation, Interactive Viz, Which Transformation?, Microstructure-Informed Model, Tracking the Entire Process, General Problem with Property Models, Pinyon: Toolkit for Managing Model Creation, Tracking Individual Decisions, Jupyter: Docs and Code in One File, Hardness Analysis Workflow, Workflow for Aging Models, and conclusions.

36 MATERIALS SCIENCE↗

Data automation at light sources

Rapidly growing data volumes at light sources demand increasingly automated data collection, distribution, and analysis processes, in order to enable new scientific discoveries while not overwhelming finite human capabilities. We present here the case for automating and outsourcing light source science using cloud-hosted data automation and enrichment services, institutional computing resources, and high- performance computing facilities to provide cost-effective, scalable, and reliable implementations of such processes. We discuss three specific services that accomplish these goals for data distribution, automation, and transformation. In the first, Globus cloud-hosted data automation services are used to implement data capture, distribution, and analysis workflows for Advanced Photon Source and Advanced Light Source beamlines, leveraging institutional storage and computing. In the second, such services are combined with cloud-hosted data indexing and institutional storage to create a collaborative data publication, indexing, and discovery service, the Materials Data Facility (MDF), built to support a host of informatics applications in materials science. The third integrates components of the previous two projects with machine learning capabilities provided by the Data and Learning Hub for science (DLHub) to enable on-demand access to machine learning models from light source data capture and analysis workflows, and provides simplified interfaces to train new models on data from sources such as MDF on leadership scale computing resources. We draw conclusions about best practices for building next-generation data automation systems for future light sources.

Blaiszik, Ben↗

Publishing unbinned differential cross section results

Machine learning tools have empowered a qualitatively new way to perform differential cross section measurements whereby the data are unbinned, possibly in many dimensions. Unbinned measurements can enable, improve, or at least simplify comparisons between experiments and with theoretical predictions. Furthermore, many-dimensional measurements can be used to define observables after the measurement instead of before. There is currently no community standard for publishing unbinned data. While there are also essentially no measurements of this type public, unbinned measurements are expected in the near future given recent methodological advances. The purpose of this paper is to propose a scheme for presenting and using unbinned results, which can hopefully form the basis for a community standard to allow for integration into analysis workflows. This is foreseen to be the start of an evolving community dialogue, in order to accommodate future developments in this field that is rapidly evolving.

47 OTHER INSTRUMENTATION↗

Demonstration of the Advanced Dynamic System Modeling Tool TRANSFORM in a Molten Salt Reactor Application via a Model of the Molten Salt Demonstration Reactor

Liquid-fueled nuclear reactors, particularly molten salt reactors (MSRs), have recently gained significant interest in the advanced reactor community. As with all reactors, modeling and simulation are critical to advanced reactor design and licensing and will be required for MSR deployment. However, there are significant gaps in existing simulation capabilities for MSRs, particularly with the unique challenges of liquid-fueled systems (e.g., fission product transport). Furthermore, advanced reactor designers require near-term tools that are readily modifiable to perform design and analysis, including the ability to extend their analysis beyond the primary system to auxiliary systems. Transient Simulation Framework of Reconfigurable Models (TRANSFORM), a Modelica-based, system modeling library developed at Oak Ridge National Laboratory, is an advanced tool that can help meet some of the near-term needs of the advanced reactor community. Here, this paper describes advanced system modeling criteria and presents TRANSFORM to the advanced reactor community by demonstration of system modeling capabilities and support of advanced analysis workflows, i.e., the Risk Analysis Virtual Environment (RAVEN) framework from Idaho National Laboratory, using the liquid-fueled Molten Salt Demonstration Reactor (MSDR) as a reference design.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗

Finite Element Analysis System Workflow Tools

A collection of MATLAB functions and class definitions called System Workflow Tools (SWFT) are available to semi-automate steps in the simulation process. Some of these steps are often simple and routine for smaller finite element models, but if done directly by an analyst can quickly become labor intensive, cumbersome, and error prone for larger, system level models. Some of SWFT’s capabilities demonstrated in this report includes writing Sierra input decks and processing Quantities of Interest (QOI) from results files. SWFT also writes scripts in order to utilize other software programs such as Cubit (separating system level CAD into subassemblies and components, creating nodesets and sidesets), DAKOTA (ensemble management), and ParaView (contour plots and animations). Detailed commands and workflows from mesh generation to report generation are provided as examples for analysts to utilize SWFT capabilities.

97 MATHEMATICS AND COMPUTING↗

Visualization and Analysis for Near-Real-Time Decision Making in Distributed Workflows

Data driven science is becoming increasingly more common, complex, and is placing tremendous stresses on visualization and analysis frameworks. Data sources producing 10GB per second (and more) are becoming increasingly commonplace in both simulation, sensor and experimental sciences. These data sources, which are often distributed around the world, must be analyzed by teams of scientists that are also distributed. Enabling scientists to view, query and interact with such large volumes of data in near-real-time requires a rich fusion of visualization and analysis techniques, middleware and workflow systems. Here, this paper discusses initial research into visualization and analysis of distributed data workflows that enables scientists to make near-real-time decisions of large volumes of time varying data.

97 MATHEMATICS AND COMPUTING↗

Echo: Data and Analysis Management [Slides]

Echo software capabilities are described. Echo provides a comprehensive suite of tools for data wrangling, management, and analysis. It is an object-oriented approach to data analysis designed for robust and scalable execution of complex analysis workflows written in MATLAB.

97 MATHEMATICS AND COMPUTING↗

Integrated Zeff analysis on the DIII-D Tokamak

Determination of effective ion charge, or Zeff, from measurements of visible brems\-strahlung radiation made by multiple diagnostics is now integrated into a single analysis workflow within the plasma profile analysis module in OMFIT [Logan et al. Fus. Sci. Technol. 74 (2018) 125]. Measurements are made across the midplane of DIII-D with independent filterscope and spectrometer based diagnostics. Emission from 5215–5245 Å is measured by 16 filtered visible bremsstrahlung measurements (filterscopes) with a sampling rate of 20 kHz. The charge exchange recombination spectroscopy (CER) system utilizes scanning spectrometers, and typically measures emission near 5291 Å with approximately 20 active spatial views. Both Abel inversion and least squares minimization analyses are performed on these chordal measurements to infer the plasma emissivity profile with respect to magnetic flux coordinate ρ. Zeff is derived from this emissivity profile from a calculation involving electron density and temperature from the Thomson scattering diagnostic. This analysis has been applied for a range of plasma conditions including L-mode, H- mode and H-mode with impurity seeding. The Zeff inferred from VB data is shown to be in good agreement with the Zeff computed from CER impurity density profile measurements when C6+ is assumed to be the dominant impurity species of the plasma. This new analysis is useful for cross checking diagnostics and measuring Zeff when all important impurities cannot be monitored directly with CER measurements.

Callahan, K. J.↗