Search NASA⌕ Search

SEARCH · Search NASA

Results for “analysis workflow”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 919 records · Page 51

CMS distributed computing workflow experience

The vast majority of the CMS Computing capacity, which is organized in a tiered hierarchy, is located away from CERN. The 7 Tier-1 sites archive the LHC proton-proton collision data that is initially processed at CERN. These sites provide access to all recorded and simulated data for the Tier-2 sites, via wide-area network (WAN) transfers. All central data processing workflows are executed at the Tier-1 level, which contain re-reconstruction and skimming workflows of collision data as well as reprocessing of simulated data to adapt to changing detector conditions. This paper describes the operation of the CMS processing infrastructure at the Tier-1 level. The Tier-1 workflows are described in detail. The operational optimization of resource usage is described. In particular, the variation of different workflows during the data taking period of 2010, their efficiencies and latencies as well as their impact on the delivery of physics results is discussed and lessons are drawn from this experience. The simulation of proton-proton collisions for the CMS experiment is primarily carried out at the second tier of the CMS computing infrastructure. Half of the Tier-2 sites of CMS are reserved for central Monte Carlo (MC) production while the other half is available for user analysis. This paper summarizes the large throughput of the MC production operation during the data taking period of 2010 and discusses the latencies and efficiencies of the various types of MC production workflows. We present the operational procedures to optimize the usage of available resources and we the operational model of CMS for including opportunistic resources, such as the larger Tier-3 sites, into the central production operation.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

De-risking fault leakage risk and containment integrity for subsurface storage applications

The subsurface is pivotal in the energy transition, for the sequestration of CO 2 and energy storage. It is crucial to understand to what extent geological faults may form leakage pathways that threaten the containment integrity of these projects. Fault flow behavior has been studied in the context of hydrocarbon development, supported by observations from wells drilled through faults, but such observations are rare in geoenergy projects. Focusing on mechanical behavior as early indicator of potential leakage risks, a probabilistic Coulomb Failure Stress workflow is developed and demonstrated using data from the Decatur CO 2 sequestration project to rank faults based on their containment risk. The analysis emphasizes the importance of fault throw relative to reservoir thickness and pore pressure change in assessing reactivation risks. Integrating this mechanical assessment with geological and dynamic fault analyses contributes to derisking fault containment for geoenergy applications, providing valuable insights for the successful development of subsurface storage projects.

58 GEOSCIENCES↗

Predicting chemically-induced skin reactions. Part II: QSAR models of skin permeability and the relationships between skin permeability and skin sensitization

Skin permeability is widely considered to be mechanistically implicated in chemically-induced skin sensitization. Although many chemicals have been identified as skin sensitizers, there have been very few reports analyzing the relationships between molecular structure and skin permeability of sensitizers and non-sensitizers. The goals of this study were to: (i) compile, curate, and integrate the largest publicly available dataset of chemicals studied for their skin permeability; (ii) develop and rigorously validate QSAR models to predict skin permeability; and (iii) explore the complex relationships between skin sensitization and skin permeability. Based on the largest publicly available dataset compiled in this study, we found no overall correlation between skin permeability and skin sensitization. In addition, cross-species correlation coefficient between human and rodent permeability data was found to be as low as R{sup 2} = 0.44. Human skin permeability models based on the random forest method have been developed and validated using OECD-compliant QSAR modeling workflow. Their external accuracy was high (Q{sup 2}{sub ext} = 0.73 for 63% of external compounds inside the applicability domain). The extended analysis using both experimentally-measured and QSAR-imputed data still confirmed the absence of any overall concordance between skin permeability and skin sensitization. This observation suggests that chemical modifications that affect skin permeability should not be presumed a priori to modulate the sensitization potential of chemicals. The models reported herein as well as those developed in the companion paper on skin sensitization suggest that it may be possible to rationally design compounds with the desired high skin permeability but low sensitization potential. - Highlights: • It was compiled the largest publicly-available skin permeability dataset. • Predictive QSAR models were developed for skin permeability. • No concordance between skin sensitization and skin permeability has been found. • Structural rules for optimizing sensitization and penetration were established.

60 APPLIED LIFE SCIENCES↗

Parallel variable selection for effective performance prediction

Large data analysis problems often involve a large number of variables, and the corresponding analysis algorithms may examine all variable combinations to find the optimal solution. For example, to model the time required to complete a scientific workflow, we need to consider the impact of dozens of parameters. To reduce the model building time and reduce the likelihood of overfitting, we look to variable selection methods to identify the critical variables for the performance model. In this work, we create a combination of variable selection and performance prediction methods that is as effective as the exhaustive search approach when the exhaustive search could be completed in a reasonable amount of time. To handle the cases where the exhaustive search is too time consuming, we develop the parallelized variable selection algorithm. Additionally, we develop a parallel grouping mechanism that further reduces the variable selection time by 70%.As a case study, we exercise the variable selection technique with the performance measurement data from the Palomar Transient Factory (PTF) workflow. The application scientists have determined that about 50 variables and parameters are important to the performance of the workflows. Our tests show that the Sequential Backward Selection algorithm is able to approximate the optimal subset relatively quickly. By reducing the number of variables used to build the model from 50 to 4, we are able to maintain the prediction quality while reducing the model building time by a factor of 6. Using the parallelization and grouping techniques we developed in this work, the variable selection process was reduced from over 18 hours to 15 minutes while ending up with the same variable subset.

Wang, J↗

MVP: a modular viromics pipeline to identify, filter, cluster, annotate, and bin viruses from metagenomes

While numerous computational frameworks and workflows are available for recovering prokaryote and eukaryote genomes from metagenome data, only a limited number of pipelines are designed specifically for viromics analysis. With many viromics tools developed in the last few years alone, it can be challenging for scientists with limited bioinformatics experience to easily recover, evaluate quality, annotate genes, dereplicate, assign taxonomy, and calculate relative abundance and coverage of viral genomes using state-of-the-art methods and standards. Here, we describe Modular Viromics Pipeline (MVP) v.1.0, a user-friendly pipeline written in Python and providing a simple framework to perform standard viromics analyses. MVP combines multiple tools to enable viral genome identification, characterization of genome quality, filtering, clustering, taxonomic and functional annotation, genome binning, and comprehensive summaries of results that can be used for downstream ecological analyses. Overall, MVP provides a standardized and reproducible pipeline for both extensive and robust characterization of viruses from large-scale sequencing data including metagenomes, metatranscriptomes, viromes, and isolate genomes. As a typical use case, we show how the entire MVP pipeline can be applied to a set of 20 metagenomes from wetland sediments using only 10 modules executed via command lines, leading to the identification of 11,656 viral contigs and 8,145 viral operational taxonomic units (vOTUs) displaying a clear beta-diversity pattern. Further, acting as a dynamic wrapper, MVP is designed to continuously incorporate updates and integrate new tools, ensuring its ongoing relevance in the rapidly evolving field of viromics. MVP is available at https://gitlab.com/ccoclet/mvp and as versioned packages in PyPi and Conda.

59 BASIC BIOLOGICAL SCIENCES↗

Building Efficiency Targeting Tool for Energy Retrofits (BETTER) Application Programing Interface (API) (BETTER API) v1.0

The BETTER API allows advanced users and developers to access BETTER's analytical functionalities via HTTP requests from their own software platform without using the BETTER web-interface. API users prepare the input data and analysis configurations in a JSON format and send it to the web application via an HTTP request. The web application runs the analytical workflow on the back end and returns an HTTP response in JSON format. The user can then decide to render the results or further process it.

Szum, Carolyn↗

From Mouse to Human: Cellular Morphometric Subtype Learned From Mouse Mammary Tumors Provides Prognostic Value in Human Breast Cancer

Mouse models of cancer provide a powerful tool for investigating all aspects of cancer biology. In this study, we used our recently developed machine learning approach to identify the cellular morphometric biomarkers (CMB) from digital images of hematoxylin and eosin (H&E) micrographs of orthotopic Trp53-null mammary tumors (n = 154) and to discover the corresponding cellular morphometric subtypes (CMS). Of the two CMS identified, CMS-2 was significantly associated with shorter survival (p = 0.0084). We then evaluated the learned CMB and corresponding CMS model in MMTV-Erbb2 transgenic mouse mammary tumors (n = 53) in which CMS-2 was significantly correlated with the presence of metastasis (p = 0.004). We next evaluated the mouse CMB and CMS model on The Cancer Genome Atlas breast cancer (TCGA-BRCA) cohort (n = 1017). Kaplan–Meier analysis showed significantly shorter overall survival (OS) of CMS-2 patients compared to CMS-1 patients (p = 0.024) and added significant prognostic value in multi-variable analysis of clinical and molecular factors, namely, age, pathological stage, and PAM50 molecular subtype. Thus, application of CMS to digital images of routine workflow H&E preparations can provide unbiased biological stratification to inform patient care.

60 APPLIED LIFE SCIENCES↗

World Spatiotemporal Analytics and Mapping Project (WSTAMP): Discovering, Exploring, and Mapping Spatiotemporal Patterns across the World’s Largest Open Source Geographic Data Sets

The application of spatiotemporal (ST) analytics to integrated data from major sources such as the World Bank, United Nations, and dozens of others holds tremendous potential for shedding new light on the evolution of cultural, health, economic, and geopolitical landscapes on a global level. Realizing this potential first requires an ST data model that addresses challenges in properly merging data from multiple authors, with evolving ontological perspectives, semantical differences, and changing attributes, as well as content that is textual, numeric, categorical, and hierarchical. Equally challenging is the development of analytical and visualization approaches that provide a serious exploration of this integrated data while remaining accessible to practitioners with varied backgrounds. The WSTAMP project at Oak Ridge National Laboratory has yielded two major results in addressing these challenges: 1) development of the WSTAMP database, a significant advance in ST data modeling that integrates 10,000+ attributes covering over 200 nation states spanning over 50 years from over 30 major sources and 2) a novel online ST exploratory and analysis tool providing an array of modern statistical and visualization techniques for analyzing these data temporally, spatially, and spatiotemporally under a standard analytic workflow. We discuss the status of this work and report on major findings.Acknowledgment Prepared by Oak Ridge National Laboratory, P.O. Box 2008, Oak Ridge, Tennessee 37831-6285, managed by UT-Battelle, LLC for the U. S. Department of Energy under contract no. DEAC05-00OR22725. Copyright This manuscript has been authored by employees of UT-Battelle, LLC, under contract DE-AC05-00OR22725 with the U.S. Department of Energy. Accordingly, the United States Government retains and the publisher, by accepting the article for publication, acknowledges that the United States Government retains a non-exclusive, paid-up, irrevocable, world-wide license to publish or reproduce the published form of this manuscript, or allow others to do so, for United States Government purposes.

analytics↗

In-Situ Statistical Analysis of Autotune Simulation Data using Graphical Processing Units

Developing accurate building energy simulation models to assist energy efficiency at speed and scale is one of the research goals of the Whole-Building and Community Integration group, which is a part of Building Technologies Research and Integration Center (BTRIC) at Oak Ridge National Laboratory (ORNL). The aim of the Autotune project is to speed up the automated calibration of building energy models to match measured utility or sensor data. The workflow of this project takes input parameters and runs EnergyPlus simulations on Oak Ridge Leadership Computing Facility’s (OLCF) computing resources such as Titan, the world’s second fastest supercomputer. Multiple simulations run in parallel on nodes having 16 processors each and a Graphics Processing Unit (GPU). Each node produces a 5.7 GB output file comprising 256 files from 64 simulations. Four types of output data covering monthly, daily, hourly, and 15- minute time steps for each annual simulation is produced. A total of 270TB+ of data has been produced. In this project, the simulation data is statistically analyzed in-situ using GPUs while annual simulations are being computed on the traditional processors. Titan, with its recent addition of 18,688 Compute Unified Device Architecture (CUDA) capable NVIDIA GPUs, has greatly extended its capability for massively parallel data processing. CUDA is used along with C/MPI to calculate statistical metrics such as sum, mean, variance, and standard deviation leveraging GPU acceleration. The workflow developed in this project produces statistical summaries of the data which reduces by multiple orders of magnitude the time and amount of data that needs to be stored. These statistical capabilities are anticipated to be useful for sensitivity analysis of EnergyPlus simulations

accelerators↗

Tracking Files in the Kepler Provenance Framework

Workflow Management Systems (WFMS), such as Kepler, are proving to be an important tool in scientific problem solving. They can automate and manage complex processes and huge amounts of data produced by petascale simulations. Typically, the produced data need to be properly visualized and analyzed by scientists in order to achieve the desired scientific goals. Both run-time and post analysis may benefit from, even require, additional meta-data - provenance information. One of the challenges in this context is the tracking of the data files that can be produced in very large numbers during stages of the workflow, such as visualizations. The Kepler provenance framework collects all or part of the raw information flowing through the workflow graph. This information then needs to be further parsed to extract meta-data of interest. This can be done through add-on tools and algorithms. We show how to automate tracking specific information such as data files locations.

99 GENERAL AND MISCELLANEOUS↗

Genome Extraction from Shotgun Metagenome Sequence Data

Uncultivated Bacteria and Archaea comprise the vast majority of species on Earth, but obtaining their genomes directly from the environment, using shotgun sequencing, has only recently become possible. To realize the hope of capturing Earth’s microbial genetic complement, technologies that accelerate recovery of high-quality genomes are necessary. We present a series of analysis steps and data products for the extraction of high quality metagenome-assembled genomes (MAGs) from microbiomes using the U.S. Department of Energy Systems Biology Knowledgebase (KBase) platform (http://www.kbase.us/). In KBase, the process is end-to-end, allowing a user to go from the initial sequencing reads all the way through to MAG genomes, which can then be analyzed with other KBase capabilities such as phylogenetic placement, functional assignment, metabolic modeling, pangenome functional profiling, RNA-Seq, and others. While portions of such capabilities are individually available from other resources, the combination of the intuitive usability, data interoperability, and integration of tools in a freely available compute resource makes KBase a uniquely powerful platform for obtaining MAGs from microbiomes. While this workflow offers tools for each of the key steps in the genome extraction process, it also provides a scaffold that can be easily extended, with additional MAG recovery and analysis tools, via the KBase SDK (Software Development Kit).

Chivian, Dylan↗

Moab Desert Crust - Sample 4E

Uncultivated Bacteria and Archaea comprise the vast majority of species on Earth, but obtaining their genomes directly from the environment, using shotgun sequencing, has only recently become possible. To realize the hope of capturing Earth’s microbial genetic complement, technologies that accelerate recovery of high-quality genomes are necessary. We present a series of analysis steps and data products for the extraction of high quality metagenome-assembled genomes (MAGs) from microbiomes using the U.S. Department of Energy Systems Biology Knowledgebase (KBase) platform (http://www.kbase.us/). In KBase, the process is end-to-end, allowing a user to go from the initial sequencing reads all the way through to MAG genomes, which can then be analyzed with other KBase capabilities such as phylogenetic placement, functional assignment, metabolic modeling, pangenome functional profiling, RNA-Seq, and others. While portions of such capabilities are individually available from other resources, the combination of the intuitive usability, data interoperability, and integration of tools in a freely available compute resource makes KBase a uniquely powerful platform for obtaining MAGs from microbiomes. While this workflow offers tools for each of the key steps in the genome extraction process, it also provides a scaffold that can be easily extended, with additional MAG recovery and analysis tools, via the KBase SDK (Software Development Kit).

Chivian, Dylan↗

Preliminary report on applications of machine learning techniques to the Nevada play fairway analysis

We are applying machine learning (ML) techniques, including training set augmentation and artificial neural networks, to mitigate key challenges in the Nevada play fairway project. The study area includes ~85 active geothermal systems as potential training sites and >12 geologic, geophysical, and geochemical features. The main goal is to develop an algorithmic approach to identify new geothermal systems in the Great Basin region. Major objectives include: 1) integrate ML techniques into the geothermal community; 2) develop open community datasets, whereby all play fairway and ML datasets and algorithms are publicly released and available for modification by various user groups; 3) identify data acquisition targets with high value for future work; 4) identify new signatures to detect blind geothermal systems; and 5) foster new capabilities for characterizing subsurface temperature and permeability. Initially, ML techniques are being applied to the same play fairway datasets and workflow. ML will then be applied to both enhanced and additional datasets, with modification of the PFA workflow to incorporate the new datasets. Finally, ML will be applied to define new workflows using the enhanced and additional datasets. An algorithmic approach that empirically learns to estimate weights of influence for diverse parameters can potentially scale and perform better than the play fairway analysis. Initial work on this project has involved 1) evaluating potential positive and negative training sites, 2) transformation of datasets into formats suitable for ML, and 3) initial development and testing of ML techniques.

58 GEOSCIENCES↗

Experimental Analysis of File Transfer Rates over Wide-Area Dedicated Connections

File transfers over dedicated connections, supported by large parallel file systems, have become increasingly important in high-performance computing and big data workflows. It remains a challenge to achieve peak rates for such transfers due to the complexities of file I/O, host, and network transport subsystems, and equally importantly, their interactions. We present extensive measurements of disk-to-disk file transfers using Lustre and XFS file systems mounted on multi-core servers over a suite of 10 Gbps emulated connections with 0-366 ms round trip times. Our results indicate that large buffer sizes and many parallel flows do not always guarantee high transfer rates. Furthermore, large variations in the measured rates necessitate repeated measurements to ensure confidence in inferences based on them. We propose a new method to efficiently identify the optimal joint file I/O and network transport parameters using a small number of measurements. We show that for XFS and Lustre with direct I/O, this method identifies configurations achieving 97% of the peak transfer rate while probing only 12% of the parameter space.

Rao, Nageswara S.↗

Land Model Testbed: Accelerating Development, Benchmarking and Analysis of Land Surface Models

A Land Model Testbed (LMT), designed to provide a computational framework for systematically assessing model fidelity and supporting rapid development of complex multiscale models, offers a general-purpose workflow for conducting large ensemble simulations of multiple land surface models, post-processing large volumes of model output, and evaluating model results. It leverages existing tools for launching model simulations and the International Land Model Benchmarking (ILAMB) package for assessing model fidelity through comparison with best-available observational datasets. Increased complexity and proliferation of uncertain parameters in process representations in land surface models has driven the need for frequent and intensive testing and evaluating of models to quantify uncertainties and optimize parameters such that results are consistent with observations. The LMT described here meets these needs by providing tools to run thousands of ensemble simulations simultaneously and post-process their output files, by automating execution of an enhanced version of ILAMB with site-specific benchmarks and multivariate functional relationships, and by offering ensemble diagnostics and a customizable dashboard for displaying model performance metrics and associated graphics. We envision the LMT capabilities will serve as a foundational computational resource for a proposed user facility focused on terrestrial multiscale model--data integration.

Sreepathi, Sarat↗

A Hybrid Energy System Workflow for Energy Portfolio Optimization

This manuscript develops a workflow, driven by data analytics algorithms, to support the optimization of the economic performance of an Integrated Energy System. The goal is to determine the optimum mix of capacities from a set of different energy producers (e.g., nuclear, gas, wind and solar). A stochastic-based optimizer is employed, based on Gaussian Process Modeling, which requires numerous samples for its training. Each sample represents a time series describing the demand, load, or other operational and economic profiles for various types of energy producers. These samples are synthetically generated using a reduced order modeling algorithm that reads a limited set of historical data, such as demand and load data from past years. Numerous data analysis methods are employed to construct the reduced order models, including, for example, the Auto Regressive Moving Average, Fourier series decomposition, and the peak detection algorithm. All these algorithms are designed to detrend the data and extract features that can be employed to generate synthetic time histories that preserve the statistical properties of the original limited historical data. The optimization cost function is based on an economic model that assesses the effective cost of energy based on two figures of merit: the specific cash flow stream for each energy producer and the total Net Present Value. An initial guess for the optimal capacities is obtained using the screening curve method. The results of the Gaussian Process model-based optimization are assessed using an exhaustive Monte Carlo search, with the results indicating reasonable optimization results. The workflow has been implemented inside the Idaho National Laboratory’s Risk Analysis and Virtual Environment (RAVEN) framework. The main contribution of this study addresses several challenges in the current optimization methods of the energy portfolios in IES: First, the feasibility of generating the synthetic time series of the periodic peak data; Second, the computational burden of the conventional stochastic optimization of the energy portfolio, associated with the need for repeated executions of system models; Third, the inadequacies of previous studies in terms of the comparisons of the impact of the economic parameters. The proposed workflow can provide a scientifically defendable strategy to support decision-making in the electricity market and to help energy distributors develop a better understanding of the performance of integrated energy systems.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗

Benchtop-compatible sample processing workflow for proteome profiling of < 100 mammalian cells

Extending proteomics to smaller samples can enable the mapping of protein expression across tissues with high spatial resolution and can reveal sub-group heterogeneity. However, despite the continually improving sensitivity of LC-MS instrumentation, in-depth profiling of samples containing low-nanogram amounts of protein has remained challenging due to analyte losses incurred during preparation and analysis. To address this, we recently developed nanodroplet processing in one pot for trace samples (nanoPOTS), a robotic/microfluidic platform that generates ready-to-analyze peptides from cellular material in ~200 nL droplets with greatly reduced sample losses. In combination with ultrasensitive LC-MS, nanoPOTS has enabled >3000 proteins to be confidently identified from as few as 10 cultured human cells and ~700 proteins from single cells. However, the nanoPOTS platform requires a highly skilled operator and a costly in-house-built robotic nanopipetting instrument. In this work, we sought to evaluate the extent to which the benefits of nanodroplet processing could be preserved when upscaling reagent dispensing volumes by a factor of 10 to those addressable by commercial micropipette. We characterized the resulting platform, termed microdroplet processing in one pot for trace samples (µPOTS), for the analysis of as few as ~25 cultured HeLa cells (4 ng total protein) or 50 µm square mouse liver tissue thin sections and found that ~1800 and ~1200 unique proteins were respectively identified with high reproducibility. As such, the reduced equipment requirements should facilitate broad dissemination of nanoproteomics workflows by obviating the need for a capital-intensive custom liquid handling system.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Open Reproducible Electron Microscopy Data Analysis

Electron microscopy (EM) is a cornerstone technique in the materials and biological sciences capable of imaging structures at nano- to atomic-scale resolution. Advances in technologies mean that one acquires datasets at increasing data rates and sizes. These advancements present enormous opportunities for researchers to understand complex systems. However, processing the resulting large-scale, complex data in a reproducible and shareable way is a real challenge for researchers. The building, managing, and maintaining complex workflows in a reproducible manner requires extensive knowledge in several areas outside the researchers’ core skill sets, such as software engineering, data science, and high-performance computing (HPC). Our work demonstrates an innovative approach to solving these problems, enabling reproducible EM data analysis through container encapsulated pipelines. Using modern container technologies, we encapsulate processing elements and connect them using shared memory. We expose user-friendly, advanced algorithms and tools to allow end users to utilize without expert programming skills. The platform enables reproducible, scalable, shareable pipelines for the analysis and visualization of EM data. Focusing on interoperability, we leverage the DOE and other agencies’ existing investments to provide a powerful software platform for EM data analysis.

Harris, Christopher↗