Search NASA⌕ Search

SEARCH · Search NASA

Results for “computational workflow”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 613 records · Page 34

A computational framework for automation of point defect calculations

We have developed a complete and rigorously validated open-source Python framework to automate point defect calculations using density functional theory. Furthermore, the framework provides an effective and efficient method for defect structure generation, and creation of simple yet customizable workflows to analyze defect calculations. This package provides the capability to compute widely-accepted correction schemes to overcome finite-size effects, including (1) potential alignment, (2) image-charge correction, and (3) band filling correction to shallow defects. Using Si, ZnO and In2O3 as test examples, we demonstrate the package capabilities and validate the methodology.

97 MATHEMATICS AND COMPUTING↗

Towards FAIR Workflows for Federated Experimental Sciences

A de-centralized, peer-to-peer AI metadata framework is demonstrated which can enable end-to-end metadata & lineage tracking for distributed Machine Learning pipelines spanning edge, High Performance Computing, and cloud environments. With a specific example of end-to-end microscopy algorithm and datasets, the proposed method shows how to enable reproducibility, audit trail, provenance of metadata artifacts. The emerging needs of automation in experimental sciences, ML-centric workflows, and FAIR metadata management across federated compute environments is addressed.

machine learning↗

Collaborative Exploration of Scientific Datasets Using Immersive and Statistical Visualization

We discuss the value of collaborative, immersive visualization for the exploration of scientific datasets and review techniques and tools that have been developed and deployed at the National Renewable Energy Laboratory (NREL). We believe that collaborative visualizations linking statistical interfaces and graphics on laptops and high-performance computing (HPC) with 3D visualizations on immersive displays (head-mounted displays and large-scale immersive environments) enable scientific workflows that further rapid exploration of large, high-dimensional datasets by teams of analysts. We present a framework, PlottyVR, that blends statistical tools, general-purpose programming environments, and simulation with 3D visualizations. To contextualize this framework, we propose a categorization and loose taxonomy of collaborative visualization and analysis techniques. Finally, we describe how scientists and engineers have adopted this framework to investigate large, complex datasets.

collaborative visualization↗

Performance characterization of scientific workflows for the optimal use of Burst Buffers

Scientific discoveries are significantly rely on the analysis of large volumes of data from observations and simulations of complex phenomena. Scientists compose the complex analyses as workflows and execute them on large-scale HPC systems. The workflow structures are in contrast with monolithic single simulations that have often been the primary use case on HPC systems. At the same time, new storage paradigms such as Burst Buffers are becoming available on HPC platforms. In this paper, we analyze the performance characteristics of a Burst Buffer and two representative scientific workflows with the aim of optimizing the usage of a Burst Buffer, extending our previous analyses (Daley et al., 2016). Our key contributions are (a) developing a performance analysis methodology pertinent to Burst Buffers, (b) improving the use of a Burst Buffer in workflows with bandwidth-sensitive and metadata-sensitive I/O workloads, (c) highlighting the key data management challenges when incorporating a Burst Buffer in the studied scientific workflows.

97 MATHEMATICS AND COMPUTING↗

Hybrid Analysis of Fusion Data for Online Understanding of Complex Science on Extreme Scale Computers

The current practice for fusion scientists running first principle simulations on high performance computing plat-forms is to either run their simulations and output their data for post-hoc analysis, or to place in situ analytics into their code. In this paper we examine a complex workflow using XGC fusions simulation run on the Oak Ridge Leadership Computing Facility's supercomputer Summit, which also involve three anal-yses as part of the results necessary for scientific discovery. We discuss the challenges faced when implementing these algorithms and present an original hybrid staging technique to help enable the physicists to make discoveries during the execution of the simulation. By creating this infrastructure, we can examine complicated physics results, which may not have been possible without the infrastructure. For example, our work enables the online visualization of turbulent homoclinic tangle around the magnetic X-point, breaking the last confinement surface. This visualization could help fusion scientists to better understand and improve the turbulence spread of plasma exhaust heat, which is crucial toward realizing plasmas beyond the currently accessible physics regimes of present-day tokamak reactors. The physics of turbulent homoclinic tangle will be reported in a future physics publication, by utilizing the original online analysis/visualization framework presented in this paper.

Suchyta, Eric↗

The Future Looks Bright for Teraflop Computing

Wouldn’t it be great to have a teraflop of computing power sitting in your lab, desktop workstation, or remote instrument server? Talk about simplifying workflows, eliminating competition for HPC resources, and allowing more scientists and technicians to get more work done! Well, the computer industry is marketing that capability now in the form of high-end video cards – and for a bargain price – with more and better technology on the market horizon. As the industry evolves to become more oriented toward multi-core and multi-threaded hardware; video card manufacturers are attempting to transition from a niche to multi-purpose market. One of the products currently getting attention is the Nvidia Tesla family of products based on the Tesla GPGPU (general purpose graphics processing unit). This card contains 128 processor computing core engines advertised as having the ability to deliver an aggregate 518 billion single-precision floating operations per second (518 Gflop), which is being introduced at a $1499 MSRP price-point. Nvidia also offers other commodity graphics cards, such as the GeForce 8800, which appear on paper to have roughly the same performance for roughly half the price – although with half the memory (768M vs the Tesla 1.5 GB). This highlights how the Tesla GPGPUs are essentially redesigned graphics cards (with no video capability, increased memory, and clock changes) that fit into PCI-Express slots in your motherboard. If you believe Nvidia’s claims, two Tesla cards will - for the right applications - turn your lab workstation into a teraflop capable supercomputer. Double-precision versions are projected for a late 2007 introduction with expected 2008 delivery. The Nvidia Tesla GPGPU is one step forward in the many-core revolution that is happening in the computer industry. Instead of making two or four processing cores available to the user, many-core processors offer tens or hundreds of processing cores. Many-core processors promise to provide very high performance-per-dollar and performance-per-watt for many computational workloads. Intel is working on their version of many-core processors but delivery dates appear to be several years in the future. Last year Intel made a large splash with their proof-of-concept teraflop 80-core chip, which they announced might be available sometime in 2011. Intel is also working on something similar to the Nvidia Tesla – codename Larrabee – which will perform in the teraflop range and has a release date of sometime around 2009 or 2010. Larrabee is supposed to have 16 – 24 cores and several nice features. Bottom line: A teraflop lab computer is feasible today as the programmable Nvidia GeForce 8 and Quadro family of graphics cards are available now, Tesla cards will be shipping, and exciting many-core architectures are on the horizon from a number of vendors. Definitely, the potential for parallel processing systems is huge, and GPGPUs certainly provide parallel processing, but are there enough applications out there to take them mainstream and make it more appealing to businesses other than just research firms? Only time will tell as more applications are developed to utilize this computational capability. Right now, programming is required. Recently Google purchased PeakStream, a firm that engaged in abstracting the task of running multiple threads to software with specific GPGPU applicability. However, Google is a visionary software company. Instrument vendors and much of the software industry are still in the early stages of the transition to multi-threaded many-core data processing. Applications that exploit the full potential of parallel processing systems, and GPGPUs in particular, really don’t exist in today’s market. The development of Matlab plug-ins is a very positive sign for the future of GPGPUs and is indicative of Nvidia’s sense of where the market is headed.

99 GENERAL AND MISCELLANEOUS↗

Quantum/AI Topology-Aware Latency-Adaptive HPC Workflow Scheduling Optimization

The growing demand for more powerful high-performance computing (HPC) systems has led to a steady rise in energy consumption by supercomputing worldwide. This study is focused on comparing our Application-Topology Mapper (ATMapper) to the popular Simple Linux Utility for Resource Management (SLURM) for the purpose of exploring methods that can further optimize job-scheduling within HPC systems. ATMapper is an Artificial-Intelligence based approach to job-scheduling that is currently being enhanced with quantum annealing (QA) to generate optimal schedules faster. We are applying QA to speedup our ATMapper process to achieve higher computing efficiency, thereby reducing HPC energy consumption. Here, we examine how four job-scheduling approaches perform in processor node assignment when using an example network architecture of 4 interconnected nodes. Using a specialized script, we are assessing the schedule of a computation flow with 11 interdependent tasks. The data movements among nodes were tracked to count for the number of interactions (network hops) between nodes needed to complete the tasks. The total number of hops and the job completion time were then used to quantify the efficiency of the different mapping approaches. In addition to SLURM, we also compare our ATMapper to the QA-enabled LBNL TIGER and the D-Wave Distributed Computing processor assignment approaches. The preliminary results showed that our topology-aware, latency-adaptive ATMapper is significantly more efficient when compared to the other scheduling approaches due to its load-imbalance network allocation. The scheduler displayed a computing efficiency of 53% by performing significantly fewer network hops than its alternatives. By reducing the number of hops, ATMapper was able to perform all 11 tasks by using only 3 nodes out of given 4. This research indicates the potential to use QA/AI for HPC job-scheduling. Later, we will test a SLURM simulator program to draw further comparisons on the effectiveness of ATMapper's scheduling approach. The results of this comparison will serve as a baseline for later improving SLURM's performance using a QA-enhanced ATMapper approach.

Caraveo, Braulio [University of Huston - Clear Lak↗

EMSL-Computing/CoreMS-Portal

Software that orchestrate data processing and management for mass spectrometry workflows and data products. Stack presented contains a web application, data processing job scheduling, and data processing workers for data processing and management.

Corilo, Yuri↗

Uncertainty Quantification of Metal Additive Manufacturing Processing Conditions Through the use of Exascale Computing

Metal additive manufacturing (AM) is a disruptive manufacturing technology that opens the design space for parts outside those possible from traditional manufacturing methods. In order to accelerate industry and R&D needs to certify AM parts, the Exascale Additive Manufacturing project (ExaAM) has developed a suite of exascale-ready computational tools to model the process-to-structure-to-properties (PSP) relationship for additively manufactured metal components. One such tool is an uncertainty quantification (UQ) pipeline to quantify the effect that uncertainty in processing conditions has on local mechanical responses. We present an overview of this pipeline and its required simulation and workflow codes. Using the Oak Ridge National Laboratory’s (ORNL) exascale computer, Frontier, we utilize this pipeline to cross multiple length and time scales to predict the local mechanical response of a location within a complex AM bridge part, AMB2018-01 produced by the National Institute of Standards and Technology (NIST) as part of their 2018 AM-Bench test series. Our results are then compared to experimental mechanical tests of parts from the NIST build to quantify the error in the ExaAM UQ workflow.

Carson, Robert↗

Building a Consistent and Reproducible Database for Adsorption Evaluation in Covalent–Organic Frameworks

We present a workflow that traces the path from the bulk structure of a crystalline material to assessing its performance in carbon capture from coal's postcombustion flue gases. This workflow is applied to a database of 324 covalent-organic frameworks (COFs) reported in the literature, to characterize their CO 2 adsorption properties using the following steps: (1) optimization of the crystal structure (atomic positions and unit cell) using density functional theory, (2) fitting atomic point charges based on the electron density, (3) characterizing the pore geometry of the structures before and after optimization, (4) computing carbon dioxide and nitrogen isotherms using grand canonical Monte Carlo simulations with an empirical interaction potential, and finally, (5) assessing the CO 2 parasitic energy via process modeling. The full workflow has been encoded in the Automated Interactive Infrastructure and Database for Computational Science (AiiDA). Both the workflow and the automatically generated provenance graph of our calculations are made available on the Materials Cloud, allowing peers to inspect every input parameter and result along the workflow, download structures and files at intermediate stages, and start their research right from where this work has left off. In particular, our set of CURATED (Clean, Uniform, and Refined with Automatic Tracking from Experimental Database) COFs, having optimized geometry and high-quality DFT-derived point charges, are available for further investigations of gas adsorption properties. We plan to update the database as new COFs are being reported.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Finding Electronic Structure Machine Learning Surrogates without Training

A myriad of phenomena in materials science and chemistry rely on quantum-level simulations of the electronic structure in matter. While moving to larger length and time scales has been a pressing issue for decades, such large-scale electronic structure calculations are still challenging despite modern software approaches and advances in high-performance computing. The silver lining in this regard is the use of machine learning to accelerate electronic structure calculations – this line of research has recently gained growing attention. The grand challenge therein is finding a suitable machine-learning model during a process called hyperparameter optimization. This, however, causes a massive computational overhead in addition to that of data generation. We accelerate the construction of machine-learning surrogate models by roughly two orders of magnitude by circumventing excessive training during the hyperparameter optimization phase. We demonstrate our workflow for Kohn-Sham density functional theory, the most popular computational method in materials science and chemistry.

36 MATERIALS SCIENCE↗

Virtual Engineering: Python framework for engineering process design

Virtual Engineering (VE) is a Python software framework designed to accelerate the research and development of engineering processes that are fundamentally defined by multiple unit operations executed in series. VE supports a wide variety of different multi-physics models and integrates them to simulate a complete end-to-end process. To automate the execution of this model sequence, VE provides (i) a robust method to communicate between models, (ii) a high-level, user-friendly interface to set model parameters and enable optimization, and (iii) an overall model-agnostic approach that allows new computational units to be swapped in and out of workflows. Although the VE framework was developed to support the biochemical conversion of biomass to fuel, we have designed each component to easily accommodate new domains and unit models.

09 BIOMASS FUELS↗

Integration Of PanDA Workload Management System With Supercomputers for ATLAS and Data Intensive Science

The.LHC, operating at CERN, is leading Big Data driven scientific explorations. Experiments at the LHC explore the fundamental nature of matter and the basic forces that shape our universe. ATLAS, one of the largest collaborations ever assembled in the sciences, is at the forefront of research at the LHC. To address an unprecedented multi-petabyte data processing challenge, the ATLAS experiment is relying on a heterogeneous distributed computational infrastructure. The ATLAS experiment uses PanDA (Production and Data Analysis) Workload Management System for managing the workflow for all data processing on over 150 data centers. Through PanDA, ATLAS physicists see a single computing facility that enables rapid scientific breakthroughs for the experiment, even though the data centers are physically scattered all over the world. While PanDA currently uses more than 250,000 cores with a peak performance of 0.3 petaFLOPS, LHC data taking runs require more resources than grid can possibly provide. To alleviate these challenges, LHC experiments are engaged in an ambitious program to expand the current computing model to include additional resources such as the opportunistic use of supercomputers. We will describe a project aimed at integration of PanDA WMS with supercomputers in United States, in particular with Titan supercomputer at Oak Ridge Leadership Computing Facility. Current approach utilizes modified PanDA pilot framework for job submission to the supercomputers batch queues and local data management, with light-weight MPI wrappers to run single threaded workloads in parallel on LCFs multi-core worker nodes. This implementation was tested with a variety of Monte-Carlo workloads on several supercomputing platforms for ALICE and ATLAS experiments and it is in full pro duction for the ATLAS since September 2015. We will present our current accomplishments with running PanDA at supercomputers and demonstrate our ability to use PanDA as a portal independent of the computing facilities infrastructure for High Energy and Nuclear Physics as well as other data-intensive science applications, such as bioinformatics and astro-particle physics.

97 MATHEMATICS AND COMPUTING↗

INTEGRATION OF PANDA WORKLOAD MANAGEMENT SYSTEM WITH SUPERCOMPUTERS

Abstract The Large Hadron Collider (LHC), operating at the international CERN Laboratory in Geneva, Switzerland, is leading Big Data driven scientific explorations. Experiments at the LHC explore the funda- mental nature of matter and the basic forces that shape our universe, and were recently credited for the dis- covery of a Higgs boson. ATLAS, one of the largest collaborations ever assembled in the sciences, is at the forefront of research at the LHC. To address an unprecedented multi-petabyte data processing challenge, the ATLAS experiment is relying on a heterogeneous distributed computational infrastructure. The ATLAS experiment uses PanDA (Production and Data Analysis) Workload Management System for managing the workflow for all data processing on over 140 data centers. Through PanDA, ATLAS physicists see a single computing facility that enables rapid scientific breakthroughs for the experiment, even though the data cen- ters are physically scattered all over the world. While PanDA currently uses more than 250000 cores with a peak performance of 0.3+ petaFLOPS, next LHC data taking runs will require more resources than Grid computing can possibly provide. To alleviate these challenges, LHC experiments are engaged in an ambitious program to expand the current computing model to include additional resources such as the opportunistic use of supercomputers. We will describe a project aimed at integration of PanDA WMS with supercomputers in United States, Europe and Russia (in particular with Titan supercomputer at Oak Ridge Leadership Com- puting Facility (OLCF), Supercomputer at the National Research Center Kurchatov Institute , IT4 in Ostrava, and others). The current approach utilizes a modified PanDA pilot framework for job submission to the supercomputers batch queues and local data management, with light-weight MPI wrappers to run single- threaded workloads in parallel on Titan s multi-core worker nodes. This implementation was tested with a variety of Monte-Carlo workloads on several supercomputing platforms. We will present our current accom- plishments in running PanDA WMS at supercomputers and demonstrate our ability to use PanDA as a portal independent of the computing facility s infrastructure for High Energy and Nuclear Physics, as well as other data-intensive science applications, such as bioinformatics and astro-particle physics.

De, K↗

Integration Of PanDA Workload Management System With Supercomputers for ATLAS and Data Intensive Science

The Large Hadron Collider (LHC), operating at the international CERN Laboratory in Geneva, Switzerland, is leading Big Data driven scientific explorations. Experiments at the LHC explore the fundamental nature of matter and the basic forces that shape our universe, and were recently credited for the discovery of a Higgs boson. ATLAS, one of the largest collaborations ever assembled in the sciences, is at the forefront of research at the LHC. To address an unprecedented multi-petabyte data processing challenge, the ATLAS experiment is relying on a heterogeneous distributed computational infrastructure. The ATLAS experiment uses PanDA (Production and Data Analysis) Workload Management System for managing the workflow for all data processing on over 150 data centers. Through PanDA, ATLAS physicists see a single computing facility that enables rapid scientific breakthroughs for the experiment, even though the data centers are physically scattered all over the world. While PanDA currently uses more than 250,000 cores with a peak performance of 0.3 petaFLOPS, LHC data taking runs require more resources than Grid computing can possibly provide. To alleviate these challenges, LHC experiments are engaged in an ambitious program to expand the current computing model to include additional resources such as the opportunistic use of supercomputers. We will describe a project aimed at integration of PanDA WMS with supercomputers in United States, Europe and Russia (in particular with Titan supercomputer at Oak Ridge Leadership Computing Facility (OLCF), MIRA supercomputer at Argonne Leadership Computing Facilities (ALCF), Supercomputer at the National Research Center Kurchatov Institute , IT4 in Ostrava and others). Current approach utilizes modified PanDA pilot framework for job submission to the supercomputers batch queues and local data management, with light-weight MPI wrappers to run single threaded workloads in parallel on LCFs multi-core worker nodes. This implementation was tested with a variety of Monte-Carlo workloads on several supercomputing platforms for ALICE and ATLAS experiments and it is in full production for the ATLAS experiment since September 2015. We will present our current accomplishments with running PanDA WMS at supercomputers and demonstrate our ability to use PanDA as a portal independent of the computing facilities infrastructure for High Energy and Nuclear Physics as well as other data-intensive science applications, such as bioinformatics and astro-particle physics.

Data-intensive science↗

MaDaTS: Managing Data on Tiered Storage for Scientific Workflows

Scientific workflows are processing large amounts of data through complex simulation and analysis tasks. Meanwhile, the need to minimize I/O costs on next generation systems and the evolution of new technologies (NVRAMs, SSDs etc.) is resulting in deeper storage hierarchies on High Performance Computing (HPC) systems. A multi-tiered storage hierarchy introduces complexities in workflow and data management. There is need for simple and flexible data abstractions that can allow users to seamlessly manage workflow data and tasks on HPC systems with multiple storage tiers. MaDaTS (Managing Data on Tiered Storage for Scientific Workflows) provides an API and a command-line tool that allows users to manage their workflows and data on tiered storage (Ghoshal & Ramakrishnan (2017)).

97 MATHEMATICS AND COMPUTING↗