Search NASA⌕ Search

SEARCH · Search NASA

Results for “scripting”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 649 records · Page 36

Data processing and analysis with the autoPROC toolbox

Typical topics and problems encountered during data processing of diffraction experiments are discussed and the tools provided in the autoPROC software are described. A typical diffraction experiment will generate many images and data sets from different crystals in a very short time. This creates a challenge for the high-throughput operation of modern synchrotron beamlines as well as for the subsequent data processing. Novice users in particular may feel overwhelmed by the tables, plots and numbers that the different data-processing programs and software packages present to them. Here, some of the more common problems that a user has to deal with when processing a set of images that will finally make up a processed data set are shown, concentrating on difficulties that may often show up during the first steps along the path of turning the experiment (i.e. data collection) into a model (i.e. interpreted electron density). Difficulties such as unexpected crystal forms, issues in crystal handling and suboptimal choices of data-collection strategies can often be dealt with, or at least diagnosed, by analysing specific data characteristics during processing. In the end, one wants to distinguish problems over which one has no immediate control once the experiment is finished from problems that can be remedied a posteriori. A new software package, autoPROC, is also presented that combines third-party processing programs with new tools and an automated workflow script that is intended to provide users with both guidance and insight into the offline processing of data affected by the difficulties mentioned above, with particular emphasis on the automated treatment of multi-sweep data sets collected on multi-axis goniostats.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗

High-throughput full-automatic synchrotron-based tomographic microscopy

At the TOMCAT (TOmographic Microscopy and Coherent rAdiology experimenTs) beamline of the Swiss Light Source with an energy range of 8-45 keV and voxel size from 0.37 {micro}m to 7.4 {micro}m, full tomographic datasets are typically acquired in 5 to 10 min. To exploit the speed of the system and enable high-throughput studies to be performed in a fully automatic manner, a package of automation tools has been developed. The samples are automatically exchanged, aligned, moved to the correct region of interest, and scanned. This task is accomplished through the coordination of Python scripts, a robot-based sample-exchange system, sample positioning motors and a CCD camera. The tools are suited for any samples that can be mounted on a standard SEM stub, and require no specific environmental conditions. Up to 60 samples can be analyzed at a time without user intervention. The throughput of the system is dependent on resolution, energy and sample size, but rates of four samples per hour have been achieved with 0.74 {micro}m voxel size at 17.5 keV. The maximum intervention-free scanning time is theoretically unlimited, and in practice experiments have been running unattended as long as 53 h (the average beam time allocation at TOMCAT is 48 h per user). The system is the first fully automated high-throughput tomography station: mounting samples, finding regions of interest, scanning and reconstructing can be performed without user intervention. The system also includes many features which accelerate and simplify the process of tomographic microscopy.

43 PARTICLE ACCELERATORS↗

The accurate assessment of small-angle X-ray scattering data

Small-angle X-ray scattering (SAXS) has grown in popularity in recent times with the advent of bright synchrotron X-ray sources, powerful computational resources and algorithms enabling the calculation of increasingly complex models. However, the lack of standardized data-quality metrics presents difficulties for the growing user community in accurately assessing the quality of experimental SAXS data. Here, a series of metrics to quantitatively describe SAXS data in an objective manner using statistical evaluations are defined. These metrics are applied to identify the effects of radiation damage, concentration dependence and interparticle interactions on SAXS data from a set of 27 previously described targets for which high-resolution structures have been determined via X-ray crystallography or nuclear magnetic resonance (NMR) spectroscopy. Studies show that these metrics are sufficient to characterize SAXS data quality on a small sample set with statistical rigor and sensitivity similar to or better than manual analysis. The development of data-quality analysis strategies such as these initial efforts is needed to enable the accurate and unbiased assessment of SAXS data quality.

36 MATERIALS SCIENCE↗

GIXSGUI : a MATLAB toolbox for grazing-incidence X-ray scattering data visualization and reduction, and indexing of buried three-dimensional periodic nanostructured films

GIXSGUI is a MATLAB toolbox that offers both a graphical user interface and script-based access to visualize and process grazing-incidence X-ray scattering data from nanostructures on surfaces and in thin films. It provides routine surface scattering data reduction methods such as geometric correction, one-dimensional intensity linecut, two-dimensional intensity reshaping etc . Three-dimensional indexing is also implemented to determine the space group and lattice parameters of buried organized nanoscopic structures in supported thin films.

Jiang, Zhang↗

Interactive visualization of multi-data-set Rietveld analyses using Cinema:Debye-Scherrer [nteractive Visualization of Multi-Dataset Rietveld Analyses using Cinema:Debye-Scherrer]

A tool named Cinema:Debye-Scherrer to visualize the results of a series of Rietveld analyses is presented. The multi-axis visualization of the high-dimensional data sets resulting from powder diffraction analyses allows identification of analysis problems, prediction of suitable starting values, identification of gaps in the experimental parameter space and acceleration of scientific insight from the experimental data. The tool is demonstrated with analysis results from 59 U–Nb alloy samples with different compositions, annealing times and annealing temperatures as well as with a high-temperature study of the crystal structure of CsPbBr 3 . A script to extract parameters from a series of Rietveld analyses employing the widely used GSAS Rietveld software is also described. Finally, both software tools are available for download.

36 MATERIALS SCIENCE↗

Visualization of texture components using MTEX

Knowledge of the appearance of texture components and fibres in pole figures, in inverse pole figures and in Euler space is fundamental for texture analysis. For cubic crystal systems, such as steels, an extensive literature exists and, for example, the book by Matthies, Vinel & Helming [Standard Distributions in Texture Analysis: Maps for the Case of Cubic Orthorhomic Symmetry, (1987), Akademie-Verlag Berlin] provides an atlas to identify texture components. For lower crystal symmetries, however, equivalent comprehensive overviews that can serve as guidance for the interpretation of experimental textures do not exist. Here, this paper closes this gap by providing a set of scripts for the MTEX package [Bachmann, Hielscher & Schaeben (2010). Solid State Phenom.160, 63–68] that allow the texture practitioner to compile such an atlas for a given material system, thus aiding orientation distribution function analysis also for non-cubic systems.

36 MATERIALS SCIENCE↗

General approaches for shear-correcting coordinate transformations in Bragg coherent diffraction imaging. Part II

X-ray Bragg coherent diffraction imaging (BCDI) has been demonstrated as a powerful 3D microscopy approach for the investigation of sub-micrometre-scale crystalline particles. The approach is based on the measurement of a series of coherent Bragg diffraction intensity patterns that are numerically inverted to retrieve an image of the spatial distribution of the relative phase and amplitude of the Bragg structure factor of the diffracting sample. This 3D information, which is collected through an angular rotation of the sample, is necessarily obtained in a non-orthogonal frame in Fourier space that must be eventually reconciled. To deal with this, the approach currently favored by practitioners (detailed in Part I) is to perform the entire inversion in conjugate non-orthogonal real- and Fourier-space frames, and to transform the 3D sample image into an orthogonal frame as a post-processing step for result analysis. In this article, which is a direct follow-up of Part I, two different transformation strategies are demonstrated, which enable the entire inversion procedure of the measured data set to be performed in an orthogonal frame. The new approaches described here build mathematical and numerical frameworks that apply to the cases of evenly and non-evenly sampled data along the direction of sample rotation ( i.e. the rocking curve). The value of these methods is that they rely on the experimental geometry, and they incorporate significantly more information about that geometry into the design of the phase-retrieval Fourier transformation than the strategy presented in Part I. Finally, two important outcomes are (1) that the resulting sample image is correctly interpreted in a shear-free frame and (2) physically realistic constraints of BCDI phase retrieval that are difficult to implement with current methods are easily incorporated. Computing scripts are also given to aid readers in the implementation of the proposed formalisms.

36 MATERIALS SCIENCE↗

DLSIA: Deep Learning for Scientific Image Analysis

DLSIA (Deep Learning for Scientific Image Analysis) is a Python-based machine learning library that empowers scientists and researchers across diverse scientific domains with a range of customizable convolutional neural network (CNN) architectures for a wide variety of tasks in image analysis to be used in downstream data processing. DLSIA features easy-to-use architectures, such as autoencoders, tunable U-Nets and parameter-lean mixed-scale dense networks (MSDNets). Additionally, this article introduces sparse mixed-scale networks (SMSNets), generated using random graphs, sparse connections and dilated convolutions connecting different length scales. For verification, several DLSIA-instantiated networks and training scripts are employed in multiple applications, including inpainting for X-ray scattering data using U-Nets and MSDNets, segmenting 3D fibers in X-ray tomographic reconstructions of concrete using an ensemble of SMSNets, and leveraging autoencoder latent spaces for data compression and clustering. As experimental data continue to grow in scale and complexity, DLSIA provides accessible CNN construction and abstracts CNN complexities, allowing scientists to tailor their machine learning approaches, accelerate discoveries, foster interdisciplinary collaboration and advance research in scientific image analysis.

97 MATHEMATICS AND COMPUTING↗

Integration of TomoPy and the ASTRA toolbox for advanced processing and reconstruction of tomographic synchrotron data

The processing of tomographic synchrotron data requires advanced and efficient software to be able to produce accurate results in reasonable time. In this paper, the integration of two software toolboxes, TomoPy and the ASTRA toolbox, which, together, provide a powerful framework for processing tomographic data, is presented. The integration combines the advantages of both toolboxes, such as the user-friendliness and CPU-efficient methods of TomoPy and the flexibility and optimized GPU-based reconstruction methods of the ASTRA toolbox. It is shown that both toolboxes can be easily installed and used together, requiring only minor changes to existing TomoPy scripts. Furthermore, it is shown that the efficient GPU-based reconstruction methods of the ASTRA toolbox can significantly decrease the time needed to reconstruct large datasets, and that advanced reconstruction methods can improve reconstruction quality compared with TomoPy's standard reconstruction method.

Pelt, Daniël M.↗

Robust framework and software implementation for fast speciation mapping

One of the greatest benefits of synchrotron radiation is the ability to perform chemical speciation analysis through X-ray absorption spectroscopies (XAS). XAS imaging of large sample areas can be performed with either full-field or raster-scanning modalities. A common practice to reduce acquisition time while decreasing dose and/or increasing spatial resolution is to compare X-ray fluorescence images collected at a few diagnostic energies. In this work, several authors have used different multivariate data processing strategies to establish speciation maps. Furthermore, the theoretical aspects and assumptions that are often made in the analysis of these datasets are focused on. A robust framework is developed to perform speciation mapping in large bulk samples at high spatial resolution by comparison with known references. Two fully operational software implementations are provided: a user-friendly implementation within the MicroAnalysis Toolkit software, and a dedicated script developed under the R environment. The procedure is exemplified through the study of a cross section of a typical fossil specimen. Additionally, the algorithm provides accurate speciation and concentration mapping while decreasing the data collection time by typically two or three orders of magnitude compared with the collection of whole spectra at each pixel. Whereas acquisition of spectral datacubes on large areas leads to very high irradiation times and doses, which can considerably lengthen experiments and generate significant alteration of radiation-sensitive materials, this sparse excitation energy procedure brings the total irradiation dose greatly below radiation damage thresholds identified in previous studies. This approach is particularly adapted to the chemical study of heterogeneous radiation-sensitive samples encountered in environmental, material, and life sciences.

47 OTHER INSTRUMENTATION↗

Flexible Data-Aware Scheduling for Workflows over an In-Memory Object Store

This paper explores novel techniques for improving the performance of many-task workflows based on the Swift scripting language. We propose novel programmer options for automated distributed data placement and task scheduling. These options trigger a data placement mechanism used for distributing intermediate workflow data over the servers of Hercules, a distributed key-value store that can be used to cache file system data. We demonstrate that these new mechanisms can significantly improve the aggregated throughput of many-task workflows with up to 86x, reduce the contention on the shared file system, exploit the data locality, and trade off locality and load balance.

Duro, Francisco Rodrigo↗

Running Ensemble Workflows at Extreme Scale: Lessons Learned and Path Forward

The ever-increasing volumes of scientific data combined with sophisticated techniques for extracting information from them have led to the increasing popularity of ensemble workflows which are a collection of runs of individual workflows. A traditional approach followed by scientists to run ensembles is to rely on simple scripts to execute different runs and manage resources. This approach is not scalable and is error-prone, thereby motivating the development of workflow management systems that specialize in executing ensembles on HPC clusters. However, when the size of both the ensemble and the target system reach extreme scales, existing workflow management systems face new challenges that hamper their efficient execution. In this paper, we describe our experience scaling an ensemble workflow from the computational biology domain from the early design stages to the execution at extreme scale on Summit, a leadership class supercomputer at the Oak Ridge National Laboratory. We discuss challenges that arise when scaling ensembles to several million runs on thousands of HPC nodes. We identify challenges with composition of the ensemble itself, its execution at large scale, post-processing of the generated data, and scalability of the file system. Based on the experience acquired, we develop a generic vision of the capabilities and abstractions to add to existing workflow management systems to enable the execution of ensemble workflows at extreme scales. We believe that the understanding of these fundamental challenges will help application teams along with workflow system developers with designing the next generation of infrastructure for composing and executing extreme-scale ensemble workflows.

Mehta, Kshitij↗

Data Jockey: Automatic Data Management for HPC Multi-Tiered Storage Systems

We present the design and implementation of Data Jockey, a data management system for HPC multi-tiered storage systems. As a centralized data management control plane, Data Jockey automates bulk data movement and placement for scientific workflows and integrates into existing HPC storage infrastructures. Data Jockey simplifies data management by eliminating human effort in programming complex data movements, laying datasets across multiple storage tiers when supporting complex workflows, which in turn increases the usability of multitiered storage systems emerging in modern HPC data centers.Specifically, Data Jockey presents a new data management scheme called “goal driven data management” that can automatically infer low-level bulk data movement plans from declarative high-level goal statements that come from the lifetime of iterative runs of scientific workflows. While doing so, Data Jockey aims to minimize data wait times by taking responsibility for datasets that are unused or to be used, and aggressively utilizing the capacity of the upper, higher performant storage tiers.We evaluated a prototype implementation of Data Jockey under a synthetic workload based on a year’s worth of Oak Ridge Leadership Computing Facility’s (OLCF) operational logs. Our evaluations suggest that Data Jockey leads to higher utilization of the upper storage tiers while minimizing the programming effort of data movement compared to human involved, per-domain adhoc data management scripts.

Shin, Woong↗

HERCULES: A Pattern Driven Code Transformation System

New parallel computers are emerging, but developing efficient scientific code for them remains difficult. A scientist must manage not only the science-domain complexity but also the performance-optimization complexity. HERCULES is a code transformation system designed to help the scientist to separate the two concerns, which improves code maintenance, and facilitates performance optimization. The system combines three technologies, code patterns, transformation scripts and compiler plugins, to provide the scientist with an environment to quickly implement code transformations that suit his needs. Unlike existing code optimization tools, HERCULES is unique in its focus on user-level accessibility. In this paper we discuss the design, implementation and an initial evaluation of HERCULES.

Computer Science↗

SWIM: A Switch Manager for Datacenter Networks

As modern datacenter networks (DCNs) grow to support hundreds of thousands of servers and beyond, managing network equipment - such as routers, firewalls, and load balancers - becomes increasingly complex. Network attributes such as IP address allocations and BGP neighbor relations are scattered among various network engineering groups, which makes troubleshooting the network a cumbersome task. In addition, network vendor diversity leads to an explosion of vendor-specific management systems or single-use automation scripts, limiting network scalability while increasing the time required to perform management tasks. In this article, the authors propose a unified network management system, Switch Manager (SWIM), to cope with the growth by standardizing the language for describing network attributes and unifying the interface for executing management actions on the network equipment.

97 MATHEMATICS AND COMPUTING↗

Distribution Model Creation using Circuit Maps for Applied PV Impact Studies

Photovoltaic (PV) generation is becoming increasingly common on power distribution systems. The output fluctuations of this variable resource necessitate the creation of an accurate model to ensure that any adverse impacts to the distribution system are properly addressed. To reduce model creation time, a method of extracting data from AutoCAD drawings using scripted methods was developed. This paper will present a distribution model creation and validation process, document case studies of evaluating the impact of PV interconnections on utility distribution feeders, and discuss future applications. PV integration was found to be easier on 23 kV circuits compared to 4.16 kV circuits.

integrated circuit modeling↗

Modeling and Automation Framework for High IBR Integration in Large-Scale Power Systems

The increasing prevalence of power electronics- interfaced renewable generation sources is leading to a gradual replacement of traditional thermal generation-based synchronous machines. In this context, the modeling of a large-scale power grid that incorporates a significant number of inverter-based resources is crucial for understanding the dynamics and effects of these resources on the power system. This study investigates the positive sequence model of grid-following and grid-forming inverters. Additionally, this work explores the integration of distributed energy resources using population as an indicator of their relative geographic locations. To address challenge to integrate these inverter based resources into a realistic grid of the US Western interconnection, automation scripts are developed to streamline the process of replacing conventional generators with grid-following and grid-forming inverters, as well as allocating distributed energy resources. Different penetration levels of these inverters are considered, and their frequency regulation support following a disturbance is compared through dynamic simulations.

Lyu, Xue↗