Search NASA⌕ Search

SEARCH · Search NASA

Results for “software: data analysis”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 199 records · Page 11

STXMPy: a New Software Package for Automated Region of Interest Selection and Statistical Analysis of XANES Data

Soft X-ray spectromicroscopy based absorption near-edge structure analysis, is a spectroscopic technique useful for investigating sample composition at a nanoscale of resolution. While the technique holds great promise for analysis of biological samples, current methodologies are challenged by a lack of automatic analysis software e. g. for selection of regions of interest and statistical comparisons of sample variability. We have implemented a set of functions and scripts in Python to provide a semiautomatic treatment of data obtained using scanning transmission X-ray microscopy. The toolkit includes a novel line-by-line absorption conversion and data filtering automatically identifying image components with significant absorption. Results are provided to the user by direct graphical output to the screen and by output images and data files, including the average and standard deviation of the X-ray absorption spectrum. Using isolated mouse melanosomes as a sample biological tissue, application of STXMPy in analysis of biological tissues is illustrated. The STXMPy package allows both interactive and automated batch processing of scanning transmission X-ray microscopic data. It is open source, cross platform, and offers rapid script development using the interpreted Python language.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Deep Learning for Automated Identification of Eels in Sonar Data

Freshwater eels, such as the American eel (Anguilla rostrata) present numerous challenges related to safe downstream fish passage at hydroelectric facilities. One of those challenges is effective monitoring of their abundance, movements, and behavior to facilitate design and operation of eel protection and passage facilities. A previous EPRI study documented the ability of human analysts to reliably identify American eels in data obtained with a 1100/1800 kHz, multibeam sonar. This report describes a project to develop deep learning (a subset of artificial intelligence) tools to automate the time-consuming, subjective process of eel identification in multibeam sonar data. The project exploited new data collected in the laboratory and the existing data from the prior EPRI field study to develop and test deep learning and other data analytic tools, including wavelet filtering, differencing for static object removal, and convolutional neural network analysis. The analysis of the laboratory data demonstrated feasibility of the approach, revealed object characteristics observed with the sonar that distinguish eels from similarly sized and shaped acoustic targets, and provided additional data for algorithm selection and training. Deep learning algorithms trained and tested on the laboratory data alone achieved accuracy rates of greater than 98% when classifying acoustic images of eels and similar-sized neutrally buoyant sticks. The algorithm trained and tested on the pre-existing field data alone, and yielded classification accuracy of 9.3% false positives and 13.3% false negatives when distinguishing between eels and sticks/PVC pipes based on video clips (i.e., multiple, consecutive images). This performance is comparable to the classification accuracy achieved by human analysts in the prior study. The deep learning algorithm trained on a combination of video clips obtained in the laboratory and the field and tested on video clips from the field, was able to distinguish eels from sticks and PVC pipes (a river debris analog) of similar size with 100% accuracy. Outreach to the hardware, software, and end-user communities early in the project helped to identify needs and specify the application space. Outreach to those communities at the end of the project communicated project results and opportunities for further development. The project achieved proof of concept for automated identification of eel in multibeam sonar data. Future work should focus on acquisition of additional data for more robust algorithm training and testing; modification of the software tools to accommodate multiple acoustic targets in the acoustic field at a given time; identification of additional object classes; incorporation of motion in the object identification and classification algorithms; operationalizing the software tools, including integration with other existing sonar data analysis tools; and partnering with hardware and software providers for distribution of the software tools with their commercial products.

13 HYDRO ENERGY↗

Scalable and Power Efficient Data Analytics for Hybrid Exascale Systems

This project developed a generic and optimized set of core data analytics functions. These functions organically consolidate a broad constellation of high performance analytical pipelines. As the architectures of emerging HPC systems become inherently heterogeneous, there is a need to design algorithms for data analysis kernels accelerated on hybrid multi-node, multi-core HPC architectures comprised of a mix of CPUs, GPUs, and SSDs. Furthermore, the power-aware trend drives the advances in our performance-energy tradeoff analysis framework which enables our data analysis kernels algorithms and software to be parameterized so that users can choose the right power-performance optimizations.

97 MATHEMATICS AND COMPUTING↗

Managing Software Provenance to Enhance Reproducibility in Computational Research

Scientific processes rely on software as an important tool for data acquisition, analysis, and discovery. Over the years, sustainable software development practices have made progress in being considered as an integral component of research. However, management of computation-based scientific studies is often left to individual researchers who design their computational experiments based on personal preferences and the nature of the study. Here, we believe that the quality, efficiency, and reproducibility of computation-based scientific research can be improved by explicitly creating an execution environment that allows researchers to provide a clear record of traceability. This is particularly relevant to complex computational studies in high-performance computing (HPC) environments. In this article, we review the documentation required to maintain a comprehensive record of HPC computational experiments for reproducibility. We also provide an overview of tools and practices that we have developed to perform such studies around Flash-X, a multiphysics scientific software.

97 MATHEMATICS AND COMPUTING↗

Data from: "Towards CONUS-Wide ML-Augmented Conceptually-Interpretable Modeling of Catchment-Scale Precipitation-Storage-Runoff Dynamics"

This data package was generated to support the manuscript “Towards CONUS-Wide Machine Learning-Augmented Conceptually Interpretable Modeling of Catchment-Scale Precipitation-Storage-Runoff Dynamics.” It provides input files, model outputs, plotting data, scripts, notebooks, and documentation used to develop, evaluate, and reproduce Mass-Conserving Perceptron (MCP)-based hydrologic modeling experiments across 513 selected Catchment Attributes and Meteorology for Large-sample Studies in the United States (CAMELS-US) basins. The files are organized by modeling component and analysis purpose, including rainfall–runoff experiments, snow module experiments, coupled hydrologic-snow experiments, Long Short-Term Memory (LSTM) benchmark results, model skill metrics, initialization and epoch records, cell-state normalization files, Akaike Information Criterion (AIC)-based model comparison files, and data used to generate manuscript figures. Tabular files can be opened using standard spreadsheet software or Python/R data-analysis tools. Python scripts, Jupyter notebooks, and selected MATLAB scripts are included for model execution, postprocessing, plotting, and statistical analysis. Quality assurance and quality control were conducted through the source-data selection and modeling workflow. Meteorological forcing, streamflow, and static catchment attributes were derived from the CAMELS-US dataset, and snow water equivalent data were derived from the University of Arizona (UA) Snow Water Equivalent dataset. Selected basins and time periods were screened during the associated research workflow to avoid missing observations or poor-quality cases. Static geospatial features were processed primarily using Quantum Geographic Information System (QGIS) and Geospatial Data Abstraction Library (GDAL) workflows. Additional details are provided in the associated manuscript and documentation.

ESS-DIVE CSV File Formatting Guidelines Reporting ↗

Advanced grazing-incidence techniques for modern soft-matter materials analysis

The complex nano-morphology of modern soft-matter materials is successfully probed with advanced grazing-incidence techniques. Based on grazing-incidence small- and wide-angle X-ray and neutron scattering (GISAXS, GIWAXS, GISANS and GIWANS), new possibilities arise which are discussed with selected examples. Due to instrumental progress, highly interesting possibilities for local structure analysis in this material class arise from the use of micro- and nanometer-sized X-ray beams in micro- or nanofocused GISAXS and GIWAXS experiments. The feasibility of very short data acquisition times down to milliseconds creates exciting possibilities for in situ and in operando GISAXS and GIWAXS studies. Tuning the energy of GISAXS and GIWAXS in the soft X-ray regime and in time-of flight GISANS allows the tailoring of contrast conditions and thereby the probing of more complex morphologies. In addition, recent progress in software packages, useful for data analysis for advanced grazing-incidence techniques, is discussed.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗

Development of a Magnetic Microbead Affinity Selection Screen (MagMASS) Using Mass Spectrometry for Ligands to the Retinoid X Receptor-α

To overcome limiting factors in mass spectrometry-based screening methods such as automation while still facilitating the screening of complex mixtures such as botanical extracts, magnetic microbead affinity selection screening (MagMASS) was developed. The screening process involves immobilization of a target protein on a magnetic microbead using a variety of possible chemistries, incubation with mixtures of molecules containing possible ligands, a washing step that removes non-bound compounds while a magnetic field retains the beads in the microtiter well, and an organic solvent release step followed by LC-MS analysis. Using retinoid X receptor-α (RXRα) as an example, which is a nuclear receptor and target for anti-inflammation therapy as well as cancer treatment and prevention, a MagMASS assay was developed and compared with an existing screening assay, pulsed ultrafiltration (PUF)-MS. Optimization of MagMASS involved evaluation of multiple protein constructs and several magnetic bead immobilization chemistries. The full-length RXRα construct immobilized with amylose beads provided optimum results. Additional enhancements of MagMASS were the application of 96-well plates to enable automation, use of UHPLC instead of HPLC for faster MS analyses, and application of metabolomics software for faster, automated data analysis. Performance of MagMASS was demonstrated using mixtures of synthetic compounds and known ligands spiked into botanical extracts. .

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

Cheetah : software for high-throughput reduction and analysis of serial femtosecond X-ray diffraction data

The emerging technique of serial X-ray diffraction, in which diffraction data are collected from samples flowing across a pulsed X-ray source at repetition rates of 100 Hz or higher, has necessitated the development of new software in order to handle the large data volumes produced. Sorting of data according to different criteria and rapid filtering of events to retain only diffraction patterns of interest results in significant reductions in data volume, thereby simplifying subsequent data analysis and management tasks. Meanwhile the generation of reduced data in the form of virtual powder patterns, radial stacks, histograms and other meta data creates data set summaries for analysis and overall experiment evaluation. Rapid data reduction early in the analysis pipeline is proving to be an essential first step in serial imaging experiments, prompting the authors to make the tool described in this article available to the general community. Originally developed for experiments at X-ray free-electron lasers, the software is based on a modular facility-independent library to promote portability between different experiments and is available under version 3 or later of the GNU General Public License.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Dosimetric and Late Radiation Toxicity Comparison Between Iodine-125 Brachytherapy and Stereotactic Radiation Therapy for Juxtapapillary Choroidal Melanoma

Purpose: To compare the dose distributions and late radiation toxicities for {sup 125}I brachytherapy (IBT) and stereotactic radiation therapy (SRT) in the treatment of juxtapapillary choroidal melanoma. Methods: Ninety-four consecutive patients with juxtapapillary melanoma were reviewed: 30 have been treated with IBT and 64 with SRT. Iodine-125 brachytherapy cases were modeled with plaque simulator software for dosimetric analysis. The SRT dosimetric data were obtained from the Radionics XKnife RT3 software. Mean doses at predetermined intraocular points were calculated. Kaplan-Meier estimates determined the actuarial rates of late toxicities, and the log–rank test compared the estimates. Results: The median follow-up was 46 months in both cohorts. The 2 cohorts were balanced with respect to pretreatment clinical and tumor characteristics. Comparisons of radiation toxicity rates between the IBT and SRT cohorts yielded actuarial rates at 50 months for cataracts of 62% and 75% (P=.1), for neovascular glaucoma 8% and 47% (P=.002), for radiation retinopathy 59% and 89% (P=.0001), and for radiation papillopathy 39% and 74% (P=.003), respectively. Dosimetric comparisons between the IBT and SRT cohorts yielded mean doses of 12.8 and 14.1 Gy (P=.56) for the lens center, 17.6 and 19.7 Gy (P=.44) for the lens posterior pole, 13.9 and 10.8 Gy (P=.30) for the ciliary body, 61.9 and 69.7 Gy (P=.03) for optic disc center, and 48.9 and 60.1 Gy (P<.0001) for retina at 5-mm distance from tumor margin, respectively. Conclusions: Late radiation-induced toxicities were greater with SRT, which is secondary to the high-dose exposure inherent to the technique as compared with IBT. When technically feasible, IBT is preferred to treat juxtapapillary choroidal melanoma.

62 RADIOLOGY AND NUCLEAR MEDICINE↗

A new development on measurement and control software of SANS BATAN spectrometer (SMARTer) in Serpong, Indonesia

In 2005, the main computer for data acquisition and control system of Small-angle Neutron Scattering (SANS) BATAN Spectrometer (SMARTer) was replaced since it halted to operate the spectrometer. According to this replacement, the new software for data acquisition and control system has been developed in-house. Visual Basic programming language is used in developing the software. In the last two years, many developments have been made both in the hardware and also the software to conduct the experiment is more effective and efficient. Lately, the previous motor controller card (ISA Card) was replaced with the programmable motor controller card (PCI Card) for driving one motor of position sensitive detector (PSD), eight motors of four collimators, and six motors of six pinhole discs. This new control system software makes all motors can be moved simultaneously, then it reduces significantly the consuming time of setting up the instrument before running the experiment. Along with that development, the new data acquisition software under MS Windows operating system is also developed to drive a beam stopper in X-Y directions as well as to read the equipment status such as position of the collimators and PSD, to acquire neutron counts on monitor and PSD detectors, and also to manage 12 samples position automatically. A timer object which is set in one second to read the equipment status via serial port of the computer (RS232C), and general purpose interface board (GPIB) for reading the total counts of each pixel of the PSD from histogram memory was used in this new software. The experiment result displayed in real time on the main window, and the data is saved in the special format for further data reduction and analysis. The new software has been implemented and performed for experiment using a preset count or preset time mode for absolute scattering intensity method.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

Database Performance Monitoring for the Photovoltaic Systems

The Database Performance Monitoring (DPM) software (copyright in processes) is being developed at Sandia National Laboratories to perform quality control analysis on time series data. The software loads time indexed databases (currently csv format), performs a series of quality control tests defined by the user, and creates reports which include summary statistics, tables, and graphics. DPM can be setup to run on an automated schedule defined by the user. For example, the software can be run once per day to analyze data collected on the previous day. HTML formatted reports can be sent via email or hosted on a website. To compare performance of several databases, summary statistics and graphics can be gathered in a dashboard view which links to detailed reporting information for each database. The software can be customized for specific applications.

97 MATHEMATICS AND COMPUTING↗

AUTOMATED LEAK DETECTION OF BURIED TANKS USING GEOPHYSICAL METHODS AT THE HANFORD NUCLEAR SITE

At the Hanford Nuclear Site in Washington State, the Department of Energy oversees the containment, treatment, and retrieval of liquid high-level radioactive waste. Much of the waste is stored in single-shelled tanks (SSTs) built between 1943 and 1964. Currently, the waste is being retrieved from the SSTs and transferred into newer double-shelled tanks (DSTs) for temporary storage before final treatment. Monitoring the tanks during the retrieval process is critical to identifying leaks. An electrically-based geophysics monitoring program for leak detection and monitoring (LDM) has been successfully deployed on several SSTs at the Hanford site since 2004. The monitoring program takes advantage of changes in contact resistance that will occur when conductive tank liquid leaks into the soil. During monitoring, electrical current is transmitted on a number of different electrode types (e.g., steel cased wells and surface electrodes) while voltages are measured on all other electrodes, including the tanks. Data acquisition hardware and software allow for continuous real-time monitoring of the received voltages and the leak assessment is conducted through a time-series data analysis. The specific hardware and software combination creates a highly sensitive method of leak detection, complementing existing drywell logging as a means to detect and quantify leaks. Working in an industrial environment such as the Hanford site presents many challenges for electrical monitoring: cathodic protection, grounded electrical infrastructure, lightning strikes, diurnal and seasonal temperature trends, and precipitation, all of which create a complex environment for leak detection. In this discussion we present examples of challenges and solutions to working in the tank farms of the Hanford site.

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗

Scalable Comparative Visualization of Ensembles of Call Graphs

Optimizing the performance of large-scale parallel codes is critical for efficient utilization of computing resources. Code developers often explore various execution parameters, such as hardware configurations, system software choices, and application parameters, and are interested in detecting and understanding bottlenecks in different executions. They often collect hierarchical performance profiles represented as call graphs, which combine performance metrics with their execution contexts. The crucial task of exploring multiple call graphs together is tedious and challenging because of the many structural differences in the execution contexts and significant variability in the collected performance metrics (e.g., execution runtime). In this paper, we present Ensemble CallFlow to support the exploration of ensembles of call graphs using new types of visualizations, analysis, graph operations, and features. We introduce ensemble-Sankey , a new visual design that combines the strengths of resource-flow (Sankey) and box-plot visualization techniques. Whereas the resource-flow visualization can easily and intuitively describe the graphical nature of the call graph, the box plots overlaid on the nodes of Sankey convey the performance variability within the ensemble. Our interactive visual interface provides linked views to help explore ensembles of call graphs, e.g., by facilitating the analysis of structural differences, and identifying similar or distinct call graphs. Finally, we demonstrate the effectiveness and usefulness of our design through case studies on large-scale parallel codes.

97 MATHEMATICS AND COMPUTING↗

Modern Computational Techniques for the HMMER Sequence Analysis

This paper focuses on the latest research and critical reviews on modern computing architectures, software and hardware accelerated algorithms for bioinformatics data analysis with an emphasis on one of the most important sequence analysis applications—hidden Markov models (HMM). We show the detailed performance comparison of sequence analysis tools on various computing platforms recently developed in the bioinformatics society. The characteristics of the sequence analysis, such as data and compute-intensive natures, make it very attractive to optimize and parallelize by using both traditional software approach and innovated hardware acceleration technologies.

59 BASIC BIOLOGICAL SCIENCES↗

Scalable Proximity-Based Methods for Large-Scale Analysis of Atom Probe Data

Powered by recent advances in data acquisition technologies, today's state-of-the-art atom probe microscopes yield data sets with sizes ranging from a few million atoms to billions of atoms. Analysis of these atomic data sets within rea-sonable turnaround times is a pressing data analysis challenge for material scientists currently equipped with software systems that do not scale to these massive data sets. Here, we present the shared memory component of a larger ongoing effort to develop a multi-feature data analysis framework capable of analyzing atom probe data of all sizes and scales from desktop multicore machines to large-scale high-performance computing platforms with hybrid (shared and distributed memory) architectures. Our focus here is on a broad class of popular atom probe data analysis methods that rely on core time-consuming k-NN queries. We present a scalable, heuristic algorithm for k-NN queries using three-dimensional range trees. To demonstrate its efficacy, the k-NN algorithm is integrated with two use cases of atom probe data analysis methods and the resulting analysis times are shown to speedup by over 20X on a 32-core Cray XC40 node using workloads up to 8 million atoms, which is already beyond the at-scale capabilities of existing atom probe software. Using this k-NN algorithm, we also introduce a novel parameter estimation method for a class of cluster finding methods, called friends-of-friends (FoF) methods, to completely bypass their expensive pre-processing steps. In each case, we validate the results on a variety of control data sets.

Lu, Hao↗

Event-Based Processing of Neutron Scattering Data

Many of the world's time-of-flight spallation neutrons sources are migrating to the recording of individual neutron events. This provides for new opportunities in data processing, the least of which is to filter the events based on correlating them with logs of sample environment and other ancillary equipment. This paper will describe techniques for processing neutron scattering data acquired in event mode that preserve event information all the way to a final spectrum, including any necessary corrections or normalizations. This results in smaller final errors, while significantly reducing processing time and memory requirements in typical experiments. Results with traditional histogramming techniques will be shown for comparison.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗

py4DSTEM: A Software Package for Four-Dimensional Scanning Transmission Electron Microscopy Data Analysis

Scanning transmission electron microscopy (STEM) allows for imaging, diffraction, and spectroscopy of materials on length scales ranging from microns to atoms. By using a high-speed, direct electron detector, it is now possible to record a full two-dimensional (2D) image of the diffracted electron beam at each probe position, typically a 2D grid of probe positions. These 4D-STEM datasets are rich in information, including signatures of the local structure, orientation, deformation, electromagnetic fields, and other sample-dependent properties. However, extracting this information requires complex analysis pipelines that include data wrangling, calibration, analysis, and visualization, all while maintaining robustness against imaging distortions and artifacts. In this paper, we present py4DSTEM, an analysis toolkit for measuring material properties from 4D-STEM datasets, written in the Python language and released with an open-source license. We describe the algorithmic steps for dataset calibration and various 4D-STEM property measurements in detail and present results from several experimental datasets. We also implement a simple and universal file format appropriate for electron microscopy data in py4DSTEM, which uses the open-source HDF5 standard. We hope this tool will benefit the research community and help improve the standards for data and computational methods in electron microscopy, and we invite the community to contribute to this ongoing project.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗