Search NASA⌕ Search

SEARCH · Search NASA

Results for “software: data analysis”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 631 records · Page 35

Tracking materials science data lineage to manage millions of materials experiments and analyses

Abstract In an era of rapid advancement of algorithms that extract knowledge from data, data and metadata management are increasingly critical to research success. In materials science, there are few examples of experimental databases that contain many different types of information, and compared with other disciplines, the database sizes are relatively small. Underlying these issues are the challenges in managing and linking data across disparate synthesis and characterization experiments, which we address with the development of a lightweight data management framework that is generally applicable for experimental science and beyond. Five years of managing experiments with this system has yielded the Materials Experiment and Analysis Database (MEAD) that contains raw data and metadata from millions of materials synthesis and characterization experiments, as well as the analysis and distillation of that data into property and performance metrics via software in an accompanying open source repository. The unprecedented quantity and diversity of experimental data are searchable by experiment and analysis attributes generated by both researchers and data processing software. The search web interface allows users to visualize their search results and download zipped packages of data with full annotations of their lineage. The enormity of the data provides substantial challenges and opportunities for incorporating data science in the physical sciences, and MEAD’s data and algorithm management framework will foster increased incorporation of automation and autonomous discovery in materials and chemistry research.

Soedarmadji, Edwin↗

Skills and Knowledge for Data-Intensive Environmental Research

The scale and magnitude of complex and pressing environmental issues lend urgency to the need for integrative and reproducible analysis and synthesis, facilitated by data-intensive research approaches. However, the recent pace of technological change has been such that appropriate skills to accomplish data-intensive research are lacking among environmental scientists, who more than ever need greater access to training and mentorship in computational skills. Here, we provide a roadmap for raising data competencies of current and next-generation environmental researchers by describing the concepts and skills needed for effectively engaging with the heterogeneous, distributed, and rapidly growing volumes of available data. We articulate five key skills: (1) data management and processing, (2) analysis, (3) software skills for science, (4) visualization, and (5) communication methods for collaboration and dissemination. We provide an overview of the current suite of training initiatives available to environmental scientists and models for closing the skill-transfer gap.

54 ENVIRONMENTAL SCIENCES↗

SU-F-T-468: Efficient Scanning Data Analysis for TomoTherapy

Purpose: The standard TomoTherapy 2D scanning system requires manual analysis using spreadsheets or similar software. Manual analysis is time consuming and does not allow efficient use of the medical physicist or timely results that may indicate issues during the acquisition of data. The purpose of this study is to determine if an in-house solution can provide timely analysis of TomoTherapy scan data. Methods: A custom MATLAB program utilizing a streamlined graphical interface was developed to interpret the native scanning data acquired by the TomoTherapy Electrometer Measurement System (TEMS). The gold beam data provided by the manufacturer was used as a baseline for the acquired data. Two separate sets of scan data were used including longitudinal, transverse, and percent depth dose (PDD). The scan data were compared visually in an overlay and analytically for each beam width using the full width half max (FWHM) for longitudinal profiles and difference inside the field core for the transverse profiles as recommended by TG-148. A plot of the PDD percent error at each point along the curve was generated to show agreement. The time to analysis was measured on a laptop running an Intel i5 processor with 8GB of RAM. Results: The software program required on average less than 5 seconds to load and analyze the results. No conversion or reformatting of the data was required. This allowed real-time visualization of the comparison to the gold beam data as well as the parameters recommended by TG-148. The average percent difference was 0.3% for the longitudinal profiles and 0.1% for the transverse profiles. All PDD data was less than 1% different beyond dmax. Conclusion: We have developed a tool to analyze TomoTherapy scan data in real-time. This allows an efficient method to perform beam commissioning, annual QA, and adjust the beam parameters if necessary.

61 RADIATION PROTECTION AND DOSIMETRY↗

CHMMPP: A c++ library for constrained Hidden Markov Models

SAND2024-13027O The CHMMPP: A c++ Library for Constrained Hidden Markov Models (HMM) software supports the analysis of multivariate time series data to detect patterns using HMM. Many applications involve the detection and characterization of hidden or latent states in a complex system using observable states and variables. This software supports inference of latent states integrating both an HMM and application-specific constraints that reflect known relationships in hidden states. The CHMMPP software supports application-specific and generic methods for constrained inference. This includes a framework for customized Viterbi methods, constrained inference of hidden states with A* and integer programming methods, and various constraint-informed methods for learning HMM model parameters. CHMMPP focuses on supporting generic methods that enable the agile expression of complex sets of constraints that naturally arise in many real-world applications.

Hart, William↗

The Analysis of the Patterns of Radiation-Induced DNA Damage Foci by a Stochastic Monte Carlo Model of DNA Double Strand Breaks Induction by Heavy Ions and Image Segmentation Software

To create a generalized mechanistic model of DNA damage in human cells that will generate analytical and image data corresponding to experimentally observed DNA damage foci and will help to improve the experimental foci yields by simulating spatial foci patterns and resolving problems with quantitative image analysis. Material and Methods: The analysis of patterns of RIFs (radiation-induced foci) produced by low- and high-LET (linear energy transfer) radiation was conducted by using a Monte Carlo model that combines the heavy ion track structure with characteristics of the human genome on the level of chromosomes. The foci patterns were also simulated in the maximum projection plane for flat nuclei. Some data analysis was done with the help of image segmentation software that identifies individual classes of RIFs and colocolized RIFs, which is of importance to some experimental assays that assign DNA damage a dual phosphorescent signal. Results: The model predicts the spatial and genomic distributions of DNA DSBs (double strand breaks) and associated RIFs in a human cell nucleus for a particular dose of either low- or high-LET radiation. We used the model to do analyses for different irradiation scenarios. In the beam-parallel-to-the-disk-of-a-flattened-nucleus scenario we found that the foci appeared to be merged due to their high density, while, in the perpendicular-beam scenario, the foci appeared as one bright spot per hit. The statistics and spatial distribution of regions of densely arranged foci, termed DNA foci chains, were predicted numerically using this model. Another analysis was done to evaluate the number of ion hits per nucleus, which were visible from streaks of closely located foci. In another analysis, our image segmentaiton software determined foci yields directly from images with single-class or colocolized foci. Conclusions: We showed that DSB clustering needs to be taken into account to determine the true DNA damage foci yield, which helps to determine the DSB yield. Using the model analysis, a researcher can refine the DSB yield per nucleus per particle. We showed that purely geometric artifacts, present in the experimental images, can be analytically resolved with the model, and that the quantization of track hits and DSB yields can be provided to the experimentalists who use enumeration of radiation-induced foci in immunofluorescence experiments using proteins that detect DNA damage. An automated image segmentaiton software can prove useful in a faster and more precise object counting for colocolized foci images.

Ponomarev, Artem↗

3rd Annual Earth System Grid Federation and 3rd Annual Earth System Grid Federation and Ultrascale Visualization Climate Data Analysis Tools Face-to-Face Meeting Report December 2013

The climate and weather data science community gathered December 3–5, 2013, at Lawrence Livermore National Laboratory, in Livermore, California, for the third annual Earth System Grid Federation (ESGF) and Ultra-scale Visualization Climate Data Analysis Tools (UV-CDAT) Face-to-Face (F2F) Meeting, which was hosted by the Department of Energy, National Aeronautics and Space Administration, National Oceanic and Atmospheric Administration, the European Infrastructure for the European Network of Earth System Modelling, and the Australian Department of Education. Both ESGF and UV-CDAT are global collaborations designed to develop a new generation of open-source software infrastructure that provides distributed access and analysis to observed and simulated data from the climate and weather communities. The tools and infrastructure developed under these international multi-agency collaborations are critical to understanding extreme weather conditions and long-term climate change, while the F2F meetings help to build a stronger climate and weather data science community and stronger federated software infrastructure. The 2013 F2F meeting determined requirements for existing and impending national and international community projects; enhancements needed for data distribution, analysis, and visualization infrastructure; and standards and resources needed for better collaborations.

54 ENVIRONMENTAL SCIENCES↗

Where's My Data - WMD

WMD provides a centralized interface to access data stored in the Mission Data Processing and Control System (MPCS) GDS (Ground Data Systems) databases during MSL (Mars Science Laboratory) Testbeds and ATLO (Assembly, Test, and Launch Operations) test sessions. The MSL project organizes its data based on venue (Testbed, ATLO, Ops), with each venue's data stored on a separate database, making it cumbersome for users to access data across the various venues. WMD allows sessions to be retrieved through a Web-based search using several criteria: host name, session start date, or session ID number. Sessions matching the search criteria will be displayed and users can then select a session to obtain and analyze the associated data. The uniqueness of this software comes from its collection of data retrieval and analysis features provided through a single interface. This allows users to obtain their data and perform the necessary analysis without having to worry about where and how to get the data, which may be stored in various locations. Additionally, this software is a Web application that only requires a standard browser without additional plug-ins, providing a cross-platform, lightweight solution for users to retrieve and analyze their data. This software solves the problem of efficiently and easily finding and retrieving data from thousands of MSL Testbed and ATLO sessions. WMD allows the user to retrieve their session in as little as one mouse click, and then to quickly retrieve additional data associated with the session.

Quach, William L.↗

Near Real-time Data Analysis of Core-Collapse Supernova Simulations With Bellerophon

We present an overview of a software system, Bellerophon, built to support a production-level HPC application called CHIMERA, which simulates core-collapse supernova events at the petascale. Developed over the last four years, Bellerophon enables CHIMERA s geographically dispersed team of collaborators to perform data analysis in near real-time. Its n-tier architecture provides an encapsulated, end-to-end software solution that enables the CHIMERA team to quickly and easily access highly customizable animated and static views of results from anywhere in the world via a web-deliverable, cross-platform desktop application. In addition, Bellerophon addresses software engineering tasks for the CHIMERA team by providing an automated mechanism for performing regression testing on a variety of supercomputing platforms. Elements of the team s workflow management needs are met with software tools that dynamically generate code repository statistics, access important online resources, and monitor the current status of several supercomputing resources.

Lingerfelt, Eric J↗

Ideas on the NASA Senior Review and Mission Extension Process

Extended mission budgets fund a significant part of our community, particularly young scientists who in a relatively short time are exposed to a broad range of experiences and opportunities available in space science and other STEM fields. In particular, mission-funded science offers a relatively stable funding stream that enables meaningful mentoring efforts and interactions, including apprenticeships ranging from project management, to mission operations, to data processing and analysis, to hardware and software development, to STEM-related education and outreach. It moreover provides professional contacts that fuel diversity in mission leadership and in the science and technical workforce at-large. Last but not least, it sustains the scientific productivity of NASA’s missions, which constitute major national investments meriting their full exploitation. Budgets cuts, however, are exercising pressure on the science and operations of the extended mission fleet despite being only a small portion of the overall Division budget. The history of Heliophysics has demonstrated that maintaining the operation of missions beyond their prime phase invariably leads to a much deeper knowledge of the original mission science goals, together with new applications often not foreseen during their original design. Their operation as part of the Heliophysics System Observatory (HSO)---a coordinated, 1eliophysics, multipoint observing system of the heliosphere, is a prime example. Some of the first activities undertaken by the Parker Solar Probe and Solar Orbiter missions required recreating the larger context of their new observations by using the diversity of HSO mission capabilities. The marginal cost of maintaining this valuable, working science resource, in the form of both continuing high quality data access and expertise, is small compared to the cost of the original missions or of new missions to achieve similar objectives, making the operation of extended missions, the best “science per dollar” investment in Heliophysics. We argue that reducing both extended mission operations and science budgets will have a ripple effect across the community, including reducing employment and training opportunities for young scientists, to lowering the quality/impact of the Heliophysics Systems Observatory. Instead, we suggest that the proposal process for mission extension should be re-examined. We offer several ideas for reducing costs of extended missions toward maintaining science output and streamlining the management process.

A. Vourlidas↗

The OME Framework for genome-scale systems biology

The life sciences are undergoing continuous and accelerating integration with computational and engineering sciences. The biology that many in the field have been trained on may be hardly recognizable in ten to twenty years. One of the major drivers for this transformation is the blistering pace of advancements in DNA sequencing and synthesis. These advances have resulted in unprecedented amounts of new data, information, and knowledge. Many software tools have been developed to deal with aspects of this transformation and each is sorely needed. However, few of these tools have been forced to deal with the full complexity of genome-scale models along with high throughput genome-scale data. This particular situation represents a unique challenge, as it is simultaneously necessary to deal with the vast breadth of genome-scale models and the dizzying depth of high-throughput datasets. It has been observed time and again that as the pace of data generation continues to accelerate, the pace of analysis significantly lags behind. It is also evident that, given the plethora of databases and software efforts, it is still a significant challenge to work with genome-scale metabolic models, let alone next-generation whole cell models. We work at the forefront of model creation and systems scale data generation. The OME Framework was borne out of a practical need to enable genome-scale modeling and data analysis under a unified framework to drive the next generation of genome-scale biological models. Here we present the OME Framework. It exists as a set of Python classes. However, we want to emphasize the importance of the underlying design as an addition to the discussions on specifications of a digital cell. A great deal of work and valuable progress has been made by a number of communities towards interchange formats and implementations designed to achieve similar goals. While many software tools exist for handling genome-scale metabolic models or for genome-scale data analysis, no implementations exist that explicitly handle data and models concurrently. The OME Framework structures data in a connected loop with models and the components those models are composed of. This results in the first full, practical implementation of a framework that can enable genome-scale design-build-test. Over the coming years many more software packages will be developed and tools will necessarily change. However, we hope that the underlying designs shared here can help to inform the design of future software.

59 BASIC BIOLOGICAL SCIENCES↗

Coupling of high-resolution mass spectrometer and photosynthesis system for comprehensive leaf volatile metabolite profiling

Background Leaf-level biogenic volatile organic compounds (BVOCs) emissions represent a major source of organic gases in the atmosphere, influencing both climate and air quality. These emissions are strongly driven by environmental perturbations, which affect individual plant- to ecosystem-level processes. Uncovering all the BVOCs and understanding how their emissions respond to altered environmental conditions provide critical insights into vegetation-driven changes in atmospheric chemistry. We developed a tandem instrumentation setup that integrates a proton transfer reaction time-of-flight mass spectrometer (PTR-ToF-MS) with parts-per-trillion detection limits and a photosynthetic infrared gas exchange system for the untargeted survey of all the BVOCs. This novel system enables simultaneous, real-time monitoring of BVOC emissions and photosynthetic parameters at the leaf level, offering new opportunities to disentangle the physiological and environmental drivers of VOC release. Furthermore, we established the VOC Analysis and Processing Optimization Resource (VAPOR), an open-access software tool designed for rapid data post-processing and the analysis of the variability of hundreds of BVOCs. We assessed the performance of the tandem system under varying background conditions, using standard gas mixtures and a range of environmental factors. Results Blank emissions were substantially lower for major BVOCs (e.g., isoprene) compared to those observed in plant emissions. Despite this, the observation of background-level VOCs highlights the importance of routinely acquiring and accounting for blank measurements in analyses using the coupled instrumentation. Introduction of known VOC concentrations to the system demonstrated a linear response across different compounds with varying molecular compositions, indicating minimal gas loss regardless of chemical moieties within the coupled instrumentation. We applied the optimized system to investigate the physiological mechanisms driving BVOC emissions across different genotypes of poplar and pennycress. The high mass resolution capabilities of the PTR-ToF-MS, coupled with comprehensive VAPOR-driven data analysis, enabled the identification of several important BVOCs, including methanol and methanethiol; these BVOCs displayed substantial variation across pennycress genotypes and showed concentrations ~ 100–350% higher than the blank. Moreover, isoprene emissions varied significantly among poplar genotypes grown in different potting media. Conclusions Tandem instrumentation offers a powerful tool for profiling volatile molecular markers and elucidating their genetic and environmental underpinnings. This approach enhances our ability to predict BVOC emissions in response to genotype by environmental interactions and contributes to a deeper understanding of vegetation responses to environmental changes.

Biogenic volatile organic compounds↗

Building an Efficient Cluster Cosmology Software Package for Modeling Cluster Counts and Lensing

We introduce a software suite developed for galaxy cluster cosmological analysis with the Dark Energy Survey Data. Cosmological analyses based on galaxy cluster number counts and weak-lensing measurements need efficient software infrastructure to explore an increasingly large parameter space, and account for various cosmological and astrophysical effects. Our software package is designed to model the cluster observables in a wide-field optical survey, including galaxy cluster counts, their averaged weak-lensing masses, or the cluster's averaged weak-lensing radial signals. To ensure maximum efficiency, this software package is developed in C++ in the CosmoSIS software framework, making use of the CUBA integration library. We also implement a testing and validation scheme to ensure the quality of the package. We demonstrate the effectiveness of this development by applying the software to the Dark Energy Survey Year 1 galaxy cluster cosmological data sets, and acquired cosmological constraints that are consistent with the fiducial Dark Energy Survey analysis.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

NASA Tech Briefs, March 2014

Topics include: Data Fusion for Global Estimation of Forest Characteristics From Sparse Lidar Data; Debris and Ice Mapping Analysis Tool - Database; Data Acquisition and Processing Software - DAPS; Metal-Assisted Fabrication of Biodegradable Porous Silicon Nanostructures; Post-Growth, In Situ Adhesion of Carbon Nanotubes to a Substrate for Robust CNT Cathodes; Integrated PEMFC Flow Field Design for Gravity-Independent Passive Water Removal; Thermal Mechanical Preparation of Glass Spheres; Mechanistic-Based Multiaxial-Stochastic-Strength Model for Transversely-Isotropic Brittle Materials; Methods for Mitigating Space Radiation Effects, Fault Detection and Correction, and Processing Sensor Data; Compact Ka-Band Antenna Feed with Double Circularly Polarized Capability; Dual-Leadframe Transient Liquid Phase Bonded Power Semiconductor Module Assembly and Bonding Process; Quad First Stage Processor: A Four-Channel Digitizer and Digital Beam-Forming Processor; Protective Sleeve for a Pyrotechnic Reefing Line Cutter; Metabolic Heat Regenerated Temperature Swing Adsorption; CubeSat Deployable Log Periodic Dipole Array; Re-entry Vehicle Shape for Enhanced Performance; NanoRacks-Scale MEMS Gas Chromatograph System; Variable Camber Aerodynamic Control Surfaces and Active Wing Shaping Control; Spacecraft Line-of-Sight Stabilization Using LWIR Earth Signature; Technique for Finding Retro-Reflectors in Flash LIDAR Imagery; Novel Hemispherical Dynamic Camera for EVAs; 360 deg Visual Detection and Object Tracking on an Autonomous Surface Vehicle; Simulation of Charge Carrier Mobility in Conducting Polymers; Observational Data Formatter Using CMOR for CMIP5; Propellant Loading Physics Model for Fault Detection Isolation and Recovery; Probabilistic Guidance for Swarms of Autonomous Agents; Reducing Drift in Stereo Visual Odometry; Future Air-Traffic Management Concepts Evaluation Tool; Examination and A Priori Analysis of a Direct Numerical Simulation Database for High-Pressure Turbulent Flows; and Resource-Constrained Application of Support Vector Machines to Imagery.

Source record↗

Generic and ML Workloads in an HPC Datacenter: Node Energy, Job Failures, and Node-Job Analysis

HPC datacenters offer a backbone to the modern digital society. Increasingly, they run Machine Learning (ML) jobs next to generic, compute-intensive workloads, supporting science, business, and other decision-making processes. However, understanding how ML jobs impact the operation of HPC datacenters, relative to generic jobs, remains desirable but understudied. In this work, we leverage long-term operational data, collected from a national-scale production HPC datacenter, and statistically compare how ML and generic jobs can impact the performance, failures, resource utilization, and energy consumption of HPC datacenters. Our study provides key insights, e.g., ML-related power usage causes GPU nodes to run into temperature limitations, median/mean runtime and failure rates are higher for ML jobs than for generic jobs, both ML and generic jobs exhibit highly variable arrival processes and resource demands, significant amounts of energy are spent on unsuccessfully terminating jobs, and concurrent jobs tend to terminate in the same state. We open-source our cleaned-up data traces on Zenodo (https://doi. org/10.5281/zenodo.13685426), and provide our analysis toolkit as software hosted on GitHub (https://github.com/atlarge-research/2024-icpads-hpc-workload-characterization). This study offers multiple benefits for data center administrators, who can improve operational efficiency, and for researchers, who can further improve system designs, scheduling techniques, etc.

crossanalysis↗

All Source Sensor Analysis

ASSA is a software application that processes binary data into summarized index tables that can be used to organize features contained within the data. ASSA's index tables can also be used to search for user specified features. ASSA is designed to organize and search for patterns in unstructured binary data streams or archives, such as video, images, audio, and network traffic. ASSA is basically a very general search engine used to search for any pattern in any binary data stream. It has uses in video analytics, image analysis, audio analysis, searching hard-drives, monitoring network traffic, etc.

Trease, Harold↗

Graphs, matrices, and the GraphBLAS: Seven good reasons

The analysis of graphs has become increasingly important to a wide range of applications. Graph analysis presents a number of unique challenges in the areas of (1) software complexity, (2) data complexity, (3) security, (4) mathematical complexity, (5) theoretical analysis, (6) serial performance, and (7) parallel performance. Implementing graph algorithms using matrix-based approaches provides a number of promising solutions to these challenges. The GraphBLAS standard (istcbigdata.org/GraphBlas) is being developed to bring the potential of matrix based graph algorithms to the broadest possible audience. The GraphBLAS mathematically defines a core set of matrix-based graph operations that can be used to implement a wide class of graph algorithms in a wide range of programming environments. This paper provides an introduction to the GraphBLAS and describes how the GraphBLAS can be used to address many of the challenges associated with analysis of graphs.

97 MATHEMATICS AND COMPUTING↗

PlantCV v2: Image analysis software for high-throughput plant phenotyping

Systems for collecting image data in conjunction with computer vision techniques are a powerful tool for increasing the temporal resolution at which plant phenotypes can be measured non-destructively. Computational tools that are flexible and extendable are needed to address the diversity of plant phenotyping problems. We previously described the Plant Computer Vision (PlantCV) software package, which is an image processing toolkit for plant phenotyping analysis. The goal of the PlantCV project is to develop a set of modular, reusable, and repurposable tools for plant image analysis that are open-source and community-developed. Here in this paper we present the details and rationale for major developments in the second major release of PlantCV. In addition to overall improvements in the organization of the PlantCV project, new functionality includes a set of new image processing and normalization tools, support for analyzing images that include multiple plants, leaf segmentation, landmark identification tools for morphometrics, and modules for machine learning.

59 BASIC BIOLOGICAL SCIENCES↗

Visualization and Data Analysis at the Exascale

The scope of our working group is scientific visualization and data analysis. Scientific visualization refers to the process of transforming scientific simulation and experimental data into images to facilitate visual understanding. Data analysis refers to the process of transforming data into an information-rich form via mathematical or computational algorithms to promote better understanding. We share scope on data management with the Storage group. Data management refers to the process of tracking, organizing and enhancing the use of scientific data. The purpose of our work is to enable scientific discovery and understanding. Visualization and data analysis has a broad scope as an integral part of scientific simulations and experiments, it is also a distinct separate service for scientific discovery and documentation purposes. Our scope includes an exascale software and hardware infrastructure that effectively supports visualization and data analysis.

ALGORITHMS↗