Search NASASearch

SEARCH · Search NASA

Results for “Python applications”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 127 records · Page 7

Connecting Users and Applications with Po.daac Hosted GHRSST Data

The 80+ GHRSST public datasets represent a rich resource for sea surface temperature research and applications given their time series length, resolution, spatial coverage, varying measurement types and processing levels, and availability in the full spectrum of PO.DAAC tools and services ecosystem. The PO.DAAC has created a publicly accessible recipe suite for the user community to perform straightforward yet powerful computations on GHRSST data using python recipes, Jupyter notebooks, R, Matlab, and the NCO programming language. These recipes include numerical computations for regional and global SST trends, anomaly derivations, EOF analysis, climate signal reproduction, and ocean phenology. For example, one recipe reproduces a famous SST based warming figure from the Fourth National Climate Assessment (USA) while another focuses on quantifying the regional changes in ocean SST phenology. Most are python-based while some contain hybrid calls and leverage the NCO programming interface too. All are available on the PO.DAAC user forum (https://podaac.jpl.nasa.gov/forum/) and/or via the open source NASA GitHub repository (https://github.com/nasa/podaac_tools_and_services). Several are available in the Jupyter notebook framework including podaacypy (https://github.com/nasa/podaacpy), a recipe for GHRSST granule metadata discovery and application, and more recently a Jupyter notebook developed to support data analysis and visualization of a cloud-based Zarr formatted Level 4 MUR dataset in the AWS Open Data Registry. Throughout the summer of 2020, the PO.DAAC intends to add and migrate more of its numerical recipes to the Jupyter notebook framework and publish them on its open source GitHub repository.

Gentemann, Chelle

Derivation of Effective Properties Based on Porous Scale Simulations Using Filtering Techniques

This study presents a method for derivation of effective properties at the interface and in-depth of porous materials. The method defines a Representative Elementary Volume (REV) and applies filtering techniques to computer effective properties such as porosity and flow quantities, such as velocity and pressure. The script, developed to process the data was tested on the VTK type files that contain the mesh information and the flow solution. The method allows to choose between two types of filters, such as cellular and top-hat and define the size of the REV and number of samples along the domain. Extraction of the REV from the domain is performed to exact boundaries requested for the user. This is done using a triangulation technique and cutting through the cells to comply to the requested boundaries of the volume. The method can be applied to both structured and unstructured meshes. Filtering the material porosity and flow quantities involves integration of the numerical data. The algorithm provides three integration methods, such as Riemann sum, Monte Carlo and Quadrature rule to perform the integration. The Monte-Carlo technique permits the use of either uniform or linearly spaced distribution of points. The Quadrature rule is currently applicable to tetrahedral element types. The Monte Carlo and Quadrature rule methods require interpolation of the flow quantities at the sample points. For interpolation, two methods were tested and are readily available, Gaussian interpolation and re-sampling. It has been shown that re-sampling method has better consistency and acceptable accuracy in interpolation of the data. The algorithm was written in Python language and uses a number of modules. The major module besides numpy is PyVista. It is used to process the computational domain, clip the REV and interpolate the data. Quadrature rule integration was performed using a quadpy module. ParaView software was used externally to convert the flow solution to the VTK (or more specifically VTU) format. Integration of ParaView in the same environment with PyVista encountered problems and could not be implemented in this work. The developed algorithm is expected to be applicable to unstructured meshes and more complex porous structures as soon as the data can be passed in VTK type format. With the report is provided Python script for filtering the solution and a Matlab script for simple generation and processing of 2-D and 3-D porous channel geometries. The two scripts don't communicate.

Alexsander Zibitsker

A Fortran-Python Interface for Integrating Machine Learning Parameterization into Earth System Models

Parameterizations in Earth System Models (ESMs) are subject to biases and uncertainties arising from subjective empirical assumptions and incomplete understanding of the underlying physical processes. Recently, the growing representational capability of machine learning (ML) in solving complex problems has spawned immense interests in climate science applications. Specifically, ML-based parameterizations have been developed to represent convection, radiation and microphysics processes in ESMs by learning from observations or high-resolution simulations, which have the potential to improve the accuracies and alleviate the uncertainties. Previous works have developed some surrogate models for these processes using ML. These surrogate models need to be coupled with the dynamical core of ESMs to investigate the effectiveness and their performance in a coupled system. In this study, we present a novel Fortran-Python interface designed to seamlessly integrate ML parameterizations into ESMs. This interface showcases high versatility by supporting popular ML frameworks like PyTorch, TensorFlow, and Scikit-learn. We demonstrate the interface's modularity and reusability through two cases: a ML trigger function for convection parameterization and a ML wildfire model. We conduct a comprehensive evaluation of memory usage and computational overhead resulting from the integration of Python codes into the Fortran ESMs. By leveraging this flexible interface, ML parameterizations can be effectively developed, tested, and integrated into ESMs.

54 ENVIRONMENTAL SCIENCES

A Fortran–Python interface for integrating machine learning parameterization into earth system models

Abstract. Parameterizations in earth system models (ESMs) are subject to biases and uncertainties arising from subjective empirical assumptions and incomplete understanding of the underlying physical processes. Recently, the growing representational capability of machine learning (ML) in solving complex problems has spawned immense interests in climate science applications. Specifically, ML-based parameterizations have been developed to represent convection, radiation, and microphysics processes in ESMs by learning from observations or high-resolution simulations, which have the potential to improve the accuracies and alleviate the uncertainties. Previous works have developed some surrogate models for these processes using ML. These surrogate models need to be coupled with the dynamical core of ESMs to investigate the effectiveness and their performance in a coupled system. In this study, we present a novel Fortran–Python interface designed to seamlessly integrate ML parameterizations into ESMs. This interface showcases high versatility by supporting popular ML frameworks like PyTorch, TensorFlow, and scikit-learn. We demonstrate the interface's modularity and reusability through two cases: an ML trigger function for convection parameterization and an ML wildfire model. We conduct a comprehensive evaluation of memory usage and computational overhead resulting from the integration of Python codes into the Fortran ESMs. By leveraging this flexible interface, ML parameterizations can be effectively developed, tested, and integrated into ESMs.

54 ENVIRONMENTAL SCIENCES

wastewater_virus

This repo contains software used to clean and assemble high-throughput sequencing data containing viruses. The input is raw illumina sequencing reads and the output is a database of high-quality viral genomes. The specific application is to wastewater viral concentrates but it is not restricted to that sample type. The software is composed of Nextflow workflows and a set of custom Python and bash scripts that call publicly available bioinformatics tools to accomplish obvious tasks in data analysis in a high performance computing environment. For detailed information, please see the repo's README file.

Kantor, Rose [Lawrence Livermore National Laborato

HITMAN

HITMAN (Hermite Interpolation of Trajectories and Measurement Synthesis for Analysis of Navigators) interpolates—or estimates the unknown values between known values—flight trajectories and generates synthetic inertial measurement unit (IMU) data using Hermite splines. This Python library provides modeling and simulation capabilities to synthesize inertial measurements from discrete trajectory points, enabling researchers to create exemplar datasets for evaluating navigation algorithms in various applications, including consumer devices like smartphones and vehicles. Sandia National Laboratories is a multimission laboratory managed and operated by National Technology & Engineering Solutions of Sandia, LLC, a wholly owned subsidiary of Honeywell International Inc., for the U.S. Department of Energy’s National Nuclear Security Administration under contract DE-NA0003525.

Walker II, Michael [Sandia National Lab. (SNL-CA),

REDESIGNING A PERFORMANCE MONITORING SOFTWARE FOR SUPERCOMPUTERS

The objective of this project was to improve upon the existing Watchr software that charts performance test metrics from the Trilinos project run on supercomputers at Sandia and elsewhere across the DOE complex. Software was iteratively designed and developed using Python Pandas and Dash data visualization to improve the extensibility and user experience of Watchr. Documentation is being maintained for future developers who want to extend the application.

Camacho, Dane Joseph [Sandia National Laboratories

SAGE III/ISS Rapid Data Analysis Through Dashboarding with Jupyter Notebooks

Spaceborne remote sensing observations of Earth’s atmosphere produce significant quantities of data over the life of each mission. In the case of the Stratospheric Aerosol and Gas Experiment III on the International Space Station (SAGE III/ISS) nearly four years of vertical profiles of atmospheric ozone, water vapor, and nitrogen dioxide concentrations as well as aerosol extinction coefficients have been released. The dichotomy of the desire for both long-term trends in the atmospheric state alongside the assessment of short-term impacts of major disruptive events such as volcanic eruptions and pyrocumulus injections requires agile tools to handle these cases in near real-time as new data are produced. The analysis landscape is further complicated by the desire to compare results between the numerous contemporary observations available for a given dataset. The SAGE III/ISS team has developed a suite of tools leveraging modern web-based frameworks allowing members to interact with a dashboard-style interface to load the data record, assess new profiles as they are generated and in ensemble, compare between species, and additionally add in measurements observed by other platforms as necessary. Leveraging a commonly packaged data format of NetCDF alongside the Python Jupyter Notebook framework, the data can be served to interested parties from an analysis server while still runnable on personal systems if required. This presentation illustrates the ecosystem developed by the SAGE III/ISS team, the applicability to measurements made by any limb-observing platform, and the benefit to transforming routine analyses into readily accessible dynamic plots. Frameworks currently exist at larger scales with projects such as GIOVANNI, and this illustration seeks to show that similar frameworks are accessible and possible within the local research environment while simultaneously unloading human processing cycles for more specialized analysis tasks.

Dashboarding

rcsb-api : Python Toolkit for Streamlining Access to RCSB Protein Data Bank APIs

The Protein Data Bank (PDB) was founded in 1971 as the first open-access digital data resource in biology to serve as the single global archive for three-dimensional (3D) macromolecular structure data. Current PDB holdings exceed 230,000 experimentally determined structures of proteins, nucleic acids, viruses, and macromolecular machines. The RCSB Protein Data Bank RCSB.org research-focused web portal facilitates search, analyses, and visualization of every PDB structure along with more than one million Computed Structure Models from AlphaFold DB and the ModelArchive. It is powered by a set of publicly available Application Programming Interfaces (APIs) that both support RCSB.org users and provide programmatic access to PDB data. Given the breadth and levels of granularity encompassed in this rich data collection, efficiently accessing the information programmatically may be challenging for new users. RCSB PDB has developed a Python software package, rcsb-api , that facilitates easy and efficient use of RCSB PDB APIs within a Python environment. This software tool is designed to streamline access to the extensive corpus of data housed within the PDB, enabling researchers to search, retrieve, and analyze 3D biostructure data seamlessly. Its use will accelerate research in structural biology, molecular biology and biochemistry, drug discovery, and bioinformatics by providing more efficient tools for data integration and analysis. The new toolkit is available on GitHub (github.com/rcsb/py-rcsb-api) and published to the public Python package repository (PyPI) to foster wider usage and support basic and applied research in fundamental biology, biomedicine, and the energy sciences.

FAIR principles

High-Resolution Gridded Level 3 Aerosol Optical Depth Data from MODIS

The state-of-art satellite observations of atmospheric aerosols over the last two decades from NASA's MODIS instruments have been extensively utilized in climate change and air quality research and applications. The operational algorithms now produce level 2 aerosol data at varying spatial resolutions (1, 3, and 10 km) and level 3 data at 1 degree. The local and global applications have been benefited from the coarse resolution gridded data sets (i.e., level 3, 1 degree), as it is easier to use since data volume is low and, several online and offline tools are readily available to access and analyze the data with minimal computing resources. At the same time, researchers who require data at much finer spatial scales have to go through a challenging process of obtaining, processing, and analyzing larger volumes of data sets that require high-end computing resources and coding skills. Therefore, we have created a high spatial resolution (HRG, 0.1x0.1 degree) daily and monthly aerosol optical depth (AOD) product by combining two MODIS operational algorithms, namely Deep Blue (DB) and Dark Target (DT). The new HRG AODs meets the accuracy requirements of level 2 AOD data and provide either the same or more spatial coverage on daily and monthly scales. The data sets are provided in daily and monthly files through open Ftp server with python scripts to read and map the data. The reduced data volume with an easy to use format and tools to access the data will encourage more users to utilize the data for research and applications.

aerosol

AI for Earthquake Physics

The core LANL program sponsored by Office of Science, Basic Energy Science, Chemical Sciences, Geosciences, and Biosciences (DOE-BES-CSGB) and led by PI Johnson aims to research earthquake faults to advance fault physics and earthquake hazards. All work completed is required to be made publicly available through publications and open-source codes supporting the published results. All routines are/will-be written in open source python and applied to publicly available data sets. These routines will format data from input into models, develop and test modeling frameworks for the problems addressed, and produce figures applicable to peer-reviewed manuscripts. All work is reviewed for Los Alamos Unlimited Release before submitting to a journal. This summary encompasses recently completed work and work to be complete for the duration of the program.

Johnson, Christopher

PV Degradation Modeling: Applying Geospatial Workflows with "PVDeg"

Accurate degradation modeling is essential for predicting photovoltaic (PV) module performance, estimating longevity and informing design decisions. With degradation rates varying significantly by location, geospatial analysis is critical for PV and broader applications, such as agrivoltaics, weathering and environmental data analysis. This work presents PVDeg, an open-source tool designed for geospatial degradation analysis. PVDeg integrates meteorological data from global sources, including the National Solar Radiation Database (NSRDB) and Photovoltaic Geographical Information System (PVGIS), with degradation models. The toolkit enables users to customize geospatial workflows by integrating weather data, material parameters, and user-defined Python functions. It facilitates accelerated downloads of NSRDB and PVGIS datasets and optimizes geospatial point selection to preserve data density in regions of interest. Additionally, PVDeg provides a local database for storage and spatial queries, supporting large-scale analyses without the need for high-performance computing (HPC) resources. PVDeg provides a foundational workflow that extends its utility beyond PV applications, enabling researchers to analyze geospatial processes across discipline.

14 SOLAR ENERGY

Applying Generative-AI to NASA Documentation and Processes

This research and development project leverages generative-AI to assist in the generation of software process documentation based on NASA standards. By utilizing fine-tuned AI models, the proposed system will analyze NASA's software guidelines, helping to translate them into well-structured, compliant process documents. This assistance can reduce the manual effort required to produce such documentation, enhance consistency, and assure alignment with NASA's stringent software development and operational requirements. In addition to assisting in the generation of software process documentation, the project explores how generative-AI can help create audit checklists as well as assess the compliance of NASA provider documentation against applicable NASA standards. This approach would support the compliance auditing process, providing real-time insights and assessments. The intended result will be a streamlined process, potentially including a Python-based tool and database, that improves audit efficiency, reduces human error, lowers manpower costs and required manhours, and assures continuous compliance with NASA and industry evolving standards for safety-critical software development. Future task might be to investigate the software industry approach and standards for potential collaboration.

NASA Standards

QEpsilon v0.1.0

QEpsilon is a Python package designed to minimize the effort required to build a data-driven quantum master equation of an open quantum system and to perform time evolution of the master equation. Applications of QEpsilon span from quantum computing to condensed matter systems.

Xie, Pinchen [Lawrence Berkeley National Laborator

buhito

buhito is a Python library for graph analysis and machine learning. Graphs can represent networks with objects as nodes and their relationships as edges. buhito focuses on graphlet methods that study graphs through enumerating their component subgraphs to enable interpretable and fast models of complex systems. The package provides tools for different algorithmic designs for computing, analyzing, and applying graphlets to research problems such as machine learning, data compression, and anomaly detection in graph-structured data. A central feature is performing decomposition data analysis on graphs for machine learning models. Implemented in Python and built upon open-source scientific libraries such as NetworkX, NumPy, and SciPy, buhito provides high-performance methods for researchers exploring the mathematical and computational foundations of graphlet analysis applicable to systems of different sizes.

Pimonova, Yulia

Kamodo’s Model-Agnostic Satellite Flythrough: Lowering the Utilization Barrier for Heliophysics Model Outputs

Heliophysics model outputs are increasingly accessible, but typically are not usable by the majority of the community unless directly collaborating with the relevant model developers. Prohibitive factors include complex file output formats, cryptic metadata, unspecified and often customized coordinate systems, and non-linear coordinate grids. Some pockets of progress exist, giving interfaces to various simulation outputs, but only for a small set of outputs and typically not with open-source, freely available packages. Additionally, the increasing array of tools built upon these sporadic interfaces are typically model-specific. We present Kamodo’s model-agnostic satellite flythrough capabilities as the solution to the utilization barrier for heliophysics model outputs. Developed at the Community Coordinated Modeling Center, these flythrough capabilities are built in Python upon a network of model-agnostic interfaces developed in collaboration with model developers, providing interpolation results the community can trust. Kamodo’s flythrough capabilities present the user with a growing variety of flythrough tools based upon a rapidly expanding library of heliophysics model outputs in several domains, currently including a variety of Ionosphere-Thermosphere-Mesosphere and global magnetosphere model outputs. Each capability is designed to be easily accessible via simplistic model-agnostic syntax, with the entire package freely available in the cloud on Github. Here, we describe the tools developed, include several sample applications for common science questions, demonstrate interoperability with selected packages, and summarize ongoing developments.

Software

Code associated with Publication “Analytic Solutions and Field-Scale Application for Verification of Coupled Thermo-Hydro-Mechanical Processes in Subsurface Fractured Media”

As part of a submitted paper, which is collection of previously published analytical solutions to coupled thermo-hydro-mechanical problems in subsurface flow and transport, we have prepared a collection of python scripts to compute and plot those analytic solutions. All code to be released implements existing methods; there are no novel algorithms nor any major innovations to existing software.

Hyman, Jeffrey

xesn: Echo state networks powered by Xarray and Dask

Xesn is a Python package that allows scientists to easily design Echo State Networks (ESNs) for forecasting problems. ESNs are a Recurrent Neural Network architecture introduced by Jaeger (2001) that are part of a class of techniques termed Reservoir Computing. One defining characteristic of these techniques is that all internal weights are determined by a handful of global, scalar parameters, thereby avoiding problems during backpropagation and reducing training time significantly. Because this architecture is conceptually simple, many scientists implement ESNs from scratch, leading to questions about computational performance. Xesn offers a straightforward, standard implementation of ESNs that operates efficiently on CPU and GPU hardware. The package leverages optimization tools to automate the parameter selection process, so that scientists can reduce the time finding a good architecture and focus on using ESNs for their domain application. Importantly, the package flexibly handles forecasting tasks for out-of-core, multi-dimensional datasets, eliminating the need to write parallel programming code. Xesn was initially developed to handle the problem of forecasting weather dynamics, and so it integrates naturally with Python packages that have become familiar to weather and climate scientists such as Xarray (Hoyer & Hamman, 2017). However, the software is ultimately general enough to be utilized in other domains where ESNs have been useful, such as in signal processing (Jaeger & Haas, 2004).

97 MATHEMATICS AND COMPUTING