Search NASA⌕ Search

SEARCH · Search NASA

Results for “Python Package”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records
All sourcesNASA NTRSDOE OSTI
NASA NTRS2023

Pypromice: A Python Package for Processing Automated Weather Station Data

The pypromice Python package is for processing and handling observation datasets from automated weather stations (AWS). It is primarily aimed at users of AWS data from the Geological Survey of Denmark and Greenland (GEUS), which collects and distributes in situ weather station observations to the cryospheric science research community. Functionality in pypromice is primarily handled using two key open-source Python packages, xarray (Hoyer & Hamman, 2017) and pandas (The pandas development team, 2020). A defined processing workflow is included in pypromice for transforming original AWS observations (Level 0, L0) to a usable, CF-convention-compliant dataset (Level 3, L3) (Figure 1). Intermediary processing levels (L1,L2) refer to key stages in the workflow, namely the conversion of variables to physical measurements and variable filtering (L1), cross-variable corrections and user-defined data flagging and fixing (L2), and derived variables (L3). Information regarding the station configuration is needed to perform the processing, such as instrument calibration coefficients and station type (one-boom tripod or two-boom mast station design, for example), which are held in a toml configuration file. Two example configuration files are provided with pypromice , which are also used in the package’s unit tests. More detailed documentation of the AWS design, instrumentation, and processing steps are described in Fausto et al. (2021).

pypromice↗
NASA NTRS2016

Polarbrdf: A General Purpose Python Package for Visualization Quantitative Analysis of Multi-Angular Remote Sensing Measurements

The Bidirectional Reflectance Distribution Function (BRDF) is a fundamental concept for characterizing the reflectance property of a surface, and helps in the analysis of remote sensing data from satellite, airborne and surface platforms. Multi-angular remote sensing measurements are required for the development and evaluation of BRDF models for improved characterization of surface properties. However, multi-angular data and the associated BRDF models are typically multidimensional involving multi-angular and multi-wavelength information. Effective visualization of such complex multidimensional measurements for different wavelength combinations is presently somewhat lacking in the literature, and could serve as a potentially useful research and teaching tool in aiding both interpretation and analysis of BRDF measurements. This article describes a newly developed software package in Python (PolarBRDF) to help visualize and analyze multi-angular data in polar and False Color Composite (FCC) forms. PolarBRDF also includes functionalities for computing important multi-angular reflectance/albedo parameters including spectral albedo, principal plane reflectance and spectral reflectance slope. Application of PolarBRDF is demonstrated using various case studies obtained from airborne multi-angular remote sensing measurements using NASA's Cloud Absorption Radiometer (CAR). Our visualization program also provides functionalities for untangling complex surface/atmosphere features embedded in pixel-based remote sensing measurements, such as the FCC imagery generation of BRDF measurements of grasslands in the presence of wild fire smoke and clouds. Furthermore, PolarBRDF also provides quantitative information of the angular distribution of scattered surface/atmosphere radiation, in the form of relevant BRDF variables such as sunglint, hotspot and scattering statistics.

CAR↗
NASA NTRSDate not supplied

Functionality of the Python Packages for the HERMES Mission

The Heliophysics Environmental and Radiation Measurement Experiment Suite (HERMES) is a set of four instruments that will fly on the Lunar Gateway, an orbital outpost which will support Artemis lunar operations. HERMES will focus on understanding the causes of space-weather variability as driven by the Sun and modulated by the magnetosphere. In this talk, we will discuss the open source approach of the HERMES Science Operation Center (SOC) team being implemented in a number of Python packages which all work together. We will describe the core package which contains Python interfaces for the loading, calibrating, plotting, validating, and saving of measurement data through Common Data Format (CDF) files making use of pycdf provided by spacepy. Each instrument also has a Python package developed using a package template and will provide specific calibration and processing functionality to each instrument. The packages make extensive use of the scientific Python ecosystem and maintain compatibility with PyHC standards. The abstraction of intricate, high heritage data formats, such as CDF files, in Python enables easier analysis and opens doors for greater participation in heliophysics science.

hermes↗
NASA NTRSDate not supplied

ncompare: A Python Package for Comparing netCDF Structures

Earth science researchers and data engineers have a common problem: they often need to compare data files to see what is different between them. A lot of time is spent developing code to test differences. When it comes to comparing multidimensional data file formats like netCDFs (Network Common Data Form), this is particularly challenging and time-consuming, since there is frequently a need to evaluate the differences between dimension sizes, variable structures, and variable attributes, especially for regression testing. Since netCDFs are widely used in Earth science — with climate models, oceanographic or atmospheric reanalyses, and observational data — improved means of evaluating netCDF files can help enable a wide range of applications. We have developed a reusable open source approach through `ncompare`, which is a Python package for comparing netCDF structures [[https://github.com/nasa/ncompare]]. The `ncompare` tool compares the structure of two Network Common Data Form (NetCDF) files at the command line. It facilitates rapid comparisons by generating a formatted display of the matching and non-matching groups, variables, and associated metadata between two NetCDF datasets. The user has the option to colorize the terminal output for ease of viewing, and `ncompare` can optionally save comparison reports in text, comma-separated value (CSV), and/or Microsoft Excel formats. Despite the availability of tools (such as ncmpidiff or nccmp) that compare the values of variables, there was not previously a readily available, Python-based tool for rapid visual comparisons of group and variable structures, attributes, and chunking. `ncompare` was developed at NASA’s Atmospheric Science Data Center (ASDC) and is a collaboration with NASA Openscapes [[https://nasa-openscapes.github.io]] mentors across 11 of NASA’s data centers. Openscapes’ overarching vision is to support scientific researchers using NASA Earthdata as they migrate their workflows to the cloud. Relevant links: - https://github.com/nasa/ncompare - https://github.com/pyOpenSci/software-submission/issues/146 - https://nasa-openscapes.github.io

Daniel Kaufman↗
NASA NTRS2019

PyDDA: A New Pythonic Wind Retrieval Package

PyDDA (Pythonic Direct Data Assimilation) is a new community framework aimed at wind retrievals that depends only upon utilities in the SciPy ecosystem such as scipy, numpy, and dask. It can support retrievals of winds using information from weather radar networks constrained by high resolution forecast models over grids that cover thousands of kilometers at kilometer-scale resolution. Unlike past wind retrieval packages, this package can be installed using anaconda for easy installation and, with a focus on ease of use can retrieve winds from gridded radar and model data with just a few lines of code. The package is currently available for download at https://github.com/openradar/PyDDA.

Radar↗
NASA NTRS2024

Xstar Atomic Database: The PyXstar Package

We present a progress report on the development of PyXstar, a Python package to manage the data (input, output, intermediate, atomic database, and model-grids) associated with the XSTAR code for treating photoionized and collisionally ionized plasmas. The PyXstar modular structure and database retrieval scheme are described, and its functionality is illustrated with Python functions and classes for performing database searches. We briefly compare PyXstar with two other Python spectrum modeling tools: PyNeb and PyAtomDB.

Claudio Mendoza↗
NASA NTRS2023

The SunPy Project: An Interoperable Ecosystem for Solar Data Analysis

The SunPy Project is a community of scientists and software developers creating an ecosystem of Python packages for solar physics. The project includes the sunpy core package as well as a set of affiliated packages. The sunpy core package provides general purpose tools to access data from different providers, read image and time series data, and transform between commonly used coordinate systems. Affiliated packages perform more specialized tasks that do not fall within the more general scope of the sunpy core package. In this article, we give a high-level overview of the SunPy Project, how it is broader than the sunpy core package, and how the project curates and fosters the affiliated package system. We demonstrate how components of the SunPy ecosystem, including sunpy and several affiliated packages, work together to enable multi-instrument data analysis workflows. We also describe members of the SunPy Project and how the project interacts with the wider solar physics and scientific Python communities. Finally, we discuss the future direction and priorities of the SunPy Project.

Solar physics↗
NASA NTRSDate not supplied

Simplifying Analysis of Hierarchical HDF5 and NetCDF4 Files with Xarray-Datatree

NASA’s Earth Observing System Data and Information System (EOSDIS) contains thousands of Earth science datasets from satellites, models, and field campaigns. EOSDIS data are stored in formats that are well supported by the Earth Science community. These formats include the Hierarchical Data Format (HDF), with derivative flavors such as HDF-5 and the Network Common Data Format (NetCDF-4). The HDF specification allows for a directory-like hierarchy within a single file, known as "groups". Observational data and associated metadata within a single file can be distributed amongst multiple internal groups, which can also be nested to multiple levels. Working with datasets that have a group hierarchical structure can be difficult because of the nested structure of groups. Widely used packages, such as xarray, have data models that do not accommodate the hierarchical structure within HDF files, requiring users to traverse the file and open different HDF groups as separate, unrelated objects. Xarray-datatree is a Python package developed to solve the difficulty of traversing HDFs with a hierarchical group structure by creating a tree-like hierarchical data structure in xarray. The tree-like structure allows each group to be accessed once a DataTree object is instantiated. The migration of xarray-datatree into the xarray core library will reduce barriers to accessing Earth science data by eliminating the need to understand and traverse the specific hierarchy of a grouped HDF file.

Eni Awowale↗
NASA NTRS2023

BatAnalysis - A Comprehensive Python Pipeline for Swift BAT Survey Analysis

The Swift Burst Alert Telescope (BAT) is a coded aperture gamma-ray instrument with a large field of view that primarily operates in survey mode when it is not triggering on transient events. The survey data consists of eighty-channel detector plane histograms that accumulate photon counts over time periods of at least 5 minutes. These histograms are processed on the ground and are used to produce the survey dataset between 14 and 195 keV. Survey data comprises >90% of all BAT data by volume and allows for the tracking of long term light curves and spectral properties of cataloged and uncataloged hard X-ray sources. Until now, the survey dataset has not been used to its full potential due to the complexity associated with its analysis and the lack of easily usable pipelines. Here, we introduce the BatAnalysis python package , a wrapper for HEASoftpy, which provides a modern, open-source pipeline to process and analyze BAT survey data. BatAnalysis allows members of the community to use BAT survey data in more advanced analyses of astrophysical sources including pulsars, pulsar wind nebula, active galactic nuclei, and other known/unknown transient events that may be detected in the hard X-ray band. We outline the steps taken by the python code and exemplify its usefulness and accuracy by analyzing survey data from the Crab Pulsar, NGC 2992, and a previously uncataloged MAXI Transient. The BatAnalysis package allows for ∼ 18 years of BAT survey to be used in a systematic way to study a large variety of astrophysical sources.

Tyler Parsotan↗
NASA NTRS2021

Landslide Mapping Using Object-Based Image Analysis and Open Source Tools

Availability of high-resolution optical imagery and advances in image processing technologies have significantly improved our ability to map landslides. In recent years object-based image analysis (OBIA) has been gaining in popularity for landslide mapping due to its ability to incorporate spectral, textural, morphological and topographical properties. Many studies have been conducted based on commercial software. In this study, we create an open source Semi-Automatic Landslide Detection (SALaD) system utilizing OBIA and machine learning. Configured to run in Linux environment, it uses various opensource Python packages and modules. This system was tested in 575 km2 area along the Pasang Lhamu Highway, Nepal where large numbers of landslides were triggered by the 2015 Gorkha earthquake. Comparison with a manual inventory highlighted that this system was able to detect 70% of the landslide area. The speed and efficiency with which this system was able to detect landslides makes it a viable alternative to manual techniques for landslide mapping over large areas, when establishing approximate landslide locations is of prime importance.

Open source↗
NASA NTRSDate not supplied

The Satellite Flythrough and Reconstruction Softwares at CCMC

The next-generation of model data visualization to be offered at the Community Coordinated Modeling Center (CCMC) will be based on Kamodo, an open source python package. To increase the usefulness of our services, we are also developing new capabilities based on this software, called the satellite flythrough and the reconstruction tools, to be offered both as packages for offline analysis and through an online interface (coming soon). The satellite flythrough software ‘flies’ a satellite trajectory, whether real or imaginary, through either model data hosted at CCMC or on a personal machine. This service greatly simplifies the complexity of users’ access to model data, abstracting away the time-consuming details of model data formats and interpolation. We demonstrate execution times of a few seconds to a few minutes for several example flythroughs of a trajectory stretching over a few days, depending on the parameters chosen. We also demonstrate a reconstruction tool built on top of the satellite flythrough software, for use with mission planning and model-data comparisons. This tool, based on reconstructions provided for the GDC Science and Technology Definition Team, converts multiple, simultaneous satellite flythroughs into two-dimensional reconstructions. The reconstruction tool provides a software capability for satellite constellations to determine how many satellites are needed and in what configuration to resolve the desired features in the model data. Both tools are currently available through GitHub for a selection of CCMC-hosted ITM models. Finally, we present initial results from work in progress and plans for future work, including an expansion of the reconstruction tool to provide 3D reconstruction capabilities and a line-of-sight calculation tool.

software, python↗
NASA NTRS2017

PandExo: A Community Tool for Transiting Exoplanet Science with JWST and HST

As we approach the James Webb Space Telescope (JWST) era, several studies have emerged that aim to (1) characterize how the instruments will perform and (2) determine what atmospheric spectral features could theoretically be detected using transmission and emission spectroscopy. To some degree, all these studies have relied on modeling of JWST's theoretical instrument noise. With under two years left until launch, it is imperative that the exoplanet community begins to digest and integrate these studies into their observing plans, as well as think about how to leverage the Hubble Space Telescope (HST) to optimize JWST observations. To encourage this and to allow all members of the community access to JWST & HST noise simulations, we present here an open-source Python package and online interface for creating observation simulations of all observatory-supported timeseries spectroscopy modes. This noise simulator, called PandExo, relies on some aspects of Space Telescope Science Institute's Exposure Time Calculator, Pandeia. We describe PandExo and the formalism for computing noise sources for JWST. Then we benchmark PandExoʼs performance against each instrument team's independently written noise simulator for JWST, and previous observations for HST. We find that PandExo is within 10% agreement for HST/WFC3 and for all JWST instruments.

Batalha, Natasha E.↗
NASA NTRS2019

Citizen Science Twitter Data Management for Earth Science Applications

Social media data can provide useful real-time and historical information relating to the natural world, but managing this data poses challenges. Scientists at GES DISC are exploring the potential of Twitter data to augment precipitation data from the Global Precipitation Measurement (GPM) mission. However, the format of Twitter data is unconventional in the context of NASA data centers, resulting in frustration for scientists who need to work with the data. This study investigated procedures and standards needed to properly manage Twitter data to make them compatible with these data centers. After comparing databases, the study found that the MongoDB database was best suited for the storage of raw Twitter data due to its flexibility, ability to be accessed by multiple users, and querying functionality. The study used the Python package Zarr to transform processed Twitter data into a gridded format similar to that of satellite data. Each Tweet was mapped onto a time-space grid; each grid location contained information about Tweet attributes and precipitation. The study developed a pipeline for downloading, storing, and gridding Twitter data and transformed Twitter data into an understandable format for users of NASA satellite data.

Li, Rachel↗