Search NASA⌕ Search

SEARCH · Search NASA

Results for “Jupyter”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records
All sourcesNASA NTRSDOE OSTI
NASA NTRS2020

A Jupyter Notebook Environment For Multibody Dynamics

DARTS is a rigid/flexible multibody dynamics toolkit for themodeling and simulation of aerospace and robotic vehicles forengineering applications. In this paper we describe an on-line,browser-based environment using Jupyter notebooks to supporttraining needs for the DARTS software. The suite of curated tutorial notebooks is organized into different topic areas, and intomultiple themes within each topic area. The notebooks within atheme use a progression of examples for users to expand theirunderstanding of the software. The topic areas include one onthe DARTS multibody dynamics software and another one on thetheory underlying the multibody dynamics formulation. We alsodescribe a number of Jupyter extensions that were used - andsome developed in house - to enhance the notebook interface foruse with the dynamics simulation software. One significant extension we implemented allows the embedding of live 3D visualizations within simulation notebooks.

Gaut, Aaron↗
NASA NTRSDate not supplied

SAGE III/ISS Rapid Data Analysis Through Dashboarding with Jupyter Notebooks

Spaceborne remote sensing observations of Earth’s atmosphere produce significant quantities of data over the life of each mission. In the case of the Stratospheric Aerosol and Gas Experiment III on the International Space Station (SAGE III/ISS) nearly four years of vertical profiles of atmospheric ozone, water vapor, and nitrogen dioxide concentrations as well as aerosol extinction coefficients have been released. The dichotomy of the desire for both long-term trends in the atmospheric state alongside the assessment of short-term impacts of major disruptive events such as volcanic eruptions and pyrocumulus injections requires agile tools to handle these cases in near real-time as new data are produced. The analysis landscape is further complicated by the desire to compare results between the numerous contemporary observations available for a given dataset. The SAGE III/ISS team has developed a suite of tools leveraging modern web-based frameworks allowing members to interact with a dashboard-style interface to load the data record, assess new profiles as they are generated and in ensemble, compare between species, and additionally add in measurements observed by other platforms as necessary. Leveraging a commonly packaged data format of NetCDF alongside the Python Jupyter Notebook framework, the data can be served to interested parties from an analysis server while still runnable on personal systems if required. This presentation illustrates the ecosystem developed by the SAGE III/ISS team, the applicability to measurements made by any limb-observing platform, and the benefit to transforming routine analyses into readily accessible dynamic plots. Frameworks currently exist at larger scales with projects such as GIOVANNI, and this illustration seeks to show that similar frameworks are accessible and possible within the local research environment while simultaneously unloading human processing cycles for more specialized analysis tasks.

Dashboarding↗
NASA NTRS2017

GES DISC Data Recipes in Jupyter Notebooks

The Earth Science Data and Information System (ESDIS) Project manages twelve Distributed Active Archive Centers (DAACs) which are geographically dispersed across the United States. The DAACs are responsible for ingesting, processing, archiving, and distributing Earth science data produced from various sources (satellites, aircraft, field measurements, etc.). In response to projections of an exponential increase in data production, there has been a recent effort to prototype various DAAC activities in the cloud computing environment. This, in turn, led to the creation of an initiative, called the Cloud Analysis Toolkit to Enable Earth Science (CATEES), to develop a Python software package in order to transition Earth science data processing to the cloud. This project, in particular, supports CATEES and has two primary goals. One, to transition data recipes created by the Goddard Earth Science Data and Information Service Center (GES DISC) into an interactive and educational environment using JupyterNotebooks. Two, to acclimate Earth scientists to cloud computing. To accomplish these goals, we create JupyterNotebooks to compartmentalize the different steps of data analysis and help users obtain and parse data from the command line. We also develop a Docker container, comprised of Jupyter Notebooks, Python dependencies, and command line tools, and configure it into an easy-to-deploy package. The end result is an end-to-end product that simulates the use case of end users working in the cloud computing environment.

discoverability↗
NASA NTRS2020

Connecting Users and Applications with Po.daac Hosted GHRSST Data

The 80+ GHRSST public datasets represent a rich resource for sea surface temperature research and applications given their time series length, resolution, spatial coverage, varying measurement types and processing levels, and availability in the full spectrum of PO.DAAC tools and services ecosystem. The PO.DAAC has created a publicly accessible recipe suite for the user community to perform straightforward yet powerful computations on GHRSST data using python recipes, Jupyter notebooks, R, Matlab, and the NCO programming language. These recipes include numerical computations for regional and global SST trends, anomaly derivations, EOF analysis, climate signal reproduction, and ocean phenology. For example, one recipe reproduces a famous SST based warming figure from the Fourth National Climate Assessment (USA) while another focuses on quantifying the regional changes in ocean SST phenology. Most are python-based while some contain hybrid calls and leverage the NCO programming interface too. All are available on the PO.DAAC user forum (https://podaac.jpl.nasa.gov/forum/) and/or via the open source NASA GitHub repository (https://github.com/nasa/podaac_tools_and_services). Several are available in the Jupyter notebook framework including podaacypy (https://github.com/nasa/podaacpy), a recipe for GHRSST granule metadata discovery and application, and more recently a Jupyter notebook developed to support data analysis and visualization of a cloud-based Zarr formatted Level 4 MUR dataset in the AWS Open Data Registry. Throughout the summer of 2020, the PO.DAAC intends to add and migrate more of its numerical recipes to the Jupyter notebook framework and publish them on its open source GitHub repository.

Gentemann, Chelle↗
NASA NTRSDate not supplied

A Novel Architecture of JupyterHub on Amazon Elastic Kubernetes Service for Open Data Cube Sandbox

The Open Data Cube (ODC) initiative, with support from the Committee on Earth Observation Satellites (CEOS) System Engineering Office (SEO) has developed a state-of-the-art suite of software tools and products to facilitate the analysis of Earth Observation data. This paper presents a short summary of our novel architecture approach in a project related to the Open Data Cube (ODC) community that provides users with their own ODC sandbox environment. Users can have a sandbox environment all to themselves for the purpose of running Jupyter notebooks that leverage the ODC. This novel architecture layout will remove the necessity of hosting multiple users on a single Jupyter notebook server and provides better management tooling for handling resource usage. In this new layout each user will have their own credentials which will give them access to a personal Jupyter notebook server with access to a fully deployed ODC environment enabling exploration of solutions to problems that can be supported by Earth observation data.

Open Data Cube↗
NASA NTRS2022

Automating Surface Attitude Positioning and Pointing Operations for Mars 2020

The Surface Attitude Positioning and Pointing (SAPP) subsystem of the Mars Perseverance rover keeps track of the rover’s position and attitude on the surface of Mars. The SAPP Downlink Engineering Operations team members receive data from the rover on a daily basis. They must interpret the data to make sure the rover is staying safe and to support uplink planning. The SAPP team keeps track of the error growth in the rover’s attitude estimate due to noise in the Rover Inertial Measurement Unit’s (RIMU) gyroscopes used to propagate that attitude estimate whenever the rover is moving. Whenever this error grows to a particular threshold, SAPP is responsible for updating the onboard attitude knowledge using the RIMU’s accelerometers to estimate rover roll and pitch and sun imaging to estimate rover yaw, thereby reducing this attitude estimation error. Accurate attitude estimation is required so that the rover can successfully point its High Gain Antenna (HGA) to receive information from Earth and as a backup to the Mars orbiters used for sending data from the rover to Earth, point instruments on its Remote Sensing Mast (RSM), and support safe movement and placement of instruments by the rover’s ARM relative to the Martian surface. The Mars 2020 Engineering Operations team has been working to increase the operational efficiency of the mission and eventually move to a five-hour timeline for daily operations. In pursuit of this goal, the SAPP Engineering Operations team has automated their downlink process by developing a centralized Jupyter notebook to analyze the data received daily from the rover. The SAPP downlink Jupyter notebook automatically collects the data relevant to the SAPP subsystem and visualizes this information in plots and tables that can be easily read by downlink operators to aid them in assessing the status of the subsystem. Various Application Programming Interfaces (APIs) have been incorporated into the downlink daily notebook to automate the collection and posting of data, such as gathering and posting data products to the cloud. The SAPP team has also developed a SAPP downlink software library that includes functions to aid the notebook in processing data. In addition to assessing the SAPP subsystem on a daily basis, operators need to assess the long-term trending behavior of the subsystem over time. An automated trending process has been developed to collect information from the daily notebooks in order to plot and analyze that data in a centralized place. These daily and trending processes have expedited the SAPP downlink assessment and laid the groundwork to completely automate the SAPP downlink process so that SAPP operators are unnecessary unless something unexpected occurs. This paper will provide an overview of the functions that the SAPP subsystem carries out on a daily basis, and will then dive into the automations that have been developed for daily and trending downlink assessment. An assessment of the downlink efficiency will be provided, along with a summary of lessons learned and work to go. Finally, the authors will discuss how these types of automated spacecraft health assessments could be more broadly used within mission operations.

Zarifian, Anais↗
NASA NTRSDate not supplied

Open-source Numerical Modeling of Solidification Cracking Susceptibility: Application to Refractory Alloy Systems

Introduction. Alloys such as aluminum, nickel-base, and austenitic stainless steels are susceptible to solidification cracking during welding and 3D printing. Compositional optimization is one method used to effectively mitigate solidification cracking of those alloy systems. With the surge in hypersonic and in-space propulsion activities, refractory metals (Nb, Mo, Ta, W, and Re) and their alloy derivatives are increasing in importance due to their extreme high melting point and retention of high-temperature strength; however, their chemistry was most typically optimized to promote ductility during mechanical operations such as drawing and forming. Welding of such alloys has been a challenge due to a number of issues including solidification cracking, atmospheric contamination (O, C, and N), as well as a shift in ductile-to-brittle transition to higher temperature following grain growth induced by welding. Compositional optimization of refractory alloys for solidification cracking resistance in particular is desirable as their usage increases with the advent of advanced manufacturing methods such as 3D printing. This work evaluates the effect of compositional variation in refractory metal systems on the solidification cracking susceptibility with the goals of optimizing existing alloys and joining process techniques, and formulating new alloys with increased solidification cracking resistance. Experimental Procedures. A python code was developed in a Jupyter notebook environment (Michael and Sowards, 2023) to facilitate the calculation of crack susceptibility index proposed by Kou (2015). Composition is entered as a single point, or as a 1-D or 2-D array. The notebook calls pycalphad (Otis and Liu, 2017 and Bocklund et al, 2020) to calculate the evolution of fraction solid as a function of temperature (under either Scheil or equilibrium assumptions) and then evaluates steepness of the fraction solid curve near the terminal stage of solidification to predict solidification cracking resistance. Open source thermodynamic databases available at online repositories are used (van de Walle). The process is setup in an automated fashion to generate plots that show variation in solidification cracking susceptibility according to composition on 1-D line plots or 2-D contour plots. The Jupyter notebook and crack susceptibility algorithm was also integrated with a widely used commercial CALPHAD code for validation and alloy exploration. Results and Discussion. The crack susceptibility model was first validated against a series of refractory alloy compositions evaluated in past work which utilized a specialized Varestraint test built inside a vacuum chamber environment (Lessman and Gold, 1971). The alloys tested in the Varestraint apparatus included T-111 (Ta-8W-2Hf), ASTAR-811C (Ta-8W-1Re-0.7Hf-0.025C), FS-85 (Nb-27Ta-10W-1Zr), T-222 (Ta-9.6W-2.4Hf-0.01C), Ta-10W, B-66 (Nb-5Mo-5V-1Zr), and SCb-291 (Nb-10W-10Ta). The initial test of the model showed a strong correlation with empirical Varestraint data, i.e., a Spearman rank correlation between model predictions and hot cracking measurements was observed to be greater than 0.8. Following the validation, a set of refractory metal binary mixtures was investigated to evaluate sensitivity of Nb, Mo, W, and Ta to C, N, and O content. A series of plots were produced that suggest ppmw ranges of C, N, and O where solidification cracking increases significantly and reaches a maximum. Also comparative ranking of each primary refractory metal to each interstitial was produced. For example C produces greater cracking response in Mo whereas O produces greater cracking response in Ta and Nb. Such compositional values have utility in setting limits on pickup of these interstitial elements during welding and printing rather than using a one-size-fits-all approach. Furthermore, the results have use in determining additive powder recycling requirements, which is especially pertinent for refractory metal powders due to their high cost compared to conventional alloys. Another application created thousands of hypothetical alloys within the nominal specified composition range of two widely used refractory alloys C103 (Nb-10Hf-1Ti) and TZM (Mo-0.5Ti-0.1Zr). The cracking index was calculated for the alloys and results were fed into machine learning regression techniques including Multiple Linear Regression, Ridge Regression, and Lasso Regression to determine relative potency each alloying element had on computed solidification cracking index. A series of linear equations were produced that relate composition of C103 and TZM to solidification cracking index. The crack susceptibility of C103 for example is described by an equation of the form: cracking index ~ O + 0.667*C + 0.635*N + 0.00037*Ta – 0.0008*Hf (in wt.%) From that equation, it is clear that O has strong propensity to induce solidification cracking. Interestingly, Hf is shown to reduce calculated cracking response. Finally, realizing the potential of this method to discover new refractory alloy formulations across the period table that have low solidification cracking sensitivity, the code was applied to new untested alloy systems including W-Zr-C, W-Ta-C, and others. Conclusions. In summary, an open source numerical method has been developed using Python code to calculate Kou’s crack susceptibility index. The method was applied to refractory metals which are inherently difficult to study from a weldability testing standpoint since inert shielding gas is not sufficient and welding is typically done in vacuum, especially in light of findings presented here where oxygen has profound influence on solidification cracking. This work revealed the effect of compositional variations on a series of refractory metals and showed the framework defined here will be useful in 1) the development of new alloys that have improved weldability and 3D printability, 2) placing compositional limits on existing alloys, and 3) ensuring adequate controls of manufacturing processes such as 3D printing where powder reuse is critical. Keywords. pycalphad; Python; refractory metals; solidification cracking. References. B. Bocklund et. al. (2020) http://doi.org/10.5281/zenodo.3630657. S. Kou. (2015) https://doi.org/10.1016/j.actamat.2015.01.034. G.G. Lessmann and R.E. Gold. Welding Journal, issue 1, pp. 1-s – 8-s (1971). F.N. Michael and J.W. Sowards. NASA/TM-20230002218 (2023). R. Otis and Z.-K. Liu. (2017) http://doi.org/10.5334/jors.140. A. Van de Wallle et. al. (2018) https://doi.org/10.1016/j.calphad.2018.04.003.

pycalphad↗
NASA NTRS2025

PACE Water Resources: Demonstrating the Use of NASA's PACE Hyperspectral Ocean Color Instrument Data for Enhanced Coastal Management

This project developed tools to support the future use of Plankton, Aerosol, Cloud, ocean Ecosystem (PACE) hyperspectral imagery in water resource monitoring and research by NASA DEVELOP teams and members of the PACE applications community. We sought to address a need for support in processing and visualizing hyperspectral PACE Ocean Color Instrument (OCI) data among researchers and decision-makers working in coastal water quality management and harmful algal bloom (HAB) monitoring. To supplement the day of simulated PACE imagery available, we used Aqua MODIS earth observations with Level 3 processing from March 2022 to build a Python graphical user interface (GUI) for visualizing ocean biogeochemical parameters relevant to the early detection and monitoring of HABs. We used simulated PACE OCI Level 2 data derived from the Python Top of Atmosphere Simulation Tool (PyTOAST) to build Jupyter Notebooks for band subset and selection. The Level 3 PACE Viewer components support users with quick visualizations as well as the creation of geoTIFFs and time-series. The Level 2 Jupyter Notebooks address users’ concerns over the volume and complexity of hyperspectral imagery. The PACE Viewer is useful for visual inspection and netCDF data processing but should not be used for geospatial analysis. Once PACE launches, this tool will alleviate the technical burdens of working with hyperspectral data and support the early detection and monitoring of HABs using PACE satellite imagery.

Python Top of Atmosphere Simulation Tool↗
NASA NTRS2017

Engine Icing Data - An Analytics Approach

Engine icing researchers at the NASA Glenn Research Center use the Escort data acquisition system in the Propulsion Systems Laboratory (PSL) to generate and collect a tremendous amount of data every day. Currently these researchers spend countless hours processing and formatting their data, selecting important variables, and plotting relationships between variables, all by hand, generally analyzing data in a spreadsheet-style program (such as Microsoft Excel). Though spreadsheet-style analysis is familiar and intuitive to many, processing data in spreadsheets is often unreproducible and small mistakes are easily overlooked. Spreadsheet-style analysis is also time inefficient. The same formatting, processing, and plotting procedure has to be repeated for every dataset, which leads to researchers performing the same tedious data munging process over and over instead of making discoveries within their data. This paper documents a data analysis tool written in Python hosted in a Jupyter notebook that vastly simplifies the analysis process. From the file path of any folder containing time series datasets, this tool batch loads every dataset in the folder, processes the datasets in parallel, and ingests them into a widget where users can search for and interactively plot subsets of columns in a number of ways with a click of a button, easily and intuitively comparing their data and discovering interesting dynamics. Furthermore, comparing variables across data sets and integrating video data (while extremely difficult with spreadsheet-style programs) is quite simplified in this tool. This tool has also gathered interest outside the engine icing branch, and will be used by researchers across NASA Glenn Research Center. This project exemplifies the enormous benefit of automating data processing, analysis, and visualization, and will help researchers move from raw data to insight in a much smaller time frame.

Engine Icing↗
NASA NTRS2019

OPeNDAP Clients, Aggregation and S3

In this talk, we will discuss our work for testing OPeNDAP client access of data stored in the Amazon S3 cloud storage using a set of common analysis tools including Panoply, Jupyter Notebooks with Python xarray, NCO command line tool package, ArcGIS, and GDAL. We will also discuss our ongoing work on improving performance in Hyrax aggregation functionality.

Amazon S3↗
NASA NTRS2019

GeoNEX: A Cloud Gateway for Near Real-time Processing of Geostationary Satellite Products

The emergence of a new generation of geostationary satellite sensors provides land andatmosphere monitoring capabilities similar to MODIS and VIIRS with far greater temporal resolution (5-15 minutes). However, processing such large volume, highly dynamic datasets requires computing capabilities that (1) better support data access and knowledge discovery for scientists; (2) provide resources to enable real-time processing for emergency response (wildfire, smoke, dust, etc.); and (3) provide reliable and scalable services for the broader user community. This paper presents an implementation of GeoNEX (Geostationary NASA-NOAA Earth Exchange) services that integrate scientific algorithms with Amazon Web Services (AWS) to provide near realtime monitoring (~5 minute latency) capability in a hybrid cloud-computing environment. It offers a user-friendly, manageable and extendable interface and benefits from the scalability provided by Amazon Web Services. Four use cases are presented to illustrate how to (1) search and access geostationary data; (2) configure computing infrastructure to enable near real-time processing; (3) disseminate and utilize research results, visualizations, and animations to concurrent users; and (4) use a Jupyter Notebook-like interface for data exploration and rapid prototyping. As an example of (3), the Wildfire Automated Biomass Burning Algorithm (WF_ABBA) was implemented on GOES-16 and -17 data to produce an active fire map every 5 minutes over the conterminous US. Details of the implementation strategies, architectures, and challenges of the use cases are discussed.

GeoNEX↗
NASA NTRS2019

TPSAS-NF1676L-32493-DND

The Committee on Earth Observation Satellites (CEOS) System Engineering Office (SEO) has supported the Open Data Cube (ODC) initiative to provide a data architecture solution that has value to its global users and increases the impact of EO satellite data. ODC is an open-source platform for processing satellite data. We have developed software products and tools around the core ODC that would help users perform machine learning on EO satellite data. The recent United Nations (UN) Sustainable Development Agenda provides a shared blueprint for peace and prosperity for people and for the planet, considering our current situation and helping to create a plan. The core of this agenda is a set of seventeen Sustainable Development Goals (SDGs), which represent an urgent call for action by all countries - both developed and developing - in a global partnership. The CEOS SEO team has recently developed and released a set of innovative Jupyter notebooks addressing UN SDGs 6.6.1 (spatial extents of water-related ecosystems), 11.3.1 (ratio of land consumption rate to population growth rate), and 15.3.1 (proportion of land that is degraded over total land area). These notebooks empower users by providing features that will assist with streamlining analysis ready data retrieval, processing, and visualization. We have recently incorporated several machine learning techniques in these notebooks. In this paper, we present the lessons learned from our experience on classifying land using supervised and unsupervised machine learning techniques using ODC framework for UN SDGs. We identify the current limitations of ODC to seamlessly support machine learning techniques. We propose features that would help machine learning, specifically within the ODC framework. We propose a thematic indexing/loading of data for both unsupervised learning as well as data annotation/labeling pipeline. Currently, ODC supports machine learning by separating data-management from the analysis process. It works as a mechanism to load cubes of data. ODC does not natively support features that are vital in machine learning such as validation splits, fair/balanced sampling, establishing load size constraints, etc. We believe that our proposed features will empower users by providing features that bring machine learning techniques closed to ODC. Enhancements to ODC to better accommodate machine learning techniques can assist in fulfilling UN SDGs such as 6.3.2, 6.4.2, 6.6.1, 11.3.1, 14.1.1, 15.1.1, 15.3.1, and 15.4.2.

Syed R Rizvi↗