Search NASASearch

SEARCH · Search NASA

Results for “dask”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

Accelerated Simulation of Air Pollution Using NVIDIA RAPIDS

Atmospheric chemistry models are a central tool to study and forecast the impact of air pollution on the environment, vegetation, and human health. However, the numerical simulation of chemical kinetics is computationally expensive due to the stiffness of the system of ordinary differential equations that describes atmospheric chemistry. Here we present an alternative approach to the computation of atmospheric chemistry based on machine learning. Our training data set is produced using the NASA Goddard Earth Observing System (GEOS) model with GEOS-Chem chemistry, run on the NASA Center for Climate Simulation (NCCS) Discover supercomputing cluster on 384 Intel Xeon Haswell cores. This model spends more than 50% of total run time on solving atmospheric chemistry. The data set contains as input features the air pollution concentrations before solving the differential equations, together with some key physical parameters such as temperature and sun intensity. As target variables we define the air pollution concentrations after solving the differential equations. Using Dask-cuDF and Dask-XGBoost on the NVIDIA RAPIDS platform on 8 Tesla V100 GPUs, we generate from this training set gradient boosted decision tree models that can reproduce the simulation of chemical kinetics. We do this on the NCCS Advanced Data Analytics Platform (ADAPT) science cloud environment. Our application takes full advantage of recent advances in Dask-XGBoost, such as multi-node and multi-GPU scaling for distributed training with large data sets. The increase in training data size enabled by this is critical to capture the full range of chemical environments encountered across the globe and all annual seasons.The boosted tree models offer good predictability and show many of the features of the full chemistry reference simulation. Further improvements can be achieved through mass balance considerations and by accounting for error correlations. We incorporate the boosted tree models into the GEOS reference model using XGBoost's C API. This enables a seamless integration of the GPU trained models into GEOS-Chem, which is written in Fortran and optimized for use in a massively parallel CPU environment. We show the benefits of this approach and discuss the potential speedup of this machine learning accelerated atmospheric chemistry model.

Keller, Christoph A.

PyDDA: A New Pythonic Wind Retrieval Package

PyDDA (Pythonic Direct Data Assimilation) is a new community framework aimed at wind retrievals that depends only upon utilities in the SciPy ecosystem such as scipy, numpy, and dask. It can support retrievals of winds using information from weather radar networks constrained by high resolution forecast models over grids that cover thousands of kilometers at kilometer-scale resolution. Unlike past wind retrieval packages, this package can be installed using anaconda for easy installation and, with a focus on ease of use can retrieve winds from gridded radar and model data with just a few lines of code. The package is currently available for download at https://github.com/openradar/PyDDA.

Radar

A New Architecture for Parallelization of Complex Spacecraft Trajectory Optimization Scans

This paper describes CopScanner, a new component of the Copernicus ecosystem for spacecraft trajectory design and optimization. CopScanner is a Python library being developed at the NASA JSC which enables easy parallelization of Copernicus scans. CopScanner is currently being developed and implemented for production of Copernicus trajectory scans for upcoming Artemis Missions (Artemis II and beyond). On the backend, CopScanner utilizes Dask, an open-source Python library for parallel computing which enables parallelization over both multi-core local machines and large-scale distributed computing clusters. CopScanner abstracts the trajectory scanning process into a DAG which is constructed using a chain of individual subscans. Each node in the DAG executes a python module, called the callable, for which there are built-in defaults, or users may specify their own. Support for custom callables makes CopScanner a versatile trajectory optimization software. All output files and associated metadata from a CopScanner scan are compressed and stored in a two-file output, collectively called the FileStore, consisting of a SQLite database and a compressed JSON MessagePack file, for which CopScanner provides a Python class for interaction.

Quentin Moore