Search NASA⌕ Search

SEARCH · Search NASA

Results for “numpy”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 55 records · Page 3
All sourcesNASA NTRSDOE OSTI
DOE OSTI2021

The Polar System Analysis Package (Ver. 1 Specifications)

The Polar System Analysis Package (POLeSTAr) is a modern data fusion tool being designed to benchmark,evaluate and compare the polar physics and biogeochemistry of Earth System Models (ESMs). It is being written in Python to make use of libraries including Xarray, Matplotlib, SciPy, NumPy and PyTorch within algorithms developed specifically for polar system modeling. Initial POLeSTAr algorithms will be adapted from the MATLAB Ridgepack package that was designed to analyze the Model for Prediction Across Scales (MPAS) sea ice component of the Energy Exascale Earth System Model (E3SM) and sea ice model CICE within the Community Earth System Model (CESM).

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Guide to runquic.py Python Script

The runquic.py Python script uses Python 3.8 or newer and requires several Python packages to be installed including: NumPy, SciPy, Pandas, Geopandas, and Shapely. The script assumes that the batch QUIC of project folders with all of the input files (except for any input files interpolated from WRF output files) have already be saved to the drive and are ready to be run. If the project is set to use WRF model output, it will be interpolated at runtime.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Fast Semi-automated Filtration Method for Non-targeted LC-QTOF Data of Aged Nitroplasticizer Samples

A full dataset of aged nitroplasticizer (NP) is composed of more than 2000 unique mass-to-charges (m/z) when combining the non-targeted data obtained from both positive and negative electrospray ionization modes in time-of-flight mass spectrometry. Therefore, manual processing of these data often takes days, weeks, or even months to scrutinize for mechanistic insights. To effectively extract meaningful signals that represent vital degradation intermediates in the early NP degradation mechanism, a semi-automated postprocessing workflow for data filtering, tailored to the aging experiment of NP, has been developed. The automated portion of this workflow is written in a Python code (using pandas, numpy, and matplotlib libraries), which removes more than 65% of potential false signals within seconds via four threshold-based adjustable filters: signal sensitivity, coefficient of variation, number of measurements, and retention time variability. As for the manual portion, a pattern-based inspection method is employed to reduce another 23% or more false positives, which greatly simplifies data visualization and results in less than 3% of potential candidate m/z needing in-depth data interpretation. As a positive control, known compounds are verified. Using this semi-automated data reduction method, the amount of time required is reduced to a matter of hours for data filtering in the non-targeted datasets of aged NP, which saves more time and effort for compound identification.

36 MATERIALS SCIENCE↗
DOE OSTI2021

Feature Extraction: Improving Remote Sensor Classification of Non-Proliferation

This research focuses on developing algorithms for nuclear non-proliferation detection using remote sensor modeling. To improve the performance of classification models, we implemented a data pipeline with feature extraction. This pipeline takes raw data and transforms it into smaller data points called features that still describe the model. Improving this classification works towards the departments of energy’s missions of ensuring American’s security and prosperity by creating technology that addresses nuclear challenges. To conduct this analysis, we used the Python programming language and some key packages, including tsfresh and TSFEL. Originally tsfresh was selected because it has the most statistical features out of all the packages. Later TSFEL was incorporated due to the additional features it can extract from data, such as temporal and spectral. However, feature extraction becomes challenging in the presence of missing values. In this case, two additional Python packages were added to our workflow, NumPy and pandas, allowing for the feature extraction process to handle unknown values. Our data pipeline was tested on data collected from a simulation that describes the process state of a physical example. The results show the pipeline’s capability to consume and extract a total 17 features from tabular data. Future work includes producing classifications using decision tree-based models such as XGBoost and improving data collection by analyzing feature importance.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2025

Improving Runtime Performance of Tensor Computations using Rust From Python

In this work, we investigate improving the runtime performance of key computational kernels in the Python Tensor Toolbox (pyttb), a package for analyzing tensor data across a wide variety of applications. Recent runtime performance improvements have been demonstrated using Rust, a compiled language, from Python via extension modules leveraging the Python C API—e.g., web applications, data parsing, data validation, etc. Using this same approach, we study the runtime performance of key tensor kernels of increasing complexity, from simple kernels involving sums of products over data accessed through single and nested loops to more advanced tensor multiplication kernels that are key in low-rank tensor decomposition and tensor regression algorithms. In numerical experiments involving synthetically generated tensor data of various sizes and these tensor kernels, we demonstrate consistent improvements in runtime performance when using Rust from Python over 1) using Python alone, 2) using Python and the Numba just-in-time Python compiler (for loop-based kernels), and 3) using the NumPy Python package for scientific computing (for pyttb kernels).

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Comparing the Performance of Julia on CPUs versus GPUs and Julia-MPI versus Fortran-MPI: a case study with MPAS-Ocean (Version 7.1)

Abstract. Some programming languages are easy to develop at the cost of slow execution, while others are fast at runtime but much more difficult to write. Julia is a programming language that aims to be the best of both worlds – a development and production language at the same time. To test Julia's utility in scientific high-performance computing (HPC), we built an unstructured-mesh shallow water model in Julia and compared it against an established Fortran-MPI ocean model, the Model for Prediction Across Scales–Ocean (MPAS-Ocean), as well as a Python shallow water code. Three versions of the Julia shallow water code were created: for single-core CPU, graphics processing unit (GPU), and Message Passing Interface (MPI) CPU clusters. Comparing identical simulations revealed that our first version of the Julia model was 13 times faster than Python using NumPy, where both used an unthreaded single-core CPU. Further Julia optimizations, including static typing and removing implicit memory allocations, provided an additional 10–20× speed-up of the single-core CPU Julia model. The GPU-accelerated Julia code was almost identical in terms of performance to the MPI parallelized code on 64 processes, an unexpected result for such different architectures. Parallelized Julia-MPI performance was identical to Fortran-MPI MPAS-Ocean for low processor counts and ranges from 2× faster to 2× slower for higher processor counts. Our experience is that Julia development is fast and convenient for prototyping but that Julia requires further investment and expertise to be competitive with compiled codes. We provide advice on Julia code optimization for HPC systems.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2025

Dataset for "Climatic and socioeconomic drivers of water use and their spatio-temporal patterns for small and mid-sized cities in the Contiguous United States"

This dataset contains all code for calibrating and analyzing machine learning models for "Climatic and socioeconomic drivers of water use and their spatio-temporal patterns for small and mid-sized cities in the Contiguous United States". Please unzip the folders and follow the instructions from 'README.txt'. Required python modulessklearn=1.2.2numpy=1.23.3xgboost=2.0.2joblib=1.2.0 Required R libraryshapFlex:devtools::install_github("nredell/shapFlex")library(shapFlex)

Dave, Hari [Civil and Environmental Engineering De↗
DOE OSTI2026

rustpix

rustpix is a high-performance, open-source Rust library with first-class Python bindings (via PyO3) for processing pixel-detector data in neutron imaging. It targets time-stamping detectors such as Timepix3 (TPX3) at ORNL's Spallation Neutron Source (VENUS beamline), where each detected neutron deposits charge across a cluster of pixels within a very high-rate event stream (96M+ hits/sec). rustpix parses TPX3 event data in parallel using memory-mapped I/O, offers four interchangeable clustering algorithms (ABS adjacency-based search, DBSCAN, graph/union-find connected components, and a parallel grid method), and extracts weighted, super-resolved centroids to produce neutron-event lists. A streaming architecture lets it process files larger than available memory. rustpix is distributed as a pip-installable Python package (with NumPy integration), Rust crates, a command-line tool, and an interactive GUI; it writes HDF5, Apache Arrow, and CSV; and it is designed to extend to TPX4 and other detector types. Released as open-source under the MIT License.

Zhang, Chen [Oak Ridge National Laboratory (ORNL),↗
DOE OSTI2026

Pythia8 Quark and Gluon Jets (float32)

A float32 (single-precision) version of the quark and gluon jet dataset originally published by Komiske, Metodiev, and Thaler (Zenodo record 3164691). Only the 20-file subset without charm and bottom quark jets is included here. All simulation parameters and jet selection criteria are identical to the original: Pythia 8.226, √s = 14 TeV Quarks from WeakBosonAndParton:qg2gmZq, gluons from WeakBosonAndParton:qqbar2gmZg with the Z decaying to neutrinos FastJet 3.3.0, anti-k_t jets with R = 0.4 p_T^jet ∈ [500, 550] GeV, |y^jet| < 1.7 There are 20 files, each in compressed NumPy format (QG_jets_fp32_0.npz through QG_jets_fp32_19.npz). Each file contains two arrays: X: (100000, M, 4) — 50k quark and 50k gluon jets, randomly sorted, padded to max multiplicity M, with particle features (pt, rapidity, azimuthal angle, pdgid) stored as float32 y: (100000,) — jet labels, gluon = 0, quark = 1 The original dataset stores X in float64. Here X has been cast to float32, approximately halving file size. The y labels are unchanged. If you use this dataset, please cite the original Zenodo record and its associated paper: Komiske, Metodiev, Thaler, Energy Flow Networks: Deep Sets for Particle Jets, JHEP 01 (2019) 121, arXiv:1810.05165

energyflow↗
DOE OSTI2026

Pythia8 Quark and Gluon Jets (float16)

A float16 (half-precision) version of the quark and gluon jet dataset originally published by Komiske, Metodiev, and Thaler (Zenodo record 3164691). Only the 20-file subset without charm and bottom quark jets is included here. All simulation parameters and jet selection criteria are identical to the original: Pythia 8.226, √s = 14 TeV Quarks from WeakBosonAndParton:qg2gmZq, gluons from WeakBosonAndParton:qqbar2gmZg with the Z decaying to neutrinos FastJet 3.3.0, anti-k_t jets with R = 0.4 p_T^jet ∈ [500, 550] GeV, |y^jet| < 1.7 There are 20 files, each in compressed NumPy format (QG_jets_fp32_0.npz through QG_jets_fp32_19.npz). Each file contains two arrays: X: (100000, M, 4) — 50k quark and 50k gluon jets, randomly sorted, padded to max multiplicity M, with particle features (pt, rapidity, azimuthal angle, pdgid) stored as float32 y: (100000,) — jet labels, gluon = 0, quark = 1 The original dataset stores X in float64. Here X has been cast to float16, approximately halving file size. The y labels are unchanged. Users should be aware that float16 has limited dynamic range and precision. If you use this dataset, please cite the original Zenodo record and its associated paper: Komiske, Metodiev, Thaler, Energy Flow Networks: Deep Sets for Particle Jets, JHEP 01 (2019) 121, arXiv:1810.05165

energyflow↗
DOE OSTI2019

COFFEA - Columnar Object Framework For Effective Analysis [Slides]

The COFFEA Framework provides a new approach to HEP analysis, via columnar operations, that improves time-to-insight, scalability, portability, and reproducibility of analysis. It is implemented with the Python programming language and commodity big data technologies such as Apache Spark and NoSQL databases. To achieve this suite of improvements across many use cases, COFFEA takes a factorized approach, separating the analysis implementation and data delivery scheme. All analysis operations are implemented using the NumPy or awkward-array packages which are wrapped to yield user code whose purpose is quickly intuited. Various data delivery schemes are wrapped into a common front-end which accepts user inputs and code, and returns user defined outputs. We will present published results from analysis of CMS data using the COFFEA framework along with a discussion of metrics and the user experience of arriving at those results with columnar analysis.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

Code to map Xray reflectivity data from real space to reciprocal space

Code to map X-ray reflectivity data from real space to reciprocal space. The code is written as a Jupyter notebook. Example data are included as a separate file (example_data.csv). These data were collected on a Panalytical XRD system. As a first step, run all cells in the notebook. Opportunities are identified for users to make their own choices for choosing the boundaries of the region of interest to estimate background. A simple algorithm is used to estimate background from the outer regions of the data collection region, interpolating the result inside the region of interest. The background is removed and then the data mapped to Qx-Qz (reciprocal space) coordinates. The results are written as NumPy binary and Matlab formatted files.

Fitzsimmons, Michael [Oak Ridge National Lab. (ORN↗
DOE OSTI2020

Tools to perform data reduction of time-of-flight data taken on BL4A (MagRef)—a Jupyter notebook approach

A collection of modules or tools to produce reflectivity profiles from data acquired with unpolarized and polarized neutron beams using the BL4A (MagRef) instrument at the Spallation Neutron Source. The Jupyter notebook approach offers transparency to the user on what is done with the data, and once the event mode data are stored as NumPy binary files empowers the user to perform the steps from their own laptop without the need for Mantid. The notebook offers two options for background subtraction, applies corrections for the wavelength dependence of the polarizer and flipper efficiencies, accounts for Zeeman energy splitting of the spin-flip channels, Qz-bins the data along contours of constant Qz, propagates errors and writes results in text format suitable for analysis with tools such as GenX.

Fitzsimmons, Michael [Oak Ridge Associated Univ., ↗
DOE OSTI2022

HAMR - Heterogeneous Accelerator Memory Resource (HAMR) v1.0

HAMR is a library defining an accelerator technology agnostic memory model that bridges between accelerator technologies (CUDA, HIP, ROCm, OpenMP, Sycl, OpenCL, Kokos, etc) and traditional CPUs in heterogeneous computing environments. HAMR is light weight and implemented in modern C++. HAMR can be used to manage memory with in a single code or as a data model for coupling codes in a technologically agnostic way. HAMR provides a Python module for coupling C++ and Python codes which implements zero-copy data transfers to and from Python using the Numpy array interface and Numba CUDA array interface protocols.

Loring, Burlen↗
DOE OSTI2023

Xarray Climate Data Analysis Tools

xCDAT is an extension of xarray for climate data analysis on structured grids. It serves as a modern successor to the Community Data Analysis Tools (CDAT) library. Xarray is an "open source project and Python package that introduces labels in the form of dimensions, coordinates, and attributes on top of raw NumPy-like arrays, which allows for more intuitive, more concise, and less error-prone user experience. Xarray includes a large and growing library of domain-agnostic functions for advanced analytics and visualization with these data structures" (source: https://xarray.dev/). The goal of xCDAT is to provide generalizable features and utilities for simple and robust analysis of climate data. xCDAT's design philosophy is focused on reducing the overhead required to accomplish certain tasks in xarray. Some key xCDAT features are inspired by or ported from the core CDAT library, while others leverage powerful libraries in the xarray ecosystem (e.g., xESMF and cf_xarray) to deliver robust APIs.

Vo, Tom↗
DOE OSTI2020

PyDDA: A Pythonic Direct Data Assimilation Framework for Wind Retrievals

This software assimilates data from an arbitrary number of weather radars together with other spatial wind fields (eg numerical weather forecasting model data) in order to retrieve high resolution three dimensional wind fields. PyDDA uses NumPy and SciPy’s optimization techniques combined with the Python Atmospheric Radiation Measurement (ARM) Radar Toolkit (Py-ART) in order to create wind fields using the 3D variational technique (3DVAR). PyDDA is hosted and distributed on GitHub at https://github.com/openradar/PyDDA. PyDDA has the potential to be used by the atmospheric science community to develop high resolution wind retrievals from radar networks. These retrievals can be used for the evaluation of numerical weather forecasting models and plume modelling. This paper shows how wind fields from 2 NEXt generation RADar (NEXRAD) WSR-88D radars and the High Resolution Rapid Refresh can be assimilated together using PyDDA to create a high resolution wind field inside Hurricane Florence.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2023

Legacy Analysis of Dark Matter Annihilation from the Milky Way Dwarf Spheroidal Galaxies with 14 Years of Fermi-LAT Data: Data Products

https://arxiv.org/abs/2311.04982This repository contains data products from the 14 year Fermi-LAT analysis of the Milky Way dSphs as described in McDaniel et al (2023) https://arxiv.org/abs/2311.04982. The included data products are the SED fits files and 2D TS profiles in the WIMP mass and cross section space. These are available for dSphs as well as the blank-field analysis, using both the standard likelihood and the weighted likelihood approach (see Appendix A). CSV data files are included containing relevant information about the dSphs (see table 1 of McDaniel+2024) and blank fields (RA & Dec). Also included is a python Jupyter Notebook to show basic usage of the data products. For Example, plotting the SED likelihoods, converting the SED likelihood to DM space, creating a combined TS profile, obtaining upper limits, etc. This is also stored as a static html file for easier viewing. SEDs are stored as fits files in the output format of fermipy (see https://fermipy.readthedocs.io/en/latest/advanced/sed.html) TS profiles are stored as numpy arrays covering 40 logarithmically spaced mass values over the 1 GeV - 1 TeV mass range and 60 logarithmically spaced cross-section values covering the $10^{-28}$ to $10^{-22}$ cm$^3$/s cross section range. TS profiles including the J-factor prior and without are available, and are labeled with "Jprior" or "noprior" respectively.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2025

Data release for A search for extremely-high-energy neutrinos and first constraints on the ultra-high-energy cosmic-ray proton fraction with IceCube

<h1 id="summary">Summary</h1> <p>Title: Data Release for A search for extremely-high-energy neutrinos and first constraints on the ultra-high-energy cosmic-ray proton fraction with IceCube</p> <p>The IceCube observatory analyzed 12.6 years of data in search of extremely-high-energy (EHE) neutrinos above 5 PeV. The resultant limit of the search (Fig 1), and the effective area of the event selection (Fig 7), are provided in this data release.</p> <h1 id="contents">Contents</h1> <ul> <li><p>README file: this file</p> </li> <li><p><code>differential_limit_and_sensitivity.csv</code>: a comma separated value file, giving the observed experimental differential limit, and sensitivity, of the search as a function of neutrino energy. This is the content of Fig 1 in the paper. The first column is the neutrino energy in GeV. The second column is the limit in units of GeV/cm2/s/sr. The third column is the sensitivity in units of GeV/cm2/s/sr.</p> </li> <li><p><code>effective_area.csv</code>: a comma separated value file, giving the effective area of the search as a function of energy. This is the content of Fig 7 in the paper. The first column is the neutrino energy in GeV. The second column is the total effective area of the search, summed across neutrino flavors, and averaged across neutrinos and antineutrinos, in meters-squared. The third column is the effective area of the search for the average of electron neutrino and electron antineutrinos in units of meters-squared. The fourth column is the same as the third, but for muon-flavor neutrinos. The fifth column is the same as the third and fourth, but for tau-flavor neutrinos.</p> </li> <li><p><code>demo.py</code>: a short python script to demonstrate how to read the files. Run like <code>python demo.py</code>. A standard base python installation is sufficient, as the only dependencies are numpy and matplotlib.</p> </li> </ul> <h1 id="contacts">Contacts</h1> <p>For any questions about this data release, please write to analysis@icecube.wisc.edu</p>

Astronomy and Astrophysics↗