Search NASA⌕ Search

SEARCH · Search NASA

Results for “Python API”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 199 records · Page 11

Bringing heterogeneity to the CMS software framework

The advent of computing resources with co-processors, for example Graphics Processing Units (GPU) or Field-Programmable Gate Arrays (FPGA), for use cases like the CMS High-Level Trigger (HLT) or data processing at leadership-class supercomputers imposes challenges for the current data processing frameworks. These challenges include developing a model for algorithms to offload their computations on the co-processors as well as keeping the traditional CPU busy doing other work. The CMS data processing framework, CMSSW, implements multithreading using the Intel Threading Building Blocks (TBB) library, that utilizes tasks as concurrent units of work. In this paper we will discuss a generic mechanism to interact effectively with non-CPU resources that has been implemented in CMSSW. In addition, configuring such a heterogeneous system is challenging. In CMSSW an application is configured with a configuration file written in the Python language. The algorithm types are part of the configuration. The challenge therefore is to unify the CPU and co-processor settings while allowing their implementations to be separate. We will explain how we solved these challenges while minimizing the necessary changes to the CMSSW framework. We will also discuss on a concrete example how algorithms would offload work to NVIDIA GPUs using directly the CUDA API.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

tSFM 1.0: tRNA Structure–Function Mapper

Structure-conditioned information statistics have proven useful to predict and visualize tRNA Class-Informative Features (CIFs) and their evolutionary divergences. Although permutation P-values can quantify the significance of CIF divergences between two taxa, their naive Monte Carlo approximation is slow and inaccurate. The Peaks-over-Threshold approach of Knijnenburg et al. (2009) promises improvements to both speed and accuracy of permutation P-values, but has no publicly available API. Here, we present tRNA Structure–Function Mapper (tSFM) v1.0, an open-source, multi-threaded application that efficiently computes, visualizes and assesses significance of single- and paired-site CIFs and their evolutionary divergences for any RNA, protein, gene or genomic element sequence family. Multiple estimators of permutation P-values for CIF evolutionary divergences are provided along with confidence intervals. tSFM is implemented in Python 3 with compiled C extensions and is freely available through GitHub (https://github.com/tlawrence3/tSFM) and PyPI.

59 BASIC BIOLOGICAL SCIENCES↗

Localized Actual Meteorological Year File Creator (LAF): A tool for using locally observed weather data in building energy simulations

The Localized Actual Meteorological Year File Creator (LAF) application provides web-based access to real meteorological data and processes it into a weather file suitable for building energy modeling. Building energy consumption is affected by what is inside the building (such as occupants, appliances, HVAC systems, etc.) and by what is outside: the weather conditions the building is exposed to. However, freely available weather data files are limited to a number of specific locations, usually airports, which are often located away from city centers where buildings are concentrated. The authors have developed a new tool that supports the investigation and quantification of micro-climate conditions on building energy consumption. LAF is built on the Python open-source programming language and has a Graphical User Interface (GUI) that allows users to create custom weather data files for building energy simulations. Many sets of actual meteorological year weather data for long time periods are publicly available online (such as the MesoWest database) for thousands of locations in the US. LAF selects weather data according to user specifications and automatically processes it through an API across multiple weather stations and multiple time periods. The user may easily select a specific location, time frame, and time step that best meets their needs. This article presents a useful tool for energy modelers, building designers and operators to assist with building performance analysis and optimization.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗

Accelerating the Inference of the Exa.TrkX Pipeline

Recently, graph neural networks (GNNs) have been successfully used for a variety of particle reconstruction problems in high energy physics, including particle tracking. The Exa.TrkX pipeline based on GNNs demonstrated promising performance in reconstructing particle tracks in dense environments. It includes five discrete steps: data encoding, graph building, edge filtering, GNN, and track labeling. All steps were written in Python and run on both GPUs and CPUs. In this work, we accelerate the Python implementation of the pipeline through customized and commercial GPU-enabled software libraries, and develop a C++ implementation for inferencing the pipeline. The implementation features an improved, CUDA-enabled fixed-radius nearest neighbor search for graph building and a weakly connected component graph algorithm for track labeling. GNNs and other trained deep learning models are converted to ONNX and inferenced via the ONNX Runtime C++ API. The complete C++ implementation of the pipeline allows integration with existing tracking software. We report the memory usage and average event latency tracking performance of our implementation applied to the TrackML benchmark dataset.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

Python Materials Genomics (pymatgen): A robust, open-source python library for materials analysis

We present the Python Materials Genomics (pymatgen) library, a robust, open-source Python library for materials analysis. A key enabler in high-throughput computational materials science efforts is a robust set of software tools to perform initial setup for the calculations (e.g., generation of structures and necessary input files) and post-calculation analysis to derive useful material properties from raw calculated data. The pymatgen library aims to meet these needs by (1) defining core Python objects for materials data representation, (2) providing a well-tested set of structure and thermodynamic analyses relevant to many applications, and (3) establishing an open platform for researchers to collaboratively develop sophisticated analyses of materials data obtained both from first principles calculations and experiments. The pymatgen library also provides convenient tools to obtain useful materials data via the Materials Project's REpresentational State Transfer (REST) Application Programming Interface (API). As an example, using pymatgen's interface to the Materials Project's RESTful API and phasediagram package, we demonstrate how the phase and electrochemical stability of a recently synthesized material, Li 4 SnS 4 , can be analyzed using a minimum of computing resources. Here, we find that Li 4 SnS 4 is a stable phase in the Li-Sn-S phase diagram (consistent with the fact that it can be synthesized), but the narrow range of lithium chemical potentials for which it is predicted to be stable would suggest that it is not intrinsically stable against typical electrodes used in lithium-ion batteries.

36 MATERIALS SCIENCE↗

CrossLink: Geometry API [Slides]

The mesh generation process is very challenging and time consuming when working with complex CAD models. The process of creating and sorting geometric entities into groups appropriate for meshing is labor intensive and prone to error. In addition, the common data exchange formats such as STEP and IGES do not propagate information such as entity names that may be defined in the original model. Finally, entity counts change frequently with parameter variation as a result of tolerance-based geometry operations. Thus, sorting by index does not provide a robust and repeatable means for grouping. xGeom is a geometry library that enables the creation of NURBS curves and surfaces via a python scripting interface. xGeom is ideal for studying relatively simple models and is fully integrated with CrossLink’s mesh generation capabilities. For more complex models, xCAD is a python-based Creo Parametric CAD model driver that enables the model to be generated, queried, parametrically modified, regenerated, and exported without data loss and in a fully repeatable manner.

97 MATHEMATICS AND COMPUTING↗

Multi-language Struct Support in Babel

Babel is an open-source language interoperability framework tailored to the needs of high-performance scientific computing. As an integral element of the Common Component Architecture (CCA) it is used in a wide range of research projects. In this paper we describe how we extended Babel to support interoperable tuple data types (structs). Structs are a common idiom in scientific APIs; they are an efficient way to pass tuples of nonuniform data between functions, and are supported natively by most programming languages. Using our extended version of Babel, developers of scientific code can now pass structs as arguments between functions implemented in any of the supported languages. In C, C++ and Fortran 2003, structs can be passed without the overhead of data marshaling or copying, providing language interoperability at minimal cost. Other supported languages are Fortran 77, Fortran 90, Java and Python. We will show how we designed a struct implementation that is interoperable with all of the supported languages and present benchmark data compare the performance of all language bindings, highlighting the differences between languages that offer native struct support and an object-oriented interface with getter/setter methods.

97 MATHEMATICS AND COMPUTING↗

The SENSEI Generic In Situ Interface: Tool and Processing Portability at Scale [Book Chapter]

One key challenge when doing in situ processing is the investment required to add code to numerical simulations needed to take advantage of in situ processing. Such instrumentation code is often specialized, and tailored to a specific in situ method or infrastructure. Then, if a simulation wants to use other in situ tools, each of which has its own bespoke API [4], then the simulation code team will quickly become overwhelmed with having a different set of instrumentation APIs, one per in situ tool or method. In an ideal situation, such instrumentation need happen only once, and then the instrumentation API provides access to a large diversity of tools. In this way, a data producer’s instrumentation need not be modified if the user desires to take advantage of a different set of in situ tools. The SENSEI generic in situ interface addresses this challenge, which means that SENSEI-instrumented codes enjoy the benefit of being able to use a diversity of tools at scale, tools that include Libsim, Catalyst, Ascent, as well as user-defined methods written in C++ or Python. SENSEI has been shown to scale to greater than 1M-way concurrency on HPC platforms, and provides support for a rich and diverse collection of common scientific data models. Furthermore, this chapter presents the key design challenges that enable tool and processing portability at scale, some performance analysis, and example science applications of the methods.

Bethel, E. Wes↗

Boosting RDataFrame performance with transparent bulk event processing

RDataFrame is ROOT’s high-level interface for Python and C++ data analysis. Since it first became available, RDataFrame adoption has grown steadily and it is now poised to be a major component of analysis software pipelines for LHC Run 3 and beyond. Thanks to its design inspired by declarative programming principles, RDataFrame enables the development of highperformance, highly parallel analyses without requiring expert knowledge of multi-threading and I/O: user logic is expressed in terms of self-contained, small computation kernels tied together by a high-level API. This design completely decouples analysis logic from its actual execution, and opens several interesting avenues for workflow optimization. In particular, in this work we explore the benefits of moving internal data processing from an event-by-event to a bulkby-bulk loop. This refactoring dramatically reduces the framework’s runtime overheads; in collaboration with the I/O layer it improves data access patterns; it exposes information that optimizing compilers might use to auto-vectorize the invocation of user-defined computations; finally, while existing user-facing interfaces remain unaffected, it becomes possible to additionally offer interfaces that explicitly expose bulks of events, useful e.g. for the injection of GPU kernels into the analysis workflow. In order to inform similar future R&D, design challenges will be presented, as well as an investigation of the relevant timememory trade-off backed by novel performance benchmarks.

Guiraud, Enrico↗

TH-D-BRB-04: Pinnacle Scripting: Improving Efficiency While Maintaining Safety

Scripting capabilities and application programming interfaces (APIs) are becoming commonly available in modern treatment planning systems. These links to the treatment planning system (TPS) allow users to read data from the TPS, and in some cases use TPS functionality and write data back to the TPS. Such tools are powerful extensions, allowing automation of routine clinical tasks and supporting research, particularly research involving repetitive tasks on large patient populations. The data and functionality exposed by scripting/API capabilities is vendor dependent, as are the languages used by script/API engines, such as the Microsoft .NET framework or Python. Scripts deployed in a clinical environment must be commissioned and validated like any other software tool. This session will provide an overview of scripting applications and a discussion of best practices, followed by a practical introduction to the scripting capabilities of three commercial treatment planning systems. Learning Objectives: Understand the scripting capabilities available in several treatment planning systems Learn how to get started using scripting capabilities Understand the best practices for safe script deployment in a clinical environment R. Popple, Varian Medical Systems has provided research support unrelated to the topic of this session.R. Cardan, Varian Medical Systems for grant research, product evaluation, and teaching honorarium.

60 APPLIED LIFE SCIENCES↗

Xarray Climate Data Analysis Tools

xCDAT is an extension of xarray for climate data analysis on structured grids. It serves as a modern successor to the Community Data Analysis Tools (CDAT) library. Xarray is an "open source project and Python package that introduces labels in the form of dimensions, coordinates, and attributes on top of raw NumPy-like arrays, which allows for more intuitive, more concise, and less error-prone user experience. Xarray includes a large and growing library of domain-agnostic functions for advanced analytics and visualization with these data structures" (source: https://xarray.dev/). The goal of xCDAT is to provide generalizable features and utilities for simple and robust analysis of climate data. xCDAT's design philosophy is focused on reducing the overhead required to accomplish certain tasks in xarray. Some key xCDAT features are inspired by or ported from the core CDAT library, while others leverage powerful libraries in the xarray ecosystem (e.g., xESMF and cf_xarray) to deliver robust APIs.

Vo, Tom↗

HC-NIDS: signatures and simulations for detecting cyber-attacks aiming to cause damage against cyber-physical energy systems

Physical device safety is typically implemented locally using embedded controllers, while operations safety is primarily performed in control centers. Safe operations can be enhanced by correct design of device-level control algorithms, and protocols, procedures and operator training at the control-room level, but all can fail. Moreover, these elements exchange data and issue commands via vulnerable communication layers. In order to secure these gaps and enhance operational safety, we believe monitoring of command sequences must be combined with an awareness of physical device limitations and automata models that capture safety mechanisms. One way of doing this is by leveraging specification-based intrusion detection to monitor for physical constraint violations. The method can also verify that physical infrastructure state is consistent with monitoring information and control commands exchanged between field devices and control centers. This additional security layer enhances protection from both outsider attacks and insider mistakes. We implemented specification-based SCADA command analyzers using physical constraint algorithms directly in the Bro framework and Broccoli APIs for three separate scenarios: a water heater, an automated distribution system, and an over-current protection scheme. To accomplish this, we added low-level analyzers capable of examining control system-specific protocol packets for both Modbus TCP and DNP3, and also higher-level analyzers able to interpret device command and data streams within the context of each device's physical capabilities and present operational state. Thus the software that we are making available includes the Bro/Broccoli scripts for these three scenarios, as well as simulators, written in C, of those scenarios that generate sample traffic that is monitored by the Bro/Broccoli scripts. In addition, we have also implemented systems to directly pull cyber-physical information from the OSIsoft PI historian system. We have included the Python scripts used to perform that monitoring.

Source record↗

HC-NIDS: signatures and simulations for detecting cyber-attacks aiming to cause damage against cyber-physical energy systems

Physical device safety is typically implemented locally using embedded controllers, while operations safety is primarily performed in control centers. Safe operations can be enhanced by correct design of device-level control algorithms, and protocols, procedures and operator training at the control-room level, but all can fail. Moreover, these elements exchange data and issue commands via vulnerable communication layers. In order to secure these gaps and enhance operational safety, we believe monitoring of command sequences must be combined with an awareness of physical device limitations and automata models that capture safety mechanisms. One way of doing this is by leveraging specification-based intrusion detection to monitor for physical constraint violations. The method can also verify that physical infrastructure state is consistent with monitoring information and control commands exchanged between field devices and control centers. This additional security layer enhances protection from both outsider attacks and insider mistakes. We implemented specification-based SCADA command analyzers using physical constraint algorithms directly in the Bro framework and Broccoli APIs for three separate scenarios: a water heater, an automated distribution system, and an over-current protection scheme. To accomplish this, we added low-level analyzers capable of examining control system-specific protocol packets for both Modbus TCP and DNP3, and also higher-level analyzers able to interpret device command and data streams within the context of each device's physical capabilities and present operational state. Thus the software that we are making available includes the Bro/Broccoli scripts for these three scenarios, as well as simulators, written in C, of those scenarios that generate sample traffic that is monitored by the Bro/Broccoli scripts. In addition, we have also implemented systems to directly pull cyber-physical information from the OSIsoft PI historian system. We have included the Python scripts used to perform that monitoring.

Source record↗

HAPI: An API Standard for Accessing Heliophysics Time Series Data

Heliophysics data analysis often involves combining diverse science measurements, many of them captured as time series. Although there are now only a few commonly used data file formats, the diversity in mechanisms for automated access to and aggregation of such data holdings can make analysis that requires intercomparison of data from multiple data providers difficult. The Heliophysics Application Programmer's Interface (HAPI) is a recently developed standard for accessing distributed time series data to increase interoperability. The HAPI specification is based on the common elements of existing data services, and it standardizes the two main parts of a data service: the request interface and the response data structures. The interface is based on the REpresentational State Transfer (REST) or RESTful architecture style, and the HAPI specification defines five required REST endpoints. Data are returned via a streaming format that hides file boundaries; the metadata is detailed enough for the content to be scientifically useful, e.g., plotted with appropriate axes layout, units, and labels. Multiple mature HAPI-related open-source projects offer server-side implementation tools and client-side libraries for reading HAPI data in multiple languages (IDL, Java, MATLAB, and Python). Multiple data providers in the US and Europe have added HAPI access alongside their existing interfaces. Based on this experience, data can be served via HAPI with little or no information loss compared to similar existing web interfaces. Finally, HAPI has been recommended as a COSPAR standard for time series data delivery.

Robert S. Weigel↗

Scientific Computational Imaging Code (SCICO)

Scientific Computational Imaging Code (SCICO) is a Python package for solving the inverse problems that arise in scientific imaging applications. Its primary focus is providing methods for solving ill-posed inverse problems by using an appropriate prior model of the reconstruction space. SCICO includes a growing suite of operators, cost functionals, regularizers, and optimization routines that may be combined to solve a wide range of problems, and is designed so that it is easy to add new building blocks. SCICO is built on top of JAX rather than NumPy, enabling GPU/TPU acceleration, just-in-time compilation, and automatic gradient functionality, which is used to automatically compute the adjoints of linear operators. An example of how to solve a multi-channel tomography problem with SCICO is shown in Figure 1. The SCICO source code is available from GitHub, and pre-built packages are available from PyPI. It has extensive online documentation, including API documentation and usage examples, which can be run online at Google Colab and binder.

97 MATHEMATICS AND COMPUTING↗

CROCUS Low Cost All-in-One Weather Station AMB-001 Data Argonne National Laboratory Prairie Site

The Ambient Weather WS-2902D (AMB) is a low cost weather station that has become very useful for filling data gaps in harder to deploy locations. These low cost weather stations collect 13 second data, which is averaged to a five minute data output available to users through an API key. The data files contain measurements for precipitation, temperature, wind chill/heat index, relative humidity, dew point, UV index, solar radiation, wind speed, wind direction, wind gust, and with an external particulate matter 2.5 (PM 2.5) sensor. Having all of these measurements in one condense system allows for fast deploying and dense network capabilities. Three of the AMB weather stations were deployed at the Argonne Testbed for Multiscale Observational Science (ATMOS), a 20-acre prairie site at Argonne National Laboratory in Lemont, Illinois. The instruments are denoted by their three digit identifier (CMS-AMB-xxx) format. The data is presented as daily NetCDF (.nc) files, each containing approximately 24 hours of observations. Files follow the naming convention of: the project (CROCUS), location (atmos), instrument name (CMS-AMB-001), data level (raw, a1), and date (year, month, day). The NetCDF format can be accessed using common scientific software such as Python using xarray, netCDF4 or ACT-DOE.

54 ENVIRONMENTAL SCIENCES↗

CROCUS Low Cost All-in-One Weather Station AMB-002 Data Argonne National Laboratory Prairie Site

The Ambient Weather WS-2902D (AMB) is a low cost weather station that has become very useful for filling data gaps in harder to deploy locations. These low cost weather stations collect 13 second data, which is averaged to a five minute data output available to users through an API key. The data files contain measurements for precipitation, temperature, wind chill/heat index, relative humidity, dew point, UV index, solar radiation, wind speed, wind direction, wind gust, and with an external particulate matter 2.5 (PM 2.5) sensor. Having all of these measurements in one condense system allows for fast deploying and dense network capabilities. Three of the AMB weather stations were deployed at the Argonne Testbed for Multiscale Observational Science (ATMOS), a 20-acre prairie site at Argonne National Laboratory in Lemont, Illinois. The instruments are denoted by their three digit identifier (CMS-AMB-xxx) format. The data is presented as daily NetCDF (.nc) files, each containing approximately 24 hours of observations. Files follow the naming convention of: the project (CROCUS), location (atmos), instrument name (CMS-AMB-002), data level (raw, a1), and date (year, month, day). The NetCDF format can be accessed using common scientific software such as Python using xarray, netCDF4 or ACT-DOE.

54 ENVIRONMENTAL SCIENCES↗

CROCUS Low Cost All-in-One Weather Station AMB-004 Data Argonne National Laboratory Prairie Site

The Ambient Weather WS-2902D (AMB) is a low cost weather station that has become very useful for filling data gaps in harder to deploy locations. These low cost weather stations collect 13 second data, which is averaged to a five minute data output available to users through an Application Programming Interface (API) key. The data files contain measurements for precipitation, temperature, wind chill/heat index, relative humidity, dew point, UV index, solar radiation, wind speed, wind direction, wind gust, and with an external particulate matter 2.5 (PM 2.5) sensor. Having all of these measurements in one condense system allows for fast deploying and dense network capabilities. Three of the AMB weather stations were deployed at the Argonne Testbed for Multiscale Observational Science (ATMOS), a 20-acre prairie site at Argonne National Laboratory in Lemont, Illinois. The instruments are denoted by their three digit identifier (CMS-AMB-xxx) format. The data is presented as daily NetCDF (.nc) files, each containing approximately 24 hours of observations. Files follow the naming convention of: the project (CROCUS), location (atmos), instrument name (CMS-AMB-004), data level (raw, a1), and date (year, month, day). The NetCDF format can be accessed using common scientific software such as Python using xarray, netCDF4 or ACT-DOE.

EARTH SCIENCE > ATMOSPHERE > AEROSOLS > PARTICULAT↗