Search NASA⌕ Search

SEARCH · Search NASA

Results for “Data manipulation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 55 records · Page 3

UNH TDP - Voltsys Raw Data and Processing Scripts - Fall 2021

This submission contains raw Voltsys rectifier data and processing scripts associated with MHKDR submission 394 (UNH TDP - Concurrent Measurements of Inflow, Power Performance, and Loads for a Grid-Synchronized Vertical Axis Cross-Flow Turbine Operating in a Tidal Estuary, DOI: 10.15473/1973860) from the University of New Hampshire and Atlantic Marine Energy Center (AMEC) turbine deployment platform. The user is directed to the MHKDR submission 394 for relevant context and detail of this deployment; see link below. The 394_READ_ME file here provides the description from that submission for quick reference. The READ_ME file for this specific instrument from the 394 submission is also available here. This submission contains a zipped folder structure containing raw data in its original format and MATLAB (2019a) processing scripts used to process and manipulate the data into its final form. The final data products are submitted in the 394 submission.

16 TIDAL AND WAVE POWER↗

UNH TDP - Shark Meter Raw Data and Processing Scripts - Fall 2021

This submission contains raw Shark Power Meter data and processing scripts associated with MHKDR submission 394 (UNH TDP - Concurrent Measurements of Inflow, Power Performance, and Loads for a Grid-Synchronized Vertical Axis Cross-Flow Turbine Operating in a Tidal Estuary, DOI: 10.15473/1973860) from the University of New Hampshire and Atlantic Marine Energy Center (AMEC) turbine deployment platform. The user is directed to the MHKDR submission 394 for relevant context and detail of this deployment; see link below. The 394_READ_ME file here provides the description from that submission for quick reference. The READ_ME file for this specific instrument from the 394 submission is also available here. This submission contains a zipped folder structure containing raw data in its original format and MATLAB (2019a) processing scripts used to process and manipulate the data into its final form. The final data products are submitted in the 394 submission.

16 TIDAL AND WAVE POWER↗

UNH TDP - IMU Raw Data and Processing Scripts - Fall 2021

This submission contains raw Yost IMU (Inertial Measurement Unit) data and processing scripts associated with MHKDR submission 394 (UNH TDP - Concurrent Measurements of Inflow, Power Performance, and Loads for a Grid-Synchronized Vertical Axis Cross-Flow Turbine Operating in a Tidal Estuary, DOI: 10.15473/1973860) from the University of New Hampshire and Atlantic Marine Energy Center (AMEC) turbine deployment platform. The user is directed to the MHKDR submission 394 for relevant context and detail of this deployment; see link below. The 394_READ_ME file here provides the description from that submission for quick reference. The READ_ME file for this specific instrument from the 394 submission is also available here. This submission contains a zipped folder structure containing raw data in its original format and MATLAB (2019a) processing scripts used to process and manipulate the data into its final form. The final data products are submitted in the 394 submission.

16 TIDAL AND WAVE POWER↗

UNH TDP - Load Cell Raw Data and Processing Scripts - Fall 2021

This submission contains raw Load Cell data and processing scripts associated with MHKDR submission 394 (UNH TDP - Concurrent Measurements of Inflow, Power Performance, and Loads for a Grid-Synchronized Vertical Axis Cross-Flow Turbine Operating in a Tidal Estuary, DOI: 10.15473/1973860) from the University of New Hampshire and Atlantic Marine Energy Center (AMEC) turbine deployment platform. The user is directed to the MHKDR submission 394 for relevant context and detail of this deployment; see link below. The 394_READ_ME file here provides the description from that submission for quick reference. The READ_ME file for this specific instrument from the 394 submission is also available here. This submission contains a zipped folder structure containing raw data in its original format and MATLAB (2019a) processing scripts used to process and manipulate the data into its final form. The final data products are submitted in the 394 submission.

16 TIDAL AND WAVE POWER↗

Practical Guide to Chemometric Analysis of Optical Spectroscopic Data

The methodology and mathematical treatment of several classic multivariate methods for the analysis of spectroscopic data is demonstrated in a straightforward way that can be used as a basis for teaching an undergraduate introductory course on chemometric analysis. The multivariate techniques of classical least squares (CLS), principal component regression (PCR), and partial least squares (PLS), as well as the univariate Beer’s law method have been described and compared, building students’ understanding by starting with the univariate method and progressing step by step into the multivariate methods. Equations for the production of regression vectors from training set spectral data is described and their use demonstrated for the prediction of constituent concentrations on a separate validation set of spectra. Extreme care is taken to ensure consistency in variable formatting of data matrices. This provides a key foundation to understanding how spectral data are manipulated using these different mathematical approaches for building quantitative regression models. Each method is applied to a real-world data set, and the results are discussed to show students the types of information that can be gleaned from each method. A training set comprised of 20 infrared absorbance spectra containing 3 constituents (benzene, polystyrene, and gasoline) of known composition are used to demonstrate the matrix operations for each regression method. A separate set of 12 real-world napalm samples (containing benzene, polystyrene and gasoline) are used as a validation set to demonstrate the ability to utilize the regression models on an unknown dataset. A toolbox (PNNL Chemometric Toolbox) written in MATLAB language is supplied in the Supplemental Information file and can be used as a companion for understanding the development and deployment of the chemometric algorithms described in this paper. The datasets of the infrared spectra are also supplied, allowing users to build and inspect the chemometric models on their own. Finally, the Toolbox includes scripts to assist users in loading their own datasets into MATLAB and performing CLS, PCR, and PLS on their data.

Upper-Division Undergraduate, Analytical Chemistry↗

AbacusSummit: Cosmological N-body Halos, Light Cones, Particles, Merger Trees, Initial Conditions, and Power Spectra

This is the primary data release of the AbacusSummit, a suite of large, high-accuracy cosmological N-body simulations. Documentation describing the suite and code, including links to refereed publications and instructions for manipulating the data contained in this release, may be found at the following URL: https://abacussummit.readthedocs.io/ The primary data products presented in this release are halo catalogs, light cones, and particle data. Several supplemental products are also presented: power spectra, initial conditions, and merger trees. Cleaned halo catalogs are also provided for the subset of simulations that were used in refereed publications at the time of this DOI. This release also contains several large scale-free and high-redshift simulations run as part of the same allocation used to produce AbacusSummit.

79 ASTRONOMY AND ASTROPHYSICS↗

On the integration of molecular dynamics, data science, and experiments for studying solvent effects on catalysis

Computational workflows that combine molecular dynamics (MD) simulations and emerging data-centric (DC) methods can accelerate the screening and analysis of solvent systems experimentally and computationally. Here, MD simulations provide atomic positions and velocities of reactant, solvent, and catalyst materials that can be manipulated into data representations that in turn can be used by DC techniques to conduct predictive modeling, feature extraction, and experimental design. For liquid-phase catalytic applications, emerging DC techniques such as Convolutional and Graph Neural Networks (CNN/GNN), Topological Data Analysis (TDA), and Active Learning (AL) can leverage MD and experimental data to quickly predict solvent effects on reaction outcomes. For instance, in recent studies, 3D solvent environments obtained with MD have been exploited by CNNs to predict experimental reaction rates for homogeneous acid-catalyzed lignocellulosic processes. In this perspective, we discuss basic principles of DC methods and how these can be combined with MD to enable high-throughput screening of solvent selection for diverse catalysis applications.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Dataset for "A primer on forest structure measurement with lidar for ecologists"

This repository includes data and code accompanying the case study included in the manuscript "A primer on forest structure measurement with lidar for ecologists" (submitted to Ecosphere). We compiled lidar datasets from multiple platforms in a common area to: 1. Demonstrate how differences in sensor characteristics influence density and resolution of lidar data. 2. Provide open-source, co-located datasets for users to further inspect differences in lidar data. 3. Provide example code to perform basic lidar analysis. This case study is meant to allow readers to get hands-on experience with real-world data from different platforms. This case study is not meant to be a rigorous comparison of derived ecological metrics among all sensors; such comparisons can be found throughout other publications referenced throughout the main manuscript. Code includes basic functions in R commonly used to visualize and manipulate lidar data accessible with a normal laptop computer; more sophisticated algorithms for advanced users are also referenced throughout the main manuscript. Terrestrial laser scanning (TLS), mobile laser scanning (MLS), UAS laser scanning (ULS), airborne laser scanning (ALS), and spaceborne laser scanning (SLS) data were collected within the Smithsonian Environmental Research Center (SERC) forest dynamics plot in Maryland, USA. TLS, MLS, and ALS data were collected within 1 month of the 2021 growing season; ULS data were collected in November 2020 (“leaf-off” data) and July 2022 (“leaf-on” data).

54 ENVIRONMENTAL SCIENCES↗

Robust, automated sleep scoring by a compact neural network with distributional shift correction

Studying the biology of sleep requires the accurate assessment of the state of experimental subjects, and manual analysis of relevant data is a major bottleneck. Recently, deep learning applied to electroencephalogram and electromyogram data has shown great promise as a sleep scoring method, approaching the limits of inter-rater reliability. As with any machine learning algorithm, the inputs to a sleep scoring classifier are typically standardized in order to remove distributional shift caused by variability in the signal collection process. However, in scientific data, experimental manipulations introduce variability that should not be removed. For example, in sleep scoring, the fraction of time spent in each arousal state can vary between control and experimental subjects. We introduce a standardization method, mixture z-scoring, that preserves this crucial form of distributional shift. Using both a simulated experiment and mouse in vivo data, we demonstrate that a common standardization method used by state-of-the-art sleep scoring algorithms introduces systematic bias, but that mixture z-scoring does not. We present a free, open-source user interface that uses a compact neural network and mixture z-scoring to allow for rapid sleep scoring with accuracy that compares well to contemporary methods. This work provides a set of computational tools for the robust automation of sleep scoring.

60 APPLIED LIFE SCIENCES↗

The Tiny Median Filter: A Small Size, Flexible Arbitrary Percentile Finder Scheme Suitable for FPGA Implementation

This document reports the design, implementation and testing of a small silicon resource usage, very flexible arbitrary percentile finding scheme called the Tiny Median Filter. It can be used not only as a median filter in image processing with square filtering windows, but also for applications of any percentile filter or maximum or minimum finder with any size of data set as long as the number of bits of the data is finite. It opens possibilities for image processing tasks with non-square or irregular filter windows. In this scheme, data swapping or data bit manipulating are avoided and high functional efficiency of the logic components is applied to save silicon resources. Some logic functions are absorbed into other functions to further reduce the complexity. The combinational logic paths are designed to be sufficiently short so that the firmware can be compiled to the maximum operating frequency allowed by the block memories of the FPGA devices. The Tiny Median Filter receives, processes and output data in non-stop manner with no irregular timing which helps to simplify design of surrounding stages.

Wu, Jinyuan [Fermilab] (ORCID:0000000344329521)↗

Oak Ridge National Laboratory Pilot Demonstration of an Attestation and Anomaly Detection Framework using Distributed Ledger Technology for Power Grid Infrastructure

This report summarizes the design and pilot demonstration of a framework called Grid Guard that was created to provide increased data and device trustworthiness to electric grid devices by leveraging distributed ledger technology (DLT), specifically blockchain. Grid Guard contains a combination of core cryptographic methods such as the secure hash algorithm (SHA), and asymmetric cryptography, private permissioned blockchain, baselining configuration data, consensus algorithm (Raft) and the Hyperledger Fabric (HLF) framework. The system implements a low energy, fast, and robust enhancement to system trustworthiness within and across electric grid systems such as substations, control centers and metering infrastructures. Blockchain is a distributed database structured that provides a practically unalterable (immutable) timeline of stored transactions. By relying on hashing and the Raft consensus algorithm, if an entity tries to illegitimately alter a record at one instance of the database the other ledger nodes are not altered. They work to cross-reference each other and easily locate any incorrectly added data and remove it. The bulk raw data is stored in an off-chain storage (outside of the blockchain ledger) and a hash of this baseline data is stored in the Blockchain ledger via hashing windows of time-series and configuration data, after aggregation and filtering. The bulk off-chain data repository is then considered to be trust-anchored using the hashes stored in the blockchain. To secure the electric grid testbed devices and data, device configuration baselines were compared to those baselines that had been previously stored in the ledger. Statistical baselines for device configurations, network communication patterns, and high-speed sensor data are calculated and then stored off-chain and hashes stored in the ledger. Measurements such as three-phase voltage and current, frequency, breaker status, protection scheme settings, network configuration settings (and other device configuration artifacts) and network traffic features (packet interarrival times) are compared every minute or other selected time windows. During phase 1 of the Grid Guard DLT project different DLT technologies were studies, and an assessment was performed on DLT technology vulnerabilities, uses, and key characteristics. DLT consensus protocols were studies (e.g., RAFT, named after Reliable, Replicated, Redundant, And Fault-Tolerant). Also, cryptography, public, private and permissioned or permissionless systems were assessed. Grid Guard implements a permissioned private DLT. Consensus algorithm selection and choice of DLT implementation depended heavily on the use-case. For this use-case, parameters were selected to measure performance and existing tools for assessment. Benchmarking was performed theoretically and practically. During phase 2 hashed transactions/blocks were inserted into the ledger every second. During phase 2 of the Grid Guard DLT project, a prototype framework was developed and demonstrated for attestation of critical substation devices and data using precision timing systems that use PTP and IRIG-B protocols) on a testbed of operational devices that emulated a distribution substation, control center, and power metering infrastructure using real Operational Technology (OT). The testbed includes OT devices such as protective relays, human machine interfaces (HMI), and power meters. To determine when to collect and compare system and network baselines, an initial examination of an anomaly detection capability to identify malicious manipulation of data streams was conducted. The resulting anomaly detection was demonstrated in a set of experiments and leveraged to trigger device artifact attestation checks. Attestation checks occur against device configuration baselines when compared with the immutable blockchain-stored baselines, which provided a cryptographically supported means by which to store baselines. The electrical substation-grid testbed was created to test the Grid Guard framework. The testbed emulates the operations of a portion of a power grid and SCADA systems as closely as possible. The testbed integrates real protocols, mainly IEC 61850 standard protocols, such as the Sampled Value (SV) and the GOOSE protocols. The testbed also supports DNP3 and other layer 2 and layer 3 protocols such as Telnet, SSH, SFTP/FTP and other proprietary protocols needed to connect to industrial control system equipment. The testbed emulates real power conditions using the OpalRT hardware-in-the-loop (HIL) device which can create fault situations that cannot be easily tested on real systems. The electrical substation-grid testbed was created using real measurement, communication, and protection devices that electrical utilities commonly use.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Semantic embedding for quantum algorithms

The study of classical algorithms is supported by an immense understructure, founded in logic, type, and category theory, that allows an algorithmist to reason about the sequential manipulation of data irrespective of a computation’s realizing dynamics. As quantum computing matures, a similar need has developed for an assurance of the correctness of high-level quantum algorithmic reasoning. Parallel to this need, many quantum algorithms have been unified and improved using quantum signal processing (QSP) and quantum singular value transformation (QSVT), which characterize the ability, by alternating circuit ansätze, to transform the singular values of sub-blocks of unitary matrices by polynomial functions. However, while the algebraic manipulation of polynomials is simple (e.g., compositions and products), the QSP/QSVT circuits realizing analogous manipulations of their embedded polynomials are non-obvious. This work constructs and characterizes the runtime and expressivity of QSP/QSVT protocols where circuit manipulation maps naturally to the algebraic manipulation of functional transforms (termed semantic embedding). In this way, QSP/QSVT can be treated and combined modularly, purely in terms of the functional transforms they embed, with key guarantees on the computability and modularity of the realizing circuits. We also identify existing quantum algorithms whose use of semantic embedding is implicit, spanning from distributed search to proofs of soundness in quantum cryptography. The methods used, based in category theory, establish a theory of semantically embeddable quantum algorithms, and provide a new role for QSP/QSVT in reducing sophisticated algorithmic problems to simpler algebraic ones.

Physics↗

Manipulation of Geographic Information in Global Seismology

Geographic data, such as seismic event locations, station locations, etc., are generally given in geographic latitude Φ ’, longitude θ , and depth below sea level, ζ , using the WGS84 ellipsoid as a reference. In software systems that use this type of geographic data, it is necessary to manipulate the data mathematically in order to perform such tasks as finding the angular distance or azimuth from one point to another, to find an array of points along a great circle, to rotate a point about a pole of rotation, to move a point some angular distance in a specified direction, to find the intersections of two great circles or to find the intersections of a great circle and a small circle. In this paper, equations are presented that convert geographic locations first to geocentric coordinates and then to Earth-centered Cartesian coordinates where many mathematical manipulations can be performed conveniently and efficiently.

58 GEOSCIENCES↗

At the interfaces of the hydrologic sciences: Connecting water, elements, ecosystems, and people through the major contributions of Dr. Emily Bernhardt

Here, in this paper, we describe the major contributions of Professor Emily Bernhardt to the hydrologic sciences. Dr. Bernhardt’s work addresses how carbon, nutrient, and contaminant dynamics respond to a wide range of environmental perturbations that alter hydrologic dynamics within and connectivity among ecosystems. Her research leverages intensive and extensive field sampling, experimental manipulations, macroscale data harmonization and exploration, and continental to global-scale synthesis activities to uncover key drivers and patterns of the impacts human perturbations have on water and elemental cycles. Dr. Bernhardt’s research program is defined by her ability to ask questions and use approaches that explicitly consider connectivity and interfaces in a variety of ways. Here, we highlight significant contributions from Dr. Bernhardt’s work, organized by connectivity, interfaces, and interactions among and across (1) elemental cycles, (2) ecosystems, (3) watersheds, (4) scales, and (5) disciplines. We conclude with a section on Dr. Bernhardt’s impact on the hydrologic sciences and beyond through her exceptional dedication to mentorship, engagement, and service.

58 GEOSCIENCES↗

A Photovoltaic MPPT Charge Controller Real-Time Testbed for Cybersecurity Applications

The increasing deployment of distributed energy resources (DER) over the last decade is a great ally to combat climate change and strengthen the grid during increasingly common extreme weather events. However, DER systems, combined with the ongoing transition to a digital power grid, also pose substantial cybersecurity threats. One of the most common communication protocols used in DER integration is the Distributed Network Protocol 3 (DNP3), which is known to have many security vulnerabilities. Thus, it is essential to investigate cyberattack behaviors and mitigation on power systems using DNP3. In this paper, we designed and implemented a cybersecurity testbed for a simulated photovoltaic (PV) maximum power point tracking (MPPT) charge controller. Our testbed uses an MPPT charge controller simulated on a Typhoon HIL602+ real-time simulator with a real DNP3 communication connection over TCP/IP, allowing for safe and efficient monitoring and manipulation of data traffic between the simulated hardware and supervisory control and data acquisition (SCADA) systems.

14 SOLAR ENERGY↗

Computational Physics Overview [Slides]

Computational physics is an important part of the overall investment in National Security Science at LANL. Computational physics is the study and implementation of numerical analysis to solve problems in physics for which a quantitative theory already exists. Historically, computational physics was the first application of modern computers in science. There are three key elements to computational physics: mathematical models of physical phenomena and conservation equations, computer codes that implement these models, and computer platforms that execute the code instructions and manipulate the data.

45 MILITARY TECHNOLOGY, WEAPONRY, AND NATIONAL DEF↗

Real-time evaluation of cybersecurity threats to DER inverter grid-support functions

In this project we aim to contribute to the understanding of the type and severity of potential cybersecurity attacks to the grid-support functionalities of DER systems interconnected to the AC distribution grid via inverters. Our preliminary work focused on developing a small-scale testbed allowing to study cybersecurity threats to an isolated photovoltaic-battery system using a real-time simulator (Typhoon HIL602+) with a real DNP3 communication connection over TCP/IP, allowing for safe and efficient monitoring and manipulation of data traffic between the simulated hardware and supervisory control and data acquisition (SCADA) system. In this project we propose to expand upon this development by utilizing a) a recently acquired NovaCor RTDS (Real Rime digital Simulator) to emulate the DER-inverter-grid topology including main grid-support functions as defined by IEEE Std. 1547-2018, and b) an industrial control and automation device to enable realistic evaluation of control functions and utilization of communication protocols for real-time data transmission.

24 POWER TRANSMISSION AND DISTRIBUTION↗

New Perspectives on the Exoplanet Radius Gap from a Mathematica Tool and Visualized Water Equation of State

Recent astronomical observations obtained with the Kepler and TESS missions and their related ground-based follow-ups revealed an abundance of exoplanets with a size intermediate between Earth and Neptune (1 R ⊕ ≤ R ≤ 4 R ⊕ ). A low occurrence rate of planets has been identified at around twice the size of Earth (2 × R ⊕ ), known as the exoplanet radius gap or radius valley. We explore the geometry of this gap in the mass–radius diagram, with the help of a Mathematica plotting tool developed with the capability of manipulating exoplanet data in multidimensional parameter space, and with the help of visualized water equations of state in the temperature–density (T–ρ) graph and the entropy–pressure (s–P) graph. We show that the radius valley can be explained by a compositional difference between smaller, predominantly rocky planets (<2 × R ⊕ ) and larger planets (>2 × R ⊕ ) that exhibit greater compositional diversity including cosmic ices (water, ammonia, methane, etc.) and gaseous envelopes. In particular, among the larger planets (>2 × R ⊕ ), when viewed from the perspective of planet equilibrium temperature (T eq ), the hot ones (T eq ≳ 900 K) are consistent with ice-dominated composition without significant gaseous envelopes, while the cold ones (T eq ≲ 900 K) have more diverse compositions, including various amounts of gaseous envelopes.

79 ASTRONOMY AND ASTROPHYSICS↗