Search NASA⌕ Search

SEARCH · Search NASA

Results for “process data”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 325 records · Page 18
All sourcesNASA NTRSDOE OSTI
DOE OSTI2020

Data for Post-Access Report: Tidal Energy Resource Characterization and Model Validation via the Assessment of Cost-effective Spatially Distributed Current Measurements from microFloat Swarms

This project evaluated how high-resolution, spatially distributed field data can be used to refine and validate site-scale hydrodynamic simulations of tidal channels. Use of such spatially-distributed field observations or site-scale hydrodynamic simulations will be needed for producing accurate predictions of tidal energy production over larger arrays of tidal turbines. The data contained in this archive includes field observations of water velocity from a series of short surveys in Agate Pass taken on August 20th, 2020 using station-keeping ADCP (acoustic doppler current profiler) measurements, drifting downward-looking ADCPs, and microFloats, gathered by researchers at APL-UW under a previous project. It also includes high-resolution hydrodynamic model runs for the site during that time generated under TEAMER support. An explanatory file (.pdf format) describes the survey and field data structure. Processed data is stored in .mat format. Survey summary: start time, number of floats, water level. microFloat data: - Cleaned data for each float in survey, - Time series of position (lat., lon.), depth (relative to surface), and horizontal velocity (u, v) Drifting ADCP data: - Cleaned data for each drifting ADCP in survey - Time series of position (lat., lon.), sample depth (relative to surface), and observed water velocity (u,v,w) Stationary ADCP data: - Cleaned data for each station-keeping data set in survey - Time series of position (lat., lon.), sample depth (relative to surface), and observed water velocity (u,v,w) FVCOM (Finite Volume Coastal Ocean Model) model outputs: Water level and velocity, as well as derived variables (e.g., power density) from the final refined model simulations for the Agate Pass subdomain. Contained in NetCDF and ASCII files for the duration of August 18th and 20th 2020 field surveys. Additional scripts for processing the data and generating the figures in the report are included.

16 TIDAL AND WAVE POWER↗
DOE OSTI2022

Impacts of processing decisions on TNSL cross sections and their applications

Thermal neutron scattering law (TNSL) data describe low-energy neutrons scattering off of bound materials, and can have a significant impact on modeling any system with slow neutrons, including nuclear reactors. Previous work to introduce TNSL data to neutron transport codes at LLNL focused on COG and TART, with the limitation that these codes require highly specialized data processing and formatting. We have recently increased efforts to process TNSL data with the LLNL nuclear data processing code FUDGE. FUDGE reads and writes the evaluated and processed files using the generalized nuclear database structure (GNDS). This process uncovered some significant difficulties in processing TNSL data, and unearthed assumptions made in current TNSL data processing that we have found inadequate. (authors)

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2021

On single-crystal total scattering data reduction and correction protocols for analysis in direct space

Data reduction and correction steps and processed data reproducibility in the emerging single-crystal total-scattering-based technique of three-dimensional differential atomic pair distribution function (3D-ΔPDF) analysis are explored. All steps from sample measurement to data processing are outlined using a crystal of CuIr 2 S 4 as an example, studied in a setup equipped with a high-energy X-ray beam and a flat-panel area detector. Computational overhead as pertains to data sampling and the associated data-processing steps is also discussed. Various aspects of the final 3D-ΔPDF reproducibility are explicitly tested by varying the data-processing order and included steps, and by carrying out a crystal-to-crystal data comparison. Situations in which the 3D-ΔPDF is robust are identified, and caution against a few particular cases which can lead to inconsistent 3D-ΔPDFs is noted. Although not all the approaches applied herein will be valid across all systems, and a more in-depth analysis of some of the effects of the data-processing steps may still needed, the methods collected herein represent the start of a more systematic discussion about data processing and corrections in this field.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗
DOE OSTI2024

Streaming Readout and Data-Stream Processing With ERSAP

With the exponential growth in the volume and complexity of data generated at high-energy physics and nuclear physics research facilities, there is an imperative demand for innovative strategies to process this data in real or near-real-time. Given the surge in the requirement for high-performance computing, it becomes pivotal to reassess the adaptability of current data processing architectures in integrating new technologies and managing streaming data. This paper introduces the ERSAP framework, a modern solution that synergizes flow-based programming with the reactive actor model, paving the way for distributed, reactive, and high performance in data stream processing applications. Additionally, we unveil a novel algorithm focused on time-based clustering and event identification in data streams. The efficacy of this approach is further exemplified through the data-stream processing outcomes obtained from the recent beam tests of the EIC prototype calorimeter at DESY.

Vardan, Gyurjyan↗
DOE OSTI2021

Comparative Assessment of Data-driven Process Models in Health Information Technology

Process mining for conformance analysis consists of comparing a reference process model against a data-driven process model generated via log files from information technology systems. However, in the absence of a complete reference process model, we found no suggested approaches in the literature to address the need for evaluating process conformance among different healthcare facilities to assess standardization of care. Our goal is to find similarities and dissimilarities in data-driven process models among US Veterans Health Administration (VHA) facilities that can be indicative of patient safety issues. Our hypothesis was that the analysis would not produce statistically significant differences in outcome. We present a unique implementation of conformance analysis in process mining that consists of combining process mining, process mapping and statistical metrics. We illustrate our approach by applying it to the analysis of two clinical radiology order process models generated from healthcare data provided by two similar facilities in the VHA. The comparative assessment showed that about 70% of the orders completed successfully and 30% were not completed due to policy and duplications. Our analysis found a good statistical correlation between both facilities, as the Spearman’s correlation coefficient between facilities for the frequency of cases per total hours was 0.87879, for the frequency of cases by state transition was 0.79702 and for the throughput time per state transition was 0.63582. Additional statistical analyses using the Mann-Whitney U test and the root mean square error both produced values that were not significant. The foregoing approach validated our hypothesis by demonstrating a good statistical correlation of data describing the flow of clinical radiology orders absent a credible reference model. Finding good agreement between both facilities was important in confirming that the clinical orders flow in a similar manner, suggesting standardization of care.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

Automatic Classification of Biological Targets in a Tidal Channel Using a Multibeam Sonar

Multibeam sonars are widely used for environmental monitoring of fauna at marine renewable energy sites. However, they can rapidly accrue vast volumes of data, which poses a challenge for data processing. Here, using data from a deployment in a tidal channel with peak currents of 1–2 m s –1 , we demonstrate the data-reduction benefits of real-time automatic classification of targets detected and tracked in multibeam sonar data. First, we evaluate classification capabilities for three machine learning algorithms: random forests, support vector machines, and k-nearest neighbors. For each algorithm, a hill-climbing search optimizes a set of hand-engineered attributes that describe tracked targets. Here, the random forest algorithm is found to be most effective—in postprocessing, discriminating between biological and nonbiological targets with a recall rate of 0.97 and a precision of 0.60. In addition, 89% of biological targets are correctly classified as either seals, diving birds, fish schools, or small targets. Model dependence on the volume of training data is evaluated. Second, a real-time implementation of the model is shown to distinguish between biological targets and nonbiological targets with nearly the same performance as in postprocessing. From this, we make general recommendations for implementing real-time classification of biological targets in multibeam sonar data and the transferability of trained models.

16 TIDAL AND WAVE POWER↗
DOE OSTI2023

Ultrahigh performance LC/FT-MS non-targeted screening for biomass burning organic aerosol with MZmine2 and MFAssignR

In recent years, ultrahigh performance liquid chromatography Fourier transform mass spectrometry (LC/FT-MS) based non-targeted screening (NTS) methods have become increasingly popular for comprehensive analysis of complex organic mixtures. However, applying these methods for environmental complex mixture analysis is challenging due to the extreme complexity of natural samples and a lack of standard samples or surrogates for environmental complex mixtures. Furthermore, limited molecular markers in the databases and insufficient data processing software workflows make the application of these methods more challenging for environmental complex mixtures. In this work, we implement a new NTS data processing workflow to process data collected from ultrahigh performance liquid chromatography and Fourier transform Orbitrap Elite Mass Spectrometry (LC/FT-MS) by combining MZmine2 and MFAssignR, two opensource data processing tools and commercial Mesquite liquid smoke as a surrogate for biomass burning organic aerosol. MZmine2.53 data extraction followed MFAssignR molecular formula assignment offered noise free and highly accurate 1733 individual molecular formulas presented in liquid smoke with 4906 molecular species, including isomers. The results of this new approach were consistent with the results of direct infusion FT-MS analysis confirming its reliability. Over 90% of the molecular formulas presented in mesquite liquid smoke were matched with the molecular formulas of ambient biomass burning organic aerosol. This suggests the potential use of commercial liquid smoke as a surrogate for biomass burning organic aerosol research. Furthermore, the presented method significantly improves the identification of the molecular composition of biomass burning organic aerosol by successfully addressing some of the limitations related to the data analysis and giving a semi quantitative insight into the analysis.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2022

EMSL-Computing/CoreMS-Portal

Software that orchestrate data processing and management for mass spectrometry workflows and data products. Stack presented contains a web application, data processing job scheduling, and data processing workers for data processing and management.

Corilo, Yuri↗
DOE OSTI2023

Towards a Comparative Assessment of Data-Driven Process Models in Health Information Technology

Process mining for conformance analysis focuses on comparing a reference process model against a data-driven process model that is generated via log files from information technology systems. While this approach is helpful when there is an existing process model in an organization, it leaves the question of what to do in the absence of a complete reference process model unanswered. In this paper, we present a comparative assessment approach that combines process mining, process mapping for dimensionality reduction, and statistical analysis. Our goal is to find similarities and dissimilarities in data-driven process models among U.S. Veterans Health Administration (VHA) facilities to assess process conformance among different healthcare facilities, which can help assess the standardization of care. We illustrate our approach by applying it to two clinical radiology order process models generated by two similar facilities. Our results demonstrate statistical similarities in the standardization of care among those two facilities.

Klasky, Hilda↗
DOE OSTI2022

Data reduction through optimized scalar quantization for more compact neural networks

Raw data generation for several existing and planned large physics experiments now exceeds TB/s rates, generating untenable data sets in very little time. Those data often demonstrate high dimensionality while containing limited information. Meanwhile, Machine Learning algorithms are now becoming an essential part of data processing and data analysis. Those algorithms can be used offline for post processing and post data analysis, or they can be used online for real time processing providing ultra low latency experiment monitoring. Both use cases would benefit from data throughput reduction while preserving relevant information: one by reducing the offline storage requirements by several orders of magnitude and the other by allowing ultra fast online inferencing with low complexity Machine Learning models. Moreover, reducing the data source throughput also reduces material cost, power and data management requirements. In this work we demonstrate optimized nonuniform scalar quantization for data source reduction. This data reduction allows lower dimensional representations while preserving the relevant information of the data, thus enabling high accuracy Tiny Machine Learning classifier models for online fast inferences. We demonstrate this approach with an initial proof of concept targeting the CookieBox, an array of electron spectrometers used for angular streaking, that was developed for LCLS-II as an online beam diagnostic tool. We used the Lloyd-Max algorithm with the CookieBox dataset to design an optimized nonuniform scalar quantizer. Optimized quantization lets us reduce input data volume by 69% with no significant impact on inference accuracy. When we tolerate a 2% loss on inference accuracy, we achieved 81% of input data reduction. Finally, the change from a 7-bit to a 3-bit input data quantization reduces our neural network size by 38%.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Even Higher-Level Synthesis: An Exploration of AI Hardware Accelerators using HLS4ML

With the rise of artificial intelligence, the popularization of deep learning, and a constantly evolving industry, the demand for flexible and efficient tools has never been greater. As algorithms grow more complex, their runtime and energy consumption increase exponentially. Customized hardware accelerators, long used for specific mathematical operations, remain essential for managing modern applications' computational and power demands. Hardware accelerators can speed up complex computations by orders of magnitude, but their manual design and verification processes are often challenging and time-consuming. High-Level Synthesis (HLS) provides a solution by transforming high-level algorithm descriptions, typically written in C++ or SystemC, into synthesizable RTL suitable for hardware implementation. This approach reduces development time for RTL engineers while offering flexibility beyond what traditional handwritten RTL can provide. We extended this capability to the machine-learning domain with the open-source framework hls4ml, which allows neural networks trained in Python frameworks like Tensorflow or PyTorch to be synthesized into efficient hardware representations for the traditional FPGA and ASIC flows. This breakthrough addresses the growing need for reduced design turnaround and easy verification of ML hardware accelerators with low latency and power efficiency constraints. During this tutorial, we will demonstrate how Python complements HLS by simplifying the ML design process, bridging the gap between software and hardware development. Attendees will explore how we translate neural networks modeled in Python into fixed-point C++ models suitable for HLS workflows. We will dive into strategies like Value-Range Analysis and Quantization-Aware Training, which optimize these designs for deployment and evaluate their accuracy, power consumption, and energy efficiency. To exemplify these concepts, experts from Fermilab will share their experiences applying this technology to high-energy physics experiments, where real-time, low-latency processing is critical. Over the years, Fermilab engineers have demonstrated how deep neural networks, optimized for hardware using hls4ml, can meet the stringent requirements of trigger systems at the CERN Large Hadron Collider. These systems rely on rapid decision-making to process immense data volumes while retaining only the most relevant events for further analysis. The application of hls4ml has also been extended to innovative technologies like smart pixel arrays. These smart pixels integrate ML inference capabilities directly into sensor devices, enabling localized data processing at the pixel level. This approach drastically reduces the need to transmit raw data to external processing units, significantly decreasing power consumption and latency. By embedding neural networks within the pixel architecture, the smart pixels can identify and prioritize relevant data in real time, providing a highly efficient solution for edge computing in scenarios such as particle detectors and imaging systems. Fermilab's work highlights the potential of hardware-accelerated ML in scenarios where both speed and power efficiency are mission-critical. Through this tutorial, attendees will gain valuable insights into the challenges and solutions of deploying ML in hardware. Understanding how HLS and hls4ml streamline the development of neural network-based hardware accelerators is fundamental for the industry's future. Participants will learn how these technologies are shaping the future of AI and scientific computing.

Di Guglielmo, Giuseppe [Fermilab]↗
DOE OSTI2025

Cardinal: Seismic and Geoacoustic Array Processing

Data collected via seismic and infrasound array deployments are leveraged in the geosciences to detect and characterize a myriad of natural and anthropogenic sources. These deployments consist of numerous sensors placed in a predetermined configuration to amplify signal strength and improve the efficacy of array processing techniques used to measure signal directionality and waveform coherence. High‐fidelity feature extraction is often predicated on interstation distance as well as the frequency content and wavelength of an incident signal. Numerous array processing softwares analyze data in sequential frequency bands to obtain a more detailed characterization of a signal. However, current algorithms are limited in their ability to determine optimal array configuration for each band. We introduce an open‐source Python code, called Cardinal, to process seismic and infrasound array data in discretized time–frequency space with the option of applying an adaptive array design to determine optimal subarray configuration for each frequency band. To reduce computational time, the array processing step can be run in parallel using multithreading. Furthermore, the software has the capability to aggregate array processing results from different time–frequency pixels to produce separate sets of detections, or families, with added utility via the application of an adaptive semblance threshold, which aids in isolating signals‐of‐interest from coherent background noise. Upon appropriate configuration, Cardinal exhibits the potential to combine distinct seismic and infrasound phases into separate families.

Adaptive Array↗
DOE OSTI2021

UNH TDP - Voltsys Raw Data and Processing Scripts - Fall 2021

This submission contains raw Voltsys rectifier data and processing scripts associated with MHKDR submission 394 (UNH TDP - Concurrent Measurements of Inflow, Power Performance, and Loads for a Grid-Synchronized Vertical Axis Cross-Flow Turbine Operating in a Tidal Estuary, DOI: 10.15473/1973860) from the University of New Hampshire and Atlantic Marine Energy Center (AMEC) turbine deployment platform. The user is directed to the MHKDR submission 394 for relevant context and detail of this deployment; see link below. The 394_READ_ME file here provides the description from that submission for quick reference. The READ_ME file for this specific instrument from the 394 submission is also available here. This submission contains a zipped folder structure containing raw data in its original format and MATLAB (2019a) processing scripts used to process and manipulate the data into its final form. The final data products are submitted in the 394 submission.

16 TIDAL AND WAVE POWER↗
DOE OSTI2021

UNH TDP - Load Cell Raw Data and Processing Scripts - Fall 2021

This submission contains raw Load Cell data and processing scripts associated with MHKDR submission 394 (UNH TDP - Concurrent Measurements of Inflow, Power Performance, and Loads for a Grid-Synchronized Vertical Axis Cross-Flow Turbine Operating in a Tidal Estuary, DOI: 10.15473/1973860) from the University of New Hampshire and Atlantic Marine Energy Center (AMEC) turbine deployment platform. The user is directed to the MHKDR submission 394 for relevant context and detail of this deployment; see link below. The 394_READ_ME file here provides the description from that submission for quick reference. The READ_ME file for this specific instrument from the 394 submission is also available here. This submission contains a zipped folder structure containing raw data in its original format and MATLAB (2019a) processing scripts used to process and manipulate the data into its final form. The final data products are submitted in the 394 submission.

16 TIDAL AND WAVE POWER↗
DOE OSTI2022

Canopy spectra, Feb2017, PA-SLZ: Panama

Canopy spectra of sunlit canopy of Guarea kunthiana, Brosimum utile, Terminalia amazonia (TERMAM), Vochysia ferruginea (VOCHFE), Miconia borealis (MICOBO) and Guatteria dumetorum (GUATDU) species from the Smithsonian Tropical Research Institute (STRI) canopy crane site in the San Lorenzo National Park, Republic of Panama (PA-SLZ: Bosque Protector San Lorenzo). Canopy spectra were measured at 1:20 – 2 pm on 22 February 2017, using a SVC spectroradiometer. This data package includes the raw SVC data (*.sig), processed data of individual spectra, and processed spectra averaged over the canopy of each of the six trees measured (*.csv). The package also includes photographs of the canopy spectral targets, metadata and the instrument manual. This data was collected as part of the 2017 Brookhaven National Laboratory – Smithsonian Tropical Research Institute leaf traits by age campaign.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2021

Pycheron: A Python-Based Seismic Waveform Data Quality Control Software Package

Supplementing an existing high-quality seismic monitoring network with openly available station data could improve coverage and decrease magnitudes of completeness; however, this can present challenges when varying levels of data quality exist. Without discerning the quality of openly available data, using it poses significant data management, analysis, and interpretation issues. Incorporating additional stations without properly identifying and mitigating data quality problems can degrade overall monitoring capability. If openly available stations are to be used routinely, a robust, automated data quality assessment for a wide range of quality control (QC) issues is essential. To meet this need, we developed Pycheron, a Python-based library for QC of seismic waveform data. Pycheron was initially based on the Incorporated Research Institutions for Seismology’s Modular Utility for STAtistical kNowledge Gathering but has been expanded to include more functionality. Pycheron can be implemented at the beginning of a data processing pipeline or can process stand-alone data sets. Its objectives are to (1) identify specific QC issues; (2) automatically assess data quality and instrumentation health; (3) serve as a basic service that all data processing builds on by alerting downstream processing algorithms to any quality degradation; and (4) improve our ability to process orders of magnitudes more data through performance optimizations. This article provides an overview of Pycheron, its features, basic workflow, and an example application using a synthetic QC data set.

58 GEOSCIENCES↗
DOE OSTI2021

UNH TDP - Shark Meter Raw Data and Processing Scripts - Fall 2021

This submission contains raw Shark Power Meter data and processing scripts associated with MHKDR submission 394 (UNH TDP - Concurrent Measurements of Inflow, Power Performance, and Loads for a Grid-Synchronized Vertical Axis Cross-Flow Turbine Operating in a Tidal Estuary, DOI: 10.15473/1973860) from the University of New Hampshire and Atlantic Marine Energy Center (AMEC) turbine deployment platform. The user is directed to the MHKDR submission 394 for relevant context and detail of this deployment; see link below. The 394_READ_ME file here provides the description from that submission for quick reference. The READ_ME file for this specific instrument from the 394 submission is also available here. This submission contains a zipped folder structure containing raw data in its original format and MATLAB (2019a) processing scripts used to process and manipulate the data into its final form. The final data products are submitted in the 394 submission.

16 TIDAL AND WAVE POWER↗
DOE OSTI2023

Unified Software Architecture for Advanced Materials and Manufacturing Technologies Data Management and Processing: FY 2023 Multidimensional Data Correlation Platform

This report details the various digital manufacturing activities ongoing at ORNL as part of the Advanced Materials and Manufacturing Technologies (AMMT) program. The AMMT program is exploring a data-driven approach to demonstrate the use of AM for the fabrication of components for nuclear applications, with the goal of providing a greater understanding of manufacturing quality outcomes that would pave the way toward the development of standards for certification and qualification. The objective of this work package is to establish a digital manufacturing discipline common to all participants of the AMMT program to improve the performance, reliability, and lifetime of nuclear components. As part of this effort, we will develop a unified software architecture for AMMT data management and processing, deploy the digital platform across AMMT participants’ facilities, and generate pedigreed datasets in a common format across multiple labs and facilities. To this end, the MDDC work package has focused on three activities during FY23. First, the MDF Digital Tool was overhauled to better serve the needs of the AMMT program. Next, multiple laser powder bed fusion (L-PBF) systems at the MDF were upgraded to a common sensor package for collecting comparable in situ data across machines. Finally, various improvements relevant to the AMMT program were implemented in the ORNL-developed software tool, Peregrine. This report marks the completion of FY23 milestone M3CR-22OR0403051: Report Describing the Architecture of the Digital Platform to Support AMMT Activities.

36 MATERIALS SCIENCE↗