DEVELOPMENT OF BIG-DATA EDGE-COMPUTING ANALYTIC FRAMEWORK FOR DISTRIBUTED FIBER OPTIC SENSORS
Explore the source record for details and available documents.
02 PETROLEUM↗
SEARCH · Search NASA
Results for “Data Analytics”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Collaborative: in situ visual analytics technologies for extreme scale combustion simulations
This project aims to drastically enhance the usability of in situ analysis and visualization for extreme-scale scientific simulations. Current exascale computing capabilities promise to offer greater predictive ability of simulations and to further push the frontiers of science and technology. However, to validate the simulation output at extreme scale, examine the modeled phenomena, and discover previously unknowns from the output data, the output must be reduced or transformed in situ as it is being generated during the simulation such that the amount of data to examine and store is kept to a minimum. Such in situ approaches allow us to process and analyze the data and any embedded geometry to an extent that would be prohibitively expensive, if not impossible, to perform as a post hoc task. While in situ processing has been demonstrated to be a feasible and promising approach, its full potential has not yet been leveraged. In this project, we have developed comprehensive enhancements to in situ technology based on probability distributions in data. Our research focuses on jointly developing new ways of interacting with massive statistical samples while creatively utilizing new state-of-the-art computational resources to push the boundaries of in situ exploration. Moreover, we have developed new time-dependent techniques to enable previously unattainable capabilities in areas such as intelligent simulation steering and precise feature identification. We have experimentally studied our design and implementation at NERSC and OLCF, and are able to leverage existing in situ infrastructures whenever possible. While the exemplar in this project is combustion, many other fields for which turbulent transport is important, e.g., fusion, climate, astrophysics among others, encounter similar issues as simulations scale up to the exascale. This project shows its potential to generate high impact on DOE missions since the resulting technology promises to improve scientists’ ability to rapidly and correctly interpret and tune extreme-scale simulations, leading to new scientific understanding and advancements.
97 MATHEMATICS AND COMPUTING↗
Characterization Techniques for TRU Waste from the Analytical Laboratory Hot Cells at Idaho National Laboratory - 19610
Since the first reactor went hot in 1942, there has been a never ending desire to improve the nuclear reactor and the fuel that it burns. The Idaho National Laboratory has been a key player in reactor and fuel development. As more research projects begin, the spectrum of waste parameters grows. Over the years, the Analytical laboratory (AL) at the INL Materials and Fuels Complex (MFC) has supported a large range of various research projects. Each project brings its own challenge and analytical technique that, in turn, yields its own unique waste parameters. The AL Hot Cells at INL consist of 6 small cells that can support around 12 different fuel research projects at a time along with infrastructure and environmental samples. Each different project yields different levels of transuranic (TRU) material, RCRA constituents, dose, and contamination levels. The waste parameters from each project dictate a path forward for the waste as Low-Level Waste (LLW), Contact Handled TRU (CH-TRU), Mixed Contact Handled TRU (M-CH-TRU), Remote Handled TRU (RH-TRU), or Mixed Remote Handled TRU (M-RHTRU). With visual examination techniques and process knowledge as an aid, there have been waste handling methods developed that allow debris and sample material to be segregated. Unique characterization mechanisms have been applied for each debris and sample material that are generally more accurate than smear and surface contamination data. For debris found in the Hot Cells, analytical data is gathered and conservatively summarized for each fuel type and research method. This consists of data from Quadrupole Inductively Coupled Plasma Mass Spectrometry (Q-ICP-MS), Gamma Spectroscopy, and some data verification through Thermal Ionization Mass Spectrometry (TIMS) and Multi-Collector Inductively Coupled Plasma Mass Spectrometry (MC-ICP-MS). This data gives a range of TRU isotope concentrations. Fuel age is taken into account to determine the dominant TRU activity isotopes. Cesium concentrations are also gathered. Each parameter/fuel type then ties a measured Cesium number to TRU content based on historical analytical data. Debris in the hot cells is segregated by project and packaged in individual 2 gallon metal paint cans. Each can is counted by gamma spectroscopy in a calculated and measured geometry. Using process knowledge to determine the fuel or project type, newly counted cesium numbers can be applied to the historic analytical data and TRU content can be estimated. The TRU content allows waste debris to be segregated between CH-TRU and RH-TRU. Packaging techniques can allow for the minimization of RHTRU waste. Sample material received into the AL must be dissolved into a liquid matrix in order to be analyzed in most cases. When analysis is complete, these liquids go through a consolidation, complexation, neutralization, and solidification process where real time analytical data can be used to calculate TRU content and RCRA constitutes. These calculations allow for no further analytical data to be gathered. The waste matrix is considered stable and acceptable by all Acceptable Knowledge (AK) documents. The unique techniques used in characterizing and handling the Analytical Laboratories Hot Cell waste can be applied to various situations and various laboratories. They allow for more accurate data for waste constituents and also for the minimization of RH-TRU waste. (authors)
12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗
A Visual Analytics Framework for Reviewing Streaming Performance Data
Understanding and tuning the performance of extreme-scale parallel computing systems demands a streaming approach due to the computational cost of applying offline algorithms to vast amounts of performance log data. Analyzing large streaming data is challenging because the rate of receiving data and limited time to comprehend data make it difficult for the analysts to sufficiently examine the data without missing important changes or patterns. To support streaming data analysis, we introduce a visual analytic framework comprising of three modules: data management, analysis, and interactive visualization. The data management module collects various computing and communication performance metrics from the monitored system using streaming data processing techniques and feeds the data to the other two modules. The analysis module automatically identifies important changes and patterns at the required latency. In particular, we introduce a set of online and progressive analysis methods for not only controlling the computational costs but also helping analysts better follow the critical aspects of the analysis results. Finally, the interactive visualization module provides the analysts with a coherent view of the changes and patterns in the continuously captured performance data. Through a multi-faceted case study on performance analysis of parallel discrete-event simulation, we demonstrate the effectiveness of our framework for identifying bottlenecks and locating outliers.
Human-centered computing↗
A Processing and Analytics System for Microscopy Data Workflows: The Pycroscopy Ecosystem of Packages
Major advancements in fields as diverse as biology and quantum computing have relied on a multitude of microscopy techniques. Despite the considerable proliferation of these instruments, significant bottlenecks remain in terms of processing, analysis, storage, and retrieval of the acquired datasets. Aside from lack of file standards, individual domain-specific analysis packages are often disjoint from the underlying datasets, and thus keeping track of analysis and processing steps remains tedious for the end-user, hampering reproducibility. Here, in this study, the pycroscopy ecosystem of packages is introduced, an open-source python-based ecosystem underpinned by a common data model. The data model, termed the N-dimensional spectral imaging data format, is realized in pycroscopy's sidpy package. This package is built on top of dask arrays, thus leveraging dask array attributes, but expanding them to accelerate microscopy relevant analysis and visualization. Several examples of the use of the pycroscopy ecosystem to create workflows for data ingestion and analysis of scanning transmission electron microscopy (STEM) and scanning probe microscopy data are shown. Adoption of such standardized routines will be critical to usher in the next generation of autonomous instruments where processing, computation, and meta-data storage will be critical to overall experimental operations.
97 MATHEMATICS AND COMPUTING↗
Cross-property deep transfer learning framework for enhanced predictive analytics on small materials data
Abstract Artificial intelligence (AI) and machine learning (ML) have been increasingly used in materials science to build predictive models and accelerate discovery. For selected properties, availability of large databases has also facilitated application of deep learning (DL) and transfer learning (TL). However, unavailability of large datasets for a majority of properties prohibits widespread application of DL/TL. We present a cross-property deep-transfer-learning framework that leverages models trained on large datasets to build models on small datasets of different properties. We test the proposed framework on 39 computational and two experimental datasets and find that the TL models with only elemental fractions as input outperform ML/DL models trained from scratch even when they are allowed to use physical attributes as input, for 27/39 (≈ 69%) computational and both the experimental datasets. We believe that the proposed framework can be widely useful to tackle the small data challenge in applying AI/ML in materials science.
36 MATERIALS SCIENCE↗
Data from: 'Abiotic influences on continuous conifer forest structure across a subalpine watershed'
This package archives the core data used for analysis and inference in 'Abiotic influences on continuous conifer forest structure across a subalpine watershed' (Worsham et al., 2025). All data were collected in the East River, Washington Gulch, Slate River, and Coal Creek watersheds of Colorado. In the paper, we quantified the relative influence of climate, topographic, edaphic, and geologic factors on conifer stand structure and composition, and their functional relationships, at the watershed scale. We used waveform LiDAR data to derive spatially continuous stand structure metrics. We fused these with a species-level classification map to estimate tree species abundance. We applied generalized additive and generalized boosted models to evaluate the covariability of structural and compositional metrics with abiotic variables. The package contains the essential products required for reproducing our analysis and the tables and figures reported in the publication. The products comprise four classes: (1) geospatial data, (2) tabular data used for inferential analysis, (3) tabular data describing analytical results and performance statistics, and (4) a data user guide. (1) includes discretized waveform LiDAR data, locations and attributes of individual tree crowns, sampling locations and domain boundaries, a canopy height model, and raster files of estimated forest structural and compositional metrics at 100 m grid scale. (2) includes all response and explanatory variable values applied in inferential models. Response variables include conifer forest stand density, basal area, 95th percentile height, quadratic mean diameter, and others. Explanatory variables include climatic water deficit, actual evapotranspiration, elevation, heat load, soil available water content, and others. (3) includes results of training and testing several individual tree detection (ITD) algorithms, as well as inferential modeling results. (4) is a PDF user guide for this data package, including detailed descriptions and data dictionaries for all files. The data package root contains 17 assets: 8 compressed tape archive (.tar.gz) files, 5 comma-separated values (.csv) files, 3 Geographic Tagged Image File Format (GeoTIFF) (.tif) files, and 1 Portable Document Format (.pdf) file. The compressed .tar.gz archives contain ESRI shapefiles (.shp) .tif, compressed LASer (.laz), and .csv files. The archives must first be decompressed using the widely distributed command-line software utility TAR. All other files, including constituent files within the .tar.gz archives, can be opened in the open-source R statistical computing environment. Alternatively, .csv files may also be read in any simple text editor software or Microsoft Excel. Geospatial files including .shp and .tif files can also be opened in GIS software, such as QGIS (open-source) or ESRI ArcGIS (proprietary). The .pdf Data User Guide can be read with Adobe Acrobat Reader or other compatible readers.
2018 NEON and 2025 CHESS Campaigns↗
Analytical methods for online data quality assessment
This chapter provides a comprehensive overview of the main steps for algorithmic sensor signal quality assessment, which can enhance the decision-making process for water resource recovery facility (WRRF) operation and optimization. It introduces the concept of redundancy as the basis for data quality assessment. It also explains the typical data processing pipeline, which consists of preliminary analysis, data pre-processing, and specific algorithmic approaches. Each of these processes is presented and discussed in three separate sections. Importantly, this chapter introduces the main approaches for data quality assessment, provides guidelines for selecting the most suitable one and the key performance indicators to evaluate them and explains how to collect metadata through such an algorithmic approach.
Aguado, Daniel↗
Advanced Analytics of Rig Parameter Data Using Rock Reduction Model Constraints for Improved Drilling Performance.
Abstract not provided.
Raymond, David W.↗
Advanced Analytics of Rig Parameter Data Using Rock Reduction Model Constraints for Improved Drilling Performance.
Abstract not provided.
Raymond, David W.↗
Translytics: A Novel Approach for Runtime Selection of Database Layout Based on User’s Context
Currently, organizations have to maintain separate systems for transactions and analytics inside the company. Notably, different vendors provide the capabilities for either of these tasks that require specialized hardware or software. Data engineers are required to retrieve data from one source and transform it into another format to obtain the maximum benefit in a minimum time. Organizations strive for a competitive advantage that is achieved by fetching data from their customers and getting insights earliest for timely decision making. Present practices do not permit the view of the latest data for analytics since the first data have to be fetched from the source, transformed, and loaded to other systems to be utilized for analysis by relevant teams. This paper introduces a single system for both transactions and analytics. Our proposed solution would permit companies to seamlessly adapt our solution without the need to shift all of their data to newer systems and allow all the teams. It would grant all the teams to have a view of the latest available data without extra expertise and budget.
Tanvir, Muhammad Makhshif↗
Incorporating historical data and past analyses for improved tensile property prediction of 9% Cr steel
Data-driven analytical clustering and visualization techniques were applied to the dataset of 9% Cr experimental alloy data generated through the eXtremeMAT project. Techniques and results were compared with the resulting clusters obtained through similar analytical techniques on previous and reduced versions of the dataset. The principal components were generated in order to reduce the dimensionality of the complex dataset and to visualize the underlying trends in the data. Partitioning around medoids was performed on the resulting principal components to determine relevant clusters. Domain knowledge labels were further applied to the principal components to compare the labels with the trends identified through the clustering methods. The resulting clusters show the addition of the new data into the dataset, and show where alignment between the domain knowledge labels and partitioning around medoids (PAM) clusters show where the labels are appropriate. The clusters can be used to compare the tensile properties of the alloys, and to reduce the variation in the dataset.
Alloy design, Mechanical Properties, data analysis↗
From multivariate to functional data analysis: Fundamentals, recent developments, and emerging areas
Functional data analysis (FDA), which is a branch of statistics on modeling infinite dimensional random vectors resided in functional spaces, has become a major research area for Journal of Multivariate Analysis. We review some fundamental concepts of FDA, their origins and connections from multivariate analysis, and some of its recent developments, including multi-level functional data analysis, high-dimensional functional regression, and dependent functional data analysis. Here, we also discuss the impact of these new methodology developments on genetics, plant science, wearable device data analysis, image data analysis, and business analytics. Two real data examples are provided to motivate our discussions.
97 MATHEMATICS AND COMPUTING↗
Power System Waveform Datasets for Machine Learning
The desire for increased visibility across the electricity grid will necessarily increase the deployment of sensing and measurement devices and associated data management needs to unprecedented levels. For the existing sensing and measurement infrastructure, there remains a great amount of “value” yet to be extracted through advanced data management and analytics. Availability of more data will not, by itself, lead to changes in grid visibility, security, and resiliency. To create the predictive and prescriptive environment required to enable new markets and transactions for customer revenue and a reliable grid, the data must be collected, organized, evaluated, and analyzed using sophisticated algorithms to provide actionable information allowing operators and customers to reliably manage an increasingly complex grid. Progress in artificial intelligence (AI) has been largely driven by large, publicly available datasets that can be used to train AI algorithms such as MNIST, a database of handwritten images of digits, and ImageNet, an image database of everyday objects. These types of publicly available databases of real-world training datasets have been largely credited for advancement of image processing, computer vision, and deep learning algorithms that these use cases deploy. However, in the power systems industry to date, there are few databases with proper event labeling, and data access to a publicly available collection of power system event waveforms that will allow users to interact with grid signature data. Publicly available datasets of power system event waveforms, such as the DOE/EPRI dataset, often lack critical metadata or contain limited examples of each event type, and data formats vary widely across these datasets.
32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
Pollution inequality 50 years after the Clean Air Act: the need for hyperlocal data and action
Fifty years ago the Clean Air Act amendments of 1970 were the first major US legislation that authorized regulation of air pollutants, creating National Ambient Air Quality Standards (NAAQSs) to protect public health and the environment. While US air quality has improved, with average PM 2.5 concentrations in 2016 a third of 1981 levels, air pollution remains a major health risk in the US and globally. Moreover, air pollution impacts are still uneven, with the most polluted US communities of 50 years ago still so today. Air pollution “hot spots” result in disproportionate exposure at neighborhood scales within cities, particularly in disadvantaged communities, but an in-depth understanding at these scales is lacking. This policy perspective discusses how trends in sensor technology, spatial data collection, analytics and retrieval are converging to enable the production of hyperlocal air pollution data, and that this needs to be done in a manner that enables marginalized communities to shape decision making.
29 ENERGY PLANNING, POLICY, AND ECONOMY↗
Initial Characterization and Optimization of the Liquid Sampling-Atmospheric Pressure Glow Discharge Ionization Source Coupled to an Orbitrap Mass Spectrometer for the Determination of Plutonium
Plutonium measurements are essential to the nuclear forensics and safeguards community. The liquid sampling-atmospheric pressure glow discharge (LS-APGD) microplasma ionization source coupled with an Orbitrap mass spectrometer is a proven platform for uranium isotope ratio determinations. Here, this work expands the LS-APGD-Orbitrap platform capabilities by reporting the first-ever analysis of plutonium with the LS-APGD and the first-ever measurement of elemental plutonium with an Orbitrap mass spectrometer. This coupling has the potential to dramatically reduce the complex sample manipulations required for traditional analysis techniques employed for actinide isotope ratio determinations. As a first step toward the goal of simultaneous uranium and plutonium isotope ratio determinations, the initial characterization and optimization of the platform for the detection of plutonium are reported. Collision-induced dissociation modality settings were optimized to reduce water-related and other molecular clusters containing plutonium, maximizing 242 Pu 16 O 2 + responses. A design of experiments study was conducted to optimize the discharge conditions of the dual-electrode LS-APGD toward the responsivity of 242 Pu 16 O 2 +. The measurement sensitivity was determined from a Pu response curve, yielding a limit of detection of 10 fg (absolute) of total analyte when data was collected and processed with a Spectroswiss FTMS Booster X2 data acquisition system. Additionally, plutonium and uranium were measured in a simultaneous acquisition, and each analyte remained unaffected by the other. It is believed that the LS-APGD-Orbitrap platform could be a valuable addition to the nuclear forensics’ toolbox and, indeed, other scientific disciplines and regulatory communities in which rapid, high-resolution plutonium determinations are paramount.
37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
Operational Analytics Studies for ATLAS Distributed Computing: Data Popularity Forecast and Utilization of the WLCG Centers
Operational analytics is the direction of research related to the analysis of the current state of computing processes and the prediction of future states in order to anticipate imbalances and take timely measures to stabilize a complex system. There are two relevant areas in ATLAS Distributed Computing that are currently the focus of studies: user physics analysis including the forecast of popularity of data samples among users, and evaluating WLCG centers for their readiness to process user analysis payloads. Studying these areas is challenging due to the complexity involved, as it requires a comprehensive understanding of numerous boundary conditions typically found in large-scale distributed computing infrastructures. Forecasts of data popularity are problematic without the categorization of user tasks by their types (data transformation or physics analysis), which do not always appear on the surface but may induce noise, which introduces significant distortions for predictive analysis. Evaluating the WLCG resources by their analysis workloads is also a challenging task as it is necessary to find a balance between the workload of the resource, its performance, the waiting time for jobs on it, as well as the volume of jobs that it processes. This is especially difficult in a heterogeneous computing environment, where legacy resources are used along with modern high-performance machines. We will look at these areas of research in detail and discuss what tools and methods are used in our work, demonstrating results already obtained.
72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
Hanford SMART System: Calculation of Radionuclide Emissions, Ambient Air Concentrations and Error Propagation (Final Report)
This document, authored by Radiological Assessment Corporation (RAC), details the calculations used to convert laboratory radiological analytical sample data and field measurement data for both ambient air samplers and stack air samples to concentration values with associated uncertainty. The RAC supported an effort by Mission Support Alliance (MSA) to replace the Automated Bar Coding of All Samples at Hanford (ABCASH) and Environmental Release Summary (ERS) database software with a commercial off-the-shelf (COTS) data management software system. The ABCASH and ERS databases were not compatible with the Hanford system standards Windows 10 Operating System deployed in calendar year 2018. The replacement software, Sample Management and Analytical Results Tracking (SMART), is used for environmental data collection, analyses, and regulatory reporting for the Hanford Environmental Surveillance and Effluent Monitoring programs.
54 ENVIRONMENTAL SCIENCES↗