Search NASA⌕ Search

SEARCH · Search NASA

Results for “Open data”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 235 records · Page 13
All sourcesNASA NTRSDOE OSTI
DOE OSTI2022

MZA: A Data Conversion Tool to Facilitate Software Development and Artificial Intelligence Research in Multidimensional Mass Spectrometry

Modern mass spectrometry-based workflows employing hybrid instrumentation and orthogonal separations collect multidimensional data, potentially allowing deeper understanding in omics studies through adoption of artificial intelligence methods. However, the large volume of these rich data challenges existing data storage and access technologies, therefore precluding informatics advancements. Here we present MZA™ (pronounced m-za), the mass-to-charge (m/z) generic data storage and access tool designed to facilitate software development and artificial intelligence research in multidimensional mass spectrometry measurements. Composed by a data conversion tool and a simple file structure based on the HDF5 format, MZA provides easy, cross-platform and efficient programmatic access to raw MS-data, enabling fast development of new tools in data science programming languages such as Python and R. The software executable and example Python and R scripts are freely available at https://github.com/PNNL-m-q/mza.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2023

Panorama 360 (Final Report)

This is the final technical report for the DOE-funded Panorama 360 project. Panorama 360 provided a resource for the collection, analysis, and sharing of performance data about end-to-end scientific workflows executing on DOE facilities. The work focused on workflows that include experimental data generation at DOE facilities. The main activities of Panorama 360 include the development of: 1. A distributed repository that stores different types of workflow execution data (e.g., point and time series performance traces at fine- and coarse-grained levels); 2. A set of open-source data capture, curation, and publishing tools fully integrated with a state-of-the-art workflow management system that automates data ingestion to the repository and enables users to discover, query, and process data from the repository; 3. A set of analysis algorithms and machine learning based tools to perform analysis and characterization of the gathered data, which can be used to detect anomalous performance or system faults; and 4. Best practices and recommendations for workflow evaluation, analysis, execution, and architectures.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Panorama 360 (Final Report)

This final technical report from the lead institution, USC grant #DE-SC0012636, serves as the final technical report for collaborative institution UNC-CH grant #DE-SC0012390. The goal was to develop a repository and associated capabilities for data collection, ingestion, and analysis for a broad class of DOE applications that span experimental and simulation science workflows. In particular, this work focuses on workflows that include experimental data generation at DOE facilities. The main activities of Panorama 360 include the development of: (1) A distributed repository that stores different types of workflow execution data (e.g., point and time series performance traces at fine- and coarse-grained levels); (2) A set of open-source data capture, curation, and publishing tools fully integrated with a state-of-the-art workflow management system that automates data ingestion to the repository and enables users to discover, query, and process data from the repository; (3) A set of analysis algorithms and machine learning based tools to perform analysis and characterization of the gathered data, which can be used to detect anomalous performance or system faults; and (4) Best practices and recommendations for workflow evaluation, analysis, execution, and architectures.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

Efficient Data Management in Neutron Scattering Data Reduction Workflows at ORNL

Oak Ridge National Laboratory (ORNL) experimental neutron science facilities produce 1.2 TB a day of raw event-based data that is stored using the standard metadata-rich NeXus schema built on top of the HDF5 file format. Performance of several data reduction workflows is largely determined by the amount of time spent on the loading and processing algorithms in Mantid, an open-source data analysis framework used across several neutron sciences facilities around the world. The present work introduces new data management algorithms to address identified input output (I/O) bottlenecks on Mantid. First, we introduce an in-memory binary-tree metadata index that resemble NeXus data access patterns to provide a scalable search and extraction mechanism. Second, data encapsulation in Mantid algorithms is optimally redesigned to reduce the total compute and memory runtime footprint associated with metadata I/O reconstruction tasks. Results from this work show speed ups in wall-clock time on ORNL data reduction workflows, ranging from 11% to 30% depending on the complexity of the targeted instrument-specific data. Nevertheless, we highlight the need for more research to address reduction challenges as experimental data volumes increase.

Godoy, William↗
DOE OSTI2020

Temporal Analysis and Scene Change Detection in Multispectral Overhead Imagery

Scene change detection can be a tedious and time consuming process especially when concerning large geographical areas, and the process can be even more cumbersome when analyzing changes in an area over large spans of time. Developing a useful way to help analysts recognize at what points in time significant changes to a scene have occurred can allow them to better focus their efforts in characterizing events. Applications include: Facility monitoring, Construction chronology, Monitoring of vehicle/aircraft activity, Characterization of larger sequences of events. In large areas exceeding hundreds to thousands of square kilometers in size, it can be difficult localizing when scene changes have occurred. Analysts can spend hours going through imagery to try to identify new construction, monitor facility activities, monitor vehicle movement, etc. where the object of interest may only be a few square meters. Our goal is to help cut down this time by giving analysts change maps with hot spots of change, allowing them to focus on regions that have experienced actual change in time frames they're interested in. Additionally, by combining these change maps into layers within a data cube, analysts can examine the change maps from a temporal perspective, allowing events to be characterized over spans of time. By opening the data cube in an imaging software capable of separating the layers, we can analyze the change maps sequentially, allowing us to examine scene changes occurring over time. As an example, we examined overhead imagery from Planet Labs of what appears to be a parking lot on Fort Irwin over the course of a year using ENVI, a geospatial satellite imagery analysis software. Using ENVI, we generate a graph of changes over time, and notice a particular segment near the end of our analysis window where no changes are detected. Examination of the actual satellite imagery reveals that during this time span, the parking lot was empty. This could be due to facility shutdown for maintenance or upgrades, or possibly even total workforce/vehicle fleet movement. Information like this could help analysts better characterize events, as well as to help create clearer timelines in larger sequences of events. Workflow steps: - Collect multiple maps of the same AOI (Area of Interest) during a time span of interest; - Generate change maps from AOI maps; - Generate data cube from change maps. An analyst can use the data cube to help inspect an AOI for activities within a time span of interest. If an event of interest is discovered, the analyst can then refer to the maps corresponding to the appropriate dates and times in the data cube to see precisely what is transpiring. The biggest objective being worked on is improving the change detection methodology employed. We currently use PCA-EM (Principal Component Analysis with Expectation Maximization), but we are currently focusing on implementing IR-MAD (Iteratively Reweighted Multivariate Alteration Detection) to be used in conjunction with PCA-EM in an effort to decrease false positivity and noise in the change maps we generate.

42 ENGINEERING↗
DOE OSTI2025

Optimizing Deep Learning Models for Climate-Related Natural Disaster Detection from UAV Images and Remote Sensing Data

This research study utilized artificial intelligence (AI) to detect natural disasters from aerial images. Flooding and desertification were two natural disasters taken into consideration. The Climate Change Dataset was created by compiling various open-access data sources. This dataset contains 6334 aerial images from UAV (unmanned aerial vehicles) images and satellite images. The Climate Change Dataset was then used to train Deep Learning (DL) models to identify natural disasters. Four different Machine Learning (ML) models were used: convolutional neural network (CNN), DenseNet201, VGG16, and ResNet50. These ML models were trained on our Climate Change Dataset so that their performance could be compared. DenseNet201 was chosen for optimization. All four ML models performed well. DenseNet201 and ResNet50 achieved the highest testing accuracies of 99.37% and 99.21%, respectively. This research project demonstrates the potential of AI to address environmental challenges, such as climate change-related natural disasters. This study’s approach is novel by creating a new dataset, optimizing an ML model, cross-validating, and presenting desertification as one of our natural disasters for DL detection. Three categories were used (Flooded, Desert, Neither). Our study relates to AI for Climate Change and Environmental Sustainability. Drone emergency response would be a practical application for our research project.

AI↗
DOE OSTI2023

Strong Axiality in a Dysprosium(III) Bis(borolide) Complex Leads to Magnetic Blocking at 65 K

Substituted dysprosocenium complexes of the type [Dy(Cp R ) 2 ] + exhibit slow magnetic relaxation at cryogenic temperatures and have emerged as top-performing single-molecule magnets. The remarkable properties of these compounds derive in part from the strong axial ligand field afforded by the cyclopentadiene anions, and the design of analogous compounds with even stronger ligand fields is one promising route toward identifying new single-molecule magnets that retain a magnetic memory at even higher temperatures. Here, we report the synthesis and characterization of a dysprosium bis(borolide) compound, [K(18-crown-6)][Dy(BC 4 Ph 5 ) 2 ] (1), featuring the dysprosocenate anion [Dy(BC 4 Ph 5 ) 2 ] - with a pseudoaxial coordination environment afforded by two dianionic pentaphenyl borolide ligands. Variable-field magnetization data reveal open magnetic hysteresis up to 66 K, establishing 1 as a top-performing single-molecule magnet among its dysprosocenium analogues. Ac magnetic susceptibility data indicate that 1 relaxes via an Orbach mechanism above ~80 K with U eff = 1500(100) cm -1 and τ 0 = 10 -12.0(9) s, whereas Raman relaxation and quantum tunneling of the magnetization dominate at lower temperatures. Compound 1 exhibits a 100 s blocking temperature of 65 K, among the highest reported for dysprosium-based single-molecule magnets. Ab initio spin dynamics calculations support the experimental U eff and τ 0 values and enable a quantitative comparison of the relaxation dynamics of 1 and two representative dysprosocenium cations, yielding additional insights into the impact of the crystal field splitting and vibronic coupling on the observed relaxation behavior. Importantly, compound 1 represents a step toward the development of alternatives to substituted dysprosocenium single-molecule magnets with increased axiality.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2024

Carbon Storage Site Mapping Inquiry Tool (MapIT)

To date, 48 projects, consisting of 139 wells, are currently under review with the Environmental Protection Agency’s (EPA) Underground Injection Control (UIC) Program for Class VI – wells used for geologic sequestration of carbon dioxide. The number of applications submitted is expected to increase in coming years with the increase of the 45Q tax credit available to projects that initiate construction prior to 2033. The amount of data collected to submit a Class VI permit is vast, and often disparate, coming from state, federal, and commercial entities, as well as field-specific data collected within an area of interest. When preparing for site selection and permitting, the initial aggregation of relevant public data can be time intensive. The Carbon Storage Site Mapping Inquiry tool (MapIT) was created to support and accelerate the discovery and accessibility of open-source data and information available across the USA. Data was aggregated and organized based on data types described within the EPA UIC Class VI permit documentation. The online tool enables users to explore hundreds of geospatial data layers and connect to additional external resources, leveraging API and REST services where possible to ensure updates to data in real time. MapIT enables users to explore state and federal data related to geologic, geophysical, structural, hydrologic, and contextual information. In addition to displaying spatial data and linking to external resources, MapIT leverages custom widgets to ensure that internal data and external data are discoverable and accessible. The widgets connect users to resources such as the USGS publications and the USGS Earthquake Catalog based on a user-defined location. This talk will describe data aggregation workflows, data types, data preparation, and tool development for MapIT. The Carbon Storage Site Mapping Inquiry Tool and underlying database are valuable, intuitive resources that empower government, academic, commercial and industry stakeholders to explore, analyze, and acquire carbon storage related data.

Morkner, Paige↗
DOE OSTI2022

The United States COVID-19 Forecast Hub dataset

Academic researchers, government agencies, industry groups, and individuals have produced forecasts at an unprecedented scale during the COVID-19 pandemic. To leverage these forecasts, the United States Centers for Disease Control and Prevention (CDC) partnered with an academic research lab at the University of Massachusetts Amherst to create the US COVID-19 Forecast Hub. Launched in April 2020, the Forecast Hub is a dataset with point and probabilistic forecasts of incident cases, incident hospitalizations, incident deaths, and cumulative deaths due to COVID-19 at county, state, and national, levels in the United States. Included forecasts represent a variety of modeling approaches, data sources, and assumptions regarding the spread of COVID-19. The goal of this dataset is to establish a standardized and comparable set of short-term forecasts from modeling teams. These data can be used to develop ensemble models, communicate forecasts to the public, create visualizations, compare models, and inform policies regarding COVID-19 mitigation. These open-source data are available via download from GitHub, through an online API, and through R packages.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2022

Validation of standardized data formats and tools for ground-level particle-based gamma-ray observatories

Context. Ground-based γ-ray astronomy is still a rather young field of research, with strong historical connections to particle physics. This is why most observations are conducted by experiments with proprietary data and analysis software, as is usual in the particle physics field. However, in recent years, this paradigm has been slowly shifting toward the development and use of open-source data formats and tools, driven by upcoming observatories such as the Cherenkov Telescope Array (CTA). In this context, a community-driven, shared data format (the gamma-astro-data-format, or GADF) and analysis tools such as Gammapy and ctools have been developed. So far, these efforts have been led by the Imaging Atmospheric Cherenkov Telescope community, leaving out other types of ground-based γ-ray instruments. Aims. We aim to show that the data from ground particle arrays, such as the High-Altitude Water Cherenkov (HAWC) observatory, are also compatible with the GADF and can thus be fully analyzed using the related tools, in this case, Gammapy. Methods. We reproduced several published HAWC results using Gammapy and data products compliant with GADF standard. We also illustrate the capabilities of the shared format and tools by producing a joint fit of the Crab spectrum including data from six different γ-ray experiments. Results. We find excellent agreement with the reference results, a powerful confirmation of both the published results and the tools involved. Conclusions. The data from particle detector arrays such as the HAWC observatory can be adapted to the GADF and thus analyzed with Gammapy. A common data format and shared analysis tools allow multi-instrument joint analysis and effective data sharing. To emphasize this, a sample of Crab nebula event lists is made public with this paper. Because of the complementary nature of pointing and wide-field instruments, this synergy will be distinctly beneficial for the joint scientific exploitation of future observatories such as the Southern Wide-field Gamma-ray Observatory and CTA.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2023

Understanding and Improving Energy Efficiency of Regional Mobility Systems Leveraging System-Level Data

Increased congestion required urban Americans to travel 6.8 billion hours more and purchase 3.1 billion gallons of fuel for a congestion cost of $\$$153 billion, according to the 2019 Urban Mobility Report. How to effectively manage the regional mobility system and improve the energy efficiency presents a big challenge to public agencies. Recent years have witnessed massive multi-jurisdictional multi-modal system-level data from various sources, which provides an unprecedented opportunity to improve the mobility system and its energy efficiency. However, implications of system-level data for mobility and energy efficiency are unclear. Those system-level data sets are siloed, spatially and temporally sparse, biased, not unified, and lacking of insights for system management. Consequently, there is a real need to acquire, fuse, mine and learn from multi-source system-level data to prepare public agencies to deal more effectively with large-scale energy efficiency modeling, management and planning. This project proposes to intensively review inexpensive, replicable and openly-accessible data from multi-modal systems, develop a data-driven system-level modeling framework enabled and validated by data, identify the energy inefficiencies of mobility systems from infrastructure, vehicles, passenger systems, and quantify the benefits of system-level strategies to improve mobility/energy efficiency. In addition, this research develops models to effectively estimate energy consumption and emissions from various types of vehicles on the roadway networks, with high granularity and high fidelity. Traditional models often heavily rely on aggregated infrastructure or vehicle/passenger data, for example, the census survey, land-use, and traffic counts of one or several classes, which may lead to research gaps considering the emerging vehicle technologies. Those models do not contain individual vehicular information. We propose an integrated data-driven method that combines multiple network modeling components, featuring the utilization of state-wide vehicle registration data. The additional vehicle registration data improve the model performance, and produce high-resolution vehicle-specific estimates of emissions and network performance metrics. Two case studies on the Pittsburgh and Philadelphia regional network show that the proposed method can efficiently and effectively estimate the emissions of a large-scale network, and provide valuable information for evaluating common management strategies.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2022

Accuracy of predictions made by machine learned models for biocrude yields obtained from hydrothermal liquefaction of organic wastes

Hydrothermal liquefaction (HTL) has potential for converting abundant wet organic wastes into renewable fuels. Because HTL consists of a complex reaction network, deterministic, physics-based prediction of its biocrude yield is prohibitively difficult. Data-driven methods provide an alternative to the physics-based approach; however, rigorous testing must be performed to ensure the accuracy of predictions made by data-driven methods. To this end, a data set was assembled consisting of 570 data points appearing in the open literature. The data set was divided into training, validation, and test sub-sets and used for evaluating different machine learning regression approaches to predict biocrude yield. Among the tested algorithms, Random Forest and eXtreme Gradient Boosting (XGBoost) predicted biocrude yields in a test set that had not been used for training with the greatest accuracy, with root mean square errors (RMSE) of 8.34 and 8.57, respectively. Further refinement of the Random Forest model reduced its RMSE to 8.07. In comparison, predictions of a series of literature models resulted in RMSE ranging from 9.16 in the most accurate case to 27.6 in the least accurate; most literature models yielded RMSE values > 10. Using biocrude yield predictions from the most accurate Random Forest model and a probabilistic economic analysis found that the model accuracy is sufficient to prioritize allocation of resources based on projected minimum fuel selling price. In our report the models and analysis represent a major advance in the ability to use readily available data to predict biocrude yields on new feedstocks that have not previously been studied.

42 ENGINEERING↗
DOE OSTI2023

Schneider Springs Fire Study 2023 for Ecosystem Respiration Rates: Surface Water Chemistry and Hydrologic Sensor Data across the Yakima River Basin, Washington, USA (v2)

This dataset supports a broader study examining the drivers of spatial variability in wildfire impacts across the Yakima River Basin. Data provided within this dataset were generated from sample collection across 17 total sites (8 sites affected by a recent wildfire, 9 sites unaffected by a recent wildfire) within multiple rivers throughout the Yakima River Basin in Washington, USA from May-July 2023. Fire affected sites are defined as those affected by the 2021 Schneider Springs Fire, based on the drainage area of the streams being within the 2021 Schneider Springs Fire burn perimeter or not (Figure 1, below). The contents include surface water geochemistry data (dissolved organic carbon; total dissolved nitrogen; total suspended solids); short-term sonde data (specific conductivity; turbidity; pH; chlorophyll A; temperature); stream depth data; stream velocity; manual chamber open channel respiration data; sensor time-series data (oxygen; water pressure; barometric pressure); field metadata (including qualitative information on in stream and river corridor characteristics); and environmental context photos taken in the field. The dataset also includes a summary file of the sensor data and plots of the sensor data. Sensors were only recovered at 15 out of the 17 sites, and not all sensors were recovered at all 15 sites (see Methods section for more details), therefore all data does not exist at all sites. Data from a 2022 study at the same sites, as well as additional sites, can be found at https://data.ess-dive.lbl.gov/view/doi:10.15485/1969566. The data package was originally published in November 2023. It was updated in June 2025 (v2; modified files). See the change history section in the readme for more details. For details on how to navigate data packages generated by this project, see https://data.ess-dive.lbl.gov/portals/PNNLRiverCorridorSFA/About. This dataset is comprised of one folder with field photos and one main data folder with two subfolders. The main data folder consists of (1) file-level metadata; (2) data dictionary; (3) field metadata; (4) field protocol; (5) readme; (6) international generic sample number (IGSN) mapping file; and (7) stream depth and averages. The sensor data subfolder consists of (1) sensor installation methods summary; (2) stream velocity; and (3) six subfolders. The BarotrollAtm (barometric pressure; temperature), DepthHOBO (water pressure; temperature), MantaRiver (specific conductivity; turbidity; pH; chlorophyll A; temperature), EXO (specific conductivity; pH; temperature), miniDOT (dissolved oxygen; temperature), and miniDOTManualChamber (dissolved oxygen; temperature) contain time-series data, plots, and summary files. The sample data subfolder consists of (1) total suspended solids (TSS) data; (2) dissolved organic carbon (DOC, measured as non-purgeable organic carbon, NPOC) data and averages; (3) total dissolved nitrogen (TN) data and averages; and (4) methods codes. All files are .csv, .pdf, .jpg, .jpeg, or .mov.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2022

Visualization Tool for Comparing Low-Carbon Energy Options

This project developed a web based visualization tool for comparing current and future nuclear fuel cycle options to low-carbon and conventional energy technologies in the United States. While nuclear power is well established as the gold-standard for baseload power production and also as the technology with the lowest overall carbon intensity of any commercial form of electricity generation. This project develop a web-based tool for electricity producers and consumers to compare renewable and conventional energy technologies to the conventional and advanced nuclear fuel cycle options. The work leveraged data on land use, carbon intensity, unit cost and pricing data for renewables from the open literature. Input data from the Advanced Fuel Cycle Cost Basis report will be used for current and advanced nuclear power systems. We coupled these data to energy flow and lifecycle models for user-selected energy generation and storage types. We will display economic comparisons from the perspectives of generators and consumers, as well as carbon production, land use, and reliability comparisons.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗
DOE OSTI2025

Transfer learning of neural surrogates on multifidelity groundwater simulations

Multifidelity data used in the paper published in Advances in Water Resources 206 (2025) 105140, https://doi.org/10.1016/j.advwatres.2025.105140 The code used to process the data is openly available on GitHub at https://github.com/Model-Reduction-and-UQ-Group/Transfer_Learning_K_reconstruction Computationally inexpensive surrogates of process-based models, such as deep neural networks, enable ensemble-based computations used in risk assessment, data assimilation, etc. However, generation of large datasets required to train a neural network can be as expensive as the ensemble simulations themselves. We ameliorate this challenge by using data from multifidelity (MF) groundwater simulations and transfer learning (TL) to reduce data generation costs while maintaining model accuracy. As a computational example, we train a deep convolutional neural network (CNN) to reconstruct permeability fields from saturation maps derived from a multiphase flow model. Starting with very low- and low-fidelity data generated on increasingly coarse meshes, we pretrain the CNN, followed by output-layer training and fine-tuning using only a limited number of high-fidelity samples. We demonstrate the surrogate’s robustness when interpreting low-quality inputs—such as interpolated maps or data affected by noise—which has strong implications for the applicability in practical hydrogeological scenarios. This multilevel MF-TL strategy achieves a favorable trade-off between computational efficiency and predictive accuracy, significantly outperforming high-fidelity-only approaches under the same computational budget.

Chiofalo, Alessia [University of Bologna] (ORCID:0↗
DOE OSTI2025

Capturing Historic Reliability Performance Through Graph Databases: A Model Based System Engineering Approach

With the goal of improving the performance and reliability of high dependable technological systems such as nuclear power plants, advanced monitoring and health management systems are employed to inform system engineers on observed degradation processes and anomalous behaviors of assets and components. This information is captured in the form of large amount of data which can be heterogenous in nature (e.g., numeric, textual). Such large data availability poses challenges when system engineers are required to parse and analyze them in order to track historic reliability performance of assets and components. This paper tackles directly this challenge by providing means to organize data in the form of a graph: a knowledge graph. The presented approach distinguish itself from current knowledge graph-based methods by the fact that model-based system engineering (MBSE) models are used to “put data into context”. In particular, MBSE models are used as skeleton of a knowledge graph; numeric and textual data elements, once processed, are associated to MBSE model elements. Thus, a knowledge graph captures both system architecture (though MBSE models) and health/performance data. Such feature opens the door to new data analytics methods designed to identify causal relations between observed phenomena.

97 - MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Data Sharing as a Catalyst for Expanding the Energy Frontier

As the energy landscape evolves to include technologies such as geothermal energy, comprehensive data become essential for driving innovation and scalability, particularly with the growing use of tools like machine learning and artificial intelligence. In emerging sectors, the cost of gathering high-quality data across large spatial areas can present a significant barrier. A key solution is leveraging existing data from well-established industries like oil and gas. However, the proprietary nature of data in these industries often hinders collaboration. This paper explores how cultivating a culture of data sharing can act as a catalyst for progress, fueling breakthroughs across both conventional and renewable energy sectors. Practical compromises that protect business interests while enabling data access are proposed, and real-world success stories are highlighted, demonstrating how collaboration has accelerated advancements in geothermal, carbon capture, and other innovative technologies.

15 GEOTHERMAL ENERGY↗
DOE OSTI2022

Learning Global Proliferation Expertise Evolution Using AI-Driven Analytics and Public Information

Detecting and anticipating global proliferation expertise and capability evolution from unstructured, noisy, and incomplete public data streams is a highly desired, but extremely challenging task. Here, in this article, we present our pioneering data-driven approach to support the non-proliferation mission to detect and explain the evolution of proliferation expertise and capability development globally from terabytes of publicly available information (PAI), focusing on our knowledge extraction pipeline and descriptive analytics. We first discuss how we fuse nine open-source data streams, including multilingual data, to convert 4 TB of unstructured data to structured knowledge and encode dynamically evolving proliferation expertise representations—content and context graphs. For this, we rely on natural language processing (NLP) and deep learning (DL) models to perform information extraction, topic modeling, and distributed text representation (aka embedding) learning. We then present interactive, usable, and explainable descriptive analytics to refine domain knowledge and present it in a human-understandable form. Finally, we introduce future work avenues that will leverage our dynamic knowledge representations and descriptive analytics to enable predictive and prescriptive inferences to achieve real-time domain understanding and contextual reasoning about global proliferation expertise and capability evolution.

98 NUCLEAR DISARMAMENT, SAFEGUARDS, AND PHYSICAL P↗