Search NASA⌕ Search

SEARCH · Search NASA

Results for “Data Integration”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 469 records · Page 26

Mapping Support for Targeted Critical Minerals Exploration and Extraction

The United States’ dependency on imported minerals poses significant risks to economic stability and national security due to potential supply disruptions. Recognizing the strategic importance of critical minerals, the Department of Energy (DOE) emphasizes the need for a secure and resilient supply chain to support emissions reduction, technology development, and capitalization on clean energy opportunities. The DOE’s Office of Manufacturing and Energy Supply Chains (MESC), in collaboration with the Office of Policy (OP), addresses these vulnerabilities by focusing on upstream domestic critical minerals production, balancing extraction with social and environmental goals, including conservation, environmental justice, and respect for Tribal sovereignty. This report showcases a collaborative effort involving Idaho National Laboratory (INL), Argonne National Laboratory (Argonne), National Renewable Energy Laboratory (NREL), and the U.S. Geological Survey (USGS) to map mineral development potential along with key social and environmental datasets. A geographical information system (GIS)-based web map application was developed as a preliminary tool for environmental analysis, integrating 158 geospatial data layers such as critical habitat, land ownership, economic indicators, and environmental concerns. Data were sourced from agencies like the Bureau of Land Management (BLM) and USGS and processed using GIS technology to enhance visualization and analysis. The proposed analysis framework categorizes areas into high, mid, and low concern based on withdrawn lands, special status species, the Economic Development Capacity Index (EDCI) Mining Composite Index, and the Climate and Economic Justice Screening Tool (CEJST). While the application provides broad visualizations, it is not a substitute for detailed environmental reviews required under the National Environmental Policy Act (NEPA). Users must conduct further analyses and engage with tribal entities and other stakeholders for comprehensive planning. A case study of the Idaho Cobalt Belt (ICB) in Lemhi County, Idaho, has been provided in the report to illustrate the tool's practical use. This report introduces a GIS application and framework to support stakeholders in identifying and prioritizing areas for critical mineral exploration, promoting secure supply chains, and advancing the nation's energy independence through responsible resource stewardship.

54 ENVIRONMENTAL SCIENCES↗

Illuminating the Material World: Autonomous Microscopy to Understand Order, Disorder, and Everything In Between

Artificial intelligence (AI) holds immense promise for revolutionizing microscopy, yet its widespread adoption has been hindered by challenges ranging from user inexperience to limited model transferability and difficulties in operationalizing machine learning. This presentation showcases our approach to developing practical autonomy for materials discovery, aiming to accelerate the integration of AI into everyday microscopy workflows. As shown in Fig. 1, I will focus on three key areas: understanding order-disorder transitions, quantifying point defects, and achieving truly device-scale microscopy. First, I will demonstrate the power of multi-modal knowledge graphs for integrating diverse microscopy data. By combining imaging, spectroscopy, and diffraction data, these graphs provide a holistic view of material behavior, capturing the intricate relationships between different modalities [1,2]. I will present a case study on how these models illuminate the structural and chemical changes associated with irradiation in oxide thin films, revealing critical insights for designing materials for extreme environments like spaceflight and nuclear energy. Specifically, I will show how multi-modal analysis clarifies the evolution of order-disorder transitions under irradiation, a key factor influencing material performance in these applications. Next, I will address the challenge of quantifying point defects in 2D materials. We demonstrate the application of computer vision and transfer learning to accurately identify and classify various defect types, such as vacancies and substitutional atoms, and to quantify their concentrations. This information is crucial for understanding and tailoring the properties of 2D materials for applications in electronics, optoelectronics, and catalysis. For example, I will show how our models can characterize the topological distribution of point defects in MXene transition metal carbides, providing valuable insights for optimizing their performance in energy storage and separation science. Finally, I will discuss our progress toward autonomous device-scale microscopy [3,4]. We are fundamentally redesigning electron microscopes around the principles of machine reasoning, enabling automation beyond basic tasks like sample navigation and data acquisition to include sophisticated experimental design. This approach paves the way for truly reproducible and massively scaled analysis campaigns. I will emphasize the importance of autonomous microscopy platforms for high-throughput materials discovery and characterization, facilitating the rapid screening of materials for a broad range of applications and accelerating the development of next-generation technologies.

36 MATERIALS SCIENCE↗

One Earth Energy Static and Dynamic Reservoir Modeling

This report presents the static and dynamic reservoir modeling conducted for the CarbonSAFE Phase III Illinois Storage Corridor project to assess the feasibility of commercial-scale CO 2 storage in the Mt. Simon Sandstone at the One Earth Energy (OEE) site in McLean County, Illinois. Three-dimensional geocellular models of the Mt. Simon storage complex were developed in Petrel ® by integrating petrophysical log data, core analyses, and seismic surveys from the OEE #1 stratigraphic test well and two nearby wells, with multiple model versions created as new data became available. Dynamic reservoir simulations, performed using Landmark's Nexus software, progressed through three phases (preliminary, sensitivity, and UIC Class VI permit studies) evaluating injection scenarios across varying rates, well orientations, permeability models, and multi-well configurations. Results demonstrate that commercial-scale storage is feasible: three injection wells spaced approximately one mile apart can store a total of 90 million tonnes of CO 2 over 20 years, producing a combined plume with an equivalent radius of 3.2 miles and a maximum pressure-front-defined Area of Review of 178 mi 2 at the end of injection that diminishes to 34 mi 2 after 50 years of post-injection monitoring. Sensitivity analyses indicate that a 20% change in porosity or permeability yields approximately a 7% change in AoR radius, and that perforating the high-permeability arkosic zone minimizes the pressure front compared to injection in the upper Mt. Simon Sandstone.

09 BIOMASS FUELS↗

Chemically Enabled CO 2 -Enhanced Oil Recovery in Multi-Porosity, Hydrothermally Altered Carbonates in the Southern Michigan Basin (Final Technical Report)

This is the Final Technical Report for the project "Chemically Enabled CO 2 -Enhanced Oil Recovery in Multi-Porosity, Hydrothermally Altered Carbonates in the Southern Michigan Basin." Over the course of six years of collaboration between Battelle and project partners, all stated objectives of the program have been completed, including full geological characterization of the TBR trend (See companion report for Task2), laboratory and modeling experiments to determine the optimum composition and design of CO 2 -EOR operations in the TBR trend, execution of a field test of chemically-enhanced CO 2 in a TBR well, and integration of the data and learnings gathered during these efforts into a full-trend development plan. Detailed reporting on these activities, their outcomes, and implications for trend-wide development is provided in the report. This report and encompassed data will provide TBR field operators with detailed information on what worked, what did not work, and how to proceed with production optimization of their TBR assets using chemically-enhanced CO 2 -EOR. CO 2 -EOR is a relatively well understood and broadly implemented strategy for increasing incremental production across the oil and gas industry, but its application has been primarily focused on reservoirs with limited heterogeneity. The intention of this project was show first that the same physical mechanisms that improve recovery factors in homogeneous reservoirs (namely wettability alteration, viscosity alteration, oil swelling, and mobility control) are at play in heterogeneous reservoirs. This was proven by the project’s laboratory studies and dynamic simulations, with the potential exception of mobility control, which needs further study. The second intention was to demonstrate via direct field testing that CO 2 -EOR can work in a strongly heterogeneous reservoir. While the field test strategy implemented during this project did not succeed in producing oil, data gathered during the test sheds light on what may work for field operators who try chemically-enhanced CO 2 -EOR within their own reservoirs, significantly reducing the level of uncertainty carried by first-of-a-kind commercial efforts that could (and should) follow this test. Simultaneously, the project has identified several large-volume ethanol plants and other sources of CO 2 emissions in the region and provided a handrail that CO 2 emitters and field operators can leverage to capture, transport, and inject that CO 2 into their fields. This project has also shown that, in many cases, the economics of CO 2 -EOR in the TBR are attractive. And finally, by completing a project of this scope in the southern Michigan Basin, the project has contributed to the knowledge base and operational experience of field operators, state regulatory agencies, local service companies, and state universities, with CO 2 -EOR projects which should allow follow-on projects to proceed safely and efficiently.

02 PETROLEUM↗

Multiscale ACI Satellite Database

The SATELLITE_EAGLES_PNNL NetCDF dataset contains a suite of satellite- and reanalysis-derived atmospheric and surface parameters on a regular latitude–longitude grid. The dataset includes core geophysical fields such as land fraction, aerosol optical depth at multiple wavelengths (465, 550, 667, and 865 nm), sea surface temperature, estimated inversion strength, and various thermodynamic and dynamic quantities (e.g., relative humidity, vertical velocity, boundary-layer height, and surface fluxes) from both MERRA and ERA reanalysis products, provided as daily-mean and instantaneous values. A major component of the dataset consists of MODIS-retrieved cloud microphysical properties, including cloud droplet number concentration, cloud effective radius, optical thickness, and liquid water path, provided for three compositing regimes (“All,” “Q06,” and “G18”). Corresponding cloud-top parameters—temperature, height, and pressure—along with total and domain-mean cloud fraction fields are also included. The file further integrates additional satellite data from AMSR-E (for cloud water, rain water, and surface precipitation retrievals) and CERES (for top-of-atmosphere radiative fluxes, cloud fractions, and albedo). This dataset is designed to evaluate aerosol–cloud interactions in warm clouds, emphasizing the use of MODIS for deriving cloud droplet number concentration and liquid water path statistics. The complementary satellite and reanalysis fields are co-located and time-matched to the same instantaneous MODIS observations, enabling consistent comparisons between cloud properties, aerosol loading, and large-scale meteorological conditions. The dataset is recently featured in Christensen et al. (2025), Machine Learning Reveals Strong Grid-Scale Dependence in the Satellite Nd–LWP Relationship, EGUsphere [preprint], https://doi.org/10.5194/egusphere-2025-3850, 2025.

Christensen, Matthew [Pacific Northwest National L↗

Multiscale ACI Satellite Database

The SATELLITE_EAGLES_PNNL NetCDF dataset contains a suite of satellite- and reanalysis-derived atmospheric and surface parameters on a regular latitude–longitude grid. The dataset includes core geophysical fields such as land fraction, aerosol optical depth at multiple wavelengths (465, 550, 667, and 865 nm), sea surface temperature, estimated inversion strength, and various thermodynamic and dynamic quantities (e.g., relative humidity, vertical velocity, boundary-layer height, and surface fluxes) from both MERRA and ERA reanalysis products, provided as daily-mean and instantaneous values. A major component of the dataset consists of MODIS-retrieved cloud microphysical properties, including cloud droplet number concentration, cloud effective radius, optical thickness, and liquid water path, provided for three compositing regimes (“All,” “Q06,” and “G18”). Corresponding cloud-top parameters—temperature, height, and pressure—along with total and domain-mean cloud fraction fields are also included. The file further integrates additional satellite data from AMSR-E (for cloud water, rain water, and surface precipitation retrievals) and CERES (for top-of-atmosphere radiative fluxes, cloud fractions, and albedo). This dataset is designed to evaluate aerosol–cloud interactions in warm clouds, emphasizing the use of MODIS for deriving cloud droplet number concentration and liquid water path statistics. The complementary satellite and reanalysis fields are co-located and time-matched to the same instantaneous MODIS observations, enabling consistent comparisons between cloud properties, aerosol loading, and large-scale meteorological conditions. The dataset is recently featured in Christensen et al. (2025), Machine Learning Reveals Strong Grid-Scale Dependence in the Satellite Nd–LWP Relationship, EGUsphere [preprint], https://doi.org/10.5194/egusphere-2025-3850, 2025.

54 ENVIRONMENTAL SCIENCES↗

Total Power Factor Smart Contract with Cyber Grid Guard Using Distributed Ledger Technology for Electrical Utility Grid with Customer-Owned Wind Farm

In modern electrical grids, the numbers of customer-owned distributed energy resources (DERs) have increased, and consequently, so have the numbers of points of common coupling (PCC) between the electrical grid and customer-owned DERs. The disruptive operation of and out-of-tolerance outputs from DERs, especially owned DERs, present a risk to power system operations. A common protective measure is to use relays located at the PCC to isolate poorly behaving or out-of-tolerance DERs from the grid. Ensuring the integrity of the data from these relays at the PCC is vital, and blockchain technology could enhance the security of modern electrical grids by providing an accurate means to translate operational constraints into actions/commands for relays. This study demonstrates an advanced power system application solution using distributed ledger technology (DLT) with smart contracts to manage the relay operation at the PCC. The smart contract defines the allowable total power factor (TPF) of the DER output, and the terms of the smart contract are implemented using DLT with a Cyber Grid Guard (CGG) system for a customer-owned DER (wind farm). This article presents flowcharts for the TPF smart contract implemented by the CGG using DLT. The test scenarios were implemented using a real-time simulator containing a CGG system and relay in-the-loop. The data collected from the CGG system were used to execute the TPF smart contract. The desired TPF limits on the grid-side were between +0.9 and +1.0, and the operation of the breakers in the electrical grid and DER sides was controlled by the relay consistent with the provisions of the smart contract. The events from the real-time simulator, CGG, and relay showed a successful implementation of the TPF smart contract with CGG using DLT, proving the efficacy of this approach in general for implementing electrical grid applications for utilities with connections to customer-owned DERs.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Component Failure R&D

This presentation is to establish a scientific basis for risk and reliability analysis through integrated work with data collection, model development, and stakeholder engagement.

availability↗

Evolution of the ATLAS TDAQ online software framework towards Phase-II upgrade: Use of Kubernetes as an orchestrator of the ATLAS Event Filter computing farm

The ATLAS experiment at the LHC at CERN continuously evolves its TDAQ system to meet the challenges of new physics goals and technological advancements. As ATLAS prepares for the Phase-II Run 4 of the LHC, significant enhancements in the TDAQ Controls and Configuration (TDAQ-CC) tools have been designed to ensure efficient data collection, processing, and management. This abstract presents the evolution of ATLAS TDAQ-CC system leading up to Phase-II Run 4. As part of the evolution towards Phase-II, Kubernetes has been chosen to orchestrate the Event Filter (EF) farm. By leveraging Kubernetes, ATLAS can dynamically allocate computing resources, scale processing capacity in response to changing data taking conditions and ensure high availability of data processing services. The integration of the Kubernetes with the TDAQ Run Control framework enables perfect synchronisation between the experiment’s data acquisition components and the computing infrastructure. We will discuss the architectural considerations and implementation challenges involved in Kubernetes integration with the ATLAS TDAQ-CC system. We will highlight the benefits of using Kubernetes as an EF farm orchestrator, including improved resource utilization, enhanced fault tolerance, and simplified deployment and management of data processing workflows. In addition, we will report on the extensive testing of Kubernetes that was conducted using a farm of 2500 servers within the experiment data taking environment, demonstrating its scalability and robustness in handling the demands of the ATLAS TDAQ system for Phase-II. The adoption of Kubernetes represents a significant step forward in the evolution of ATLAS TDAQ-CC system, aligning with industry best practices in container orchestration.

Corso Radu, Alina [Univ. of California, Irvine, CA↗

Data Centers and Digital Assurance Introduction to Supply Chain and Cybersecurity for Data Centers, Session 1

The first session of the TADA (Technical Assistance for Digital Assurance) Data Centers Cohort Workshop, held on October 30, 2025, introduced foundational concepts of Digital Assurance in the context of data center and grid integration. Sponsored by the U.S. Department of Energy, the workshop brought together utilities, data center operators, developers, and vendors to address cybersecurity and supply chain vulnerabilities. The session emphasized the growing criticality of data centers within the electric grid and the need for secure, real-time, bidirectional communication. Participants explored the principles of Digital Assurance, including cybersecurity, cyber-informed engineering (CIE), and lifecycle security, and applied a threat-vulnerability-consequence framework to identify and mitigate risks at the data center–grid interface. Discussions covered a range of threats such as spoofed dispatch signals and insider threats, architectural vulnerabilities like SCADA interfaces and insecure protocols, and potential consequences including cascading grid failures. The session also raised strategic questions about business value, vendor assurance, and defining cyber boundaries and responsibilities. This foundational workshop set the stage for deeper technical analysis and the development of actionable frameworks in subsequent sessions. Session 1 of 3.

24 - POWER TRANSMISSION AND DISTRIBUTION↗

Electric Vehicle Charging Analytics and Reporting Tool (EV-ChART): Data Format and Preparation Guidance (V.5.0)

The Joint Office of Energy and Transportation maintains the Electric Vehicle Charging Analytics and Reporting Tool (EV-ChART), which provides a centralized hub for submitting electric vehicle (EV) charging infrastructure data directed by the Federal Highway Administration (23 CFR 680.112(a)-(c)). EV-ChART provides a streamlined data submission process and an integrated set of analytic tools, connects to other data sources, and empowers data sharing and access across stakeholders, including the public. Any data shared publicly will be aggregated and anonymized to stay in accordance with 23 CFR 680. This EV-ChART Data Format and Preparation Guidance provides a comprehensive overview of the data reporting requirements as authorized under 23 CFR 680.112(a)-(c)). The guidance is intended to be used alongside the EV-ChART Data Input Template, which defines the tabular data structure that these data submissions must follow. Per 23 CFR 680.112(a)-(c), the annual and quarterly data submissions are required of all National Electric Vehicle Infrastructure (NEVI) Formula Program projects, as well as projects for the construction of publicly accessible EV chargers that are funded with funds made available under Title 23, United States Code, including any EV charging infrastructure project funded with federal funds that is treated as a project on a federal-aid highway. One-time data submissions are required of both the NEVI Formula Program projects and grants awarded under 23 U.S.C. 151(f) for projects that are for EV charging stations located along and designed to serve the users of designated Alternative Fuel Corridors (AFCs). Other information and data required in 23 CFR 680, such as 23 CFR 680.112(d), 23 CFR 680.116(c), and 23 CFR 680.106(a), are not discussed in this guidance.

33 ADVANCED PROPULSION SYSTEMS↗

Supporting Information for the Integrated Methane Monitoring Platform Design (Final Report)

As the urgency for understanding methane emissions and the number of methane monitoring technologies being deployed have increased in the last two decades, there is an opportunity and a need to integrate the numerous disparate data sources to enable the detection, quantification, contextualization, and reporting of methane emissions along the oil and gas supply chain. Such an integration would enable emissions reductions through early detection of super emitters, data-driven mitigation strategies, and improvedgreenhouse gas inventories. The GTI Energy (“GTI”)project team (“the team”) worked with a multitude of industry experts, stakeholders, and subject matter experts (SMEs) to collect guidance, insights, and information to inform the requirements and subsequent engineering, design, deployment, and operations of an integrated methane monitoring platform (IMMP). This final report describes the results of the team’s effort to execute the Integrated Methane Monitoring Platform Design project, ultimately providing an engineering, design, deployment, and operating plan (EDDOP) for the IMMP. This document provides additional information supporting the findings in the final report.

03 NATURAL GAS↗

Integrated Methane Monitoring Platform Design (Final Report)

As the urgency for understanding methane emissions and the number of methane monitoring technologies being deployed have increased in the last two decades, there is an opportunity and a need to integrate the numerous disparate data sources to enable the detection, quantification, contextualization, and reporting of methane emissions along the oil and gas supply chain. Such an integration would enable emissions reductions through early detection of super emitters, data-driven mitigation strategies, and improved greenhouse gas inventories. The GTI Energy (“GTI”) project team (“the team”) worked with a multitude of industry experts, stakeholders, and subject matter experts (SMEs) to collect guidance, insights, and information to inform the requirements and subsequent engineering, design, deployment, and operations of an integrated methane monitoring platform (IMMP). This final report describes the results of the team’s effort to execute the Integrated Methane Monitoring Platform Design project, ultimately providing an engineering, design, deployment, and operating plan (EDDOP) for the IMMP. This final report summarizes and integrates the project tasks' results and outputs.

03 NATURAL GAS↗

21 cm Power Spectrum Analysis of North Celestial Pole Observations with the Tianlai Dish Pathfinder Array

The Tianlai Dish Pathfinder Array (TDPA) is a radio interferometer designed to test techniques for 21 cm intensity mapping in the post-reionization Universe as a means of measuring large-scale cosmic structure. Using nine nights of observations targeting the North Celestial Pole field, totaling approximately 107 hr of integration time, we analyze data in the frequency range 700–800 MHz (corresponding to redshift z ∼ 0.9). We do the data format conversion, radio frequency interference flagging, calibration, imaging and point source subtraction, and foreground removal via Singular Value Decomposition. The spherically averaged power spectrum Δ 2 (k) is obtained. Furthermore, this work successfully establishes and validates a comprehensive data analysis framework for the TDPA. We identify key improvements including sky model refinement, increased integration time, and pipeline optimization that will enable future detection of the 21 cm signal through auto-correlation and cross-correlation with optical galaxy surveys.

cosmology: large-scale structure of universe↗

Test of lepton flavor universality with a measurement of R ( D * ) using hadronic B tagging at the Belle II experiment

The ratio of branching fractions R ( D * ) = B ( B ¯ → D * τ − ν ¯ τ ) / B ( B ¯ → D * ℓ − ν ¯ ℓ ) , where ℓ is an electron or muon, is measured using a Belle II data sample with an integrated luminosity of 189 fb − 1 at the SuperKEKB asymmetric-energy e + e − collider. Data is collected at the ϒ ( 4 S ) resonance, and one B meson in the ϒ ( 4 S ) → B B ¯ decay is fully reconstructed in hadronic decay modes. The accompanying signal B meson is reconstructed as B ¯ → D * τ − ν ¯ τ using leptonic τ decays. The normalization decay, B ¯ → D * ℓ − ν ¯ ℓ , produces the same observable final-state particles. The ratio of branching fractions is extracted in a simultaneous fit to two signal-discriminating variables in both channels and yields R ( D * ) = 0.262 − 0.039 + 0.041 ( stat ) − 0.032 + 0.035 ( syst ) . This result is consistent with the current world average and with Standard Model predictions. Published by the American Physical Society 2024

Adachi, I. (ORCID:0000000322870173)↗

Interpretation of Ion Irradiation and Neutron Irradiation Damage in Additively Manufactured 316 Stainless Steel using Multiscale Modeling

The accelerated adoption of nuclear energy necessitates advanced manufacturing technologies, such as additive manufacturing, to meet heightened supply chain requirements and support innovative reactor technologies. Due to the unique microstructural characteristics of additively manufactured materials under distinct solidification conditions, comprehensive evaluation of their performance in reactor environments is essential. The Advanced Materials and Manufacturing Technologies program under the Department of Energy's Office of Nuclear Energy focuses on understanding the irradiation performance and damage evolution of laser powder bed fusion 316 stainless steel, with an emphasis on integrating ion and neutron irradiation data to accelerate the development and qualification of materials for advanced nuclear reactor applications. While ion irradiation is a cost- and time-effective method, modeling and simulation are required to interpret the data for the broader range of irradiation conditions encountered in advanced reactors. In fiscal year 2025, integrated multiscale modeling and simulations were conducted to assess irradiation damage in additively manufactured 316 stainless steel. Key outcomes include predictions of chromium enrichment at grain boundaries, nickel enrichment at dislocation cell walls and void surfaces, and heterogeneous void evolution under ion and neutron irradiation conditions. Cluster dynamics simulations revealed the coarsening of voids at high irradiation temperatures and the suppression of void growth by high network dislocation density, while also demonstrating significant growth and coarsening of voids and self-interstitial atom loops at low dose rates. Machine learning-accelerated atomistic simulations highlighted the impact of the local environment and chromium concentration on vacancy diffusivity, providing key insights on the influence of composition on void swelling and radiation-induced segregation. Additionally, molecular dynamics simulations demonstrated the presence of defect production bias and a significant effect of carbon content on defect cluster behavior. These combined efforts aim to predict the performance of additively manufactured materials under various reactor conditions, supporting their qualification for nuclear reactor applications by interpreting ion irradiation data. This report underscores the potential of integrated multiscale modeling to analyze ion irradiation data in the effort to accelerate the qualification of additively manufactured materials for nuclear reactor components.

316 stainless steel↗

Datum: A Scientific Metadata Catalog

The data catalog market is currently flooded with a myriad of different products, but none serve the scientific community well. There are cloud-native tools like Databricks, Snowflake,to on-premise solutions like Collibra and Datahub. The common failing of all these tools however, is their inability to serve the scientific data community directly. Most catalogs are targeted towards financial, health, or user data - not sensor or scientific domain data. They also prioritize integrations that often don’t exist or are just starting to be used in the scientific realm - all while ignoring common scientific tools and file types. Datum is a catalog which targets the scientific data directly, including the tools and networks in which those tools are used. We work with the producers and consumers of the data where they are, targeting cloud and on-premise with a focus on classified networks. Datum is an Erlang/Elixir application. Technical Features Note: The features listed below are still under development and may change, slightly, upon final delivery of the product. File Formats - Datum has the ability to read additional metadata and provides processing pipelines for the following file formats: Plain Text, PDF, LaTeX, HTML, Open Document Format (.odt), XML, CSV/TSV (and other standard delimiters), OpenDocument Database and Spreadsheets, Geo-Referenced TIFF, Common Data Format, HDF/HDF5, LabView TDMS, Excel, DeltaTables, Parquet, Apache Iceberg, Apache Hudi and many others. Metadata Collection - Scanners for the local and networked file systems and cloud storage providers. Network integration with common databases such as MSSQL and MySQL. User Plugin System - Users are able to provide either file processing, metadata extraction, or sampling plugins in the programming language of their choice. Authentication/Authorization -: OIDC integration, SCIM provisioning and EntraID integration out of the box. Full user and group management system with a “least privilege” operating mode. Governance - Customizable data governance platform; dictate and enforce required metadata, enforce data embargos, and enforce user agreements and NDAs before data access. Ability to create health checks on data, rejecting abandoned or poorly curated data and automatically removing it from the search index. Ability for users to submit corrections. Search - Semantic search is a first class citizen. No licenses to expensive, external software required. Integrated use of vectors and vector-based search allows for AI agent integration at all levels of operation. Metadata Model - Display and control data’s lineage and connections to other data and data directories. Data is modeled after a filesystem - an organization instantly recognizable and navigable by most any user. CLI and SDK - Ships with a Command Line Interface (CLI) tool and with a fully-featured Python SDK. This allows for rapid and programmatic use of Datum by every level of user. Minimal Infrastructure - Datum ships as a single executable file and can be run on any operating system and most CPU architectures. Datum has no reliance on external databases, search indexing tools, or other outside services - and it runs equally well on edge computing devices, cloud services, or in a clustered HPC environment.

darrington, john↗