Results for “Open Data Repository” — Page 7

DOE OSTI2026

Methods for evaluation and treatment of data shift

This is a code repository for a set of tools for detecting and mitigating data shifts in machine learning. The goal of the tools is to provide capabilities for determining when new data sets differ from training data sets and for adapting existing models to new data or correcting data shifts (via domain adaptation). The components will be written in Python, a high-level programming language that takes advantage of the Python ecosystem of high-quality open-source packages for machine learning and signal processing.

Parikh, Nidhi↗

DOE OSTI2021

Classified library celebrates two-year anniversary print your own commemorative history posters

Today is the National Security Research Center’s second anniversary. These two years, though, have been eight decades in the making, said NSRC Director Riz Ali. “We opened our doors in 2019, but the Center’s beginnings date back to World War II when J. Robert Oppenheimer created the Lab in Los Alamos to develop the first-ever nuclear weapons,” Ali said. “An important part of opening the Lab included standing up a technical reports library. Since then, we have amassed millions of holdings and today we are one of the top research institutions in the country.” What had long-been a repository of classified archival materials is today a dynamic, vibrant library that outsizes the collections of other similar research libraries around the country. Today, the NSRC houses the Laboratory’s classified nuclear weapons research, data and reports, as well as unclassified historical artifacts – the stories, photos, films and more from the people who have been a part of the Lab from the start.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗

DOE OSTI2021

LABS: Los Alamos Benchmark Suite - Current status and planning [Slides]

The LABS repository now has over 500 verified and revised input files with about 600 input files left to get up to speed with what we currently use. After that there are many, many more to go. We're in the process of building code infrastructure, moving to template output files, automating input file generation, outputting file data mining for calculation results, and benchmarking selection on sensitivity and other benchmark characteristics. We intend to open source LABS. We'll most likely release the input files sooner rather than later. The associated tool will probably be open sourced later.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

DOE OSTI2024

The Role of Data Filtering in Open Source Software Ranking and Selection

Faced with more than 100M open source projects, a more manageable small subset is needed for most empirical investigations. More than half of the research papers in leading venues investigated filtering projects by some measure of popularity with explicit or implicit arguments that unpopular projects are not of interest, may not even represent "real" software projects, or that less popular projects are not worthy of study. However, such filtering may have enormous effects on the results of the studies if and precisely because the sought-out response or prediction is in any way related to the filtering criteria.This paper exemplifies the impact of this common practice on research outcomes, specifically how filtering of software projects on GitHub based on inherent characteristics affects the assessment of their popularity. Using a dataset of over 100,000 repositories, we used multiple regression to model the number of stars -a commonly used proxy for popularity- based on factors such as the number of commits, the duration of the project, the number of authors and the number of core developers. Our control model included the entire dataset, while a second filtered model considered only projects with ten or more authors. The results indicated that while certain characteristics of the repository consistently predict popularity, the filtering process significantly alters the relationships between these characteristics and the response. We found that the number of commits exhibited a positive correlation with popularity in the control sample but showed a negative correlation in the filtered sample. These findings highlight the potential biases introduced by data filtering and emphasize the need for careful sample selection in empirical research of mining software repositories. We recommend that empirical work should either analyze complete datasets such as World of Code, or employ stratified random sampling from a complete dataset to ensure that filtering is not biasing the results.

Malviya Thakur, Addi↗

DOE OSTI2024

RectifHydPlus Data Pipeline

The RectifHydPlus Data Pipeline is an open source and fully reproducible data processing pipeline for creating RectifHydPlus—a dataset of historical monthly net electricity generation for all US hydropower plants (>10MW). The pipeline is coded in R, applying tidyverse libraries and code principles, and using the targets data pipeline framework. All data inputs to the RectifHydPlus Data Pipeline are available from public sources. References to all data inputs, as well as instructions for running the RectifHydPlus Data Pipeline, are available on the GitLab code repository: https://code.ornl.gov/turnersw/rectifhydplus

Turner, SeanWilliam Donald [Oak Ridge National Lab↗

DOE OSTI2025

RectifHydPlus Data Pipeline v1.1.0

The RectifHydPlus Data Pipeline is an open source and fully reproducible data processing pipeline for creating RectifHydPlus—a dataset of historical monthly net electricity generation for all US hydropower plants (>10MW). The pipeline is coded in R, applying tidyverse libraries and code principles, and using the targets data pipeline framework. All data inputs to the RectifHydPlus Data Pipeline are available from public sources. References to all data inputs, as well as instructions for running the RectifHydPlus Data Pipeline, are available on the GitLab code repository: https://code.ornl.gov/turnersw/rectifhydplus

Turner, SeanWilliam Donald [Oak Ridge National Lab↗

DOE OSTI2023

Chemical Blast Standard (1 kg)

Chemical explosions create blast waves with large overpressure disturbances. It is important to develop a standard blast model based on data to accurately predict acoustic blast-wave amplitudes near detonations and invert for explosion energy from distant observations of blast-wave signals. However, open data from large, controlled chemical explosions with reliable ground truth can be challenging to find. The lack of access to such data could limit the number of contributions to related research and potentially stifle the rate of discoveries or validation of existing models. Here, to address these data scarcity problem, we have curated and compiled a standardized set of 817 blast-wave waveforms from 19 distinct high-explosive events. The blast-wave waveforms are standardized to a 1 kg trinitrotoluene explosion using scaling laws and corrections for location effects. A brief overview of the dataset is presented along with explosion feature models as well as recommendations for extracting explosion features. The resulting dataset is distributed to an open repository in both Seismic Analysis Code and pandas DataFrame formats containing the waveforms, the scaled distances, and the sample rates.

58 GEOSCIENCES↗

DOE OSTI2023

A Collaboration Website for Muon Catalyzed Fusion and Muon Beam Production

This project sets up a website to support the nascent Muon Catalyzed Fusion collaboration including development of particle accelerators and transport beamlines for muon beams. The website is envisioned as having the general public information pages and private pages for collaboration members. Multimedia elements like images, text animations, and video lectures, covering a broad spectrum of topics will populate the educational site, covering muon facilities, to comprehensive explorations and seminal documents that define the science of Muon Catalyzed Fusion, Acceleration, Applications, Instrumentation, Beamline Design, and beam dynamics design codes. Ensuring compatibility across devices and operating systems, it also features integration with Google Docs for collaboration, a code repository (GitHub), a blog platform with comments (WordPress), the potential for ChatGPT integration and interactive graph plotting with Python Plotty to enhance data visualization. This project will maintain public and protected private pages, due to the proprietary nature of the work or research in progress. The public sections will be built to foster dissemination of information and highlight recent work within the NK Labs collaboration, including lectures, published papers, and regular blog posts with open commenting. The private section will support unpublished or nonpublic research, by facilitating collaborative efforts through integrated Google Docs and Python Plotty for shared graphing work. Ultimately, this project strives to make complex scientific knowledge more accessible to the public, foster enhanced collaboration, and serve as a platform for sharing cutting-edge research in Muon Catalyzed Fusion and Accelerators.

43 PARTICLE ACCELERATORS↗

DOE OSTI2024

MODAQ 2.0 (Modular Ocean Data AcQuisition System v2.0) [SWR-24-138]

MODAQ 2.0 is the continuation of the Modular Ocean/Offshore Data AcQuisition system which offers new benefits of being based on generally available hardware and open-source development tools. The MODAQ 2.0 project is the amalgamation of ROS packages, tools, and guidance for developing data acquisition and control applications for marine energy devices using a ROS2 based architecture. The MODAQ 2.0 Reference Design includes several packages that are targeted for the marine energy sector and allow most developers with a basic background in programming to spin up a high-quality DAQ and control system. It is comprised of multiple software repositories which can be found at the MODAQ2 GitHub Organization: https://github.com/NREL-MODAQ2

Nichols, Casey↗

DOE OSTI2023

EGS Collab Experiment 2: Shear Stimulation ERT Monitoring Data

This repository contains the electrical resistivity tomography (ERT) monitoring data that was collected before, during, and after shear stimulation attempts were conducted at EGS Collab. These tests were carried out on the SURF 4100 level during Experiment 2 in March, 2022. Flow and stimulation data corresponding to this dataset are available in a different GDR dataset, which is linked below. Also included here are the E4D input files that were used to process the ERT data. E4D is an open source ERT monitoring and inversion code which is linked for use below.

15 GEOTHERMAL ENERGY↗

DOE OSTI2024

Synthesizing realistic sand assemblies with denoising diffusion in latent space

Abstract The shapes and morphological features of grains in sand assemblies have far‐reaching implications in many engineering applications, such as geotechnical engineering, computer animations, petroleum engineering, and concentrated solar power. Yet, our understanding of the influence of grain geometries on macroscopic response is often only qualitative, due to the limited availability of high‐quality 3D grain geometry data. In this paper, we introduce a denoising diffusion algorithm that uses a set of point clouds collected from the surface of individual sand grains to generate grains in the latent space. By employing a point cloud autoencoder, the three‐dimensional point cloud structures of sand grains are first encoded into a lower‐dimensional latent space. A generative denoising diffusion probabilistic model is trained to produce synthetic sand that maximizes the log‐likelihood of the generated samples belonging to the original data distribution measured by a Kullback‐Leibler divergence. Numerical experiments suggest that the proposed method is capable of generating realistic grains with morphology, shapes and sizes consistent with the training data inferred from an F50 sand database. We then use a rigid contact dynamic simulator to pour the synthetic sand in a confined volume to form granular assemblies in a static equilibrium state with targeted distribution properties. To ensure third‐party validation, 50,000 synthetic sand grains and the 1542 real synchrotron microcomputed tomography (SMT) scans of the F50 sand, as well as the granular assemblies composed of synthetic sand grains are made available in an open‐source repository.

Vlassis, Nikolaos N.↗

DOE OSTI2022

VOLTTRON Modular Framework: Enabling flexible and scalable deployment solutions

VOLTTRON ™ is an open-source platform for distributed sensing and control. The platform provides services for collecting and storing data from buildings and devices and provides an environment for developing applications which interact with that data. The platform allows developers to build out their use cases by utilizing these frameworks and integrating new capabilities. To simplify the deployment of systems built on VOLTTRON, a new way of organizing the codebase is being explored. This document details these efforts through a new code repository layout for the VOLTTRON platform and services, and how this new layout provides targeted deployments using standard python deployment packages (wheels). In addition, this paper will discuss the development of third-party agents and how they can be integrated within the VOLTTRON ecosystem. Finally, we will discuss core platform development and direction for the modularized version of VOLTTRON.

47 OTHER INSTRUMENTATION↗

DOE OSTI2023

Machine learning interatomic potential for silicon-nitride (Si 3 N 4 ) by active learning

Silicon nitride (Si 3 N 4 ) is an extensively used material in the automotive, aerospace, and semiconductor industries. However, its widespread use is in contrast to the scarce availability of reliable interatomic potentials that can be employed to study various aspects of this material on an atomistic scale, particularly its amorphous phase. In this work, we developed a machine learning interatomic potential, using an efficient active learning technique, combined with the Gaussian approximation potential (GAP) method. Our strategy is based on using an inexpensive empirical potential to generate an initial dataset of atomic configurations, for which energies and forces were recalculated with density functional theory (DFT); thereafter, a GAP was trained on these data and an iterative re-training algorithm was used to improve it by learning on-the-fly. When compared to DFT, our potential yielded a mean absolute error of 8 meV/atom in energy calculations for a variety of liquid and amorphous structures and a speed-up of molecular dynamics simulations by 3–4 orders of magnitude, while achieving a first-rate agreement with experimental results. Our potential is publicly available in an open-access repository.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

DOE OSTI2021

Development of the IES Plug-and-Play Framework

This report discusses the status of the flexible plug-and-play framework development currently ongoing that aims to integrate Modelica/Dymola with the Risk Analysis and Virtual ENvironment (RAVEN) software in terms of both Functional Mock-Up Interface (FMI)/Functional Mock-Up Unit (FMU) construction and repository structures that aim to ease the sharing and simulation of complex dynamic models. This report aims to provide an overview of all the performed activities resolving around the deployment of methods, software infrastructures, guidelines and workflow for the construction and usage of models, encapsulated using the FMI/FMU protocols and standards. In particular, the report is organized in three main macro-subjects, which are connected to each other: - FMI/FMU adaptors for modelica models - HYBRID repository new structure and open-source deployment - RAVEN FMI/FMU exporting capabilities and Artificial Intelligence (AI)-based analysis acceleration. The first part of the report discusses the FMI/FMU adaptors that have been created within the HYBRID repository to allow users to quickly export models, such as FMUs. Several examples are shown that highlight the step-by-step process of converting an existing Modelica model into an FMU for use within the Dymola platform. Simulation results demonstrate that, while minor differences may occur, the overall control, trends, and solution integrity are maintained between standard Modelica simulation and FMU simulation results. However, it is worth noting that, for small systems, the FMU results have a slower simulation time than the Modelica only simulation. Using this process, a company can provide models that contain proprietary information to entities without disclosing any of the information about the model that could be considered business sensitive. Such an ability would allow institutions to bypass the necessity of “whitewashing” data. In the second part of the report, the new structure of the HYBRID repository is discussed with a major focus on the series of updates that has been completed. These updates include the addition of Modelica system-level regression tests and software quality assurance documentation that ensure that modifications to the Modelica models do not alter system-level model results. The third and final part of the report aims to report the work that has been performed for the deployment of methods and workflows for the construction of RAVEN AI-based models compliant with the FMI/FMU standard. Such development represents the key for the deployment of the concept of “Flexible ecosystem” since it allows for the replacement of high-fidelity modelica models (or any other FMI/FMU compliant model) with RAVEN generated AI surrogate models. Overall, extensive work has been completed on developing FMUs and FMIs from existing models, understanding the requirements and limitations of FMUs, and open-sourcing the HYBRID repository with an integrated regression system.

42 ENGINEERING↗

DOE OSTI2021

Development of the IES Plug-and-Play Framework

This report discusses the status of the flexible plug-and-play framework development currently ongoing that aims to integrate Modelica/Dymola with the Risk Analysis and Virtual ENvironment (RAVEN) software in terms of both Functional Mock-Up Interface (FMI)/Functional Mock-Up Unit (FMU) construction and repository structures that aim to ease the sharing and simulation of complex dynamic models. This report aims to provide an overview of all the performed activities resolving around the deployment of methods, software infrastructures, guidelines and workflow for the construction and usage of models, encapsulated using the FMI/FMU protocols and standards. In particular, the report is organized in three main macro-subjects, which are connected to each other: - FMI/FMU adaptors for modelica models - HYBRID repository new structure and open-source deployment - RAVEN FMI/FMU exporting capabilities and Artificial Intelligence (AI)-based analysis acceleration. The first part of the report discusses the FMI/FMU adaptors that have been created within the HYBRID repository to allow users to quickly export models, such as FMUs. Several examples are shown that highlight the step-by-step process of converting an existing Modelica model into an FMU for use within the Dymola platform. Simulation results demonstrate that, while minor differences may occur, the overall control, trends, and solution integrity are maintained between standard Modelica simulation and FMU simulation results. However, it is worth noting that, for small systems, the FMU results have a slower simulation time than the Modelica only simulation. Using this process, a company can provide models that contain proprietary information to entities without disclosing any of the information about the model that could be considered business sensitive. Such an ability would allow institutions to bypass the necessity of “whitewashing” data. In the second part of the report, the new structure of the HYBRID repository is discussed with a major focus on the series of updates that has been completed. These updates include the addition of Modelica system-level regression tests and software quality assurance documentation that ensure that modifications to the Modelica models do not alter system-level model results. The third and final part of the report aims to report the work that has been performed for the deployment of methods and workflows for the construction of RAVEN AI-based models compliant with the FMI/FMU standard. Such development represents the key for the deployment of the concept of “Flexible ecosystem” since it allows for the replacement of high-fidelity modelica models (or any other FMI/FMU compliant model) with RAVEN generated AI surrogate models. Overall, extensive work has been completed on developing FMUs and FMIs from existing models, understanding the requirements and limitations of FMUs, and open-sourcing the HYBRID repository with an integrated regression system.

14 SOLAR ENERGY↗

DOE OSTI2026

Maximizing efficiency of dataset compression for machine learning potentials with information theory

Machine learning interatomic potentials (MLIPs) balance high accuracy and lower costs compared to density functional theory calculations, but their performance often depends on the size and diversity of training datasets. Large datasets improve model accuracy and generalization but are computationally expensive to produce and train on, while smaller datasets risk discarding rare but important atomic environments and compromising MLIP accuracy/reliability. Here, we develop an information-theoretical framework to quantify the efficiency of dataset compression methods and propose an algorithm that maximizes this efficiency. By framing atomistic dataset compression as an instance of the minimum set cover (MSC) problem over atom-centered environments, our method identifies the smallest subset of structures that contains as much information as possible from the original dataset while pruning redundant information. The approach is extensively demonstrated on the GAP-20 and TM23 datasets and validated on 64 varied datasets from the ColabFit repository. Across all cases, MSC consistently retains outliers, preserves dataset diversity, and reproduces the long-tail distributions of forces even at high compression rates, outperforming other subsampling methods. Furthermore, MLIPs trained on MSC-compressed datasets exhibit reduced error for out-of-distribution data even in low-data regimes. We explain these results using an outlier analysis and show that such quantitative conclusions could not be achieved with conventional dimensionality reduction methods. The algorithm is implemented in the open-source QUESTS package and can be used for several tasks in atomistic modeling, from data subsampling, outlier detection, and training improved MLIPs at a lower cost.

36 MATERIALS SCIENCE↗

DOE OSTI2023

LCA-PyTorch

LCA-PyTorch is a code repository which contains PyTorch implementations of the Locally Competitive Algorithm (LCA), which is a biologically-plausible sparse coding model. LCA-PyTorch allows for the training, testing, and analysis of single layer LCA networks, multi-layer LCA networks, and hybrid LCA-based deep neural network models on a wide variety of applications and data types. LCA-PyTorch was developed in Python, a high-level programming language that takes advantage of the Python ecosystem of high-quality open-source packages for machine learning. LCA-PyTorch interfaces heavily with the open-source PyTorch Python package.

Teti, Michael↗

DOE OSTI2023

pvOps: a Python package for empirical analysis of photovoltaic field data

The purpose of pvOps is to support empirical evaluations of data collected in the field related to the operations and maintenance (O&M) of photovoltaic (PV) power plants. pvOps presently contains modules that address the diversity of field data, including text-based maintenance logs, current-voltage (IV) curves, and timeseries of production information. The package functions leverage machine learning, visualization, and other techniques to enable cleaning, processing, and fusion of these datasets. These capabilities are intended to facilitate easier evaluation of field patterns and extraction of relevant insights to support reliability-related decision-making for PV sites. The open-source code, examples, and instructions for installing the package through PyPI can be accessed through the GitHub repository.

14 SOLAR ENERGY↗