SEARCH · Search NASA

Results for “Open data”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 37 records · Page 2

All sources NASA NTRS DOE OSTI

DOE OSTI2025

Isolating Unisolated Upsilons with Anomaly Detection in CMS Open Data

We present the first study of anti-isolated Upsilon decays to two muons (ϒ→𝜇⁺⁢𝜇⁻) in proton-proton collisions at the Large Hadron Collider. Using a machine learning (ML)-based anomaly detection strategy, we “rediscover” the ϒ in 13 TeV CMS Open Data from 2016, despite overwhelming anti-isolated backgrounds. We elevate the signal significance to 6.4⁢𝜎 using these methods, starting from 1.6⁢𝜎 using the dimuon mass spectrum alone. Moreover, we demonstrate improved sensitivity from using an ML-based estimate of the multifeature likelihood compared to traditional “cut-and-count” methods. This is the first ever detection of anti-isolated Upsilons, which can be useful in the study of heavy-flavor fragmentation in quantum chromodynamics. Our Letter demonstrates that it is possible and practical to find real signals in experimental collider data using ML-based anomaly detection, and we distill a readily accessible benchmark dataset from the CMS Open Data to facilitate future anomaly detection developments.

machine learning↗

DOE OSTI2021

Machine Learning Using Open Data Sources for Detection of Nuclear Proliferation Activities (U)

In FY2020, Savannah River National Laboratory (SRNL) in collaboration with the Sanghani Center for Artificial Intelligence and Data Analytics (SCAIDA) at Virginia Polytechnic Institute and State University (VT) and funded by the Department of Energy’s (DOE) Defense Nuclear Nonproliferation Research and Development, began developing a demonstration prototype system that uses multiple machine learning and data analytic methods on large-scale open data sources to identify new, developing, and/or undeclared nuclear programs. Using the announcement in May 2018 of the proposed Savannah River Plutonium Processing Facility (SRPPF) as a test subject, the goal of this 2-year project is to forecast the SRPPF using only data prior to May 2018. The project work is split into a preliminary prototype development for the first year with an initial evaluation of viability followed by the second year of development to create an integrated prototype system and more extensive performance evaluation. This report documents the results of the preliminary-phase tasks.

98 NUCLEAR DISARMAMENT, SAFEGUARDS, AND PHYSICAL P↗

DOE OSTI2021

End-to-End Jet Classification of Boosted Top Quarks with CMS Open Data

We describe a novel application of the end-to-end deep learning technique to the task of discriminating top quark-initiated jets from those originating from the hadronization of a light quark or a gluon. The end-to-end deep learning technique combines deep learning algorithms and low-level detector representation of the high-energy collision event. In this study, we use lowlevel detector information from the simulated CMS Open Data samples to construct the top jet classifiers. To optimize classifier performance we progressively add low-level information from the CMS tracking detector, including pixel detector reconstructed hits and impact parameters, and demonstrate the value of additional tracking information even when no new spatial structures are added. Relying only on calorimeter energy deposits and reconstructed pixel detector hits, the end-to-end classifier achieves a ROC-AUC score of 0.975±0.002 for the task of classifying boosted top quark jets. After adding derived track quantities, the classifier ROC-AUC score increases to 0.9824±0.0013, serving as the first performance benchmark for these CMS Open Data samples.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

DOE OSTI2022

End-to-end jet classification of boosted top quarks with the CMS open data

Here we describe a novel application of the end-to-end deep learning technique to the task of discriminating top quark-initiated jets from those originating from the hadronization of a light quark or a gluon. The end-to-end deep learning technique uses low-level detector representation of high-energy collision event as inputs to deep learning algorithms. In this study, we use low-level detector information from the simulated Compact Muon Solenoid (CMS) open data samples to construct the top jet classifiers. To optimize classifier performance we progressively add low-level information from the CMS tracking detector, including pixel detector reconstructed hits and impact parameters, and demonstrate the value of additional tracking information even when no new spatial structures are added. Relying only on calorimeter energy deposits and reconstructed pixel detector hits, the end-to-end classifier achieves an area under the receiver operator curve (AUC) score of 0.975 ± 0.002 for the task of classifying boosted top quark jets. After adding derived track quantities, the classifier AUC score increases to 0.9824 ± 0.0013, serving as the first performance benchmark for these CMS open data samples.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

NASA NTRS2018

Lessons Learned and Cost Analysis of Hosting a Full Stack Open Data Cube (ODC) Application on the Amazon Web Services (AWS)

The Open Data Cube (ODC) initiative, with support from the Committee on Earth Observation Satellites (CEOS) System Engineering Office (SEO) has developed a state-of-the-art suite of software tools and products to facilitate the analysis of Earth Observation data. This paper presents a short summary and cost analysis of our experience using Amazon Web Services (AWS) to host one such software product, the CEOS Data Cube (CDC) web-based User Interface (UI). In order to provide adaptability, flexibility, scalability, and robustness, we leverage widely-adopted and well-supported technologies such as the Django web framework and the AWS Cloud platform. The UI has empowered users by providing features that assist with streamlining data preparation, data processing, data visualization, and the sub-setting of Analysis Ready Data (ARD) products in order to achieve a wide variety of Earth imaging objectives.

Rizvi, Syed R.↗

DOE OSTI2025

LinkML: an open data modeling framework

Background Scientific research relies on well-structured, standardized data; however, much of it is stored in formats such as free-text lab notebooks, nonstandardized spreadsheets, or data repositories. This lack of structure challenges interoperability, making data integration, validation, and reuse difficult. Findings LinkML (Linked Data Modeling Language) is an open framework that simplifies the process of authoring, validating, and sharing data. LinkML can describe a range of data structures, from flat, list-based models to complex, interrelated, and normalized models that utilize polymorphism and compound inheritance. It offers an approachable syntax that is not tied to any one technical architecture and can be integrated seamlessly with many existing frameworks. The LinkML syntax provides a standard way to describe schemas, classes, and relationships, allowing modelers to build well-defined, stable, and optionally ontology-aligned data structures. Once defined, LinkML schemas may be imported into other LinkML schemas. These key features make LinkML an accessible platform for interdisciplinary collaboration and a reliable way to define and share data semantics. Conclusions LinkML helps reduce heterogeneity, complexity, and the proliferation of single-use data models while simultaneously enabling compliance with FAIR (Findable, Accessible, Interoperable, and Reusable) data standards. LinkML has seen increasing adoption in various fields, including biology, chemistry, biomedicine, microbiome research, finance, electrical engineering, transportation, and commercial software development. In short, LinkML makes implicit models explicitly computable and allows data to be standardized at their origin. LinkML documentation and code are available at https://linkml.io/.

AI-ready data↗

DOE OSTI2020

Open Energy Data Initiative (OpenEDI) - Open Data Access Tools (OEDI-ODAT) [SWR 20-57]

Open Energy Data Initiative (OpenEDI): Advancing Analytics and Research Innovation through Improved Data Access. The Open Energy Data Initiative (OEDI) aims to improve and automate access of high-value energy data sets across the U.S. Department of Energy’s (DOE’s) programs, offices, and national laboratories. Sponsored by the DOE, this platform is being implemented by the National Renewable Energy Laboratory (NREL) to make data actionable and discoverable by researchers and industry to accelerate analysis and advance innovation. Partners at other national laboratories will help utilize the platform for analysis, can be directly involved in determining requirements, and will be resources for providing new datasets that can be shared with the public to expand innovation.

Rager, David↗

DOE OSTI2022

How open data and interdisciplinary collaboration improve our understanding of space weather: A risk and resiliency perspective

Space weather refers to conditions around a star, like our Sun, and its interplanetary space that may affect space- and ground-based assets as well as human life. Space weather can manifest as many different phenomena, often simultaneously, and can create complex and sometimes dangerous conditions. The study of space weather is inherently trans-disciplinary, including subfields of solar, magnetospheric, ionospheric, and atmospheric research communities, but benefiting from collaborations with policymakers, industry, astrophysics, software engineering, and many more. Effective communication is required between scientists, the end-user community, and government organizations to ensure that we are prepared for any adverse space weather effects. With the rapid growth of the field in recent years, the upcoming Solar Cycle 25 maximum, and the evolution of research-ready technologies, we believe that space weather deserves a reexamination in terms of a “risk and resiliency” framework. By utilizing open data science, cross-disciplinary collaborations, information systems, and citizen science, we can forge stronger partnerships between science and industry and improve our readiness as a society to mitigate space weather impacts. The objective of this manuscript is to raise awareness of these concepts as we approach a solar maximum that coincides with an increasingly technology-dependent society, and introduce a unique way of approaching space weather through the lens of a risk and resiliency framework that can be used to further assess areas of improvement in the field.

79 ASTRONOMY AND ASTROPHYSICS↗

DOE OSTI2020

End-to-End Physics Event Classification with CMS Open Data: Applying Image-Based Deep Learning to Detector Data for the Direct Classification of Collision Events at the LHC

This paper describes the construction of novel end-to-end image-based classifiers that directly leverage low-level simulated detector data to discriminate signal and background processes in pp collision events at the Large Hadron Collider at CERN. To better understand what end-to-end classifiers are capable of learning from the data and to address a number of associated challenges, we distinguish the decay of the standard model Higgs boson into two photons from its leading background sources using high-fidelity simulated CMS Open Data. We demonstrate the ability of end-to-end classifiers to learn from the angular distribution of the photons recorded as electromagnetic showers, their intrinsic shapes, and the energy of their constituent hits, even when the underlying particles are not fully resolved, delivering a clear advantage in such cases over purely kinematics-based classifiers.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

DOE OSTI2025

The Pierre Auger Observatory open data

The Pierre Auger Collaboration has embraced the concept of open access to their research data since its foundation, with the aim of giving access to the widest possible community. A gradual process of release began as early as 2007 when 1% of the cosmic-ray data was made public, along with 100% of the space-weather information. In February 2021, a portal was released containing 10% of cosmic-ray data collected by the Pierre Auger Observatory from 2004 to 2018, during the first phase of operation of the Observatory. The Open Data Portal includes detailed documentation about the detection and reconstruction procedures, analysis codes that can be easily used and modified and, additionally, visualization tools. Since then, the Portal has been updated and extended. In 2023, a catalog of the highest-energy cosmic-ray events examined in depth has been included. A specific section dedicated to educational use has been developed with the expectation that these data will be explored by a wide and diverse community, including professional and citizen scientists, and used for educational and outreach initiatives. This paper describes the context, the spirit, and the technical implementation of the release of data by the largest cosmic-ray detector ever built and anticipates its future developments.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

DOE OSTI2026

MINERvA s Open Data Product: A First for Neutrino Data Preservation

Access to information on neutrino nucleus interactions is critical to the success of all neutrino oscillation experiments. MINERvA's rich dataset covers a range of energies and nuclei unique amongst experiments, and as such is critical to the community in building the important shared knowledge needed to unravel the mysteries of the neutrino. In particular, its dataset provides the greatest statistical coverage in in the range of neutrino energies pertinent for DUNE until DUNE's near detector begins operation. Historically, such significant datasets in neutrino physics have been preserved primarily through their published results. While meaningful and useful, this limits the ability to explore the data to its fullest extent as new perspectives continue to form. MINERvA has undertaken a major effort to break this trend and preserve its data in a format to be as analyzable as possible from outside the collaboration. This has culminated in the officially-released MINERvA Open Data Product for the community to take advantage of and utilize. Maintaining direct access to the dataset in an analyzable form will allow new insights to continue to be extracted indefinitely. This talk will cover the contents of this product, the information included (and excluded), the tools provided to utilize the product effectively, the support MINERvA intends to provide in its use, and some lessons learned through the process.

Last, David [Rochester U.] (ORCID:0000000245147183↗

NASA NTRS1983

The UARS and open data system concept and analysis study. Executive summary

Alternative concepts for a common design for the UARS and OPEN Central Data Handling Facility (CDHF) are offered. The designs are consistent with requirements shared by UARS and OPEN and the data storage and data processing demands of these missions. Because more detailed information is available for UARS, the design approach was to size the system and to select components for a UARS CDHF, but in a manner that does not optimize the CDHF at the expense of OPEN. Costs for alternative implementations of the UARS designs are presented showing that the system design does not restrict the implementation to a single manufacturer. Processing demands on the alternative UARS CDHF implementations are discussed. With this information at hand together with estimates for OPEN processing demands, it is shown that any shortfall in system capability for OPEN support can be remedied by either component upgrades or array processing attachments rather than a system redesign.

Mittal, M.↗

DOE OSTI2021

Open Data and Deep Semantic Segmentation for Automated Extraction of Building Footprints

Advances in machine learning and computer vision, combined with increased access to unstructured data (e.g., images and text), have created an opportunity for automated extraction of building characteristics, cost-effectively, and at scale. These characteristics are relevant to a variety of urban and energy applications, yet are time consuming and costly to acquire with today’s manual methods. Several recent research studies have shown that in comparison to more traditional methods that are based on features engineering approach, an end-to-end learning approach based on deep learning algorithms significantly improved the accuracy of automatic building footprint extraction from remote sensing images. However, these studies used limited benchmark datasets that have been carefully curated and labeled. How the accuracy of these deep learning-based approach holds when using less curated training data has not received enough attention. The aim of this work is to leverage the openly available data to automatically generate a larger training dataset with more variability in term of regions and type of cities, which can be used to build more accurate deep learning models. In contrast to most benchmark datasets, the gathered data have not been manually curated. Thus, the training dataset is not perfectly clean in terms of remote sensing images exactly matching the ground truth building’s foot-print. A workflow that includes data pre-processing, deep learning semantic segmentation modeling, and results post-processing is introduced and applied to a dataset that include remote sensing images from 15 cities and five counties from various region of the USA, which include 8,607,677 buildings. The accuracy of the proposed approach was measured on an out of sample testing dataset corresponding to 364,000 buildings from three USA cities. The results favorably compared to those obtained from Microsoft’s recently released US building footprint dataset.

97 MATHEMATICS AND COMPUTING↗

NASA NTRS1983

The UARS and open data concept and analysis study

Alternative concepts for a common design for the UARS and OPEN Central Data Handling Facility (CDHF) are offered. Costs for alternative implementations of the UARS designs are presented, showing that the system design does not restrict the implementation to a single manufacturer. Processing demands on the alternative UARS CDHF implementations are then discussed. With this information at hand together with estimates for OPEN processing demands, it is shown that any shortfall in system capability for OPEN support can be remedied by either component upgrades or array processing attachments rather than a system redesign. In addition to a common system design, it is shown that there is significant potential for common software design, especially in the areas of data management software and non-user-unique production software. Archiving the CDHF data are discussed. Following that, cost examples for several modes of communications between the CDHF and Remote User Facilities are presented. Technology application is discussed.

Mittal, M.↗

DOE OSTI2025

Toward equitable environmental exposure modeling through convergence of data, open, and citizen sciences: an example of air pollution exposure modeling amidst increasing wildfire smoke

Exposure modeling is critical in environmental epidemiology and human health but may face challenges (e.g., skewed data, unequal error, context-insensitive validation, and computational demands). Modeling decisions reflect the intended use of the models and the values that modelers prioritize. We aimed to provide a conceptual framework and machine learning (ML) modeling protocols that address these issues. With 500m-gridded hourly PM 2.5 and O 3 levels in Illinois before, during, and after the 2023 Canadian wildfire season as a motivating example, we conducted modeling experiments to evaluate modeling methods, guided by three domains we propose based on theories of science: 1) Data Diversity, leveraging open and citizen science data to enhance inclusivity, parsimony, and representativeness; 2) Equitable Accuracy, ensuring fairly distributed uncertainties across subpopulations; and 3) Sustainable Modeling, balancing accuracy with reducing computational demands to promote accessibility for under-resourced researchers. Here, we found that ML with publicly available data can achieve high accuracy. Depending on methods, performance may vary substantially, even with identical input data. Large but skewed data may reduce performance. Misuse of cross-validation protocols can underestimate prediction error; although we observed R 2 s of ∼98 %, the modeled estimates varied significantly, indicating the need for careful model validation. By using new modeling protocols including representativeness-considered training and validation data and a new loss function, we achieved high agreement between estimates and ground-based measurements (e.g., R 2 = ∼90 % for PM 2.5 ; ∼80 % for O 3 ), equally distributed errors across sociodemographic strata and urban–rural divides, and reduction in computation time—from several weeks or months to a few days.

Exposure assessment↗

DOE OSTI2025

Open data sets for assessing photovoltaic system reliability

Photovoltaic (PV) systems have become a cornerstone of renewable energy strategies, particularly due to the significant reduction in solar power costs over the past decade. However, the long-term reliability of PV installations presents a persistent challenge, requiring the development of advanced monitoring and predictive maintenance strategies. A wide range of data types is used to evaluate the health of PV systems, including environmental conditions, electrical performance, and inspection imagery. These data enable methodologies such as machine learning (ML) models for lifetime prediction and computer vision techniques for defect detection. However, the acquisition of high-quality and comprehensive data is difficult, particularly in terms of long-term consistency and data variety. Publicly available data sets serve as valuable resources for addressing these challenges, but they often suffer from fragmentation and are difficult to access. This paper presents a comprehensive review of existing open-source data sets related to PV degradation, analyzing their features, functionalities, and potential applications. We categorize these data sets based on the specific aspects of PV system information they cover, such as environmental conditions, operational monitoring, image inspection and module materials, and propose relevant tools and ML models for processing them. In addition, we propose practices for future data collection and usage, while also discussing potential directions in data-driven research. Our aim is to enhance data utilization and publication among researchers and industry professionals, promoting a deeper understanding of the role of data in enhancing the performance and durability of PV systems.

14 SOLAR ENERGY↗

NASA NTRSDate not supplied

Open Data Integration (ODIN): A Concurrent, Distributed Message-Based Architecture and Framework for Disaster Response

The Runtime for Airspace Concept Evaluation (RACE) is an open-source software architecture and framework to build configurable, highly concurrent and distributed message-based systems that offer scalable, low-latency performance on commodity hardware. RACE was used in commercial aviation applications to rapidly build systems that span several machines (including synchronized displays), interface existing hardware simulators and other live data feeds, and incorporate sophisticated visualization components such as NASA WorldWind. These RACE applications validated elements of the FAA’s System Wide Information Management (SWIM) Program, handling up to 1000 messages/sec from diverse sources (SFDPS, TFM-DATA, TAIS, ASDE-X, ITWS and local ADS) for 4,500 simultaneous flights tracked in the next-generation air transportation system’s digital backbone. We have since generalized RACE to support Open Data Integration (ODIN) applications outside aviation. Systems built with RACE/ODIN can be deployed in the field, on commodity hardware, and operate with limited or intermittent connectivity to the outside world. Our primary use case is a web-server with local/persistent data storage that runs within and only serves the stakeholder network (e.g. an incident command post). We are tailoring the RACE/ODIN system to support wildland fire management for the upcoming NASA Wildland Fire Safety Demonstration Series. RACE-ODIN is under consideration for application in the Scalable Traffic Management for Emergency Response Operations project, or STEReO, which aims to create a system that can be deployed during emergencies, to coordinate multiple elements of disaster response. Such data sources predominantly come from existing services on the internet (e.g. weather and satellite data, imported from so called "edge servers") but can also include dynamic (real-time) data from computer simulations and within the stakeholder network (such as aircraft and personnel tracking information). We will present the architecture and ODIN system demonstration incorporating local data from instrumented power-line towers, interpolated weather data and geospatial data from space-based platforms.

Joseph C Coughlan↗

NASA NTRSDate not supplied

Open Data Integration (ODIN): A Concurrent, Distributed Message-Based Architecture and Framework for Disaster Response

Guillaume P Brat↗