SEARCH · Search NASA
Results for “Open data”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Ume: Unstructured Mesh Explorations
Ume is an open-source collection of data structures for unstructured computational meshes and some simple algorithms that operate on them. These algorithms mimic the memory access patterns of a common class of operations found in several of the computational physics simulation codes developed at Los Alamos National Laboratory. The intent is that Ume can be used by hardware vendors to understand the memory traffic created by complex codes in a simplified environment, and to explore new means of optimization for that traffic. Ume is provided as a source-code C++ library and includes several applications that demonstrate the use of that library.
A Primer on Phased Array Radar Technology for the Atmospheric Sciences
The scientific community has expressed interest in the potential of phased array radars (PARs) to observe the atmosphere with finer spatial and temporal scales. Although convergence has occurred between the meteorological and engineering communities, the need exists to increase access of PAR to meteorologists. Here, we facilitate these interdisciplinary efforts in the field of ground-based PARs for atmospheric studies. We cover high-level technical concepts and terminology for PARs as applied to studies of the atmosphere. A historical perspective is provided as context along with an overview of PAR system architectures, technical challenges, and opportunities. Envisioned scan strategies are summarized because they are distinct from traditional mechanically scanned radars and are the most advantageous for high-resolution studies of the atmosphere. Further, open access to PAR data is emphasized as a mechanism to educate the future generation of atmospheric scientists. Finally, a vision for the future of operational networks, research facilities, and expansion into complementary radar wavelengths is provided.
Toward a Smart Metaverse City: Immersive Realism and 3D Visualization of Digital Twin Cities
Metaverse and its related extended reality technologies can enable immersive, realistic, and participatory visualization of 3D data, and their use and the potential within a smart city can be effective for supporting urban research and urban operations management. This book chapter describes a vision for prototyping a “Smart Metaverse City” to combine the unique advantage of the Metaverse technology with the two-way connectivity of a digital twin city application. This synergy aims to create a virtual environment for immersive geovisualization to help researchers and the public understand the complex urban system through science-based and data-driven approaches. This book chapter selectively reviews past technological and paradigm advancements for collecting, analyzing, and visualizing 3D urban big data. Then we present a prototyping Geographic Information System (GIS) framework, together with some relevant data sources and open-source web technologies, to help researchers create a smart Metaverse city. We demonstrate our vision and discuss its application opportunities through a real-world example, a digital twin city developed at the Oak Ridge National Laboratory, to facilitate participatory, smart, and sustainable campus management.
Templates for developing and versioning data standards and reporting formats using GitHub
This data package contains three templates that can be used for creating README files and Issue Templates, written in the markdown language, that support community-led data reporting formats. We created these templates based on the results of a systematic review (see related references) that explored how groups developing data standard documentation use the Version Control platform GitHub, to collaborate on supporting documents. Based on our review of 32 GitHub repositories, we make recommendations for the content of README Files (e.g., provide a user license, indicate how users can contribute) and so 'README_template.md' includes headings for each section. The two issue templates we include ('issue_template_for_all_other_changes.md' and 'issue_template_for_documentation_change.md') can be used in a GitHub repository to help structure user-submitted issues, or can be modified to suit the needs of data standard developers. We used these templates when establishing ESS-DIVE's community space on GitHub (https://github.com/ess-dive-community) that includes documentation for community-led data reporting formats. We also include file-level metadata 'flmd.csv' that describes the contents of each file within this data package. Lastly, the temporal range that we indicate in our metadata is the time range during which we searched for data standards documented on GitHub.
Data from: "A guide to using GitHub for developing and versioning data standards and reporting formats"
These data are the results of a systematic review that investigated how data standards and reporting formats are documented on the version control platform GitHub. Our systematic review identified 32 data standards in earth science, environmental science, and ecology that use GitHub for version control of data standard documents. In our analysis, we characterized the documents and content within each of the 32 GitHub repositories to identify common practices for groups that version control their documents on GitHub.In this data package, there are 8 CSV files that contain data that we characterized from each repository, according to the location within the repository. For example, in 'readme_pages.csv' we characterize the content that appears across the 32 GitHub repositories included in our systematic review. Each of the 8 CSV files has an associated data dictionary file (names appended with '_dd.csv' and here we describe each content category within CSV files.There is one file-level metadata file (flmd.csv) that provides a description of each file within the data package.
Data from: “Enabling FAIR data in Earth and environmental science with community-centric (meta)data reporting formats”
This dataset contains supplementary information for a manuscript describing the ESS-DIVE (Environmental Systems Science Data Infrastructure for a Virtual Ecosystem) data repository's community data and metadata reporting formats. The purpose of creating the ESS-DIVE reporting formats was to provide guidelines for formatting some of the diverse data types that can be found in the ESS-DIVE repository. The 6 teams of community partners who developed the reporting formats included scientists and engineers from across the Department of Energy National Lab network. Additionally, during the development process, 247 individuals representing 128 institutions provided input on the formats. The primary files in this dataset are 10 data and metadata crosswalk for ESS-DIVE’s reporting formats (all files ending in _crosswalk.csv). The crosswalks compare elements used in each of the reporting formats to other related standards and data resources (e.g., repositories, datasets, data systems). This dataset also contains additional files recommended by ESS-DIVE’s file-level metadata reporting format. Each data file has an associated dictionary (files ending in _dd.csv) which provide a brief description of each standard or data resource consulted in the data reporting format development process. The flmd.csv file describes each file contained within the dataset.
xCDAT: A Python Package for Simple and Robust Analysis of Climate Data
xCDAT (Xarray Climate Data Analysis Tools) is an open-source Python package that extends Xarray (Hoyer & Hamman, 2017) for climate data analysis on structured grids. xCDAT streamlines analysis of climate data by exposing common climate analysis operations through a set of straightforward APIs. Some of xCDAT’s key features include spatial averaging, temporal averaging, and regridding. These features are inspired by the Community Data Analysis Tools (CDAT) library (Dean N. Williams et al., 2009) (D. N. Williams, 2014) (Doutriaux et al., 2019) and leverage powerful packages in the Xarray ecosystem including xESMF (Zhuang et al., 2023), xgcm (Abernathey et al., 2022), and CF xarray (Cherian et al., 2023). To ensure general compatibility across various climate models, xCDAT operates on datasets that are compliant with the Climate and Forecast (CF) metadata conventions (Hassell et al., 2017).
Sandia National Laboratories Ecosystem for Open Science: Metadata Schema v0.2 Description.
The Ecosystem for Open Science (eOS) initiative was established in 2019. Its objective is improving openness and sharing of data and information across Defense Nuclear Nonproliferation (DNN) Research and Development (R&D) activities. To support this initiative, the eOS team at Sandia National Laboratories (SNL) developed metadata and data standards and proposed a machine-readable metadata schema. The nuclear explosion monitoring field was selected as a focus area due to its the wide range of pertinent phenomenologies.We developed the DCAT-eOS-AP metadata schema extending the Data Catalog Vocabulary version 2 (DCATv2) standard using an application profile (AP), to fit the needs of multi-disciplinary NA-22 projects. The DCAT-eOS-AP metadata schema describes data at different levels of granularity ranging from general descriptions to more domain-specific granular metadata. Its implementation and serialization is flexible with the ability to include new file or data types. Thus, it will scale with the ever-increasing data management needs of government research. Due to the multitude of phenomenologies represented in the DCAT-eOS-AP schema, we anticipate that it will be easily extensible to various projects across many DOE mission areas. This document describes data management challenges faced within the DNN R&D portfolio and provides insight on how metadata and data standards/guidelines combined with a comprehensive metadata schema can add value to programs throughout the Department of Energy (DOE). It reviews the importance of metadata standards, FAIR (Findability, Accessibility, Interoperability, and Reusability) data principles, and metadata schemas. Additionally, it summarizes input from subject matter experts (SME) at SNL and other National Laboratories that resulted in metadata and data standards/guidelines encompassing domains relevant to NA-22 projects. Finally, we discuss the DCAT-eOS-AP metadata development. Implementation recommendations and future development directions are included for those keen on adopting the DCAT-eOS-AP metadata schema.
MalGen: Malware Generation with Specific Behaviors to Improve Machine Learning-based Detectors
In recent years, infections and damage caused by malware have increased at exponential rates. At the same time, machine learning (ML) techniques have shown tremendous promise in many domains, often out performing human efforts by learning from large amounts of data. Results in the open literature suggest that ML is able to provide similar results for malware detection, achieving greater than 99% classifcation accuracy [49]. However, the same detection rates when applied in deployed settings have not been achieved. Malware is distinct from many other domains in which ML has shown success in that (1) it purposefully tries to hide, leading to noisy labels and (2) often its behavior is similar to benign software only differing in intent, among other complicating factors. This report details the reasons for the diffcultly of detecting novel malware by ML methods and offers solutions to improve the detection of novel malware.
Lawrence Livermore National Laboratory Environmental Report 2021
The purposes of the Lawrence Livermore National Laboratory Environmental Report 2021 are to record Lawrence Livermore National Laboratory’s (LLNL’s) compliance with environmental standards and requirements, describe LLNL’s environmental protection and remediation programs, and present the results of environmental monitoring at the two LLNL sites—the Livermore Site and Site 300. The report is prepared for the U.S. Department of Energy (DOE) by LLNL’s Environmental Functional Area. Submittal of the report satisfies requirements under DOE Order 231.1B, “Environment, Safety and Health Reporting,” and DOE Order 458.1, “Radiation Protection of the Public and Environment.” The report is distributed electronically and is available at https://saer.llnl.gov/, the website for the LLNL annual environmental report. Previous LLNL annual environmental reports beginning with 1994 are also on the website. Some references in the electronic report text are underlined, which indicates that they are clickable links. Clicking on one of these links will open the related document, data workbook, or website. Sampling location maps throughout this report were created using ArcGIS® software by Esri. The report begins with an executive summary, which provides the purpose of the report and an overview of LLNL’s compliance and monitoring results. The first three chapters provide background information: Chapter 1 is an overview of the location, meteorology, and hydrogeology of the two LLNL sites; Chapter 2 is a summary of LLNL’s compliance with environmental regulations; and Chapter 3 is a description of LLNL’s environmental programs with an emphasis on the Environmental Management System including pollution prevention. The majority of the report covers LLNL’s environmental monitoring programs and monitoring data for 2021: effluent and ambient air monitoring and dose assessment (Chapter 4); waters, including wastewater, storm water runoff, surface water, rain, and groundwater (Chapter 5); and terrestrial, including soil, sediment, vegetation, foodstuff, ambient radiation, and special status wildlife and plants (Chapter 6). The remaining two chapters discuss LLNL’s groundwater remediation program (Chapter 7), and quality assurance for the environmental monitoring programs (Chapter 8). Complete monitoring data, which are summarized in the body of the report, are provided in Appendix A. The report uses Système International units, consistent with the federal Metric Conversion Act of 1975 and Executive Order 12770, “Metric Usage in Federal Government Programs” (1991). For ease of comparison to environmental reports issued prior to 1991, dose values and many radiological measurements are given in both metric and U.S. customary units. A conversion table is provided in the glossary. The report is the responsibility of LLNL’s Environmental Functional Area. Monitoring data were obtained through the combined efforts of the Environmental Functional Area; Environmental Restoration Department; Physical and Life Sciences Environmental Monitoring Radiological Laboratory; and the Radiation Protection Functional Area.
Terrain-Influenced Winds and Fire-Fire Interactions in Wildland Fire Simulations [Dissertation]
Ensemble-based approaches to prescribed fire planning cannot be supported by computational fluid dynamics based models like FIRETEC and the Wildland-Urban Interface Fire Dynamics Simulator (WFDS) because they are too computationally expensive and cannot leverage large eddy simulation approaches like CAWFE and WRF-SFIRE because they have too coarse of resolution. QUIC-Fire was developed to fill this gap but it cannot currently address complex terrain, that is typical for instance in the Western United States. This dissertation describes a variety of improvements made to QUIC-Fire and its various incorporated algorithms in an effort to make it a viable tool in simulating wildland and prescribed fires on terrain. The modifications made to QUIC-Fire are described in three chapters. The first chapter describes the extension of the diagnostic wind model QUIC-URB, the wind engine of QUIC-Fire, to a terrain-following coordinate system. The terraininfluenced winds it generates are analyzed and compared. In particular, this chapter presents the mathematical derivation of the wind solver leading to a linear system of equations that are solved through the successive over-relaxation method. The model is validated against a standard test used in previous works (the Askervein Hill) and against a new dataset from measurements in the Socorro Mountains, New Mexico. The terrain-following implementation captures the correct phenomenology for the isolated Askervein Hill, with a wind speed up at the top of the hill. The model agrees well with measurements on the upwind side of the peak, but overestimates speed-up on the downwind side of the hill. This is due to the inability of the model to generate flow separation and wake-eddy dynamics. On a common laptop, the divergence-free wind field is obtained in 6 s, making the solver appealing for coupled fire-atmosphere simulations. The Socorro Mountain is highly complex, with many cliff faces, peaks, and valleys. Although the model captures the magnitude and direction of inlet and outlet areas of the domain, it performs rather poorly in the valley region and in the regions near the steep cliffs. Hence, the model shows good agreement with data in areas of open sloped terrain but lacks in areas where flow separation and thermally driven effects may be present (neither effect is addressed in this work). In the second chapter the implementation of the terrain-following version of QUIC-URB into QUIC-Fire, and the necessary changes needed to include terrain are described. No changes to the underlying fire spread algorithm are made other than what is required to correctly account for the inclusion of terrain. Previously published FIRETEC results that use five different topographies that share the same centerline profile are compared to simulation results from the modified QUIC-Fire that use the same topographies and fuels. QUIC-Fire results show overall similar behaviors in terms of how the topographies affect fire shapes and trends in spread rates. Due to the terrain-following version of QUIC-URB being unable to generate flow separations at the crest of hills, fire spread rates in these regions across all topographies are over-predicted when compared to FIRETEC. Lateral fire growth shows similar trends with FIRETEC between topographies but does not capture the increase in spread due to a diagonal interface between grassland and forested fuel region of the domain. These results suggest that there are three algorithms within QUIC-Fire that could use improvement: how flame tilt angle is accounted for, the incorporation of non-local drag effects, and the inclusion of the wake-eddy parameterizations that are used in QUIC-URB. Lastly, the third chapter describes a modification to the initial guess used for the calculation of the QUIC-URB mass-conserved wind solution during fire simulations. The modification is aimed at improving fire-fire interactions in QUIC-Fire simulations. The modification consists of using the solution from the previous timestep as the starting point for the calculation of the solution for the next timestep. Fire-fire interactions is greatly improved by the change but a new source of error is introduced. Due to how plumes are modelled in QUIC-Fire the new solution contains errors where gaps in the plume structure are present. However, these errors are mostly limited to the upper atmosphere, where they do not affect fire behavior at the surface, and their magnitude isn’t significant enough to discount the amount of new fire phenomenology now captured in QUIC-Fire with the change.
Digital Signal Processing Using Deep Neural Networks
Currently there is great interest in the utility of deep neural networks (DNNs) for the physical layer of radio frequency (RF) communications. In this manuscript we describe a custom DNN specially designed to solve problems in the RF domain. Our model leverages the mechanisms of feature extraction and attention through the combination of an autoencoder convolutional network with a transformer network, to accomplish several important communications network and digital signals processing (DSP) tasks. We also present a new open dataset and physical data augmentation model that enables training of DNNs that can perform automatic modulation classification, infer, and correct transmission channel effects, and directly demodulate baseband RF signals.
Surface Complexation/Ion Exchange Hybrid Model for Radionuclide Sorption to Clay Minerals (M4SF-23LL010301062)
This progress report (Level 4 Milestone Number M4SF-23LL010301062) summarizes research conducted at Lawrence Livermore National Laboratory (LLNL) within the Argillite International Collaborations Activity Number SF-23LL01030106. The activity is focused on our long-term commitment to engaging our partners in international nuclear waste repository research. The focus of this milestone is the establishment of international collaborations for surface complexation modeling and the associated impacts of unlocking larger, community-based datasets. More specifically, we are developing a database framework for Spent Fuel and Waste and Science Technology (SFWST) that is aligned with the Helmholtz Zentrum Dresden Rossendorf (HZDR) sorption database development group in support of the database needs of the SFWST program. In our FY22 effort, we described a detailed analysis of U(VI) sorption to quartz through both traditional surface complexation modeling and through a hybrid ML framework. In FY23, effort was placed on publication of these results and expansion of the LLNL surface complexation and ion exchange database (L-SCIE) in order to assess mineral-based radionuclide retardation under a wider variety of geochemical conditions (e.g., ionic strength, varying electrolyte compositions). Efforts were initiated to expand L-SCIE to include radionuclide surface complexation and ion exchange to clays that are relevant to subsurface geochemical processes occurring at nuclear waste repositories. In particular, a large source of sorption data for clays resides at the Paul Scherrer Institute (PSI) (work primarily by Bradbury and Baeyens) and we initiated discussions on how to retrieve those data and apply FAIR principles to those datasets. In addition to L-SCIE development, two hybrid models that incorporate AI/ML were investigated and compared to discern the most promising approaches for accurate and precise estimations of radionuclide retardation. Key considerations for future model development include (1) the ability to reduce computational burden on determining retardation coefficients for PA and (2) the ability to quantify and predict radionuclide-mineral partitioning at a more efficient, rapid pace due to automated workflows. Upon the careful consideration of the most effective modeling approaches, we are identifying ways to implement these approaches into PA. Ultimately, the data science-based workflows will provide a major incentive for other institutions to adopt a FAIR-formatted, interoperable database. LLNL will play a key role in disseminating sorption data and acting as good data stewards by updating the database in a consistent format and assessing the quality of the newly assimilated data in an organized fashion. To this end, all data and workflows are open access and made available on the LLNL Seaborg research website (https://seaborg.llnl.gov/resources/geochemical-databases-modeling-codes).
Comparison of Open-Source Photovoltaic Performance Models Against Multi-Year Field Data
Explore the source record for details and available documents.