Search NASA⌕ Search

SEARCH · Search NASA

Results for “custom data sets”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 37 records · Page 2

Scoreboard

Emerging HPC machines have given rise to enhanced compute power that far outstrips the machine's ability to save large scale results for post-processing. To combat this, in situ data analysis techniques are slowly being adopted. With in situ data management favoring workflows composed of multiple simulations and analyses connected in transit on heterogeneous machines, scientists and engineers need a tool that enables them to create data extracts, visualizations, and interactively monitor and steer their simulations. Scoreboard Phase II is a next generation analysis software that supports composite in transit workflows on heterogeneous architectures and restores interactivity to in situ data analysis through simulation monitoring and computational steering. Scoreboard provides a simulation dashboard with graphs of metrics over time, controls for setting custom simulation steering parameters, controls for managing the set of data extracts being produced in the simulation, as well as the ability to explore data extracts, all from a web browser. Realizing the vision outlined in this project required research into making a system that integrates end to end from simulations all the way to the user. In situ tools generally suffer from complexity and excessive software dependencies. Scoreboard, by contrast, is easy to build and integrate into simulation codes and it provides first class FORTRAN support. The Scoreboard library is capable of in situ and in transit data analysis that can produce data extracts commonly needed for Computational Fluid Dynamics (CFD) analysis. Simulations can transparently stage data in transit to a Scoreboard Endpoint program, which can accept their data and produce the requested data extracts. This lets simulations return to their work while the Endpoint works on the analysis. Efficiently staging the data at scale was a topic of this research. Scoreboard provides the means to let the user manage data extracts and monitor/steer many simulations from a web browser. This area of the research focused on discovery of in transit network components to expose and control their steering parameters within an interactive browser-based user interface that includes: system topology, gathered metrics, notifications, dynamically-generated steering controls, and exploration of visualization data products.

Whitlock, BradJoseph [Intelligent Light] (00000001↗

Custom Accessors: Enabling Scalable Data Ingestion, (Re-)Organization, and Analysis on Distributed Systems

The emerging class of high velocity and high volume data analytic workflows comprise interwoven data ingestion, organization, and processing stages, with ingestion and organization steps often contributing comparable or even higher computational costs than actual processing steps. Since complex workflows consist of a variety of phases that view and use data differently, being able to construct efficient, scalable, distributed data structures (arrays, vectors, sets, maps, and multi-maps) is essential and requires custom methods to extend and shrink containers, analyze and position data, and, maintain globallyconsistent meta-data. In this paper, we propose a novel datastructure access paradigm based on the concept of Accessors. At a high level, accessors are customizable callable objects that can modify the behavior of insert, read, update, and delete operations for distributed containers while preserving atomicity guarantees. Accessors provide a very clean and natural way to implement a variety of programming patterns, e.g., conditional insertion/deletion and cascading computations, which would be otherwise hard (or even impossible) to express in parallel and distributed settings without using locks. We demonstrate the practicality and usefulness of our approach with two representative use cases and study the performance of these applications on a distributed High-Performance Computing system. Our analysis highlights that our proposed abstraction allows for an effective overlapping and concurrent execution of different workflow steps (e.g., data ingestion and analysis), which in a conventional analytics pipeline would execute sequentially, contributing cumulatively to the overall latency.

Castellana, Vito G. [BATTELLE (PACIFIC NW LAB)] (O↗

Product Innovation to Increase Low-to-Moderate-Income Customers' Adoption of Community Solar PV

This study aims to comprehensively analyze community solar project preferences for consumers and suppliers by conducting three distinct analyses. First we analyze the predictors of community solar contract adoption to understand how individual priorities affect the probability of adoption. Using an original data set of survey responses from potential community solar customers, we analyzed the predictors of contract adoption by employing a weighted logit model. We find that individuals who were previously familiar with community solar projects were significantly more likely to adopt than those who were not familiar. Secondly, a survey of community solar developers and financiers identified industry perceived barriers to community solar access and inclusion. Thirdly, we gathered payment performance information from community solar initiatives to measure how financial risks are perceived and how they interact with customer demographics. Our study is beneficial to the public by providing insights into the drivers and barriers of community solar adoption and sheds light on the importance of understanding individual priorities in designing effective community solar policies. The community solar industry has changed significantly since the beginning of this project. The industry continues to grow at a rapid rate, with an additional 7 gigawatts expected to come online between 2022 and 2027. With federal pressure to meet climate goals, as the harms of climate change continue to impact everybody, legislators are looking to community solar as a method to achieving their states energy policy goals. These new policies that push for low-to-moderate inclusion, coupled with the increase in community solar capacity illustrate a new era for the community solar industry. A number of policies have arisen in the last few months that push for more inclusive practices including the groundbreaking Solar for All program run by the EPA. The research created a “best practice” contract that can then be used, in conjunction with the manuscript and validated study, to pitch the industry on a more inclusive community solar product.

14 SOLAR ENERGY↗

Soil Carbon Data: long tail recovery

The software is intended to be part of an open source effort regarding soils data. The software provides customized data ingestion scripts for soil carbon related data sets and scripts for output databases that conform to common templates.

Source record↗

Soil Carbon Data: long tail recovery

The software is intended to be part of an open source effort regarding soils data. The software provides customized data ingestion scripts for soil carbon related data sets and scripts for output databases that conform to common templates.

Source record↗

Automating the Solar Interconnection Technical Evaluation Process: PREconfiguring and Controlling Inverter SEt-Points (PRECISE)

Utilities are receiving an increasing number of interconnection requests for distributed solar photovoltaic (PV) systems from their customers, requiring a solution to quickly and intelligently perform technical assessments of these requests and to manage these new renewable energy assets. In this paper, we present a stand-alone interconnection tool, PREconfiguring and Controlling Inverter SEt-points (PRECISETM), and how it overcomes the challenges of accelerating technical evaluations and leverages smart inverter functionality to provide local voltage support on an as-needed basis. The paper details the purpose and architecture of PRECISE, including tools, data source requirements, the model-based interconnection evaluation process, and technical results. PRECISE is an integrated software tool that customizes PV inverter settings for utilities by modeling and assessing impacts on local voltages and the need for advanced inverter functions (AIFs) (e.g., volt-VAR and volt-watt). PRECISE maximizes the use of readily available utility data sets to make fast online assessments of incoming PV interconnection requests and makes accept/reject recommendations along with custom settings for AIFs.

ENERGY PLANNING, POLICY, AND ECONOMY,SOLAR ENERGY↗

Localized Actual Meteorological Year File Creator (LAF): A tool for using locally observed weather data in building energy simulations

The Localized Actual Meteorological Year File Creator (LAF) application provides web-based access to real meteorological data and processes it into a weather file suitable for building energy modeling. Building energy consumption is affected by what is inside the building (such as occupants, appliances, HVAC systems, etc.) and by what is outside: the weather conditions the building is exposed to. However, freely available weather data files are limited to a number of specific locations, usually airports, which are often located away from city centers where buildings are concentrated. The authors have developed a new tool that supports the investigation and quantification of micro-climate conditions on building energy consumption. LAF is built on the Python open-source programming language and has a Graphical User Interface (GUI) that allows users to create custom weather data files for building energy simulations. Many sets of actual meteorological year weather data for long time periods are publicly available online (such as the MesoWest database) for thousands of locations in the US. LAF selects weather data according to user specifications and automatically processes it through an API across multiple weather stations and multiple time periods. The user may easily select a specific location, time frame, and time step that best meets their needs. This article presents a useful tool for energy modelers, building designers and operators to assist with building performance analysis and optimization.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗

LBNL-Load

This software package is a set of Python-based tools for forecasting future time series of electrical power demand for a granular set of customer types in California, based on historical data on individual customers' demand, combined with aggregate forecasts of demand growth.

Gerke, Brian↗

Exploration of Metagenome Assemblies with an Interactive Visualization Tool

Metagenomics, one of the fastest growing areas of modern genomic science, is the genetic profiling of the entire community of microbial organisms present in an environmental sample. Elviz is a web-based tool for the interactive exploration of metagenome assemblies. Elviz can be used with publicly available data sets from the Joint Genome Institute or with custom user-loaded assemblies. Elviz is available at genome.jgi.doe.gov/viz

Cantor, Michael↗

PSY (PowerSystems.jl) [SWR-23-105]

PowerSystems.jl is a package to organize and manipulate data for the study of energy systems with diverse modeling requirements. This software serves two main purposes: to reduce the burden of large power system data set development, and to promote reproducible research and simulation. PowerSystems.jl implements an abstract hierarchy to represent and customize power systems data and includes data containers for quasi-static and dynamic simulation applications. Key features include efficient management of large quantities of time series data, optimized serialization, and comprehensive validation capabilities. For further information regarding the Sienna modeling framework at NREL, see: https://www.nrel.gov/analysis/sienna.html

Lara Aguilar, Jose Daniel↗

Off-line tracking of series parameters in distribution systems using AMI data

Electric distribution systems have historically lacked measurement points, and equipment is often operated to its failure point, resulting in customer outages. The widespread deployment of sensors at the distribution level is enabling observability. This paper presents an off-line parameter value tracking procedure that takes advantage of the increasing number of measurement devices being deployed at the distribution level to estimate changes in series impedance parameter values over time. The tracking of parameter values enables non-diurnal and non-seasonal change to be flagged for investigation. The presented method uses an unbalanced Distribution System State Estimation (DSSE) and a measurement residual-based parameter estimation procedure. Measurement residuals from multiple measurement snapshots are combined in order to increase the effective local redundancy and improve the robustness of the calculations in the presence of measurement noise. Data from devices on the primary distribution system and from customer meters, via an AMI system, form the input data set. Results of simulations on the IEEE 13-Node Test Feeder are presented to illustrate the proposed approach applied to changes in series impedance parameters. A 5% change in series resistance elements can be detected in the presence of 2% measurement error when combining less than 1 day of measurement snapshots into a single estimate.

Williams, Tess L.↗

Portfolio-Scale Optimization of Customer Energy Efficiency Incentive and Marketing: Cooperative Research and Development Final Report, CRADA Number CRD-13-535

North East utility National Grid (NGrid) is developing a portfolio-scale application of OpenStudio designed to optimize incentive and marketing expenditures for their energy efficiency (EE) programs. NGrid wishes to leverage a combination of geographic information systems (GIS), public records, customer data, and content from the Building Component Library (BCL) to form a JavaScript Object Notation (JSON) input file that is consumed by an OpenStudio-based expert system for automated model generation. A baseline model for each customer building will be automatically tuned using electricity and gas consumption data, and a set of energy conservation measures (ECMs) associated with each NGrid incentive program will be applied to the model. The simulated energy performance and return on investment (ROI) will be compared with customer hurdle rates and available incentives to A) optimize the incentive required to overcome the customer hurdle rate and B) determine if marketing activity associated with the specific ECM is warranted for that particular customer. Repeated across their portfolio, this process will enable NGrid to substantially optimize their marketing and incentive expenditures, targeting those customers that will likely adopt and benefit from specific EE programs.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

The Case for Custom TMY's: Examples Using the NSRDB

A typical meteorological year (TMY) data set essentially represents an hourly compilation of median months constructed using multi-year datasets. Although TMY data sets are generated from irradiance data in the horizontal plane, they are used in photovoltaic (PV) modeling for systems inclined to various angles. This paper demonstrates that a POA TMY (plane of array TMY) generated by selecting median months from a multi-year POA irradiance timeseries dataset produces significantly different results than a POA TMY generated by transposing a TMY dataset constructed from horizontal data. In some months the differences can be more than 3%. These results point to the need for generating TMY's using POA irradiance timeseries representing the orientation at which PV panels will be deployed.

14 SOLAR ENERGY↗

HydroForecast Long-term: Improving hydropower’s resilience to climate change through accurate climate-scale

With hydrologic patterns and water availability across the globe shifting due to climate change, advancements in hydrologic prediction systems can help significantly reduce the uncertainties that utilities and water supply entities have in their decision making. Understanding and estimating hydrology at the climate scale is critical for managing water resources under changing climate scenarios. This project focuses on integrating state-of-the-art neural network modeling with downscaled climate projections to deliver the reliable water supply projections decades into the future to meet an urgent need from hydropower operators and water utilities. In this Phase 1 DOE SBIR proposal, we developed and validated a theory-guided neural network model, HydroForecast Long-term, for climate-scale hydrology and implemented the model within existing HydroForecast infrastructure. HydroForecast Long-term combines the most accurate streamflow modeling system with a flexible and scalable data architecture to generate water supply projections out to the year 2100. This report illustrates that we have achieved our four objectives: 1) create a prototype of HydroForecast Long-term, building the neural network prediction model, 2) build an automated data input pipeline that processes large amounts of data from the latest global temperature and precipitation climate models; 3) benchmark the accuracy of the hydrologic model over the recent two decades over a large set of diverse basins, and 4) create a set of output visuals and summary metrics informed by customer feedback that connect the data to critical decision points. This work empowers water users to make data-informed decisions supporting a resilient, renewable-powered grid and water system. The results advance the Department of Energy’s mission by addressing critical gaps in water supply planning under climate change.

13 HYDRO ENERGY↗

Measuring Distribution Performance? Benchmarking Warrants Your Attention

Identifying, designing, and measuring performance metrics is critical to securing customer value, but can be a difficult task. This article examines the use of benchmarks based on publicly available performance data to set challenging, yet fair, metrics and targets.

2017 customer value ranking↗

Systems and methods for customizing kernel machines with deep neural networks

A method including receiving an input data set. The input data set can include one of a feature domain set or a kernel matrix. The method also can include constructing dense embeddings using: (i) Nyström approximations on the input data set when the input data set comprises the kernel matrix, and (ii) clustered Nyström approximations on the input data set when the input data set comprises the feature domain set. The method additionally can include performing representation learning on each of the dense embeddings using a multi-layer fully-connected network for each of the dense embeddings to generate latent representations corresponding to each of the dense embeddings. The method further can include applying a fusion layer to the latent representations corresponding to the dense embeddings to generate a combined representation. The method additionally can include performing classification on the combined representation. Other embodiments of related systems and methods are also disclosed.

Song, Huan↗

MetaboDirect: an analytical pipeline for the processing of FT-ICR MS-based metabolomic data

Background: Microbiomes are now recognized as the main drivers of ecosystem function ranging from the oceans and soils to humans and bioreactors. However, a grand challenge in microbiome science is to characterize and quantify the chemical currencies of organic matter (i.e., metabolites) that microbes respond to and alter. Critical to this has been the development of Fourier transform ion cyclotron resonance mass spectrometry (FT-ICR MS), which has drastically increased molecular characterization of complex organic matter samples, but challenges users with hundreds of millions of data points where readily available, user-friendly, and customizable software tools are lacking. Results: Here, we build on years of analytical experience with diverse sample types to develop MetaboDirect, an open-source, command-line-based pipeline for the analysis (e.g., chemodiversity analysis, multivariate statistics), visualization (e.g., Van Krevelen diagrams, elemental and molecular class composition plots), and presentation of direct injection high-resolution FT-ICR MS data sets after molecular formula assignment has been performed. When compared to other available FT-ICR MS software, MetaboDirect is superior in that it requires a single line of code to launch a fully automated framework for the generation and visualization of a wide range of plots, with minimal coding experience required. Among the tools evaluated, MetaboDirect is also uniquely able to automatically generate biochemical transformation networks (ab initio) based on mass differences (mass difference network-based approach) that provide an experimental assessment of metabolite connections within a given sample or a complex metabolic system, thereby providing important information about the nature of the samples and the set of microbial reactions or pathways that gave rise to them. Finally, for more experienced users, MetaboDirect allows users to customize plots, outputs, and analyses. Conclusion: Application of MetaboDirect to FT-ICR MS-based metabolomic data sets from a marine phage-bacterial infection experiment and a Sphagnum leachate microbiome incubation experiment showcase the exploration capabilities of the pipeline that will enable the research community to evaluate and interpret their data in greater depth and in less time. It will further advance our knowledge of how microbial communities influence and are influenced by the chemical makeup of the surrounding system. The source code and User’s guide of MetaboDirect are freely available through (https://github.com/Coayala/MetaboDirect) and (https://metabodirect.readthedocs.io/en/latest/), respectively.

54 ENVIRONMENTAL SCIENCES↗

Mobilizing the anti-duck brigade: technology and market pathways for load-shifting demand response in California

With rapidly increasing renewable power generation, the California electricity system is experiencing the impacts of the so-called “duck curve,” with occasional over-generation in the middle of the day, followed by rapid ramping of net demand producing a sharp evening peak. Mitigating this pattern will require novel demand-side management approaches to shifting demand away from high net demand periods and toward high renewables generation periods. In this paper, drawing from more than 200,000 individual residential, commercial, and industrial customers’ hourly smart meter data, we disaggregate load shapes for certain key end-uses and then average the results across a highly granular set of customer segments by sector, building type, demand profile and geographic region, to yield a library of characteristic daily load profiles (“clusters”) for different end-uses and seasons. By categorizing the library into particularly desirable or undesirable load shapes in the context of mitigating the duck curve, we can gain insight into specific technologies, seasonal effects, or site-level attributes that may serve as helpful levers to use in developing future load-shifting programs. Based on these insights, we develop recommendations on technologies and new market structures and incentives that can create a future demand curve that more closely matches renewable generation patterns.

24 POWER TRANSMISSION AND DISTRIBUTION↗