Search NASA⌕ Search

SEARCH · Search NASA

Results for “customer clustering”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records
All sourcesNASA NTRSDOE OSTI
DOE OSTI2019

Nonnegative tensor decomposition with custom clustering for microphase separation of block copolymers

High–dimensional datasets are becoming ubiquitous in many applications and therefore unsupervised tensor methods to interrogate them are needed. Here, we report a new unsupervised machine learning (ML) approach (NTFk) based on nonnegative tensor factorization integrated with a custom k–means clustering. We demonstrate the ability of NTFk to extracting temporal and spatial features of phase separation of copolymers as they are modeled by self–consistent field theory. Microphase separation of block copolymers has been extensively studied both experimentally and theoretically. However, the interpretation of computer simulations and/or experimental data, representing temporal and spatial changes of molecular species concentration is still a challenging task. Thus, extracting the phase diagram from simulations or experimental data as well as the interpretation of data requires discernment of the model/experimental parameters (such as, temperature, concentrations, the number of molecular species and the interaction between species) impact on the microphase separation process. An attractive and unique aspect of the introduced ML method is that it ensures the nonnegativity of the extracted latent features. Nonnegativity is an essential constraint needed to obtain interpretable and sparse latent features that are parts–based representation of the data. The custom clustering in NTFk serves to estimate the number of latent features in the data.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

A multi-level load shape clustering and disaggregation approach to characterize patterns of energy consumption behavior

This study presents representative electrical load shapes, disaggregated to the end-use level, for over 5000 customer clusters across California’s residential, commercial, industrial and agricultural sectors. We developed a novel, multi-level load shape clustering approach for residential and commercial sectors leveraging interval meter data for over 350,000 California utility customers collected as a part of the Phase 4 California Demand Response (DR) Potential Study. The clustering approach allowed us to identify typical consumption patterns and categorize customers based on their daily load shape displayed throughout the year. For example, we were able to identify customers with particular energy technologies such as electric vehicles and rooftop solar, as well as building occupancy types such as restaurants, grocery stores and even unoccupied buildings, based solely on whole-building interval data. We then combined the load shape-based clusters with other customer information including building type, climate, geographical area, total consumption and low-income status, to create a set of customer clusters based on both demographics and usage patterns. Total cluster electricity demand was then disaggregated into a wide variety of end-uses using weather normalization and other publicly available end-use load shape datasets. The resulting disaggregated cluster load shapes will be released in anonymized form as part of the Phase 4 DR Potential Study. They will have wide-ranging applications in energy research and policy analysis, including estimation of energy efficiency (EE) and DR potential on the end-use level, time-dependent valuation of EE savings, building stock modeling, and developing customer targeting strategies for EE and DR programs.

Murthy, Samanvitha↗
DOE OSTI2020

Distributed non-negative matrix factorization with determination of the number of latent features

The holistic analysis and understanding of the latent (that is, not directly observable) variables and patterns buried in large datasets is crucial for data-driven science, decision making and emergency response. Such exploratory analyses require devising unsupervised learning methods for data mining and extraction of the latent features, and non-negative matrix factorization (NMF) is one of the prominent such methods. NMF is based on compute-intense non-convex constrained minimization, which, for large datasets requires fast and distributed algorithms. However, current parallel implementations of NMF fail to estimate the number of latent features. In practice, identifying these features is both difficult and significant for pattern recognition and latent feature analysis, especially for large dense matrices. Here, we introduce a distributed NMF algorithm coupled with distributed custom clustering followed by a stability analysis on dense data, which we call DnMFk, to determine the number of latent variables. The results on synthetic data and the classical Swimmer data set demonstrate the accuracy of model determination while scaling nearly linearly across multiple processors for large data. Further, we employ DnMFk to determine the number of hidden features from a terabyte matrix.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2019

Unsupervised machine learning based on non-negative tensor factorization for analyzing reactive-mixing

Analysis of reactive-diffusion simulations representing complex mixing processes requires a large number of independent model runs. For each high-fidelity model simulation, the model inputs are varied and the predicted mixing behavior is represented by temporal and spatial changes in species concentration. It is then required to discern how the model inputs (such as diffusivity, dispersion, anisotropy, and velocity field properties) impact the mixing process. This task is challenging and typically involves interpretation of large model outputs representing temporal and spatial changes of species concentration within the model domain. Yet, the task can be automated and substantially simplified by applying Machine Learning (ML) methods. Here, we present an application of an unsupervised ML method (called NTF k ) using Non-negative Tensor Factorization (NTF) coupled with a custom clustering procedure based on k-means to reveal the temporal and spatial features in product concentrations. An attractive and unique aspect of the proposed ML method is that it ensures the extracted features are non-negative, which is important to obtain a meaningful deconstruction of the mixing processes. The ML methodology is applied to a large set of high-resolution finite-element model simulations representing anisotropic reaction-diffusion processes in perturbed vortex-based velocity fields. The applied finite-element method ensures that spatial and temporal species concentration are always non-negative, even in the case of high anisotropic contrasts. The simulated reaction is a fast irreversible bimolecular reaction A+B→C , where species A and B react to form species C. The reactive-diffusion model input parameters that control mixing include properties of the velocity field (such as vortex structures), anisotropic dispersion, and molecular diffusion. We demonstrate the applicability of the ML feature extraction method to produce a meaningful deconstruction of model outputs to discriminate between different physical processes impacting the reactants, their mixing, and the spatial distribution of the product C. The introduced ML analysis allowed us to identify additive temporal and spatial features that characterize mixing behavior. The application of the proposed NTF k approach is not limited to reactive-mixing. NTFk can be readily applied to any observed or simulated datasets that can be represented as tensors (multi-dimensional arrays) and have separable latent signatures or features.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Discovering hidden geothermal signatures using non-negative matrix factorization with customized k-means clustering

Discovery of hidden geothermal resources is challenging. It requires the mining of large datasets with diverse data attributes representing subsurface hydrogeological and geothermal conditions. The commonly used play fairway analysis approach typically incorporates subject-matter expertise to analyze regional data to estimate geothermal characteristics and favorability. We demonstrate an alternative approach based on machine learning (ML) to process a geothermal dataset from southwest New Mexico (SWNM). The study region includes low- and medium-temperature hydrothermal systems. Several of these systems are not well characterized because of insufficient existing data and limited past explorative work. This study discovers hidden patterns and relations in the SWNM geothermal dataset to improve our understanding of the regional hydrothermal conditions and energy-production favorability. This understanding is obtained by applying an unsupervised ML algorithm based on non-negative matrix factorization coupled with customized k-means clustering (NMFk). NMFk can automatically identify (1) hidden signatures characterizing analyzed datasets, (2) the optimal number of these signatures, (3) the dominant data attributes associated with each signature, and (4) the spatial distribution of the extracted signatures. Here, in this study, NMFk is applied to analyze 18 geological, geophysical, hydrogeological, and geothermal attributes at 44 locations in SWNM. Using NMFk, we find data patterns and identify the spatial associations of hydrothermal signatures within two physiographic provinces (Colorado Plateau and Basin and Range) and two sub-regions of these provinces (the Mogollon-Datil volcanic field and the Rio Grande rift) in SWNM. The ML algorithm extracted five hydrothermal signatures in the SWNM datasets that differentiate between low (<90°C) and medium (90-150°C)-temperature hydrothermal systems. The algorithm also suggests that the Rio Grande rift and northern Mogollon-Datil volcanic field are the most favorable regions for future geothermal resource discovery. NMFk also identified critical attributes to identify medium-temperature hydrothermal systems in the study area. The resulting NMFk model can be applied to predict geothermal conditions and their uncertainties at new SWNM locations based on limited data from unexplored regions. The code to execute the performed analyses as well as the corresponding data can be found at https://github.com/SmartTensors/GeoThermalCloud.jl.

15 GEOTHERMAL ENERGY↗
DOE OSTI2020

Development and Clustering of Rate-Oriented Load Metrics for Customer Price-Plan Analysis

One of the few methods electric utilities can use to motivate and change customer energy consumption is through retail rate structures. Utilities are increasingly moving toward more dynamic rate plans to encourage energy conservation, utilization of onsite renewable generation, peak demand reduction and flattening of demand profiles. This paper creates a set of rate-oriented load metrics that are the determinants of customers' bills under four unique rate plans. These metrics are not only indicative of which rate structure can provide customer bill reductions based on their load profile characteristics, but also convey useful information about load consumption behavior. With these metrics, utilities can analyze their customers and identify classes that are rewarded under each rate plan. This can help inform utilities whether the customers rewarded under each rate plans are meeting their original objectives. To develop these customer classes, we calculate these rate-oriented load metrics for each customer and perform k-means clustering. The analysis is conducted on a set of 300 customer profiles, examining four different rate plans, different numbers of clusters, customer bills and cluster load profile characteristics.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2024

Electric vehicle supply equipment location and capacity allocation for fixed-route networks

Electric vehicle (EV) supply equipment location and allocation (EVSELCA) problems for freight vehicles are becoming more important because of the trending electrification shift. Some previous works address EV charger location and vehicle routing problems simultaneously by generating vehicle routes from scratch. Although such routes can be efficient, introducing new routes may violate practical constraints, such as drive schedules, and satisfying electrification requirements can require dramatically altering existing routes. To address the challenges in the prevailing adoption scheme, we approach the problem from a fixed -route perspective. We develop a mixed -integer linear program, a clustering approach, and a metaheuristic solution method using a genetic algorithm (GA) to solve the EVSELCA problem. The clustering approach simplifies the problem by grouping customers into clusters, while the GA generates solutions that are shown to be nearly optimal for small problem cases. A case study examines how charger costs, energy costs, the value of time (VOT), and battery capacity impact the cost of the EVSELCA. Charger equipment costs were found to be the most significant component in the objective function, leading to a substantial reduction in cost when decreased. VOT costs exhibited a significant decrease with rising energy costs. Further, an increase in VOT resulted in a notable rise in the number of fast chargers. Longer EV ranges decrease total costs up to a certain point, beyond which the decrease in total costs is negligible.

33 ADVANCED PROPULSION SYSTEMS↗
DOE OSTI2019

Development and Clustering of Rate-Oriented Load Metrics for Customer Price-Plan Analysis: Preprint

One of the few methods electric utilities can use to motivate and change customer energy consumption is through their retail rate structures. Utilities are increasingly moving towards more dynamic rate-plans to achieve the objectives of motivating energy conservation, increasing self-consumption from onsite renewable generation, reducing peak demand and to flattening the demand profiles. This paper presents a set of rate-oriented customer load metrics that are the determinants of customers bill under four unique rate-plans. These metrics are not only indicative of which rate structure a customer should choose based on their load profiles, but also convey useful information about load consumption behavior. With these metrics, the utility can analyze their customers and identify customer classes that are rewarded under each rate-plan. This can help inform utilities whether the customer profiles being rewarded under each rate-plans are meeting their original objectives. To develop these customer classes, we calculate these rate-oriented load metrics for each customer and perform k-means clustering. The analysis is conducted on from a set of 300 customer profiles, examining the impact of four different rate-plans, different numbers of clusters and looking at customer bills and cluster load profile characteristics.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2020

Seismology in the Cloud: A New Streaming Workflow

Data-intensive research in seismology is experiencing a recent boom, driven in part by large volumes of available data and advances in the growing field of data science. However, there are significant barriers to processing large data volumes, such as long retrieval times from data repositories, complex data management, and limited computational resources. New tools and platforms have reduced the barriers to entry for scientific cluster computing, including the maturation of the commercial cloud as an accessible instrument for research. Here, we build a customized research cluster in the cloud to test a new workflow for large-scale seismic analysis, in which data are processed as a stream (retrieved on-the-fly and acted upon without storing), with data from the Incorporated Research Institutions for Seismology Data Management Center. We use this workflow to deploy a spectral peak detection algorithm over 5.6 TB of compressed continuous seismic data from 2074 stations of the USArray Transportable Array EarthScope network. Using a 50-node cluster in the cloud, we completed the noise survey in 80 hr, with an average data throughput of 1.7 GB per minute. By varying cluster sizes, we find the scaling of our analysis to be sublinear, due to a combination of algorithmic limitations and data center response times. The cloud-based streaming workflow represents an order-of-magnitude increase in acquisition and processing speed compared to a traditional download-store-process workflow, and offers the additional benefits of employing a flexible, accessible, and widely used computing architecture. It is limited, however, due to its reliance on Internet transfer speeds and data center service capacity, and may not work well for repeated analyses or those for which even higher data throughputs are needed. These research applications will require a new class of cloud-native approaches in which both data and analysis are in the cloud.

58 GEOSCIENCES↗
DOE OSTI2018

Improving Spectral Image Classification through a Custom Data Sparsing and Pixel Clustering Algorithm

The Underground Nuclear Explosion Signatures Experiment (UNESE) seeks to characterize non-prompt observables from underground nuclear explosions (UNE). As part of this effort, we evaluated the ability of Digital Globe’s World View-3 (WV3) to detect and map UNE signatures. WV3 has proven capabilities in the detection of large-scale, anomalous, man-made features; however, it has relatively limited spectral and spatial resolution as compared to airborne hyperspectral imaging. These limitations impede standard image classifiers from identifying UNE signatures that are spatially small and lack distinct spectral features. In order to improve image classification results, we developed a custom algorithm that increases the true positive rate, while decreasing the false positive rate. Our algorithm’s flexible front-end allows the user to choose the method used to sparse the data. These methods include band-ratio optimization, correlation coefficient threshold, or Adaptive Coherence Estimator (ACE) threshold. The resulting sparsed pixels are clustered into like-objects. A novel weighted-sum of ACE and correlation coefficient scores is used to classify each object based on the average spectrum of its constituent pixels. WV3 and AVIRIS data of Cuprite, Nevada were used as a verification data set. This algorithm improves overall pixel classification accuracy over standard classifiers by up to 14%.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2022

Differentially Private K -Means Clustering Applied to Meter Data Analysis and Synthesis

The proliferation of smart meters has resulted in a large amount of data being generated. It is increasingly apparent that methods are required for allowing a variety of stakeholders to leverage the data in a manner that preserves the privacy of the consumers. The sector is scrambling to define policies, such as the so called ‘15/15 rule’, to respond to the need. However, the current policies fail to adequately guarantee privacy. Here, in this paper, we address the problem of allowing third parties to apply K-means clustering, obtaining customer labels and centroids for a set of load time series by applying the framework of differential privacy. We leverage the method to design an algorithm that generates differentially private synthetic load data consistent with the labeled data. We test our algorithm’s utility by answering summary statistics such as average daily load profiles for a 2-dimensional synthetic dataset and a real-world power load dataset.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2021

Machine learning to discover mineral trapping signatures due to CO 2 injection

Mineral trapping is pursued as a geological CO 2 sequestration (GCS) mechanism because it permanently stores CO 2 in solid phases or minerals. However, CO 2 mineral-trapping mechanisms are poorly understood due to (1) lack of sufficient field and laboratory data characterizing these complex processes, and (2) challenges to develop site-specific reactive-transport models coupling fluid flow and geochemical reactions occurring at various temporal (from milliseconds to years) and spatial (from pore (millimeters) to field (kilometers)) scales. Reactive transport with additional complexities such as heterogeneity can make the simulation outputs even more difficult to interpret because of complex nonlinearity and multi-scale interdependencies. Furthermore, the values of model outputs such as concentrations can vary by several orders of magnitude, making it harder to correlate and characterize the impact of the variables via traditional data interpretation techniques such as exploratory data analyses. Recently, machine learning (ML) has shown promise in feature discovery and in highlighting hidden mechanisms that cannot be obtained by existing data-analytics and statistical methods. In this study, we applied an unsupervised ML approach, non-negative matrix factorization with custom -means clustering (NMF) to the data generated by reactive-transport simulations of GCS. The reactive-transport data consisted of 19 attributes, including four physio-chemical variables (pH, porosity, aqueous CO 2 , and sequestered CO 2 ), six chemical species (K + , Na + , HCO, Ca 2+ , Mg 2+ , Fe 2+ ), and four carbonate minerals (calcite, dolomite, siderite, and ankerite), a feldspar mineral (albite), and four clay minerals (illite, clinochlore, kaolinite, and smectite) over a period of 200 years of simulation time. Furthermore, the simulation data used was for Morrow B sandstone at the Farnsworth hydrocarbon unit in Texas. Data are sampled at two locations within the model domain: (1) at the injection well and (2) 200 m west of the injection well. The injection was performed for a period of 10 years. Using NMF, we estimated the temporal interdependencies among the 19 attributes over a span of 200 years. We found that NMF was able to identify four reaction stages and their dominant attributes; these cannot be directly discerned through traditional visualization (e.g., line plots, Pareto analysis, Glyph-based visualization methods) or exploratory data analysis tools of the simulation data. The four stages were: reactions in the injection phase followed by short-, mid-, and long-term reactions. The NMF analysis also revealed that 10 among the 19 attributes are dominant. These dominant attributes for mineral trapping include calcite, dolomite at injection well, siderite at 200 m away from the injection well, clinochlore, kaolinite, Na + , K + , Ca 2+ , Mg 2+ , pH, and aqeuous CO 2 . Finally, at late times (65–200 years), our results showed that calcite plays a major role in mineral trapping with insignificant contribution from siderite, ankerite, and clay minerals. These findings make the proposed unsupervised ML-model attractive for reactive-transport sensing towards real-time GCS monitoring.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2021

Discovering Hidden Geothermal Signatures using Unsupervised Machine Learning

Discovering hidden geothermal resources is a very challenging task. It requires the mining of large datasets, including various diverse data attributes representing subsurface hydrogeological and geothermal conditions. The commonly used Play Fairway Analysis (PFA) typically relies on subject-matter expertise to analyze site or regional data to estimate geothermal conditions and prospectivity. Here, we demonstrate an alternative approach based on machine learning (ML) to process a geothermal dataset of Southwest New Mexico (SWNM). The study region includes low- and medium-temperature hydrothermal systems. However, most of these systems are poorly characterized because of insufficient existing data and limited past explorative studies. This study aims to discover hidden patterns and relationships in the SWNM geothermal dataset to better understand regional hydrothermal conditions. This is achieved by applying an unsupervised machine learning algorithm based on non-negative matrix factorization coupled with customized k-means clustering (NMFk). NMFk can automatically identify (1) hidden (latent) signatures characterizing datasets, (2) the optimal number of these signatures, (3) dominant data attributes associated with each signature, and (4) spatial distribution of the extracted signatures. Here, NMFk is applied to analyze 18 geological, geophysical, hydrogeological, geothermal attributes at 44 locations in SWNM. NMFk successfully finds data patterns and identifies the spatial associations of hydrothermal signatures with the four physiographic provinces in SWNM (Colorado Plateau, Volcanic Field, Basin and Range, and the Rio Grande rift). The algorithm identified up to 5 hydrothermal signatures in the SWNM datasets that differentiate between low- and medium-temperature hydrothermal systems in different provinces. Also, the algorithm identifies two medium-temperature hydrothermal systems in SWNM that require further exploration for geothermal resource development. Based on our analyses, 12 of the attributes are important to identify medium-temperature hydrothermal systems, and the remaining six attributes are critical to characterize low-temperature hydrothermal systems. Based on the obtained results, we identify potential physiographic provinces for further exploration to characterize them as geothermal resources. The resulting NMFk model can be applied to predict geothermal conditions and their uncertainties at new SWNM locations based on limited data from unexplored areas.

58 GEOSCIENCES↗
NASA NTRSDate not supplied

Congruence of Clusters Defined By Whole Genome Sequencing and MALDI-TOF for Bacteria Isolated From Cleanrooms

Introduction: Oligotrophic conditions can render cleanrooms inhospitable to microbes. Despite these constraints, fungi and bacteria are frequently isolated from surfaces in astromaterials cleanrooms at the Johnson Space Center. Bacillus species are of particular concern because endospores belonging to this genus are resilient and can affect astromaterials. Current monitoring programs rely on 16S rRNA sequencing and the VITEK2 Compact system. These methods have limited power to resolve Bacillus species. Matrix-assisted laser desorption - time of flight mass spectrometry (MALDI-TOF MS), provides a rapid, low cost, method of identifying bacterial isolates and has a higher resolution than 16S rRNA sequencing, particularly for Bacillus species; however, few studies have compared this method to the industry gold standard, whole genome sequencing (WGS). Methods: Based on 16S rRNA classification, we selected 14 isolates for analysis with MALDI-TOF and WGS. Mass spectra were generated with MALDI-TOF MS and processed with custom scripts to identify clusters of closely related isolates and calculate a matrix of pairwise cosine similarity scores. Hybrid Illumina and Nanopore sequencing were used to generate draft genomes. Pairwise similarity scores were calculated from these genomes based on the average amino acid identity (AAI) predicted from single copy core genes. Congruence of clustering between these methods, was assessed by calculating adjusted Rand and Wallace coefficients. Results: Clusters of species generated from MALDI-TOF MS showed good agreement of phylotypes generated with WGS. Pairs of strains that were > 94% similar to each other in terms of predicted amino acid sequences consistently showed cosine similarities of mass spectra > 0.65 and, of the 9 clusters identified with WGS, 8 were identical with MALDI-TOF. This corresponds to an adjusted Rand index of 0.95 and a 95% confidence interval of 0.80 – 1.00 for adjusted Wallace coefficients. The only discordance was for a pair of isolates that were classified as Paenibacillus species. This pair showed relatively high similarity (0.84) in terms of MALDI-TOF MS but only 85% similarity in terms of AAI. Conclusion: This study shows that MALDI-TOF and WGS exhibit a similar ability to delineate Bacillus species isolated from cleanrooms and taxonomic units described by these two methods are consistent with one another. Since MALDI-TOF MS is low in cost and high in throughput, this approach appears to be an ideal option for routine microbial monitoring and identifying Bacillus species.

Farnaz Mazhari↗
DOE OSTI2022

Machine learning and shallow groundwater chemistry to identify geothermal prospects in the Great Basin, USA

This study discovers various geothermal prospects in the Great Basin, USA based on shallow groundwater chemical (geochemical) data. The geochemical data are expected to include hidden (latent) information that is a proxy for geothermal prospectivity. We processed the sparse geochemical data in the Great Basin at 14,341 locations including 18 attributes. Next, a non-negative matrix factorization with customized k-means clustering is applied to the geochemical data matrix that automatically finds three hidden geothermal signatures representing modestly, moderately, and highly confident geothermal prospects. The algorithm also evaluated the probability of occurrence of these types of resources through the studied region. There is a consistency between regional geothermal prospectivity as estimated by our ML methodology and the traditional play fairway analysis conducted over a portion of the study area. We also identify the dominant data attributes associated with each signature. Finally, our ML analyses allow us to reconstruct attributes from sparse into continuous over the study domain. The predicted continuous attributes can be used for future detailed geothermal explorations in the Great Basin.

15 GEOTHERMAL ENERGY↗
DOE OSTI2023

Machine Learning for Geothermal Resource Exploration in the Tularosa Basin, New Mexico

Geothermal energy is considered an essential renewable resource to generate flexible electricity. Geothermal resource assessments conducted by the U.S. Geological Survey showed that the southwestern basins in the U.S. have a significant geothermal potential for meeting domestic electricity demand. Within these southwestern basins, play fairway analysis (PFA), funded by the U.S. Department of Energy’s (DOE) Geothermal Technologies Office, identified that the Tularosa Basin in New Mexico has significant geothermal potential. This short communication paper presents a machine learning (ML) methodology for curating and analyzing the PFA data from the DOE’s geothermal data repository. The proposed approach to identify potential geothermal sites in the Tularosa Basin is based on an unsupervised ML method called non-negative matrix factorization with custom k-means clustering. This methodology is available in our open-source ML framework, GeoThermalCloud (GTC). Using this GTC framework, we discover prospective geothermal locations and find key parameters defining these prospects. Our ML analysis found that these prospects are consistent with the existing Tularosa Basin’s PFA studies. This instills confidence in our GTC framework to accelerate geothermal exploration and resource development, which is generally time-consuming.

15 GEOTHERMAL ENERGY↗