Search NASA⌕ Search

SEARCH · Search NASA

Results for “Self Organizing Map”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 37 records · Page 2

New avenue to the parton distribution functions: Self-organizing maps

Neural network algorithms have been recently applied to construct parton distribution function (PDF) parametrizations which provide an alternative to standard global fitting procedures. In this exploratory study we propose a technique using self-organizing maps (SOMs). SOMs are a class of clustering algorithms based on competitive learning among spatially ordered neurons. Our SOMs are trained on selections of stochastically generated PDF samples. The selection criterion for every optimization iteration is based on the features of the clustered PDFs. Our goal is a fitting procedure that, at variance with the standard neural network approaches, will allow for an increased control of the systematic bias by enabling user interaction in the various stages of the process.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

Dark Energy Survey Year 6 results: Clustering redshifts and importance sampling of self-organized-maps 𝑛⁡(𝑧) realizations for 3 × 2 ⁢pt samples

This work is part of a series establishing the redshift framework for the 3 × 2 ⁢pt analysis of the Dark Energy Survey Year 6 (DES Y6). For DES Y6, photometric redshift distributions are estimated using self-organizing maps (SOMs), calibrated with spectroscopic and many-band photometric data. To overcome limitations from color-redshift degeneracies and incomplete spectroscopic coverage, we enhance this approach by incorporating clustering-based redshift constraints (clustering-z, or WZ) from angular cross-correlations with BOSS and eBOSS galaxies and eBOSS quasar samples. We define a WZ likelihood and apply importance sampling to a large ensemble of SOM-derived 𝑛⁡(𝑧) realizations, selecting those consistent with the clustering measurements to produce a posterior sample for each lens and source bin. The analysis uses angular scales corresponding to 1.5–5 Mpc to optimize signal-to-noise ratio while mitigating modeling uncertainties and marginalizes over redshift-dependent galaxy bias and other systematics informed by the N-body simulation CARDINAL . While a sparser spectroscopic reference sample limits WZ constraining power at 𝑧 >1.1, particularly for source bins, we demonstrate that combining SOM with WZ improves redshift accuracy and enhances the overall cosmological constraining power of DES Y6. As a result, we estimate an improvement in 𝑆 8 of approximately 10% for cosmic shear and 3 ×2⁢pt analysis, primarily due to the WZ calibration of the source samples.

Cosmological parameters↗

Self-Organizing-Map Program for Analyzing Multivariate Data

SOM_VIS is a computer program for analysis and display of multidimensional sets of Earth-image data typified by the data acquired by the Multi-angle Imaging Spectro-Radiometer [MISR (a spaceborne instrument)]. In SOM_VIS, an enhanced self-organizing-map (SOM) algorithm is first used to project a multidimensional set of data into a nonuniform three-dimensional lattice structure. The lattice structure is mapped to a color space to obtain a color map for an image. The Voronoi cell-refinement algorithm is used to map the SOM lattice structure to various levels of color resolution. The final result is a false-color image in which similar colors represent similar characteristics across all its data dimensions. SOM_VIS provides a control panel for selection of a subset of suitably preprocessed MISR radiance data, and a control panel for choosing parameters to run SOM training. SOM_VIS also includes a component for displaying the false-color SOM image, a color map for the trained SOM lattice, a plot showing an original input vector in 36 dimensions of a selected pixel from the SOM image, the SOM vector that represents the input vector, and the Euclidean distance between the two vectors.

Li, P. Peggy↗

Atmospheric Drivers of Greenland Surface Melt Revealed by Self-Organizing Maps

Recent acceleration in surface melt on the Greenland ice sheet (GrIS) has occurred concurrently with a rapidly warming Arctic and has been connected to persistent, anomalous atmospheric circulation patterns over Greenland. To identify synoptic setups favoring enhanced GrIS surface melt and their decadal changes, we develop a summer Arctic synoptic climatology by employing self-organizing maps. These are applied to daily 500 hPa geopotential height fields obtained from the Modern Era Retrospective Analysis for Research and Applications reanalysis, 1979-2014. Particular circulation regimes are related to meteorological conditions and GrIS surface melt estimated with outputs from the Modèle Atmosphérique Régional. Our results demonstrate that the largest positive melt anomalies occur in concert with positive height anomalies near Greenland associated with wind, temperature, and humidity patterns indicative of strong meridional transport of heat and moisture. We find an increased frequency in a 500 hPa ridge over Greenland coinciding with a 63% increase in GrIS melt between the 1979-1988 and 2005-2014 periods, with 75.0% of surface melt changes attributed to thermodynamics, 17% to dynamics, and 8.0% to a combination. We also confirm that the 2007-2012 time period has the largest dynamic forcing relative of any period but also demonstrate that increased surface energy fluxes, temperature, and moisture separate from dynamic changes contributed more to melt even during this period. This implies that GrIS surface melt is likely to continue to increase in response to an ever warmer future Arctic, regardless of future atmospheric circulation patterns.

anomalies↗

Automated Classification of Thermal Infrared Spectra Using Self-organizing Maps

Existing and planned space missions to a variety of planetary and satellite surfaces produce an ever increasing volume of spectral data. Understanding the scientific informational content in this large data volume is a daunting task. Fortunately various statistical approaches are available to assess such data sets. Here we discuss an automated classification scheme based on Kohonen Self-organizing maps (SOM) we have developed. The SUM process produces an output layer were spectra having similar properties lie in close proximity to each other. One major effort is partitioning this output layer into appropriate regions. This is prefonned by defining dosed regions based upon the strength of the boundaries between adjacent cells in the SOM output layer. We use the Davies-Bouldin index as a measure of the inter-class similarities and intra-class dissimilarities that determines the optimum partition of the output layer, and hence number of SOM clusters. This allows us to identify the natural number of clusters formed from the spectral data. Mineral spectral libraries prepared at Arizona State University (ASU) and John Hopkins University (JHU) are used to test and evaluate the classification scheme. We label the library sample spectra in a hierarchical scheme with class, subclass, and mineral group names. We use a portion of the spectra to train the SOM, i.e. produce the output layer, while the remaining spectra are used to test the SOM. The test spectra are presented to the SOM output layer and assigned membership to the appropriate cluster. We then evaluate these assignments to assess the scientific meaning and accuracy of the derived SOM classes as they relate to the labels. We demonstrate that unsupervised classification by SOMs can be a useful component in autonomous systems designed to identify mineral species from reflectance and emissivity spectra in the therrnal IR.

Roush, Ted L.↗

Using Self-Organizing Maps to Identify Coherent CONUS Precipitation Regions

Extreme precipitation events have major societal impacts. These events are rare and can have small spatial scale, making statistical analysis difficult; both factors are mitigated by combining events over a region. A methodology is presented to objectively define “coherent” regions wherein data points have matching annual cycles. Regions are found by training self-organizing maps (SOMs) on the annual cycle of precipitation for each grid point across the contiguous United States (CONUS). Using the annual cycle for our intended application minimizes problems caused by consecutive dry periods and localized extreme events. Multiple criteria are applied to identify useful numbers of regions for our future application. Criteria assess these properties for each region: having many more events than experienced by a single grid point, good connectedness and compactness, and robustness to changing the number of regions. Our methodology is applicable across datasets and is tested here on both reanalysis and gridded observational data. Precipitation regions obtained align with large-scale geographical features and are readily interpretable. Useful numbers of regions balance two conflicting preferences: larger regions contain more events and thereby have more robust statistics, but more compact regions allow weather patterns associated with extreme events to be aggregated with confidence. For 6-h precipitation, 12–15 regions over the CONUS optimize our metrics. The regions obtained are compared against two existing region archetypes. For example, a popular set of regions, based on nine groups of states, has less coherent regions than defining the same number of regions with our SOM methodology.

54 ENVIRONMENTAL SCIENCES↗

Analysis of WRF extreme daily precipitation over Alaska using self‐organizing maps

Abstract We analyze daily precipitation extremes from simulations of a polar‐optimized version of the Weather Research and Forecasting (WRF) model. Simulations cover 19 years and use the Regional Arctic System Model (RASM) domain. We focus on Alaska because of its proximity to the Pacific and Arctic oceans; both provide large moisture fetch inland. Alaska's topography also has important impacts on orographically forced precipitation. We use self‐organizing maps (SOMs) to understand circulation characteristics conducive for extreme precipitation events. The SOM algorithm employs an artificial neural network that uses an unsupervised training process, which results in finding general patterns of circulation behavior. The SOM is trained with mean sea level pressure (MSLP) anomalies. Widespread extreme events, defined as at least 25 grid points experiencing 99th percentile precipitation, are examined using SOMs. Widespread extreme days are mapped onto the SOM of MSLP anomalies, indicating circulation patterns. SOMs aid in determining high‐frequency nodes, and hence, circulations are conducive to extremes. Multiple circulation patterns are responsible for extreme days, which are differentiated by where extreme events occur in Alaska. Additionally, several meteorological fields are composited for nodes accessed by extreme and nonextreme events to determine specific conditions necessary for a widespread extreme event. Individual and adjacent node composites produce more physically reasonable circulations as opposed to composites of all extremes, which include multiple synoptic regimes. Temporal evolution of extreme events is also traced through SOM space. Thus, this analysis lays the groundwork for diagnosing differences in atmospheric circulations and their associated widespread, extreme precipitation events.

Glisan, Justin M.↗

Cloud fraction at the ARM SGP site: Reducing uncertainty with self-organizing maps

Instrument downtime leads to uncertainty in the monthly and annual record of cloud fraction (CF), making it difficult to perform time series analyses of cloud properties and perform detailed evaluations of model simulations. As cloud occurrence is partially controlled by the large-scale atmospheric environment, this knowledge is used to reduce uncertainties in the instrument record. Synoptic patterns diagnosed from the North American Regional Reanalysis (NARR) during the period 1997–2010 are classified using a competitive neural network known as the self-organizing map (SOM). The classified synoptic states are then compared to the Atmospheric Radiation Measurement (ARM) Southern Great Plains (SGP) instrument record to determine the expected CF. A number of SOMs are tested to understand how the number of classes and the period of classifications impact the relationship between classified states and CFs. Bootstrapping is utilized to quantify the uncertainty of the instrument record when statistical information from the SOM is included. Although all SOMs significantly reduce the uncertainty of the CF record calculated in Kennedy et al. (Theor Appl Climatol 115:91–105, 2014), SOMs with a large number of classes and separated by month are required to produce the lowest uncertainty and best agreement with the annual cycle of CF. Lastly, this result may be due to a manifestation of seasonally dependent biases in NARR.

54 ENVIRONMENTAL SCIENCES↗

Connecting Large‐Scale Meteorological Patterns to Extratropical Cyclones in CMIP6 Climate Models Using Self‐Organizing Maps

Abstract Extratropical cyclones (ETCs) are responsible for the majority of cool‐season extreme events in the northeastern United States (NEUS), often leading to high‐impact weather conditions that can have wide‐ranging socioeconomic impacts. Evaluating the ability of climate models to adequately simulate ETC dynamics is essential for improving model performance and increasing confidence in future projections used by stakeholders and policymakers. ETCs are traditionally studied using techniques such as case studies and synoptic typing, however, these approaches can be time‐consuming, require subjective analysis, and do not necessarily identify the coincident large‐scale meteorological patterns (LSMPs). Here, we apply self‐organizing maps (SOMs) as an automated machine‐learning approach to characterize the LSMPs and associated frequency and intensity of discrete ETC events over NEUS. The dominant patterns of geopotential height variability are identified through SOM analysis of five reanalysis products during the last four decades. ETC events are tracked using TempestExtremes and are integrated with SOMs to classify the accumulated cyclone activity (ACA) associated with each pattern. We then evaluate the skill of CMIP6 historical experiments in simulating the LSMPs and ETC events identified in the SOM. Our results identify a robust bias toward more zonal patterns, with models struggling to reproduce the more amplified patterns typically associated with the highest cyclone activity. While model resolution has some impact on simulation credibility, model configuration appears to be more important in LSMP representation. The vast majority of CMIP6 models produce too few ETCs, although model errors are distributed around historical reanalyses when ACA is normalized by storm frequency.

54 ENVIRONMENTAL SCIENCES↗

Application of materials informatics to vapor-grown carbon nanofiber/vinyl ester nanocomposites through self-organizing maps and clustering techniques

Data mining and knowledge discovery techniques were employed herein to acquire new information on the viscoelastic, flexural, compressive, and tensile properties of vapor-grown carbon nanofiber (VGCNF)/vinyl ester (VE) nanocomposites. Formulation and processing factors (curing environment, presence or absence of dispersing agent, mixing method, VGCNF weight fraction, VGCNF type, high-shear mixing time, and sonication time) and testing temperature were utilized as inputs and the true ultimate strength, true yield strength, engineering elastic modulus, engineering ultimate strength, flexural modulus, flexural strength, storage modulus, loss modulus, and tan delta were selected as outputs. The data mining and knowledge discovery algorithms used in this study include self-organizing maps (SOMs) and clustering techniques. SOMs demonstrated that temperature and tan delta had the most significant effects on the output responses followed by the VGCNF high-shear mixing time, and sonication time. SOMs were also used to produce optimal responses using certain combination(s) of inputs. Fuzzy C-means algorithm (FCM) was also applied to discover patterns in the nanocomposite behavior subsequent to a principal component analysis (PCA), which is a dimensionality reduction technique. Utilizing these techniques, the nanocomposite specimens were separated into different clusters based on the testing temperature (30 °C and 120 °C being the most dominant responses), tan delta, high-shear mixing time, and sonication time. Furthermore, the VGCNF/VE specimens were separated into a cluster based on their viscoelastic responses (storage and loss moduli) at the same temperature. The FCM results indicate that, while all nanocomposite properties in the new framework are essential, the viscoelastic responses of the VGCNF/VE specimens are the most significant. Here this work highlights the utility of data mining and knowledge discovery techniques in the context of materials informatics for the discovery of patterns and trends in the material behavior that are not immediately known.

36 MATERIALS SCIENCE↗

Efficient Clustering of Software Vulnerabilities using Self Organizing Map (SOM)

The common vulnerabilities and exposures (CVE) database was created with a mission to ``identify, define, and catalog publicly disclosed cybersecurity vulnerabilities''. This rich body of information can be used to enable rapid and efficient response to secure and defend cyber operations and protect critical cyber infrastructure. The main goal of this paper is to develop a visual analytics tool to enable deep analysis of CVEs using unsupervised clustering techniques. We enhance our analysis by first mapping CVEs to hierarchical-classes in Common Weakness Enumeration (CWE) using information in the National Vulnerability Database (NVD). Both the mapping and the numerical representation of CVEs are enabled by V2W-BERT, which uses natural language processing of the extensive information in NVD to generate a large tabular database of 137,226 CVE entries from 1999 to 2020, where each CVE is represented by a vector of 768 numerical features. The vectorized data is processed by Self-Organizing Maps (SOM), which is an unsupervised machine learning technique for dimensionality reduction, visual representation and clustering. Using a Torus map of 6417 units, we achieve ~10-fold data compression of ~140k CVEs using SOM. The trained map is further clustered using standard K-means clustering into 138 clusters of CVEs. We conducted a brief investigation of the rich mapping of CVEs to best-matching-units to K-means clusters, as well as CVEs to CWEs. For example, this novel mapping provided insight into the role of CWE-59 and CWE-264 in several CVEs that is otherwise hard to explore in the original data. We conclude that our this novel approach will not only enable deep analysis of the complex relationships between CVEs and CWEs, but also a mechanism to quickly respond to and design mitigation actions for rapidly evolving vulnerabilities that have not been mapped to existing CWEs.

Panchal, Khyati↗

Phenotypic redshifts with self-organizing maps: A novel method to characterize redshift distributions of source galaxies for weak lensing

Wide-field imaging surveys such as the Dark Energy Survey (DES) rely on coarse measurements of spectral energy distributions in a few filters to estimate the redshift distribution of source galaxies. In this regime, sample variance, shot noise, and selection effects limit the attainable accuracy of redshift calibration and thus of cosmological constraints. We present a new method to combine wide-field, few-filter measurements with catalogues from deep fields with additional filters and sufficiently low photometric noise to break degeneracies in photometric redshifts. The multiband deep field is used as an intermediary between wide-field observations and accurate redshifts, greatly reducing sample variance, shot noise, and selection effects. Our implementation of the method uses self-organizing maps to group galaxies into phenotypes based on their observed fluxes, and is tested using a mock DES catalogue created from N-body simulations. Here, it yields a typical uncertainty on the mean redshift in each of five tomographic bins for an idealized simulation of the DES Year 3 weak-lensing tomographic analysis of σ Δz = 0.007, which is a 60 per cent improvement compared to the Year 1 analysis. Although the implementation of the method is tailored to DES, its formalism can be applied to other large photometric surveys with a similar observing strategy.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

Identification of the threshold security applied for fuel under reactivity-initiated conditions by self-organizing maps and neural networks - 14386

In recent times, we have 439 reactors distributed in 31 countries that give about 377 GW, covering around 12% of world capacity of energy. Currently, it accounts for an estimated 40 serious accidents reported in nuclear power plants. In this article, turns out the behavior of fuel rod in light-water reactors under accident conditions. Besides, we investigate deeply about reactivity-initiated accidents, by introducing Nordheim-Fuchs model as state equations. The early research showed that pre-irradiated fuels and afterward subjected to a rapid pulse of energy. It adopts the key items necessary to check the threshold of failure. These items are few limits as burning the fuel; the value applied to the peak of energy, pulse width, oxidation layer, and cladding. The kinetic equations of nuclear reactors modeled as a non-linear system. In this way, it can apply the paradigm of artificial intelligence using the self-organized maps. Where the goal is to find the cluster of failure for irradiated fuels until 75 GWd/MTU. The second model based on statistical is the logistic regression theory. Hence, the tests carried out and compared against the results found by neural networks for classification of clusters. It proofs that both approaches present capacity to find the flaws accurately of about 80% of cases. (authors)

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

Lagrangian Characterization of Surface Transport From the Equatorial Atlantic to the Caribbean Sea Using Climatological Lagrangian Coherent Structures and Self‐Organizing Maps

Abstract This study presents an assessment of the transport of suspended material by surface ocean currents, which have a critical role in determining the connectivity and distribution of living and non‐living material. Lagrangian experiments reveal pathways from the Equatorial Atlantic to 10 strategic regions within the Caribbean Sea, determined by considering the space‐time variability of climatological Lagrangian Coherent Structures, which act as recurrent attracting pathways and transport barriers. Due to windage or Stokes drift, wind forcing is a significant factor in determining the spatial locations where particles cluster and the time needed to reach the Caribbean from the Equatorial Atlantic. Pathways shift westward within the Caribbean and take less time to arrive with increasing wind influence. Depending on the wind effect, the particles show higher confluence in different areas of the Caribbean. A case study is presented for the Mexican Caribbean nearshore area, isolated from ocean‐current trajectories. Here, wind weakens the transport barrier responsible for this isolation and causes particle confluence toward that region. Spatial patterns of the Eulerian velocity identified through Self‐Organizing Maps, with time dependence given by their best matching units, can reproduce the characteristic Lagrangian patterns of surface current climate variability. Our study demonstrates the application of tools from dynamical systems and unsupervised neural networks to understand Lagrangian patterns and identify the processes that drive them. These findings improve our understanding of transport mechanisms of suspended material by surface ocean currents in the Western Atlantic and the Caribbean Sea, which is essential for managing and conserving marine ecosystems.

Allende‐Arandía, Ma. Eugenia↗

Detection of Anomalies in Gamma Background Radiation Data with K-Means and Self-Organizing Map Clustering Algorithms (Consortium on Nuclear Security Technologies (CONNECT) Q1 Report)

Environmental screening of gamma radiation consists of detecting weak nuisance and anomaly signal in the presence of strong and highly varying background. In a typical scenario, a mobile detector-spectrometer continuously measures gamma radiation spectra in short, e.g., one-second, signal acquisition intervals. The measurement data is a 2D matrix, where one dimension is gamma ray energy, and the other dimension is the number of measurements or total time. In principle, gamma radiation sources can be detected and identified from the measured data by their unique spectral lines. Detecting sources from data measured in a search scenario is difficult due to the highly varying background because of naturally occurring radioactive material (NORM), and low signal-to-noise ratio (S/N) of spectral signal measured during one-second acquisition intervals. The objective of this work is to explore unsupervised machine learning (ML) algorithms for detection and identification of weak nuisances and anomalies events in the presence of highly fluctuating background. The challenge is that spectral lines of isotopes are difficult to observe in one-second measurements. Averaging over the entire measurement campaign data set reveals spectral lines of most common background isotopes. Spectral lines of orphan sources, which might appear only in a few measurements during the campaign, will be washed out if averaging is performed over the entire measurement data set. The approach we have explored consists of extracting one-second measurements containing weak spectral features through data clustering. Averaging one-second spectra in a cluster should reveal the presence of anomaly sources. We created two ML models using K-means clustering and Neural Network Self-organizing Map (SOM). Performance of these ML models was benchmarked using search data. One data set contained 137 Cs source, and another dataset contained 131 I source.

61 RADIATION PROTECTION AND DOSIMETRY↗

Hyperspectral Image Classification using a Self-Organizing Map

The use of hyperspectral data to determine the abundance of constituents in a certain portion of the Earth's surface relies on the capability of imaging spectrometers to provide a large amount of information at each pixel of a certain scene. Today, hyperspectral imaging sensors are capable of generating unprecedented volumes of radiometric data. The Airborne Visible/Infrared Imaging Spectrometer (AVIRIS), for example, routinely produces image cubes with 224 spectral bands. This undoubtedly opens a wide range of new possibilities, but the analysis of such a massive amount of information is not an easy task. In fact, most of the existing algorithms devoted to analyzing multispectral images are not applicable in the hyperspectral domain, because of the size and high dimensionality of the images. The application of neural networks to perform unsupervised classification of hyperspectral data has been tested by several authors and also by us in some previous work. We have also focused on analyzing the intrinsic capability of neural networks to parallelize the whole hyperspectral unmixing process. The results shown in this work indicate that neural network models are able to find clusters of closely related hyperspectral signatures, and thus can be used as a powerful tool to achieve the desired classification. The present work discusses the possibility of using a Self Organizing neural network to perform unsupervised classification of hyperspectral images. In sections 3 and 4, the topology of the proposed neural network and the training algorithm are respectively described. Section 5 provides the results we have obtained after applying the proposed methodology to real hyperspectral data, described in section 2. Different parameters in the learning stage have been modified in order to obtain a detailed description of their influence on the final results. Finally, in section 6 we provide the conclusions at which we have arrived.

Martinez, P.↗