Results for “self-organizing maps” — Page 3

DOE OSTI2024

Tracking precipitation features and associated large-scale environments over southeastern Texas

Abstract. Deep convection initiated under different large-scale environmental conditions exhibits different precipitation features and interacts with local meteorology and surface properties in distinct ways. Here, we analyze the characteristics and spatiotemporal patterns of different types of convective systems over southeastern Texas using 13 years of high-resolution observations and reanalysis data. We find that mesoscale convective systems (MCSs) contribute significantly to both mean and extreme precipitation in all seasons, while isolated deep convection (IDC) plays a role in intense precipitation during summer and fall. Using self-organizing maps (SOMs), we found that convection can occur under unfavorable conditions without large-scale lifting or moisture convergence. In spring, fall, and winter, front-related large-scale meteorological patterns (LSMPs) characterized by low-level moisture convergence act as primary triggers for convection, while the remaining storms are associated with an anticyclonic pattern and orographic lifting. In summer, IDC events are mainly associated with front-related and anticyclonic LSMPs, while MCSs occur more in front-related LSMPs. We further tracked the life cycle of MCS and IDC events using the Flexible Object Tracker algorithm over southeastern Texas. MCSs frequently initiate west of Houston, traveling eastward for around 8 h to southeastern Texas, while IDC events initiate locally. The average duration of MCSs in southeastern Texas is 6.1 h, approximately 4.1 times the duration of IDC events. Diurnally, the initiation of convection associated with favorable LSMPs peaks at 11:00 UTC, 3 h earlier than that associated with anticyclones.

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2023

Variations in boundary layer stability across Antarctica: a comparison between coastal and interior sites

The range of boundary layer stability profiles, from the surface to 500 m a.g.l. (above ground level), present in radiosonde observations from two continental-interior (South Pole Station and Dome Concordia Station) and three coastal (McMurdo Station, Georg von Neumayer Station III, and Syowa Station) Antarctic sites, is examined using the self-organizing maps (SOMs) neural network algorithm. A wide range of potential temperature profiles is revealed, from shallow boundary layers with strong near-surface stability to deeper boundary layers with weaker or near-neutral stability, as well as profiles with weaker near-surface stability and enhanced stability aloft, above the boundary layer. Boundary layer regimes were defined based on the range of profiles revealed by the SOM analysis; 20 boundary layer regimes were identified to account for differences in stability near the surface as well as above the boundary layer. Strong, very strong, or extremely strong stability, with vertical potential temperature gradients of 5 to in excess of 30 K per 100 m, occurred more than 80 % of the time at South Pole and Dome Concordia in the winter. Weaker stability was found in the winter at the coastal sites, with moderate and strong stability (vertical potential temperature gradients of 1.75 to 15 K per 100 m) occurring 70 % to 85 % of the time. Even in the summer, moderate and strong stability is found across all five sites, either immediately near the surface or aloft, just above the boundary layer. While the mean boundary layer height at the continental-interior sites was found to be approximately 50 m, the mean boundary layer height at the coastal sites was deeper, around 110 m. Further, a commonly described two-stability-regime system in the Arctic associated with clear or cloudy conditions was applied to the 20 boundary layer regimes identified in this study to understand if the two-regime behavior is also observed in the Antarctic. It was found that moderate and strong stability occur more often with clear- than cloudy-sky conditions, but weaker stability regimes occur almost equally for clear and cloudy conditions.

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2025

Synoptic Weather Regime Classifications for June, July, August, and September, 2022

The synoptic weather regime classification has become a highly demanded product for the ARM site in recent years. This type of regime classification has shown applications in various studies and topics, including aerosol-cloud interactions, land-atmosphere interactions, and cloud radiative effects. The VAP employs an unsupervised machine learning method, Self-organizing map (SOM), to classify weather regimes for each day of the AMF campaigns and fixed sites, using ERA5 data. This idea is mainly based on our published study for TRACER in Wang et al. (2022, JGR-A). This dataset includes the data in June, July, August, and September; the last year of the data is 2022.

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2025

Synoptic Weather Regime Classifications for the whole year, from 2014 to 2015

The synoptic weather regime classification has become a highly demanded product for the ARM site in recent years. This type of regime classification has shown applications in various studies and topics, including aerosol-cloud interactions, land-atmosphere interactions, and cloud radiative effects. The VAP employs an unsupervised machine learning method, Self-organizing map (SOM), to classify weather regimes for each day of the AMF campaigns and fixed sites, using ERA5 data. This idea is mainly based on our published study for TRACER in Wang et al. (2022, JGR-A).

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2025

Synoptic Weather Regime Classifications for June, July, August, from 2000 to 2024

The synoptic weather regime classification has become a highly demanded product for the ARM site in recent years. This type of regime classification has shown applications in various studies and topics, including aerosol-cloud interactions, land-atmosphere interactions, and cloud radiative effects. The VAP employs an unsupervised machine learning method, Self-organizing map (SOM), to classify weather regimes for each day of the AMF campaigns and fixed sites, using ERA5 data. This idea is mainly based on our published study for TRACER in Wang et al. (2022, JGR-A).

node_som_ml↗

DOE OSTI2025

Synoptic Weather Regime Classifications for March, April and May, from 2000 to 2025

The synoptic weather regime classification has become a highly demanded product for the ARM site in recent years. This type of regime classification has shown applications in various studies and topics, including aerosol-cloud interactions, land-atmosphere interactions, and cloud radiative effects. The VAP employs an unsupervised machine learning method, Self-organizing map (SOM), to classify weather regimes for each day of the AMF campaigns and fixed sites, using ERA5 data. This idea is mainly based on our published study for TRACER in Wang et al. (2022, JGR-A).

node_som_ml↗

DOE OSTI2025

Synoptic Weather Regime Classifications for June, July and August, from 2000 to 2025

The synoptic weather regime classification has become a highly demanded product for the ARM site in recent years. This type of regime classification has shown applications in various studies and topics, including aerosol-cloud interactions, land-atmosphere interactions, and cloud radiative effects. The VAP employs an unsupervised machine learning method, Self-organizing map (SOM), to classify weather regimes for each day of the AMF campaigns and fixed sites, using ERA5 data. This idea is mainly based on our published study for TRACER in Wang et al. (2022, JGR-A).

node_som_ml↗

DOE OSTI2025

Synoptic Weather Regime Classifications for December, January and February, from 2000 to 2025

The synoptic weather regime classification has become a highly demanded product for the ARM site in recent years. This type of regime classification has shown applications in various studies and topics, including aerosol-cloud interactions, land-atmosphere interactions, and cloud radiative effects. The VAP employs an unsupervised machine learning method, Self-organizing map (SOM), to classify weather regimes for each day of the AMF campaigns and fixed sites, using ERA5 data. This idea is mainly based on our published study for TRACER in Wang et al. (2022, JGR-A).

{node_som_ml,synop_wea_reg}↗

DOE OSTI2025

Synoptic Weather Regime Classifications for September, October and November, from 2000 to 2025

The synoptic weather regime classification has become a highly demanded product for the ARM site in recent years. This type of regime classification has shown applications in various studies and topics, including aerosol-cloud interactions, land-atmosphere interactions, and cloud radiative effects. The VAP employs an unsupervised machine learning method, Self-organizing map (SOM), to classify weather regimes for each day of the AMF campaigns and fixed sites, using ERA5 data. This idea is mainly based on our published study for TRACER in Wang et al. (2022, JGR-A).

node_som_ml↗

DOE OSTI2025

Dark Energy Survey Year 6 Results: Redshift Calibration of the MagLim++ Lens Sample

In this work, we derive and calibrate the redshift distribution of the MagLim++ lens galaxy sample used in the Dark Energy Survey Year 6 (DES Y6) 3x2pt cosmology analysis. The 3x2pt analysis combines galaxy clustering from the lens galaxy sample and weak gravitational lensing. The redshift distributions are inferred using the SOMPZ method - a Self-Organizing Map framework that combines deep-field multi-band photometry, wide-field data, and a synthetic source injection (Balrog) catalog. Key improvements over the DES Year 3 (Y3) calibration include a noise-weighted SOM metric, an expanded Balrog catalogue, and an improved scheme for propagating systematic uncertainties, which allows us to generate O($10^8$) redshift realizations that collectively span the dominant sources of uncertainty. These realizations are then combined with independent clustering-redshift measurements via importance sampling. The resulting calibration achieves typical uncertainties on the mean redshift of 1-2%, corresponding to a 20-30% average reduction relative to DES Y3. We compress the $n(z)$ uncertainties into a small number of orthogonal modes for use in cosmological inference. Marginalizing over these modes leads to only a minor degradation in cosmological constraints. This analysis establishes the MagLim++ sample as a robust lens sample for precision cosmology with DES Y6 and provides a scalable framework for future surveys.

Giannini, G. [Chicago U., Astron. Astrophys. Ctr.;↗

DOE OSTI2025

Dark Energy Survey Year 6 Results: Redshift Calibration of the Weak Lensing Source Galaxies

Determining the distribution of redshifts for galaxies in wide-field photometric surveys is essential for robust cosmological studies of weak gravitational lensing. We present the methodology, calibrated redshift distributions, and uncertainties of the final Dark Energy Survey Year 6 (Y6) weak lensing galaxy data, divided into four redshift bins centered at $\langle z \rangle = [0.414, 0.538, 0.846, 1.157]$. We combine independent information from two methods on the full shape of redshift distributions: optical and near-infrared photometry within an improved Self-Organizing Map $p(z)$ (SOMPZ) framework, and cross-correlations with spectroscopic galaxy clustering measurements (WZ), which we demonstrate to be consistent both in terms of the redshift calibration itself and in terms of resulting cosmological constraints within 0.1$σ$. We describe the process used to produce an ensemble of redshift distributions that account for several known sources of uncertainty. Among these, imperfection in the calibration sample due to the lack of faint, representative spectra is the dominant factor. The final uncertainty on mean redshift in each bin is $σ_{\langle z\rangle} = [0.012, 0.008,0.009, 0.024]$. We ensure the robustness of the redshift distributions by leveraging new image simulations and a cross-check with galaxy shape information via the shear ratio (SR) method.

Yin, B. [Duke U.] (ORCID:0009000656049980)↗

DOE OSTI2025

The Power of DESI for Photometric Redshift Calibration: A Case Study with KiDS-1000

Accurate redshift estimates are a critical requirement for weak lensing surveys and one of the main uncertainties in constraints on dark energy and large-scale cosmic structure. In this paper, we study the potential to calibrate photometric redshift (photo-z) distributions for gravitational lensing using the Dark Energy Spectroscopic Instrument (DESI). Since beginning its science operations in 2021, DESI has collected more than 50 million redshifts, adding about one million monthly. In addition to its large-scale structure samples, DESI has also acquired over 256k high-quality spectroscopic redshifts (spec-zs) in the COSMOS and XMM and VVDS fields. This is already a factor of 3 larger than previous spec-z calibration compilations in these two regions. Here, we explore calibrating photo-zs for the subset of KiDS-1000 galaxies that fall into joint self-organizing map (SOM) cells overlapping the DESI COSMOS footprint using the DESI COSMOS observations. Estimating the redshift distribution in KiDS-1000 with the new DESI data, we find broad consistency with previously published results while also detecting differences in the mean redshift in some tomographic bins with an average shifts of Delta Mean(z) = -0.028 in the mean and Delta Median(z) = +0.011 in the median across tomographic bins. However, we also find that incompleteness per SOM cell, i.e., groups of galaxies with similar colors and magnitudes, can modify n(z) distributions. Finally, we comment on the fact that larger photometric catalogs, aligned with the DESI COSMOS and DESI XMM and VVDS footprints, would be needed to fully exploit the DESI dataset and would extend the coverage to nearly eight times the area of existing 9-band photometry.

Blanco, Diana [UC, Santa Cruz; UC, Santa Cruz, Ins↗

DOE OSTI2023

A Novel Resilience-Oriented Cellular Grid Formation Approach for Distribution Systems with Behind-the-Meter Distributed Energy Resources

This paper presents a resilience-oriented cellular grid formation approach to achieve scalable and reconfigurable community microgrid operations for distribution systems with behind-the-meter distributed energy resources. A set of interconnected solar photovoltaics, energy storage systems, and load is termed as a cell, implying a subset of the grid that can operate independently using its own resources. Cells are identified such that each cell inherently has sufficient energy resources to black start and can provide a certain level of backup power for its load under the loss of utility power supply. The proposed cell formation approach builds on a unique self-organizing map-based method (SomRes) to quantify a system's resilience. Using SomRes and a non-dominated sorting-based genetic algorithm (NSGA-II), a fast and efficient cell formation algorithm is developed to identify cells in a distribution system that are resilient against extreme events. The efficacy of the proposed approach is demonstrated on a numerical model of a real distribution feeder in Colorado, United States.

distributed energy resource↗

DOE OSTI2023

Land surface dynamics and meteorological forcings modulate land surface temperature characteristics

This study examines the effect of land cover, vegetation health, climatic forcings, elevation heat loads, and terrain characteristics (LVCET) on land surface temperature (LST) distribution over West Africa (WA). We employ fourteen machine-learning models, which preserve nonlinear relationships, to downscale LST and other predictands while preserving the geographical variability of WA. Our results showed that the random forest model performs best in downscaling predictands. This is important for the sub-region since it has limited access to mainframes to power multiplex machine-learning algorithms. In contrast to the northern regions, the southern regions consistently exhibit healthy vegetation. Also, areas with unhealthy vegetation coincide with hot LST clusters. The positive Normalized Difference Vegetation Index (NDVI) trends in the Sahel underscore rainfall recovery and subsequent Sahelian greening. The southwesterly winds cause the upwelling of cold waters, lowering LST in southern WA and highlighting the cooling influence of water bodies on LST. Identifying regions with elevated LST is paramount for prioritizing greening initiatives, and our study underscores the importance of considering LVCET factors in urban planning. Topographic slope-facing angles, heat loads, and diurnal anisotropic heat all contribute to variations in LST, emphasizing the need for a holistic approach when designing resilient and sustainable landscapes.

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2026

A Unified Photometric Redshift Calibration for Weak Lensing Surveys Using the Dark Energy Spectroscopic Instrument

The effective redshift distribution n(z) of galaxies is a critical component in the study of weak gravitational lensing. Here, we introduce a new method for determining n(z) for weak lensing surveys based on high-quality redshifts and neural-network-based importance weights. Additionally, we present the first unified photometric redshift calibration of the three leading stage-III weak lensing surveys, the Dark Energy Survey (DES), the Hyper Suprime-Cam (HSC) survey, and the Kilo-Degree Survey (KiDS), with state-of-the-art spectroscopic data from the Dark Energy Spectroscopic Instrument (DESI). We verify our method using a new, data-driven approach and obtain n(z) constraints with statistical uncertainties of the order of $σ_z$ ~ 0.01 and smaller. Our analysis is largely independent of previous photometric redshift calibrations and, thus, provides an important cross-check in light of recent cosmological tensions. Overall, we find excellent agreement with previously published results on the DES Y3 and HSC Y1 data sets, while there are some differences on the mean redshift with respect to the previously published KiDS-1000 results. We attribute the latter to mismatches in photometric noise properties in the COSMOS field compared to the wider KiDS self-organizing map-gold catalog. At the same time, the new n(z) estimates for KiDS do not significantly change estimates of cosmic structure growth from cosmic shear. Finally, we discuss how our method can be applied to future weak lensing calibrations with DESI data.

Lange, J. U. [American Univ., Washington, DC (Unit↗

DOE OSTI2023

Extensive 3D mapping of dislocation structures in bulk aluminum

Abstract Thermomechanical processing such as annealing is one of the main methods to tailor the mechanical properties of materials, however, much is unknown about the reorganization of dislocation structures deep inside macroscopic crystals that give rise to those changes. Here, we demonstrate the self-organization of dislocation structures upon high-temperature annealing in a mm-sized single crystal of aluminum. We map a large embedded 3D volume ( $$100\times 300\times 300$$ 100 × 300 × 300 $$\upmu$$ μ m $$^3$$ 3 ) of dislocation structures using dark field X-ray microscopy (DFXM), a diffraction-based imaging technique. Over the wide field of view, DFXM’s high angular resolution allows us to identify subgrains, separated by dislocation boundaries, which we identify and characterize down to the single-dislocation level using computer-vision methods. We demonstrate how even after long annealing times at high temperatures, the remaining low density of dislocations still pack into well-defined, straight dislocation boundaries (DBs) that lie on specific crystallographic planes. In contrast to conventional grain growth models, our results show that the dihedral angles at the triple junctions are not the predicted 120 $$^{\circ }$$ ∘ , suggesting additional complexities in the boundary stabilization mechanisms. Mapping the local misorientation and lattice strain around these boundaries shows that the observed strain is shear, imparting an average misorientation around the DB of $$\approx$$ ≈ 0.003 to 0.006 $$^{\circ }$$ ∘ .

36 MATERIALS SCIENCE↗

DOE OSTI2025

Emergent topological polarization textures in relaxor ferroelectrics

Relaxor ferroelectrics underpin high-performance actuators and sensors, yet the nature of polar heterogeneities driving their broadband dielectric response remains debated. Using a unified, multimodal structural refinement framework— simultaneously fitting complementary X-ray and neutron total scattering, X-ray absorption spectra, and diffuse scattering—we reconstruct 3D mesoscale polarization maps in the classic relaxor system PbMg 1/3 Nb 2/3 O 3 –PbTiO 3 . We uncover self-organized swirling polarization textures with half-skyrmion (meron) vortices, challenging models of independent polar nanoregions. These textures, characterized by smooth changes in the polarization direction, originate from overlapping volumes in which the projections of locally correlated polarization vectors onto each volume’s long axis share the same sign. Vortex cores correlate strongly with local charge and strain gradients imposed by compositional heterogeneities. In this work, our results suggest that chemical disorder, acting via depolarizing and strain fields, stabilizes topological vortex textures of the polarization field, offering a route for engineering new dielectric and ferroelectric functionalities.

Eremenko, Maksim [National Institute of Standards ↗

DOE OSTI2023

Self-organization of photoionized plasmas via kinetic instabilities

Abstract Self-organization in an unmagnetized collisionless plasma (in this paper) refers to formation of transient coherent structures such as collective oscillations (electrostatic waves) or magnetic fields resulting from so-called kinetic effects in the plasma. This topical review provides a comprehensive analysis of the self-organization of strong-field photoionized, non-equilibrium plasmas through kinetic instabilities. The authors propose and demonstrate a novel experimental platform that enables the formation of dense plasmas with known highly anisotropic and non-thermal electron velocity distribution functions on a timescale on the order of an inverse electron plasma frequency. We then show that such plasmas are highly susceptible to a hierarchy of kinetic instabilities, including two-stream, current filamentation and Weibel, that convert a fraction of the electron kinetic energy into electric and/or magnetic energy stored in self-organized structures. The electrostatic waves so produced are measured using a collective light (Thomson) scattering technique with femtosecond resolution as the kinetic instabilities aided by collisions eventually thermalize the plasma electrons. In addition, we describe a novel experimental technique that has made it possible to map the temporal evolution of the wavenumber spectrum of the thermal Weibel instability with picosecond resolution, which leads to the formation of quasi-static coherent magnetic fields with different topologies in photoionized plasmas. Finally, the paper summarizes the important results and discusses future directions on this topic.

Zhang, Chaojie (ORCID:0000000180353014)↗