Results for “generalized least squared” — Page 6

DOE OSTI2020

Dakota A Multilevel Parallel Object-Oriented Framework for Design Optimization Parameter Estimation Uncertainty Quantification and Sensitivity Analysis: Version 6.12 Theory Manual

The Dakota toolkit provides a flexible and extensible interface between simulation codes and iterative analysis methods. Dakota contains algorithms for optimization with gradient and nongradient-based methods; uncertainty quantification with sampling, reliability, and stochastic expansion methods; parameter estimation with nonlinear least squares methods; and sensitivity/variance analysis with design of experiments and parameter study methods. These capabilities may be used on their own or as components within advanced strategies such as surrogate-based optimization, mixed integer nonlinear programming, or optimization under uncertainty. By employing object-oriented design to implement abstractions of the key components required for iterative systems analyses, the Dakota toolkit provides a flexible and extensible problem-solving environment for design and performance analysis of computational models on high performance computers. This report serves as a theoretical manual for selected algorithms implemented within the Dakota software. It is not intended as a comprehensive theoretical treatment, since a number of existing texts cover general optimization theory, statistical analysis, and other introductory topics. Rather, this manual is intended to summarize a set of Dakota-related research publications in the areas of surrogate-based optimization, uncertainty quantification, and optimization under uncertainty that provide the foundation for many of Dakota's iterative analysis capabilities.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2021

Dakota A Multilevel Parallel Object-Oriented Framework for Design Optimization Parameter Estimation Uncertainty Quantification and Sensitivity Analysis (V.6.14) (Theory Manual)

The Dakota toolkit provides a flexible and extensible interface between simulation codes and iterative analysis methods. Dakota contains algorithms for optimization with gradient and nongradient-based methods; uncertainty quantification with sampling, reliability, and stochastic expansion methods; parameter estimation with nonlinear least squares methods; and sensitivity/variance analysis with design of experiments and parameter study methods. These capabilities may be used on their own or as components within advanced strategies such as surrogate-based optimization, mixed integer nonlinear programming, or optimization under uncertainty. By employing object-oriented design to implement abstractions of the key components required for iterative systems analyses, the Dakota toolkit provides a flexible and extensible problem-solving environment for design and performance analysis of computational models on high performance computers. This report serves as a theoretical manual for selected algorithms implemented within the Dakota software. It is not intended as a comprehensive theoretical treatment, since a number of existing texts cover general optimization theory, statistical analysis, and other introductory topics. Rather, this manual is intended to summarize a set of Dakota-related research publications in the areas of surrogate-based optimization, uncertainty quantification, and optimization under uncertainty that provide the foundation for many of Dakota's iterative analysis capabilities.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2020

Dakota, A Multilevel Parallel Object-Oriented Framework for Design Optimization, Parameter Estimation, Uncertainty Quantification, and Sensitivity Analysis: Version 6.13 Theory Manual

The Dakota toolkit provides a flexible and extensible interface between simulation codes and iterative analysis methods. Dakota contains algorithms for optimization with gradient and nongradient-based methods; uncertainty quantification with sampling, reliability, and stochastic expansion methods; parameter estimation with nonlinear least squares methods; and sensitivity/variance analysis with design of experiments and parameter study methods. These capabilities may be used on their own or as components within advanced strategies such as surrogate-based optimization, mixed integer nonlinear programming, or optimization under uncertainty. By employing object-oriented design to implement abstractions of the key components required for iterative systems analyses, the Dakota toolkit provides a flexible and extensible problem-solving environment for design and performance analysis of computational models on high performance computers. This report serves as a theoretical manual for selected algorithms implemented within the Dakota software. It is not intended as a comprehensive theoretical treatment, since a number of existing texts cover general optimization theory, statistical analysis, and other introductory topics. Rather, this manual is intended to summarize a set of Dakota-related research publications in the areas of surrogate-based optimization, uncertainty quantification, and optimization under uncertainty that provide the foundation for many of Dakota's iterative analysis capabilities.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2021

Dakota, A Multilevel Parallel Object-Oriented Framework for Design Optimization, Parameter Estimation, Uncertainty Quantification, and Sensitivity Analysis: Theory Manual (V.6.15)

The Dakota toolkit provides a flexible and extensible interface between simulation codes and iterative analysis methods. Dakota contains algorithms for optimization with gradient and nongradient-based methods; uncertainty quantification with sampling, reliability, and stochastic expansion methods; parameter estimation with nonlinear least squares methods; and sensitivity/variance analysis with design of experiments and parameter study methods. These capabilities may be used on their own or as components within advanced strategies such as surrogate-based optimization, mixed integer nonlinear programming, or optimization under uncertainty. By employing object-oriented design to implement abstractions of the key components required for iterative systems analyses, the Dakota toolkit provides a flexible and extensible problem-solving environment for design and performance analysis of computational models on high performance computers. This report serves as a theoretical manual for selected algorithms implemented within the Dakota software. It is not intended as a comprehensive theoretical treatment, since a number of existing texts cover general optimization theory, statistical analysis, and other introductory topics. Rather, this manual is intended to summarize a set of Dakota-related research publications in the areas of surrogate-based optimization, uncertainty quantification, and optimization under uncertainty that provide the foundation for many of Dakota's iterative analysis capabilities.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2021

Dakota, A Multilevel Parallel Object-Oriented Framework for Design Optimization, Parameter Estimation, Uncertainty Quantification, and Sensitivity Analysis: Version 6.16 Theory Manual

The Dakota toolkit provides a flexible and extensible interface between simulation codes and iterative analysis methods. Dakota contains algorithms for optimization with gradient and nongradient-based methods; uncertainty quantification with sampling, reliability, and stochastic expansion methods; parameter estimation with nonlinear least squares methods; and sensitivity/variance analysis with design of experiments and parameter study methods. These capabilities may be used on their own or as components within advanced strategies such as surrogate-based optimization, mixed integer nonlinear programming, or optimization under uncertainty. By employing object-oriented design to implement abstractions of the key components required for iterative systems analyses, the Dakota toolkit provides a flexible and extensible problem-solving environment for design and performance analysis of computational models on high performance computers. This report serves as a theoretical manual for selected algorithms implemented within the Dakota software. It is not intended as a comprehensive theoretical treatment, since a number of existing texts cover general optimization theory, statistical analysis, and other introductory topics. Rather, this manual is intended to summarize a set of Dakota-related research publications in the areas of surrogate-based optimization, uncertainty quantification, and optimization under uncertainty that provide the foundation for many of Dakota’s iterative analysis capabilities.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2021

Comparison of temperature adaptive calibration methods for laser induced fluorescence based fuel-in-oil instrument

Fuel dilution of engine oil (or fuel-in-oil, FiO) is an important issue as multiple and late-cycle fuel injection, integral to many combustion efficiency and emissions improvements (e.g., downsized boosted gasoline engines and catalyst thermal management) increases FiO rate. In addition to causing general wear and corrosion in engine due to decreased oil viscosity and pH buffering, FiO is also believed to cause destructive low speed pre-ignition (or super knock) in boosted SI engines. To understand the effects of engine operating conditions on the FiO rate, an optical diagnostic capable of measuring transient FiO on minute timescales was recently developed and demonstrated on a modified GM Ecotech engine system (Neupane et al., Applied Spectroscopy 2021). The measurement is based on adding a dye to the fuel and monitoring for its presence in oil via laser-induced fluorescence (LIF). The measured LIF signal is related to FiO concentration via pre-determined calibration factors using a multivariate classical least square (CLS) method.Since fluorescence quantum yield is a function of temperature, measured LIF intensity not only depends on FiO concentration but also oil temperature. To expand the applicability of the FiO diagnostic to transient oil-temperature conditions (e.g., cold start in practical engines), this study develops a method to account for oil-temperature variations. The effect of oil temperature (20°C - 95°C) on the LIF spectra of eight FiO samples ranging from ~0.8-15% was investigated. LIF intensities of the FiO samples decreased linearly with increasing temperature; the reductions being more significant at dye peaks. We develop a new calibration model (T-adaptive CLS) incorporating the temperature (T) effects on LIF intensity that enables simultaneous calculation of FiO and oil temperature. The improved FiO diagnostic with T-adaptive calibration is more robust, and applicable to varying oil-temperature conditions. For example, when strategies such as multiple/late fuel injections are applied to overcome cold-start instability due to use of low vaporization bio-based fuels such as ethanol, FiO rate is expected to be very high; the improved T-adaptive FiO diagnostic is hence relevant for engine- and fuel-system calibration and optimization. The diagnostic could also provide validation data for flow-field and spray interaction CFD models, further broadening the diagnostic’s utility for advancing engine technology and efficiency.

Neupane, Sneha↗

DOE OSTI2021

Tri-Level Scheduling Model Considering Residential Demand Flexibility of Aggregated HVACs and EVs Under Distribution LMP

Residential loads, especially heating, ventilation and air conditioners (HVACs) and electric vehicles (EVs), have great potentials to provide demand flexibility which is an attribute of grid-interactive efficient buildings (GEB). Under this new paradigm, first, EV and HVAC aggregator models are developed in this paper to represent the fleet of GEBs, in which the aggregated parameters are obtained based on a new approach of data generation and least squares parameter estimation (DG-LSPE), which can deal with heterogeneous HVACs. Then, a tri-level bidding and dispatching framework is established based on competitive distribution operation with distribution locational marginal price (DLMP). Furthermore, the first two levels form a bilevel model to optimize the aggregators’ payment and to represent the interdependency between load aggregators and the distribution system operator (DSO) using DLMP, and the third level is to dispatch the optimal load aggregation to all residents by the proposed priority list-based demand dispatching algorithm. Finally, case studies on a modified IEEE 33-Bus system illustrate three main technical reasons of payment reduction due to demand flexibility: load shift, DLMP step changes, and power losses. They can be used as general guidelines for better decision-making for future planning and operation of demand response programs.

24 POWER TRANSMISSION AND DISTRIBUTION↗

DOE OSTI2026

Regularization via f -Divergence: An Application to Multi-Oxide Spectroscopic Analysis

In this paper, we explore the application of convolutional neural networks (CNNs) for predicting the chemical composition of complex geologic samples in a simulated Martian atmospheric environment. Specifically, we aim to characterize oxide weight percentages (wt.%) of rock samples analyzed by remote Laser-Induced Breakdown Spectroscopy (LIBS), framing the problem as a multi-target regression task . Neural networks trained on LIBS spectra are prone to overfitting due to high spectral complexity, limited labeled data, and measurement noise. While regularization is critical for improving generalization, common methods (e.g., ℓ 2 regularization) impose constraints not directly tied to data distribution properties. We propose a novel regularization method based on a specific ƒ-divergence induced by a graph-based estimator, designed to constrain the distributional discrepancy between predictions and targets. This regularizer serves a dual purpose: (a) mitigating overfitting by enforcing a constraint on the distributional difference between predictions and noisy targets, and (b) acting as an auxiliary loss that penalizes large divergences. To enable backpropagation, we develop a differentiable approximation of this particular ƒ-divergence, making the method feasible for neural networks. Experiments on ChemCam and SuperCam LIBS calibration spectra show that mathematical equation-divergence regularization outperforms or matches standard regularization methods (ℓ 1 , ℓ 2 , dropout) and the classical baseline, partial least squares (PLS). Combining ƒ-divergence regularization with standard regularization yields further performance gains, indicating that distributional regularization is useful in this context giving a promising direction for robust model training in planetary science applications. Source code is publicly available at Klein and Li (2025), https://doi.org/10.11578/dc.20250530.7.

58 GEOSCIENCES↗

DOE OSTI2024

Determining the Crosslinking and Degradation Reaction Kinetics in Photovoltaic Encapsulants Using Accelerated Aging: Preprint

Degradation of photovoltaic (PV) module encapsulant mechanical characteristics that lead to embrittlement and delamination remains a cause of failure in solar installations. A multiscale reliability model based on detailed molecular degradation reaction kinetics was previously published, connecting the encapsulant mechanical properties (elastic modulus, yield strength, and adhesion energy) to the degraded molecular structure and interfacial bond density to adjacent solar cell and glass substrates. The model, developed primarily for poly(ethylene-co-vinyl acetate) (EVA) encapsulants, remains to be experimentally validated. Determining the degradation and crosslinking kinetics of alternative encapsulants, such as polyolefin elastomer (POE) and EVA/POE/EVA composites (EPE), can further generalize the model. Importantly, the activation energy for crosslinking of fully cured PV encapsulant products from temperature or UV is presently unknown or unavailable. In this work, we subject EVA, POE, and EPE encapsulants to a set of accelerated aging conditions, varying the temperature, UV intensity, and relative humidity. We use DSC, FTIR, and Soxhlet extraction (gel content) to characterize the encapsulants' changing molecular structure. This allows for determination of the photochemical degradation and crosslinking kinetics of the encapsulants. Preliminary results show an increase in gel content (crosslinking) and a decrease in crystallinity of EVA, POE, and EPE encapsulants under hot-aerobic (90 deg C, 22% RH) and hot-anaerobic (90oC, sealed in N2 air) accelerated aging, even in the absence of UV and crosslinking initiating agents. For a least squares regression with an assumed first-order crosslinking kinetics, the crosslinking rate parameters were computed for the three encapsulants under hot-aerobic and hot-anaerobic aging conditions. FTIR analysis showed insignificant encapsulant degradation for hot-aerobic and hot-anaerobic aging, demonstrating the critical role of UV and moisture in accelerating degradation. Aging conditions with UV exposure and elevated humidity are presently in progress.

adhesion↗

DOE OSTI2022

Testing the Cosmological Principle: Astrometric Limits on Systemic Motion of Quasars at Different Cosmological Epochs

A sample of 60,410 bona fide optical quasars with astrometric proper motions in Gaia Early Data Release 3 and spectroscopic redshifts above 0.5 in an oval 8400 square degree area of the sky is constructed. Using orthogonal Zernike functions of polar coordinates, the proper motion fields are fitted in a weighted least-squares adjustment of the entire sample and of six equal bins of sorted redshifts. The overall fit with 37 Zernike functions reveals a statistically significant pattern, which is likely to be of instrumental origin. The main feature of this pattern is a chain of peaks and dips mostly in the R.A. component with an amplitude of 25 μ as yr -1 . This field is subtracted from each of the six analogous fits for quasars grouped by redshifts covering the range 0.5 through 7.03, with median values of 0.72, 1.00, 1.25, 1.52, 1.83, 2.34. The resulting residual patterns are noisier, with formal uncertainties up to 8 μ as yr -1 in the central part of the area. We detect a single high-confidence Zernike term for the R.A. proper motion components of quasars with redshifts around 1.52 representing a general gradient of 30 μ as yr -1 over 150° on the sky. We do not find any small- or medium-scale systemic variations of the residual proper motion field as functions of redshift above the 2.5 σ significance level.

79 ASTRONOMY AND ASTROPHYSICS↗

DOE OSTI2025

A spline-based method to obtain spatially dependent viscosity in confined flows

Coupling chemical physics to continuum theories is a critical step to understanding multi-scale phenomena. This paper will connect non-equilibrium molecular dynamics simulations to a continuum-based Navier-Stokes equation that has relaxed the assumption of spatial uniformity in viscosity. Using a form for viscosity based on spline interpolation, viscosity as a function of position is obtained from the least squares fit of the velocity profile measured from molecular simulations of flow in a nanochannel. Viscosity can vary widely, particularly near the channel boundaries, indicating that uniform viscosity is no longer appropriate. Variations of the viscosity near the channel surfaces imply that considering solution and surface chemistry could be necessary to rigorously understand molecular-scale flows in nanochannels.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

DOE OSTI2021

Simulations and Experimental Verifications of an Algorithm for Radiation Source Mapping and Navigational Path Generation

Accurate and efficient mapping and localization of both ionizing and non-ionizing radiation sources are important across many different fields. As such, a versatile mapping and navigational path generation algorithm, which can be applied to any point source measurements that follow an inverse-square characteristic, was developed using non-linear least squares methods. Forty thousand simulations were performed on the algorithm, which located sources successfully in a 10 m × 10 m × 10 m three-dimensional space with a success rate of over 80% across different noise functions, given a proportional constant of 10 to 1,000. The algorithm was also verified experimentally with small-scale radioactive decontamination of a 70 cm × 70 cm surface and localization of a lost Wi-Fi router in a 70 m × 70 m open field. One hundred twenty-one measurements were taken from each experiment, which were then fed into the algorithm for navigation. For the radioactive 137Cs source, the estimated locations were within 7 cm × 7 cm of the answer in 79.3% of the scenarios, while the Wi-Fi router was located to within 7 m × 7 m in 57.9% of the tests. In general, the method requires much less information and data than a geographically comprehensive survey and thus shows a lot of potential for practical applications, such as lost source retrieval with unmanned aerial vehicles, small-scale decontamination, mapping undocumented Wi-Fi routers or radio towers, and radiation simulation with radio signals. Here, the different failure modes, desirable features, and potential improvements were also identified but remain as future work.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

DOE OSTI2022

Leveraging Multiple Raman Excitation Wavelength Systems for Process Monitoring of Nuclear Waste Streams

Processing nuclear waste from sites such as Hanford is a significant environmental cleanup need while being a significant logistical challenge. Integration of process monitoring tools, that can provide in situ and real-time feedback about the process, can significantly alleviate needs to collect grab samples for process control and product characterization. Raman spectroscopy paired with chemometric analysis is one process monitoring tool that can provide chemical composition information on a large number of chemical targets in nuclear waste streams. However, methods to improve limits of detection as well as drop uncertainty in quantification are needed. Optimizing instrument specifications can achieve this, here this is demonstrated by comparing limits of detection for key analytes when using Raman systems with 671 nm, 532 nm, and 405 nm excitation wavelengths. Generally, limits of detection decease (allowing the measurement of lower salt concentrations) with decreasing wavelength. Similarly, data collection times and averaging were optimized. Finally, multiple chemometric modeling approaches were leveraged, including multiblock methods that combined data from all three Raman systems to simultaneously quantify targets with improved sensitivity.

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗

DOE OSTI2020

Faster Johnson–Lindenstrauss transforms via Kronecker products

The Kronecker product is an important matrix operation with a wide range of applications in signal processing, graph theory, quantum computing and deep learning. In this work, we introduce a generalization of the fast Johnson–Lindenstrauss projection for embedding vectors with Kronecker product structure, the Kronecker fast Johnson–Lindenstrauss transform (KFJLT). The KFJLT reduces the embedding cost by an exponential factor of the standard fast Johnson–Lindenstrauss transform’s cost when applied to vectors with Kronecker structure, by avoiding explicitly forming the full Kronecker products. Here, we prove that this computational gain comes with only a small price in embedding power: consider a finite set of $p$ points in a tensor product of $d$ constituent Euclidean spaces $\bigotimes _{k=d}^{1}{\mathbb{R}}^{n_k}$, and let $N = \prod _{k=1}^{d}n_k$. With high probability, a random KFJLT matrix of dimension $m \times N$ embeds the set of points up to multiplicative distortion $(1\pm \varepsilon )$ provided $m \gtrsim \varepsilon ^{-2} \, \log ^{2d - 1} (p) \, \log N$. We conclude by describing a direct application of the KFJLT to the efficient solution of large-scale Kronecker-structured least squares problems for fitting the CP tensor decomposition.

Kronecker structure↗

DOE OSTI2022

Combining PCA and nonlinear fitting of peak models to re-evaluate C 1s XPS spectrum of cellulose

Cellulose is an example of a material that responds to XPS by the creation of new chemistry not present in the as-received sample. While improvements in instrumentation may be seen in general as beneficial to surface science, recent studies have shown that the consequences for some materials are detrimental. Here, in this work, these problems are illustrated through an analysis of cellulose spectra obtained during a degradation study. C 1s spectra are decomposed into two well-formed component curves that are open to chemical interpretation. In particular, a component-curve representative of pure cellulose is obtained as well as a second component curve that implies cellulose is degraded through the creation of carbon chemistry involving C—O, C$=$O and O—C$=$O. Since cellulose is a crystalline material, formed through the alignment of molecules under the influence of hydrogen bonds, the analysis and findings presented in this paper are relevant to any material analyzed by XPS whose properties are dependent on hydrogen bonds. The analysis techniques are based on an informed vectorial approach, which extracts directly from data spectral shapes that are used to monitor sample degradation via linear least squares optimization. Related mathematics of Principal Component Analysis and linear analysis are presented.

36 MATERIALS SCIENCE↗

DOE OSTI2025

ArborX 2.0

ArborX library tackles a problem of efficiently finding geometric objects that are close in space. Variations of this problem, such as finding the nearest neighbors of a point, or finding all objects within a certain distance, are inherent components of applications in many fields. The data may be large so that solving the problem efficiently may require significant computational resources, such as multiple processors or accelerators such as general purpose GPUs. ArborX' main advantage in its ability to solve large problems efficiently utilizing a combination of distributed and on-node parallelism. ArborX can be run efficiently on a wide variety of hardware, including GPUs from different vendors, which distinguishes it from other available libraries which typically choose only few of these. The other advantage is that it supports both types of user problems: spatial problems (useful for intersections and finding objects within certain distance), and nearest neighbor problems. ArborX also supports flexible interface in its interaction with a user. Particularly, it allows a user to call user's own function on a positive match, a functionality not rarely available in other libraries. ArborX implements construction and traversal algorithms using efficient tree structures, such as bounding volume hierarchy (BVH). At its core, ArborX uses linear BVH for its low construction cost and sufficient quality. ArborX implements both spatial and nearest-neighbor traversal algorithms. ArborX also provides several clustering algorithms (minimum spanning tree, DBSCAN, HDBSCAN*), interpolation using minimum least squares and ray tracing. ArborX is written using C++, and is parallelized using the message passing interface (MPI) for the distributed communication, and the Kokkos library for on-node parallelism. This approach allows ArborX to be run on a wide variety of hardware, from common laptops and desktops to supercomputers while using the same codebase.

Prokopenko, Andrey [Oak Ridge National Laboratory ↗

DOE OSTI2023

Novel Calibration Approach for Monitoring Aerosol Hydrogen Isotopes Using Laser-Induced Breakdown Spectroscopy for Molten Salt Reactor Off-Gas Streams

Online monitoring is a key challenge for the continued development of molten salt reactor (MSR) technology. Laser-induced breakdown spectroscopy (LIBS) has previously been demonstrated to be a viable tool for monitoring aerosolized species and noble gases in real time, but the ability to discern varying isotopes in these streams has not yet been investigated for MSR applications. Tritium will form in MSRs from ternary fission and from (n,α)-reactions occurring in lithium-containing salts. This study compares three spectrometers of varying resolutions and types for measuring hydrogen isotope shifts in LIBS spectra of wetted filter paper. For each spectrometer, multivariate models were built (i.e., principal component regression, partial least squares regression, and multivariate curve resolution) to quantify the isotope ratio. The top models were then modified and corrected to apply the models to aerosol samples with varying isotope ratios. This novel calibration strategy offers an 82% reduction in volume of the calibration samples needed and is a more viable pathway for calibrating deployable LIBS systems. Lastly, this calibration model was compared with an all-aerosol trained model for monitoring hydrogen isotopes during a real-time test where the protium/deuterium ratio, along with representative salt species (i.e., lithium, sodium, and potassium) were adjusted dynamically. Results of this test validated the predictive capabilities of the transferred model and highlighted the capabilities of LIBS for real-time monitoring of MSR effluent streams.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

DOE OSTI2021

Plateau Phenomenon in Gradient Descent Training of RELU Networks: Explanation, Quantification, and Avoidance

The ability of neural networks to provide ‘best in class’ approximation across a wide range of applications is well-documented. Nevertheless, the powerful expressivity of neural networks comes to naught if one is unable to effectively train (choose) the parameters defining the network. In general, neural networks are trained by gradient descent type optimization methods,a stochastic variant thereof. In practice, such methods result in the loss function decreases rapidly at the beginning of training but then, after a relatively small number of steps, significantly slow down. The loss may even appear to stagnate over the period of a large number of epochs, only to then suddenly start to decrease fast again for no apparent reason. This so-called plateau phenomenon manifests itself in many learning tasks. The present work aims to identify and quantify the root causes of plateau phenomenon.analysis is carried out in the setting of univariate ReLU networks. No assumptions are made on the number of neurons relative to the number of training data, and our results hold for both the lazy and adaptive regimes. Here, the main findings are: plateaux correspond to periods during which activation patterns remain constant, where activation pattern refers to the number of data points that activate a given neuron; quantification of convergence of the gradient flow dynamics; and, characterization stationary points in terms solutions of local least squares regression lines over subsets of the training data. Based on these conclusions, we propose a new iterative training method, the Active Neuron Least Squares (ANLS), characterised by the explicit adjustment of the activation pattern at each step, which is designed to enable a quick exit from a plateau. Illustrative numerical examples are included throughout.

97 MATHEMATICS AND COMPUTING↗