Search NASA⌕ Search

SEARCH · Search NASA

Results for “stochastic optimization”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 631 records · Page 35

CLEANing the Reward: Counterfactual Actions to Remove Exploratory Action Noise in Multiagent Learning

Learning in multiagent systems can be slow because agents must learn both how to behave in a complex environment and how to account for the actions of other agents. The inability of an agent to distinguish between the true environmental dynamics and those caused by the stochastic exploratory actions of other agents creates noise in each agent's reward signal. This learning noise can have unforeseen and often undesirable effects on the resultant system performance. We define such noise as exploratory action noise, demonstrate the critical impact it can have on the learning process in multiagent settings, and introduce a reward structure to effectively remove such noise from each agent's reward signal. In particular, we introduce Coordinated Learning without Exploratory Action Noise (CLEAN) rewards and empirically demonstrate their benefits

Reinforcement Learning↗

Enhancing Distribution System Resilience: A First-Order Meta-RL Algorithm for Critical Load Restoration

The increasing frequency of extreme events and the integration of distributed energy resources (DERs) into modern grids have elevated the need for resilient and efficient critical load restoration strategies in distribution systems. However, the stochastic nature of renewable DERs, limited energy resource availability and the intricate nonlinearities inherent in complex grid control problem make the problem challenging. Although reinforcement learning (RL) and warm-start RL methods have shown promising results, their performance often falls short in rapidly adapting to new, unseen situations and typically requires exhaustive problem-specific tuning. To address these gaps, we propose a First-Order Meta-based RL (FOM-RL) algorithm within an online framework for adaptive and robust critical load restoration. By harnessing local DERs as the enabling technology, FOM-RL allows the RL agent to swiftly adapt to new unseen scenarios by leveraging previously acquired knowledge of different tasks. Experimental results provide evidence that proposed algorithm learns more efficiently and showcases generalization capabilities across diverse set of operational scenarios. Moreover, a rigorous theoretical analysis yields a tight sublinear regret bound, sensitive to temporal variability, with a task-averaged optimality gap bounded by O(VM+D*/(Tsquare root(M))). These results suggest that optimality improves with task similarity and an increased number of tasks M, reaffirming the efficacy and scalability of the proposed approach in addressing the complexities of critical load restoration in distribution systems.

complexity theory↗

Benchmarking quantum trial wavefunctions for phaseless auxiliary-field quantum Monte Carlo

The phaseless auxiliary-field quantum Monte Carlo (ph-AFQMC) method is a stochastic imaginary-time projection technique for computing ground-state properties of strongly correlated quantum systems, with accuracy that depends critically on the choice of trial wavefunction. Here, we investigate ph-AFQMC with trial states prepared using parameterized quantum circuits. In this work, we present a comprehensive benchmarking study of quantum trial wavefunctions spanning unitary coupled-cluster, Hamiltonian-informed, Jastrow-inspired, and adaptively constructed ansatze. The benchmarking evaluates accuracy, expressibility, and scalability of these ansatze within the QC-AFQMC framework. We test these ansatze on linear hydrogen chains under bond stretching and find that several ansatz families produce chemically accurate ph-AFQMC energies across the dissociation curve. We have performed simulations using the CUDA-Q quantum development platform on the GPU partition of the Perlmutter supercomputer. When comparing ansatze at similar numbers of variational parameters, we find that different ansatz families yield comparable ph-AFQMC results despite exhibiting substantially different variational energies, optimization costs, and circuit depths. Our results indicate that the variational energy of an ansatz is not always a reliable indicator of its quality for ph-AFQMC and reveal instances of over-parameterization. In the strongly correlated regime, trial wavefunctions obtained from adaptive ansatze, exemplified here by ADAPT-VQE with the UCCSD operator pool, can outperform their fixed-ansatz counterparts (UCCSD) in terms of projected energies while using substantially more compact circuits, providing a flexible route to optimize quantum resources within the ph-AFQMC framework.

Rofougaran, Rod [LBNL, Berkeley; Columbia U.; PNL,↗

Approaches to optimization of SS/TDMA time slot assignment

Reduction techniques for traffic matrices are explored in some detail. These matrices arise in satellite switched time-division multiple access (SS/TDMA) techniques whereby switching of uplink and downlink beams is required to facilitate interconnectivity of beam zones. A traffic matrix is given to represent that traffic to be transmitted from n uplink beams to n downlink beams within a TDMA frame typically of 1 ms duration. The frame is divided into segments of time and during each segment a portion of the traffic is represented by a switching mode. This time slot assignment is characterized by a mode matrix in which there is not more than a single non-zero entry on each line (row or column) of the matrix. Investigation is confined to decomposition of an n x n traffic matrix by mode matrices with a requirement that the decomposition be 100 percent efficient or, equivalently, that the line(s) in the original traffic matrix whose sum is maximal (called critical line(s)) remain maximal as mode matrices are subtracted throughout the decomposition process. A method of decomposition of an n x n traffic matrix by mode matrices results in a number of steps that is bounded by n(2) - 2n + 2. It is shown that this upper bound exists for an n x n matrix wherein all the lines are maximal (called a quasi doubly stochastic (QDS) matrix) or for an n x n matrix that is completely arbitrary. That is, the fact that no method can exist with a lower upper bound is shown for both QDS and arbitrary matrices, in an elementary and straightforward manner.

Wade, T. O.↗

Autonomous frequency domain identification: Theory and experiment

The analysis, design, and on-orbit tuning of robust controllers require more information about the plant than simply a nominal estimate of the plant transfer function. Information is also required concerning the uncertainty in the nominal estimate, or more generally, the identification of a model set within which the true plant is known to lie. The identification methodology that was developed and experimentally demonstrated makes use of a simple but useful characterization of the model uncertainty based on the output error. This is a characterization of the additive uncertainty in the plant model, which has found considerable use in many robust control analysis and synthesis techniques. The identification process is initiated by a stochastic input u which is applied to the plant p giving rise to the output. Spectral estimation (h = P sub uy/P sub uu) is used as an estimate of p and the model order is estimated using the produce moment matrix (PMM) method. A parametric model unit direction vector p is then determined by curve fitting the spectral estimate to a rational transfer function. The additive uncertainty delta sub m = p - unit direction vector p is then estimated by the cross spectral estimate delta = P sub ue/P sub uu where e = y - unit direction vectory y is the output error, and unit direction vector y = unit direction vector pu is the computed output of the parametric model subjected to the actual input u. The experimental results demonstrate the curve fitting algorithm produces the reduced-order plant model which minimizes the additive uncertainty. The nominal transfer function estimate unit direction vector p and the estimate delta of the additive uncertainty delta sub m are subsequently available to be used for optimization of robust controller performance and stability.

Yam, Yeung↗

Effect of pattern transfer process on roughness of block copolymer patterns from directed self-assembly

Block copolymer-directed self-assembly (DSA) remains promising for improving pattern quality and reducing the stochastic variations that challenge high numerical aperture extreme ultraviolet lithography. Equally critical is refining pattern transfer methods for accurately transferring the rectified DSA features to the underlying substrate. We compare two atomic layer deposition (ALD)-based techniques: sequential infiltration synthesis (SIS) and dry liftoff, applied to polystyrene-block-poly(methyl methacrylate) (PS-b-PMMA) DSA patterns. Both methods utilize aluminum oxide hard masks, with one synthesized through infiltration into the PMMA domains and the other through conformal ALD coating. High-resolution scanning electron micrographs were analyzed to measure line edge, width, and placement roughness for both the line (PMMA) and space (PS) features. Although both methods yielded similar overall 3σ rms roughness, they differed significantly in the frequency-dependent power spectral density (PSD) profiles. SIS reduced line placement roughness at length scales associated with the polymer pitch, but increased space width roughness at low frequencies, whereas dry liftoff mimicked the frequency content of the original guiding pattern. This study underscores the importance of PSD evaluation in selecting optimal pattern transfer strategies for specific applications.

Block copolymers↗

On the formation of swelling and related flaws in laser powder bed fusion

Process monitoring in laser powder bed fusion additive manufacturing can provide insights into stochastic anomalies, melt pool and plume dynamics, and part quality. Swelling, a build anomaly where overbuilt material protrudes through the powder layer after recoating, is readily detectable in post-recoat visible light images of the powder bed. Here, this work identifies several of the underlying mechanisms driving swelling formation by analyzing the influence of processing parameters, laser scan paths, and build plate locations on the presence of swelling detected in situ. Swelling near the edge of the part and swelling in the internal region of the part are shown to correlate with different process conditions. Edge and internal swelling may be driven by different phenomena, with edge swelling predominately occurring on the edge of a part facing the laser module and correlated to clusters of near-surface voids (detected with X-ray computed tomography). A larger spot size, higher laser power, and lower scan velocity also increased the presence of edge swelling. Laser spot size and scan path influenced internal swelling, which occurred preferentially with a larger spot size and in regions with large melt pools, caused by localized heat accumulation due to non-optimal processing parameters or scan path strategies. For coupons processed with a slicer-defined maximum scan vector length, swelling seldom occurred at internal vector-stripe boundaries. These results provide a mechanistic understanding of how swelling can be linked to material flaws, insight into how some instances of swelling can be avoided, and evidence supporting the use of swelling as an in situ indicator for quality assurance and part qualification.

Anomaly↗

Gas-rich dwarf galaxies in dense and sparse environments

Dwarf irregular galaxies (generically labelled Im for the present purposes) pose an enigma to students of galaxy evolution. In nearby groups and the Virgo cluster, Im galaxies are at least as abundant as spiral galaxies, and their low surface brightnesses and high gas-to-stars ratios suggest that (at least in the stochastic self-propagating star formation scenario) there should be significant numbers of HI clouds with masses approaching 10(exp 8) solar mass which have undergone very little or no star formation. To date, however, no clouds with so little star formation that they would not be recognized as Im galaxies on high-quality photographic plates have been identified. There have been suggestions that such dwarfs may be tidally disrupted in regions of high galactic density, but may be prevalent in low density regions. We offer data from three parallel programs relevant to this issue. (1) A large number of Im galaxies throughout the Local Supercluster have been mapped in the HI spectral line using the Arecibo Radiotelescope, and we can establish the frequency with which HI disks much more extended than their optically visible portions are found. (2) Our extensive mapping of spiral and dwarf galaxies in the Virgo cluster allows us to set stringent limits on the density of star-free Hi clouds in that cluster. (3) We have conducted a sampling of the void in the distribution of galaxies toward the super galactic pole, optimized for finding low-mass HI clouds at redshifts out to approximately 2000 km/s.

Hoffman, G. Lyle↗

Image Mapping and Visual Attention on the Sensory Ego-Sphere

The Sensory Ego-Sphere (SES) is a short-term memory for a robot in the form of an egocentric, tessellated, spherical, sensory-motor map of the robot s locale. Visual attention enables fast alignment of overlapping images without warping or position optimization, since an attentional point (AP) on the composite typically corresponds to one on each of the collocated regions in the images. Such alignment speeds analysis of the multiple images of the area. Compositing and attention were performed two ways and compared: (1) APs were computed directly on the composite and not on the full-resolution images until the time of retrieval; and (2) the attentional operator was applied to all incoming imagery. It was found that although the second method was slower, it produced consistent and, thereby, more useful APs. The SES is an integral part of a control system that will enable a robot to learn new behaviors based on its previous experiences, and that will enable it to recombine its known behaviors in such a way as to solve related, but novel, task problems with apparent creativity. The approach is to combine sensory-motor data association and dimensionality reduction to learn navigation and manipulation tasks as sequences of basic behaviors that can be implemented with a small set of closed-loop controllers. Over time, the aggregate of behaviors and their transition probabilities form a stochastic network. Then given a task, the robot finds a path in the network that leads from its current state to the goal. The SES provides a short-term memory for the cognitive functions of the robot, association of sensory and motor data via spatio-temporal coincidence, direction of the attention of the robot, navigation through spatial localization with respect to known or discovered landmarks, and structured data sharing between the robot and human team members, the individuals in multi-robot teams, or with a C3 center.

Fleming, Katherine Achim↗

Recommended DSMC Collision Model Parameters for Planetary Entry

Hybrid simulation frameworks combining Computational Fluid Dynamics (CFD) and Direct Simulation Monte Carlo (DSMC) are frequently employed to efficiently perform high-fidelity simulations of environments consisting of both continuum and rarified flow. DSMC is a stochastic, particle-based method which solves the fundamental Boltzmann equation and is therefore necessary for high-Knudsen flow where continuum-based assumptions governing CFD break down. However, the DSMC methodology is generally computationally inefficient to model the continuum regime. In a CFD/DSMC hybrid approach, obtaining an accurate, high-fidelity solution hinges on the consistent treatment of transport properties and the thermo-chemical models employed within the two solvers. In principle, in regions where CFD and DSMC are both employed, the same gas mixture under the same conditions should have the same properties, regardless of simulation type. Observed differences should be due to non-equilibrium processes, rather than differences in physical models. The goal of this work is to provide a comprehensive DSMC transport database encompassing the vast majority of known gas species encountered during Earth or other planetary atmospheric entry. This goal is largely possible due to recently performed ab initio quantum chemistry calculations. Combined with other high-fidelity data, the planned database will consist of collision integral data for over 200 neutral and ionized species and over 20000 binary collisions. From these collision integrals, Nelder-Mead optimization is used to compute collision-specific Variable Soft Sphere (VSS) collision model parameters, fitted from 300 K to 20000 K. Initial comparisons of transport properties of relevant equilibrium gas mixtures show great agreement between CFD and DSMC-derived results. The completed database can be readily applied to model binary collisions of any gas mixture containing the included species over the specified temperature range, making it a valuable tool for future planetary probe modeling efforts. An example is shown below in Fig. 1. Equilibrium mixture transport properties for a 35-species mixture composed originally of 10% air and 90% pyrolysis species of a carbon-phenolic ablator material [4] are computed using both fitted VSS Parameters and the original CFD collision integral values. Deviations in computed properties between the two approaches are less than 5% for the entire temperature range.

M. R. Gosma↗

A Holistic DSMC Transport Database for Re-Entry and Ablation Modeling

Hybrid simulation frameworks combining Computational Fluid Dynamics (CFD) and Direct Simulation Monte Carlo (DSMC) are frequently employed to efficiently perform high-fidelity solutions of environments containing combined continuum/rarified flow. The use of DSMC, a stochastic, particle-based method, is necessary for high-Knudsen flow where continuum-based assumptions governing CFD break down. However, the DSMC methodology is generally very computationally inefficient to model the continuum regime. In a CFD/DSMC hybrid approach, obtaining an accurate, high-fidelity solution hinges on the consistent treatment of transport properties and the used thermo-chemical models employed within the two solvers. In principle, in regions where CFD and DSMC are both employed, the same gas mixture under the same conditions should have the same properties, regardless of simulation type. Observed differences should be due to non-equilibrium processes, rather than differences in physical models. While the transport models governing CFD and DSMC simulations are starkly different, they can effectively be linked via their use of reduced Chapman-Enskog collision integrals. In CFD, these integrals are typically stored as fitted polynomial expressions and used to directly compute gas transport properties via mixing rules or the full Chapman-Enskog formulation. In DSMC, they can be used to derive the collision parameters needed for the phenomenological collision cross-section models that govern particle interactions, via a Nelder-Mead optimization scheme. The goal of this work is to provide a unified DSMC transport database encompassing the vast majority of known gas species encountered during atmospheric entry, on Earth or any other Solar body. This goal is largely possible due to recently performed ab-initio quantum chemistry calculations. Combined with other high-fidelity literature sources, the planned database will consist of collision integral data for over 200 neutral and ionized species and over 17000 binary collisions. From these collision integrals, Nelder-Mead optimization is used to compute Variable Soft Sphere (VSS) collision model parameters for DSMC, fitted from 300 K to 20000 K. Initial comparisons of transport properties of relevant equilibrium gas mixtures show great agreement between CFD and DSMC-derived results. The completed database will be able to be readily applied to model binary collisions of any gas mixture containing the included species over the specified temperature range, making it a valuable tool for future planetary probe modeling efforts. An example is shown below. Equilibrium mixture transport properties for a 19-species Titan atmospheric model [4] are computed using both fitted VSS parameters and the original CFD collision integral values. Deviations in computed properties between the two approaches is less than 5% for the entire temperature range.

M R Gosma↗

Raptor

Raptor is an efficient Python-based tool for predicting the formation and morphology of stochastic lack of fusion defects in metal AM processes. A major obstacle for the qualification and certification of additively manufactured parts in critical applications continues to be performance variability caused in part by porosity-related defects. High-fidelity process models that could predict these defect features are currently too computationally expensive for component-level analysis. To address this, Raptor employs a high-performance geometric method to model the dynamic melt pool rather than relying on computationally intensive thermal fluid dynamics. This allows Raptor to rapidly identify regions of unmelted material that correspond to lack of fusion pores. The efficiency of this approach significantly reduces the time and resources needed for generating 3D defect predictions, which enables users to conduct large-scale parameter studies and evaluate how process variations affect part quality. The framework offers operational flexibility; users can execute simulations through a simple command line interface or integrate core functions as a library within larger computational workflows. Simulation outputs include 3D porosity maps for visualization and tools for quantitative morphological analysis. These results are suitable for direct comparison with experimental characterization data from methods such as X-ray computed tomography and can be used for statistical process optimization.

Subraveti, Vamsi [Vanderbilt Univ., Nashville, TN ↗

Two-Stage Distributionally Robust Conic Linear Programming over 1-Wasserstein Balls

Here, this paper studies two-stage distributionally robust conic linear programming under constraint uncertainty over type-1 Wasserstein balls. We present optimality conditions for the dual of the worst-case expectation problem, which characterizes worst-case uncertain parameters for its inner maximization problem. This condition offers an alternative proof, a counterexample, and an extension to previous works. Additionally, the condition highlights the potential advantage of a specific distance metric for out-of-sample performance, as exemplified in a numerical study on a facility location problem with demand uncertainty. Furthermore, cutting-plane-based algorithms, equipped with a unified scenario generation framework, are proposed for addressing both unbounded support and second-stage dual feasible regions, with a finite convergence proof under less stringent assumptions.

Wasserstein↗

Understanding the interplay between pilot fuel mixing and auto-ignition chemistry in hydrogen-enriched environment

The diesel-piloted dual-fuel compression ignition combustion strategy is well-suited to accelerate the decarbonization of transportation by adopting hydrogen as a renewable energy carrier into the existing internal combustion engine with minimal engine modifications. Despite the simplicity of engine modification, many questions remain unanswered regarding the optimal pilot injection strategy for reliable ignition with minimum pilot fuel consumption. The present study uses a single-cylinder heavy-duty optical engine to explore the phenomenology and underlying mechanisms governing the pilot fuel ignition and the subsequent combustion of a premixed hydrogen-air charge. The engine is operated in a dual-fuel mode with hydrogen premixed into the engine intake charge with a direct pilot injection of n-heptane as a diesel pilot fuel surrogate. Optical diagnostics used to visualize in-cylinder combustion phenomena include high-speed IR imaging of the pilot fuel spray evolution as well as high-speed HCHO* and OH* chemiluminescence as indicators of low-temperature and high-temperature heat release, respectively. Three pilot injection strategies are compared to explore the effects of pilot fuel mass, injection pressure, and injection duration on the probability and repeatability of successful ignition. The thermodynamic and imaging data analysis supported by zero-dimensional chemical kinetics simulations revealed a complex interplay between the physical and chemical processes governing the pilot fuel ignition process in a hydrogen containing charge. Hydrogen strongly inhibits the ignition of pilot fuel mixtures and therefore requires longer injection duration to create zones with sufficiently high pilot fuel concentration for successful ignition. Results show that ignition typically tends to rely on stochastic pockets with high pilot fuel concentration, which results in poor repeatability of combustion and frequent misfiring. In conclusion, this work has improved the understanding on how the unique chemical properties of hydrogen pose a challenge for maximization of hydrogen’s energy share in hydrogen dual-fuel engines and highlights a potential mitigation pathway.

33 ADVANCED PROPULSION SYSTEMS↗

Application of Sparse Identification of Nonlinear Dynamics for Physics-Informed Learning

Advances in machine learning and deep neural networks has enabled complex engineering tasks like image recognition, anomaly detection, regression, and multi-objective optimization, to name but a few. The complexity of the algorithm architecture, e.g., the number of hidden layers in a deep neural network, typically grows with the complexity of the problems they are required to solve, leaving little room for interpreting (or explaining) the path that results in a specific solution. This drawback is particularly relevant for autonomous aerospace and aviation systems, where certifications require a complete understanding of the algorithm behavior in all possible scenarios. Including physics knowledge in such data-driven tools may improve the interpretability of the algorithms, thus enhancing model validation against events with low probability but relevant for system certification. Such events include, for example, spacecraft or aircraft sub-system failures, for which data may not be available in the training phase. This paper investigates a recent physics-informed learning algorithm for identification of system dynamics, and shows how the governing equations of a system can be extracted from data using sparse regression. The learned relationships can be utilized as a surrogate model which, unlike typical data-driven surrogate models, relies on the learned underlying dynamics of the system rather than large number of fitting parameters. The work shows that the algorithm can reconstruct the differential equations underlying the observed dynamics using a single trajectory when no uncertainty is involved. However, the training set size must increase when dealing with stochastic systems, e.g., nonlinear dynamics with random initial conditions.

Corbetta, Matteo↗

Quantum-Assisted Variational Segmentation for Image-to-Image Wildfire Detection Using Satellite Data

The quantum computing community has been searching for suitable applications to demonstrate the potential of near-term quantum devices. Quantum machine learning is a potential candidate, particularly using models that cannot be efficiently simulated with classical computers [1, 2]. This work focuses on a transition phase of quantum computers where the quantum machine learning model is still simulable classically but projected not to be simulable as the size of the model grows. Ultimately quantum computers may have advantages for high-dimensional real-world problems. Due to the limited number of qubits in current noisy intermediate-scale quantum (NISQ) devices, the direct application of quantum computers in high dimensional data is not feasible. To remedy this problem, an encoder-decoder architecture can be utilized. The encoder model would transform the high-dimensional data into a compact representation, to a level that small quantum computers can be used today (or in the near future), and the decoder would take the quantum processed outputs back to the high-dimensional space. Addressing the two challenges of quantum machine learning, this work investigates a hybrid supervised generative model with a quantum Ising Born machine embedded as the latent distribution. The model contains four main parts (Figure 1.a.): (1) a U-NET architecture responsible for learning segmentation flow, (2) a Prior network responsible for learning an encoded latent distribution of the input data, (3) a Born machine which represents the latent distribution, and (4) a Posterior network in charge of learning the joint encoded latent distribution of inputs and target data. The initial model, proposed by [3], is optimized by (1) maximizing the overlap of the prior and posterior latent distributions, and (2) minimizing the segmentation loss. The proposed model is designed to be investigated in a simulation environment applied to the real-world application of wildfire segmentation. Specifically, the model is designed to solve the patchy wildfire segmentations of Moderate Resolution Imaging Spectroradiometer (MODIS) by taking the MODIS observations and using Visible Infrared Imaging Radiometer Suite’s (VIIRS) consistent wildfire product as the target. The model solves patchy wildfire segmentations and provides insight into the epistemic errors sourced from model variation. The model utilizes the Born machine as a QUBO solver to represent the latent space as a Bernoulli distribution. The proposed configuration allows the variational segmentation model to leverage the true quantum probabilistic nature and derive a more expressive latent configuration, increasing the model performance in describing wildfire segmentations. The quantum probabilistic information of the Born machine is directly incorporated in the Kullback-Leibler divergence loss in the prior and posterior distributions, forcing the Bernoulli latent distribution to maximize the overlap of input and joint input-target distributions. The proposed model is then trained and compared with a baseline only consisting of direct Bernoulli latent distribution with no Born machine representing the latent space. The models are evaluated based on the segmentation metrics, such as precision, recall, intersect of union, with uncertainty boundaries accounting for the stochastic nature of the model. Our findings show that even in low latent-dimensional space (due to the limit in computational power of the classical quantum simulator), we are able to effectively capture the latent representation and hence the model performs better than the baseline. The findings are a projection for scaling the model into higher dimensional latent space with the Born machine surpassing the baseline performance. Figure 1. Sub-figure (a) demonstrates the architecture for the training phase. The model consists of a Prior and Posterior network that encode inputs and joint input-target data into compact representations, respectively. The Born machine represents the latent distribution, and the U-NET branch learns the segmentation patterns of the data. The stochasticity is introduced to the U-NET through its last layer to create meaningful but stochastic segmentations. Sub-figure (b) represents the inference phase where the model takes the stochastic behavior from the prior network and injects that into the U-NET. Each attempt of inference will generate different but similar segmentations from the same distribution of the wildfire event. REFERENCES [1] Coyle, B., Mills, D., Danos, V., & Kashefi, E. (2020). The Born supremacy: quantum advantage and training of an Ising Born machine. npj Quantum Information, 6(1), 1-11. [2] Liu, J. G., & Wang, L. (2018). Differentiable learning of quantum circuit born machines. Physical Review A, 98(6), 062324. [3] Kohl, S., Romera-Paredes, B., Meyer, C., De Fauw, J., Ledsam, J. R., Maier-Hein, K., ... & Ronneberger, O. (2018). A probabilistic u-net for segmentation of ambiguous images. Advances in neural information processing systems, 31.

quantum machine learning↗

Practical Insights on Applying Simulation-Based Control Methods in Experimental Studies

Advanced nuclear reactors are crucial to the future of energy both in the United States and around the globe. In contrast to the current operating fleet, they are characterized as being deployable in remote locations and able to operate in semi-autonomous or autonomous fashion. This leap forward necessitates a new reactor control paradigm. Because advanced nuclear reactors are still under development in the United States, the creation of new control methods to achieve autonomous operations has been based on systems modeling and simulation. However, an important factor in successfully deploying these new control methods is the ability to seamlessly transition from simulation environments to real-world settings. Control methods tested in both simulation and experimental settings need to be investigated in the context of advanced reactor applications. This work developed a series of simple controllers for Idaho National Laboratory (INL)’s Microreactor Applications Research Validation and Evaluation (MARVEL) microreactor operating in load-following scenarios. These controllers were tested in both simulation and experimental settings, and a comparative performance analysis was performed. The simulation tests leveraged the Control and Optimization Modular Modeling Application for Nuclear Deployment (COMMAND) software developed in a previous stage of the current effort, along with the MARVEL Reactor Excursion and Leak Analysis Program (RELAP5-3D) and Monte Carlo N-Particle (MCNP) models. The experimental tests leveraged the COMMAND software, MARVEL models, and the U.S. Department of Energy Microreactor Program’s Microreactor Automated Control System (MACS). MACS was developed to serve as a control method testbed. It was customized to mirror the MARVEL microreactor, and COMMAND enabled MACS to emulate the physics of MARVEL. The load-following controller was developed using the simulation platform, with efforts to emulate real systems by introducing actuator saturation and noise. These factors were incrementally accounted for in the controller design. After finalizing the controller design, it was implemented with the experimental setup. The experimental conditions tested included an initial test under conditions similar to the final simulation test, and two additional scenarios. The first scenario introduced additional actuator saturation to account for equipment aging over time, which was unknown to the controller. The second scenario introduced sensor delay, a phenomenon anticipated with the use of remote operations or wireless communication in advanced reactors. These tests revealed several notable differences. While the controller performed well in simulation, it exhibited several limitations when transitioning to hardware. The main challenges involved maintaining the steady-state target power, as evidenced by larger error values between the true reactor power and setpoint power, as well as persistent oscillations in controlled reactor power. These issues could lead to unacceptable transient conditions in real reactor testing. Introducing actuator aging and stochastic delays in the experimental setup significantly impacted controller performance, resulting in increased overshoot and undershoot, and exacerbated error and oscillations previously mentioned. These findings underscore the importance of experimental testbeds for testing and validating control methods, as controllers developed using only theory and/or simulation may perform unexpectedly when applied to actual hardware. This research emphasizes the need for an experimental testbed for achieving such validation.

46 - INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AN↗

PRIME: A bottom-up approach to probabilistic rule development

PRIME is a system to be used by an intelligent machine to allow it to operate in an abstract but uncertain (or stochastic) environment. It maintains a model of the effects of the machine's actions in the form of a rule base, which is induced from experience. This bottom-up approach to rule development allows the model to adapt to changes in the environment. Each rule consists of a condition under which the rule is active, an action, the effect of the action on the environment, and an estimate of the probability of this effect occurring. The effect probabilities are used to model the uncertainty in the environment, permitting multiple possible effects for a single action under a particular set of conditions. The objective of the intelligent machine is to satisfy user-specified goals with maximum probability of success. PRIME fulfills this requirement in two ways: it continuously updates the rule base with the most recent information, to ensure the validity of the model; and it generates plans which have the maximum probability of achieving the goals, based on the probability estimates in the rule base. PRIME is composed of three main processes: exploration, generalization, and planning. In exploration, the machine executes various randomly chosen actions, observes the effects on the environment, and updates the rule base accordingly. This process is used to develop the rule base in simulation, as well as to supplement the current knowledge during normal operation. Generalization is the procedure used to induce general rules from experience, which is encoded in the form of specific rules. These general rules extend the machine's knowledge to situations which have not been encountered yet, thereby increasing the capability of the machine to plan effectively. Planning is the process of constructing an optimal sequence of actions to satisfy a goal, using the rule base to predict the effects of these actions and to determine the probability of success of the plan. The rule representation and many other data structures were specifically chosen to maximize the efficiency of these processes. A simulated environment was designed to test the performance of PRIME. The results of experimentation were largely negative. The main problem was that the domain coverage of the rules was inadequate for the number of rules stored in the rule base, due to redundancies in general rules and numerous rules covering ineffective actions. It was determined that a more efficient generalization, and some form of goal-directed exploration, are necessary in order to solve most of the current deficiencies in PRIME.

Miller, Scott A.↗