Generalization of ML-Based Surrogate Models to Dynamic Injection Schedules using Transfer Learning
For presentation at the FECM/ NETL Carbon Management Project Review Meeting, Pittsburgh, PA, August 28-September 1, 2023.
SEARCH · Search NASA
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
For presentation at the FECM/ NETL Carbon Management Project Review Meeting, Pittsburgh, PA, August 28-September 1, 2023.
Over the past several decades, regulation of compression ignition engine emissions has become increasingly stringent as concern about the environmental and health implications of these emissions has grown. These changing constraints have led to a series of new, alternative fuel injection strategies that aim to maintain power output while reducing in-cylinder generated emissions by operating in the low-temperature combustion (LTC) regime. These advanced injection strategies are created and retuned for individual combinations of engine geometry, fuel, and emissions constraints. Deep reinforcement learning has been shown to be an effective alternative to traditional optimization approaches for highly combinatorial control problems, such as discovering the optimal injection schedules for compression ignition engines. In this study, we deploy a previously presented deep reinforcement learning framework to iteratively optimize a series of engine geometries over a range of increasingly strict NO x emissions regulations. We then examine the resulting injection schedules. We discuss the potential for using this deep reinforcement learning framework for fuel selection screening and for discovering unique injection strategies for different engine geometries and future emissions standards.
The direct injection spark ignition (DISI) engine has received considerable attention due to its potential to increase the power density of traditional spark ignition engines while significantly improving fuel economy through lean, unthrottled combustion. However, the market introduction of DISI engines operated in a lean combustion mode is inhibited by their unsatisfactory emissions, especially during cold start conditions that make proper mixture formation more challenging. Ethanol-blended gasoline, now a widely used fuel, makes the cold start of a DISI engine more difficult, leading to higher HC and soot emissions because of the high latent heat of vaporization of ethanol relative to gasoline. This work investigated the impact of coolant temperature on the characteristics of combustion and emissions in a stratified-charge DISI engine fueled with an E30 fuel (i.e. 30% ethanol in gasoline), while the coolant temperature was alternated between four levels (45, 60, 75, and 90 °C) to simulate different conditions throughout the warm-up process. The experiments showed that the coolant temperature affected the post-spark inflammation time, as well as the speed, intensity, and stability of the combustion process in the engine. When the coolant temperature rose, the engine produced more NOX and less CO, PM and HC. In addition, high-speed direct photography was used to obtain crank-angle resolved images of fuel sprays and flames in the cylinder. As the coolant temperature rose, the liquid spray lengths became shorter, reducing the possibility of wall wetting, and reduced irradiance from soot particles also indicated less nonpremixed combustion. The in-cylinder imaging results are consistent with the observed combustion and emission characteristics and shed light on the underlying processes. Finally, some potential solutions to the emissions challenges faced here could be either raising in-cylinder temperatures by using trapped residuals or modifying the injection schedule, for example by increasing the number of injections or to inject later in the cycle into a higher-density environment.
In geological CO 2 storage, designing the optimal well control strategy for CO 2 injection to maximize CO 2 storage while minimizing the associated geomechanical risks is not trivial. This challenge arises due to pressure buildup, CO 2 plume migration, the highly nonlinear nature of geomechanical responses to rock-fluid interaction, and the high computational cost associated with coupled flow and geomechanics simulations. In this paper, we introduce a novel optimization framework to address these challenges. The optimization problem is formulated as follows: maximize total CO 2 storage while minimizing geomechanical risks by adjusting the injection schedules within bounded constraints. The geomechanical risks are primarily driven by injection-induced pressure build-up, which is characterized by ground displacement and the induced microseismicity. We used the Fourier neural operator (FNO)-based deep learning model to construct surrogate models, replacing the time-consuming coupled flow and geomechanics simulations for evaluating the aforementioned objective functions. The developed surrogate models have been incorporated into a multiobjective optimization framework through a genetic algorithm to reduce the computational burden. The proposed optimization framework reduces the computational cost from approximately 2,400 hours, when using objective function evaluations based on physics-based simulations, to around 20 minutes. A set of Pareto-optimal solutions of the proposed workflow yields nontrivial optimal decisions, reducing the microseismicity potential and the vertical displacement. This Pareto front highlights the optimal trade-offs between CO 2 storage amount, safety, and ground displacement, emphasizing the need for careful optimization and management of injection strategies to achieve a balanced outcome. The novelty of this work is twofold. First, we demonstrate the importance of incorporating the minimization of the geomechanical risks as objective functions into the CO 2 storage optimization workflow to mitigate the potential risk of induced microseismicity and ground displacement. Second, we leverage the FNO-based surrogate models to optimize a real-field CO 2 storage operation.
The Raft River Enhanced Geothermal System (EGS) demonstration project aims to improve the geothermal production by enhancing the reservoir permeability via shear stimulation in well RRG-9. In this paper, we performed a series of 3D thermo– hydro–mechanical (THM) simulations to investigate the influence of hydraulic and thermal effects on the development of this EGS. The model includes synthetic fracture populations based on borehole televiewer images and in-situ stress measurements from well RRG-9. Fracture permeability evolution is determined using an empirical permeability law developed from laboratory experiments. The model was calibrated by comparing the hydraulic response to field observations, including wellhead pressure, injection rate, and well injectivity. Particularly, we analyzed the enhancement of reservoir permeability and the spatial extent of the stimulation zone for the given injection schedule. Our results indicate that the permeability enhancement of fractured geothermal reservoir is caused by the combined effects of injection-induced cooling and fluid pressure increase. The decrease of temperature plays a dominant role in reactivation of natural fractures under the hydro-shearing mechanism, while the higher injection pressure promotes shear failure and enlarges the stimulation zone. Finally, for the specified extensional stress state, the model favors greatest permeability enhancement along the maximum horizontal principal stress, a moderate vertical enhancement, and a smaller gain along the minimum horizontal principal stress.
CO2 injection for geologic carbon sequestration involves hydromechanical processes that lead to changes in fluid pressure and stresses that can activate existing faults. This paper presents a new method and workflow of modeling fault activation considering more complex three-dimensional geometry of natural faults using the TOUGH-FLAC multiphase fluid flow and geomechanical simulator. In this method and workflow, FLAC3D mechanical interfaces and TOUGH3 finite volume elements are discretized using computer aided design and gridding software along with a tailored mesh translation routine. The method and workflow are demonstrated with a model of a curved minor fault embedded in a poro-elastic layered reservoir-caprock system. The model is used for a comprehensive sensitivity analysis of fault responses to fault length, injection mass rate, injection schedule, well-fault distance, and well locations versus fault location. Four metrics (CO2 plume, shear state of fault, pressure and stress path at fault monitoring points) are selected to assess CO2 migration, pressure change, and the reactivation of faults. The results reveal that CO2 can bypass around the tip of the minor impermeable fault, building up pressure and poro-elastic stress on both sides that tends to impede fault rupture. Our study shows the benefit of carefully designing the injection to achieve the targeted final storage volume, starting at a relatively low rate for considerable time, and then ramping up the injection rate to the full rate of injection. The initial low injection has two distinct benefits: (1) it allows for the formation of an extensive CO2 plume with a much higher mobility through a low viscosity that will result in a lower pressure for a given injection rate, and (2) it allows for gradual build-up of horizontal poro-elastic stress within the reservoir that will tend to impede activation of steeply dipping faults. The injection scenario starting at a low injection rate, denoted here as conservative injection, can significantly reduce the risk of fault activation as high fluid mobility and reservoir strengthening poro-elastic stress has been established long before reaching the peak injection rates. Moreover, simultaneous injection in two injection wells on both sides of fault can provide further reservoir strengthening through poro-elastic stress buildup acting on a fault under normal faulting stress regime. The findings presented in the paper can provide practical and effective guidance on long-term, safe, and reliable geological CO2 storage.
In recent years, systematic CO2 injection into geological reservoirs across the U.S. has gained traction as a strategy to mitigate greenhouse gas emissions. This approach necessitates precise monitoring to ensure secure containment, minimize risks, and optimize storage management. Our study leverages machine learning (ML) techniques to advance the understanding of CO2 injection processes, focusing on the Illinois Basin. Over a three-year injection period, we analyzed microseismic data, identifying 19 temporal intervals with significant bottom-hole pressure changes. By partitioning microseismic events into these intervals and estimating b-values, we revealed over 100 clusters of events related to fracture initiation or reactivation. Advanced spatial analysis highlighted horizontally-oriented fractures along the NNW-SSE axis. This quantification of fracture networks informs dynamic injection scheduling, work-over strategies, and risk assessments, enhancing carbon capture, utilization, and storage (CCUS) operations. Additionally, our methodology offers valuable insights for oil and gas operations and geothermal development, supporting fracture-based monitoring and risk mitigation.
This paper presents a deep learning-based approach for optimizing CO2 injection in carbon capture and storage (CCS) operations. We developed a multi-resolution machine learning model to significantly reduce data generation costs. Utilizing this proxy model, we implemented a multi-objective genetic algorithm to optimize well control during the CO2 injection process. The proposed approach was applied to the Illinois Basin Decatur Project (IBDP), successfully optimizing the CO2 injection schedule based on three key objectives: maximizing the amount of CO2 stored, maximizing sweep efficiency, and minimizing pressure increase. The use of the proxy model accelerated the optimization workflow by two orders of magnitude, while the cost of data generation for the proxy model was reduced by 90% by utilizing a coarse-scale model.
This Storage Field Development plan presents the Storage Complex characterization results, construction, monitoring, and operational plans, and costs associated with the proposed One Earth Sequestration Carbon Capture and Storage (OES-CCS) site in McLean County, Illinois, near Gibson City. The proposed storage complex, known as the Mt. Simon Storage Complex, comprises the Cambrian Mt. Simon Sandstone reservoir and the primary seal, the Cambrian Eau Claire Formation. The lowermost Underground Source of Drinking Water (USDW) identified for the site is the Ordovician St. Peter Sandstone. Geologic characterization of the Mt. Simon Storage Complex at the OES-CCS site was performed by the Illinois Storage Corridor CarbonSAFE Phase III project, which also prepared and submitted three UIC Class VI applications to construct three injection wells; the permit applications were submitted and are in the federal EPA review process. A characterization well, OEE #1, was drilled to collect site-specific data. These data were analyzed and used to develop the UIC Class VI applications. The OEE #1 well will be converted to an in-zone monitoring (IZM) well for the injection phase. The proposed buildout for the OES-CCS site includes (1) three injection wells (OES #1, OES #2, and OES #3), (2) two IZM wells, (3) two above confining zone (ACZ) monitoring wells, one of which will be used to monitor the lowermost USDW, (4) capture and compression facilities, and (5) transportation facilities, i.e., pipelines. A pre-operational testing program was proposed in the Class VI permit application and will be employed at the site pending approval. Additional pre-injection (baseline), syn-injection, and post-injection monitoring and site care procedures will be followed by OES to ensure that injection activities are protective of human health and the environment. Injection is scheduled to begin in 2025, distributed across the three injection wells in accordance with the permit operating conditions. One Earth Sequestration intends to inject up to 90 million tonnes of CO 2 over a period of approximately 20 years. Injection will begin at approximately 0.5 million tonnes of CO 2 annually and ramp up to a maximum of 4.5 million tonnes annually. Daily injection rates are expected to range from 1,400 to 1,500 tonnes per day initially and reach a maximum of approximately 4,225 tonnes per day, depending on site geology and injectivity at each injection well location, and CO 2 availability. The costs associated with the OES-CCS project include pre-operational costs (e. g. additional seismic data acquisition and well drilling), capture and transportation facility and equipment costs, predicted field operating expenditures (OpEx), and decommissioning and post-injection site care (PISC) costs. The risks associated with project activities, such as site construction, injection operations, and verification of secure storage were evaluated, and mitigation strategies proposed to alleviate those risks.
Conventional compression-ignition (CI) engines have long offered high thermal efficiencies and torque across a wide range of loads, but often require extensive exhaust gas treatment that decreases efficiency to meet ever-increasing emissions regulations. One strategy to decrease emissions is to split the fuel injection into a series of smaller injections. In this paper, we explore a new way of discovering optimal control strategies for the next generation of CI engines using deep reinforcement learning (DRL). We outline a DRL procedure to maximize the weighted reward of engine work while minimizing end-of-cycle NO x emissions. Through the procedure outlined in this paper, we show that the DRL agent is able to reduce NO x emissions threefold while only decreasing network by 2%. We demonstrate the use of transfer learning (TL) across hierarchies of physical models to accelerate the learning process, making this approach feasible for a range of control problems within this space. This paper presents a framework and demonstration for using DRL to design control systems in technology areas such as multi-pulse engine control where a hierarchy of models combined with multi-objective rewards are used for optimal operation.
As carbon storage technologies advance globally, methods to understand and mitigate induced earthquakes become increasingly important. Although the physical processes that relate increased subsurface pore pressure changes to induced earthquakes have long been known, reliable methods to forecast and control induced seismic sequences remain elusive. Suggested reservoir engineering scenarios for mitigating induced earthquakes typically involve modulation of the injection rate. Some operators have implemented periodic shutdowns (i.e., effective cycling of injection rates) to allow reservoir pressures to equilibrate (e.g., Paradox Valley) or shut-in wells after the occurrence of an event of concern (e.g., Basel, Switzerland). Other proposed scenarios include altering injection rates, actively managing pressures through coproduction of fluids, and preinjection brine extraction. Here, in this work, we use 3D physics-based earthquake simulations to understand the effects of different injection scenarios on induced earthquake rates, maximum event magnitudes, and postinjection seismicity. For comparability, the modeled injection considers the same cumulative volume over the project’s operational life but varies the schedule and rates of fluid injected. Simulation results show that cyclic injection leads to more frequent and larger events than constant injection. Furthermore, with intermittent injection scenario, a significant number of events are shown to occur during pauses in injection, and the seismicity rate remains elevated for longer into the postinjection phase compared to the constant injection scenario.
Water-based hydraulic fracturing fluids (HFFs) can chemically interact with formation shale, resulting in altered porosity and permeability of the host rock. Experimental investigations of spatial and temporal shale-HFF interactions are helpful in interpreting chemical compositions of the injectate, as well as predicting alteration of hydraulic properties in the reservoir due to mineral dissolution and precipitation. Most bench-top experiments designed to study shale-HFF chemical interactions, either using batch reactors or flow-through setups, are carried out assuming that the acid spearhead has already become mixed with neutral HFFs. During operations, however, HFFs are typically injected according to a sequenced pumping schedule, starting with a concentrated acid spearhead, followed by multiple additions of near-neutral pH HFFs containing chemical amendments and proppant. In this study, we use geochemical modeling to consider whether this pre-mixed experimental protocol provides results directly comparable to a sequential discrete fluid-shale interaction protocol. Our results show that for the batch system, the transient evolution in major ion concentrations is faster with the sequential procedure. After 2 h of reaction time, the two protocols converge to the same aqueous concentrations. In a flow-through geometry, the pre-mixed model predicts extensive chemical alteration close to the injection point but negligible alteration downstream. In contrast, the sequential model predicts mineral reactions over hundreds of meters along the flow path. The extent of shale alteration in the sequential model at a given location depends on shale mineralogy and where the acid spearhead resides during the shut-in period. The predictive model developed in this study can help experimentalists to design bench-top tests and operators to better translate the results of laboratory experiments into practical applications.
The objectives of this study were (1) to assess the fate and impact of CO2 injected into the Morrow B Sandstone in the Farnsworth Unit (FWU) through numerical non-isothermal reactive transport modeling, and (2) to compare the performance of three major reactive solute transport simulators, TOUGHREACT, STOMP-EOR, and GEM, under the same input conditions. The models were based on a quarter of a five-spot well pattern where CO2 was injected on a water-alternating-gas schedule for the first 25 years of the 1000 year simulation. The reservoir pore fluid consisted of water with or without petroleum. The results of the models have numerous broad similarities, such as the pattern of reservoir cooling caused by the injected fluids, a large initial pH drop followed by gradual pH neutralization, the long-term persistence of an immiscible CO2 gas phase, the continuous dissolution of calcite, very small decreases in porosity, and the increasing importance over time of carbonate mineral CO2 sequestration. The models differed in their predicted fluid pressure evolutions; amounts of mineral precipitation and dissolution; and distribution of CO2 among immiscible gas, petroleum, formation water, and carbonate minerals. The results of the study show the usefulness of numerical simulations in identifying broad patterns of behavior associated with CO2 injection, but also point to significant uncertainties in the numerical values of many model output parameters.
Hydraulic fracturing is a technique extensively used in the oil and gas industry, where water, proppant (sand) and additives are injected into unconventional reservoirs to enhance the recovery of shale hydrocarbon. Although some previous studies have developed pumping schedules that maximize gas production for a single-size proppant, there are very few studies that consider the effect of varying proppant diameters across pumping stages on shale gas production. Motivated by this, we carried out an extensive sensitivity analysis to determine the effect of different proppant diameters on the average fracture conductivity (FC), average propped surface area (PSA) and cumulative shale gas production volume. We found out that the cumulative shale gas production volume depends on both the average PSA and average FC. We also found out that small-diameter proppant resulted in higher average PSA and lower average FC, whereas large-diameter proppant resulted in lower average PSA and higher average FC. Hence, we designed a multi-size proppant pumping schedule considering both of these parameters into account for simultaneously propagating multiple fractures to maximize shale gas production from unconventional reservoirs. Since the size of injected proppant particles determines the average PSA and average FC for the propped hydraulic fractures, we developed a novel framework called Sequentially Interlinked Modeling Structure (SIMS) to predict the average PSA, average FC and cumulative shale gas production volume at the end of 10 years for a given pumping schedule. Then, we used this SIMS framework to obtain a multi-size proppant pumping schedule that maximizes shale gas production. In closing, we demonstrated that the obtained pumping schedule gives a cumulative shale gas production volume greater than the values obtained from the existing pumping schedules.
Background Immunotherapies, driven by immune-mediated antitumorigenicity, offer the potential for significant improvements to the treatment of multiple cancer types. Identifying therapeutic strategies that bolster antitumor immunity while limiting immune suppression is critical to selecting treatment combinations and schedules that offer durable therapeutic benefits. Combination oncolytic virus (OV) therapy, wherein complementary OVs are administered in succession, offer such promise, yet their translation from preclinical studies to clinical implementation is a major challenge. Overcoming this obstacle requires answering fundamental questions about how to effectively design and tailor schedules to provide the most benefit to patients. Methods We developed a computational biology model of combined oncolytic vaccinia (an enhancer virus) and vesicular stomatitis virus (VSV) calibrated to and validated against multiple data sources. We then optimized protocols in a cohort of heterogeneous virtual individuals by leveraging this model and our previously established in silico clinical trial platform. Results Enhancer multiplicity was shown to have little to no impact on the average response to therapy. However, the duration of the VSV injection lag was found to be determinant for survival outcomes. Importantly, through treatment individualization, we found that optimal combination schedules are closely linked to tumor aggressivity. We predicted that patients with aggressively growing tumors required a single enhancer followed by a VSV injection 1 day later, whereas a small subset of patients with the slowest growing tumors needed multiple enhancers followed by a longer VSV delay of 15 days, suggesting that intrinsic tumor growth rates could inform the segregation of patients into clinical trials and ultimately determine patient survival. These results were validated in entirely new cohorts of virtual individuals with aggressive or non-aggressive subtypes. Conclusions Based on our results, improved therapeutic schedules for combinations with enhancer OVs can be studied and implemented. Our results further underline the impact of interdisciplinary approaches to preclinical planning and the importance of computational approaches to drug discovery and development.
Current commercial long-range guided wave inspection and monitoring technologies for pipes usually require establishment of certain specific acoustic modes for wave propagation and use a large number of sensors in a certain geometric arrangement (for example, uniformly-spaced arrays of sensors on a collar, placed around a pipe). This paper, in contrast, presents a long-range corrosion detection method in pipes based on multi-mode guided acoustic waves, generated and received using simple, single-element sensors. In this technique, available guided wave modes in any interrogated zone of the structure are automatically down-selected by the medium, and changes in energy distribution over the selected modes are analyzed through signal processing to monitor material loss or addition. Here, to investigate the feasibility of the proposed method, we have performed both experimental and numerical simulation studies on 50.8-mm (2-in)-diameter schedule-40 carbon steel pipes. In these studies, a linear chirp signal is injected in compression mode, perpendicular to the surface of the pipe, generating multi-mode, multi-directional guided waves that travel within the pipe wall. Corrosion effects in the pipes are simulated numerically and experimentally. To reduce errors in the measurement technique due to changes in the local or global temperature of the measured media, a universal temperature compensation algorithm is introduced. The experimental results show good agreement with simulation results, indicating that the proposed multi-mode wave generation and detection in a pipe could be used for long-range detection and monitoring of corrosion effects, such as pipe wall losses or solid material deposits.
Thermal energy storage in oil and gas reservoirs leverages the existing surface and subsurface infrastructure, which can pave the way for economic production of geothermal energy. Existing studies on geothermal energy storage are focused mostly on the use of aquifers with more homogeneous rock and fluid properties. Coupling of heat and fluid flow in a multiphase-multicomponent system, such as an oil reservoir, is imperative especially if existing oil field assets need to be repurposed as required for a sustainable energy transition. The objective is to model the subsurface thermo-hydrological processes associated with reservoir performance and operational sustainability. The model evaluates formation pressure and temperature within the reservoir and at the injection/production wells during multiple charge and discharge cycles. Hot water (approximately 200 degrees C) heated by Concentrating Solar Power (CSP) at high pressure is injected into the existing oil reservoir for storage and produced as thermal energy for power generation, which will be accompanied by enhanced oil recovery. To demonstrate the coupled fluid and heat flow during the injection/production cycle in the subsurface reservoir, TOUGH3 (developed by Berkeley Lab) is used to simulate the thermo-hydrological (TH) processes in a multiphase, multicomponent system. Two well geometries are considered within the reservoir grid: 1) a single-well huff-n-puff system (same well is used for injection and production), and 2) an isolated injection-production well doublet. Seasonal charge and discharge cycling are implemented based on the scheduling specified in the model input file. The model reports pressure, temperature, enthalpy, liquid fluxes, heat fluxes, pore velocities, and changes in porosity & permeability due to temperature and pressure variations during the cyclic Reservoir Thermal Energy Storage (RTES) operations. The results from the simulations can be used to optimize the operational parameters (such as well spacing and injection/production rates) and round-trip efficiency for surface power-plants coupled with thermal energy storage over time. They can also serve as important inputs for levelized cost of storage estimations. The research will help to design and integrate surface renewable energy sources, such as concentrating solar power (CSP), with RTES to help balance out power supply and demand on the grid.
Power system operation will encounter numerous voltage variabilities as the proliferation of renewable energy continues. Real-time monitoring and communication technologies can potentially improve voltage stability by enabling the rapid detection of voltage deviations and the implementation of corrective actions. These corrective actions will only be effective in restoring stability if they are chosen in a timely and scalable manner. This paper considers the problem of power systems’ load voltage control in order to simultaneously address both magnitude and time voltage specifications. In order to comply with grid codes and avoid unnecessary relay protection actions, a model predictive control (MPC)-based control strategy employing temporal logic specifications (TLSs) is proposed. The TLSs strategy is introduced as a formalism to control the voltage variation at a critical load bus against operational bounds over time. The control objective is to schedule optimal control input signals from the available supportive energy storage systems, which provide reactive power injections, leading to satisfying the specified finite-time restoration described by the TLSs with minimal control efforts. The simulation results display that the TLSs strategy for power systems’ voltage control synthesis is extremely significant.