Strand displacement with loop-mediated isothermal amplification
Disclosed are methods for isothermal nucleic acid amplification and detection.
Engineering topics
Publications and source records attributed to Du, Yan.
Disclosed are methods for isothermal nucleic acid amplification and detection.
Model-based Deep Reinforcement Learning for Real-time Grid Emergency Voltage Control. A model-based Deep Reinforcement Learning (DRL) framework where a deep neural network (DNN)-based surrogate model is utilized within the control policy learning framework, making the learning process faster and more sample efficient
Here this article proposes a model-based deep reinforcement learning (DRL) method to design emergency control strategies for short-term voltage stability problems in power systems. Recent advances show promising results for model-free DRL-based methods in power systems control problems. But in power systems applications, these model-free methods have certain issues related to training time (clock time) and sample efficiency; both are critical for making state-of-the-art DRL algorithms practically applicable. DRL-agent learns an optimal policy via a trial-and-error method while interacting with the real-world environment. It is also desirable to minimize the direct interaction of the DRL agent with the real-world power grid due to its safety-critical nature. Additionally, the state-of-the-art DRL-based policies are mostly trained using a physics-based grid simulator where dynamic simulation is computationally intensive, lowering the training efficiency. We propose a novel model-based DRL framework where a deep neural network (DNN)-based dynamic surrogate model (SM), instead of a real-world power grid or physics-based simulation, is utilized within the policy learning framework, making the process faster and more sample efficient. However, having stable training in model-based DRL is challenging because of the complex system dynamics of large-scale power systems. We addressed these issues by incorporating imitation learning to have a warm start in policy learning, reward-shaping, and multi-step loss in surrogate model training. Finally, we achieved 97.5% reduction in samples and 87.7% reduction in training time for an application to the IEEE 300-bus test system.
Critical infrastructures such as cyber-physical energy systems (CPS-E) integrate information flow and physical operations that are vulnerable to natural and targeted failures. Safe, secure, and reliable operation and control of CPS-E is critical to ensure societal well-being and economic prosperity. Automated control is key for real-time operations and may be mathematically cast as a sequential decision-making problem under uncertainty. Emergence of data-driven techniques for decision making under uncertainty, such as reinforcement learning (RL), have led to promising advances for addressing sequential decision-making problems for risk-based robust CPS-E control. However, existing research challenges include understanding the applicability of RL methods across diverse CPS-E applications, addressing the effect of risk preferences across multiple RL methods, and development of open-source domain-aware simulation environments for RL experimentation within a CPS-E context. This article systematically analyzes the applicability of four types of RL methods (model-free, model-based, hybrid model-free and model-based, and hierarchical) for risk-based robust CPS-E control. Problem features and solution stability for the RL methods are also discussed. We demonstrate and compare the performance of multiple RL methods under different risk specifications (risk-averse, risk-neutral, and risk-seeking) through the development and application of an open-source simulation environment. Motivating numerical simulation examples include representative single-zone and multizone building control use cases. Finally, six key insights for future research and broader adoption of RL methods are identified, with specific emphasis on problem features, algorithmic explainability, and solution stability.
Previous studies have shown that models overestimate the strength of ENSO teleconnections to the North Pacific during springtime, but the underlying reasons for this bias remain unknown. Here, in this work, the relative contributions from basic-state and thermodynamic/dynamic forcing factors are disentangled through idealized experiments with the Community Earth System Model and a range of stationary wave modeling experiments. It is revealed that in CESM1 the diabatic heating biases over the tropical Indian Ocean and tropical central-western Pacific jointly favor a cyclonic (anticyclonic) circulation bias to occur in the North Pacific during the springtime of El Niño (La Niña) events. On one hand, the difference in the modeled and observed climatological basic state does not lead to the bias formation directly, as the diabatic heating biases are the primary cause. On the other hand, the springtime basic state is conducive to a more vigorous stationary wave response to the biased diabatic heating than the wintertime state, and this explains why the teleconnection bias occurs during springtime but not in winter. An iterative bias-correction approach is then implemented in the atmospheric model component of CESM1 to verify the linkage between the tropical diabatic heating bias and the teleconnection bias. Moreover, this explanation is shown to be relevant in other models of phase 5 of the Coupled Model Intercomparison Project (CMIP5) as a strong relationship is found between biases in ENSO-related tropical central-western Pacific/Indian Ocean precipitation and North Pacific circulation across models in spring.
The Pearl River estuary-coastal system in the Northern South China Sea is frequently affected by tropical cyclones (TCs) every year. Previous in-situ and remote sensing studies have found typhoons can enhance phytoplankton biomass and induce blooms in this region. However, the mechanistic links between phytoplankton blooms and typhoons have not been well elucidated due to the interplay of multiple processes along the land-ocean-atmosphere interfaces. Unravelling these interactions will have to rely on an integrated modelling system. In this work, we constructed a realistic, 3-dimensional, land-ocean-atmosphere modelling system with the marine ecosystem and sediment components for the China Great Bay Area. By using the integrated modelling system, we quantitatively investigated phytoplankton response to hydrological conditions variations under Typhoon Hato (2017), a strong typhoon case. Passive tracer experiments showed that with high river discharge induced by heavy rainfall, the residence time of Lingding Bay is as short as 15-day, less than half of that under the climatological discharge. The increase in freshwater pulse washed out the phytoplankton biomass within Lingding Bay. While for the offshore region, the source and sink terms analysis showed that the increase of phytoplankton biomass in the first week was because of the uplift of nutrient-rich subsurface water, while in the second week was because of the seaward propagated nearshore high phytoplankton biomass water. While riverine nutrients support phytoplankton growth in the third week, a large part of phytoplankton biomass was lost to zooplankton grazing, showing the system shifted from the bottom-up control to the top-down control.
The double-fed induction generator (DFIG) usually experiences high rotor current and DC capacitor link voltage during system fault events. In this paper, a novel data-driven approach is proposed to enhance the DFIG performance under fault scenarios. An advanced reinforcement learning (RL) algorithm called guided surrogate-gradient-based evolution strategy (GSES) is used to control the DFIG power and capacitor DClink voltage by adjusting the optional reference signals. This controller is able to prevent the rotor of DFIG from overcurrent risk and maintain the grid-connected operation. The proposed GSES-based control algorithm was evaluated through simulations on a 3.6-MW DFIG in PSCAD/EMTDC software. Results have validated the effectiveness of the proposed GSESbased control algorithm in improving the DFIG performance under fault scenarios.
We report that buildings account for 40% of total primary energy consumption and 30% of all CO 2 emissions worldwide. A large portion of building energy consumption is due to heating, ventilation, and air-conditioning (HVAC) systems. In the summer, for example, more than 50% of a building’s electricity consumption is used for cooling. With proper energy management, buildings can provide load shifting, peak shaving, frequency regulation, and many other demand response services.
As power systems are undergoing a significant transformation with more uncertainties, less inertia and closer to operation limits, there is increasing risk of large outages. Thus, there is an imperative need to enhance grid emergency control to maintain system reliability and security. Towards this end, great progress has been made in developing deep reinforcement learning (DRL) based grid control solutions in recent years. However, existing DRL-based solutions have two main limitations: 1) they cannot handle well with a wide range of grid operation conditions, system parameters, and contingencies; 2) they generally lack the ability to fast adapt to new grid operation conditions, system parameters, and contingencies, limiting their applicability for real-world applications. Here, in this paper, we mitigate these limitations by developing a novel deep meta-reinforcement learning (DMRL) algorithm. The DMRL combines the meta strategy optimization together with DRL, and trains policies modulated by a latent space that can quickly adapt to new scenarios. We test the developed DMRL algorithm on the IEEE 300-bus system. We demonstrate fast adaptation of the meta-trained DRL polices with latent variables to new operating conditions and scenarios using the proposed method, which achieves superior performance compared to the state-of-the-art DRL and model predictive control (MPC) methods.
Microgrids can be operated in island mode during utility grid outages to support service restoration and improve system resilience. To schedule and dispatch distributed energy resources (DERs) in an islanded microgrid, conventional model-based methods rely on accurate distribution network models and lack generalization and adaptability. Data-driven methods are promising for DER coordination but face practical challenges such as potential hazards to microgrids during online training and insufficient online training opportunities due to low outage rates. This paper presents a novel two-stage learning framework that builds on the deep deterministic policy gradient from demonstrations to identify an optimal restoration strategy. At the pre-training stage, imitation learning is applied to equip the control agent with expert experiences to guarantee acceptable initial performance. At the online training stage, action clipping, reward shaping, and expert demonstrations are leveraged to ensure safe exploration while accelerating the training process. In conclusion, the proposed method is illustrated using the IEEE 123-node system and compared with a representative model-based method and the standard deep deterministic policy gradient method to prove solution accuracy and demonstrate increased computational efficiency.
Here, in this work we propose a novel data-driven, real-time power system voltage stability control method based on the physics-informed guided meta evolutionary strategy (ES). The main objective is to quickly provide an adaptive control strategy to secure system voltage stability. The problem is challenging due to the high-dimensional feature of the power system model and the fast-changing and uncertain nature of power system operation scenarios. To this end, a model-free and derivative-free guided ES method is applied. The method is further combined with a meta-learning strategy to make the learnt control policy automatically adapted to unseen operation conditions and fault scenarios, which is highly desired for real-time emergency control. Last but not least, physical knowledge is embedded in the above method through a trainable action mask technique to rule out unnecessary load shedding actions for better learning and control performance. Case studies on the IEEE 300-bus system and comparisons with other state-of-the-art benchmark methods verify the superiority of the proposed physics-informed guided meta ES method in realizing fast and adaptive power system voltage stability control.
Spatially resolved elastic strains in the bulk interior of a laminated Ti-Al metal composite were studied during in situ tensile loading at strains up to 1.66% by a synchrotron-based micro-diffraction technique, namely differential aperture X-ray microscopy (DAXM). For both Al and Ti grains, deviatoric elastic strains were estimated based on polychromatic X-ray microbeam diffraction, while lattice strains along the normal direction of the tensile sample were directly measured using monochromatic X-ray microbeam diffraction. The estimated deviatoric strains show large spatial variations, and the mean values are consistent with the external loading conditions, i.e., increasing tensile strain along the tensile direction and increasing compressive strain along the sample normal with increasing loading. The directly measured lattice strains also show large spatial variations, although the magnitude of this variation is smaller than that for the estimated deviatoric strain. The directly measured lattice strains in Ti grains are largely consistent with the external loading, whereas those in Al grains are in contradiction with the external loading. The causes of the experimental results are discussed and related to both the laminated microstructure of the composite material and the limitations of the techniques.
In this paper, a day-ahead electricity market bidding problem with multiple strategic generation company (GEN-CO) bidders is studied. The problem is formulated as a Markov game model, where GENCO bidders interact with each other todevelop their optimal day-ahead bidding strategies. Considering unobservable information in the problem, a model-free and data-driven approach, known as multi-agent deep deterministic policy gradient (MADDPG), is applied for approximating the Nash equilibrium (NE) in the above Markov game. The MADDPG algorithm has the advantage of generalization due to the automatic feature extraction ability of the deep neural networks. The algorithm is tested on an IEEE 30-bus system with three competitive GENCO bidders in both an uncongested caseand a congested case. Comparisons with a truthful bidding strategy and state-of-the-art deep reinforcement learning methods including deep Q network and deep deterministic policy gradient (DDPG) demonstrate that the applied MADDPG algorithm can find a superior bidding strategy for all the market participants with increased profit gains. In addition, the comparison with a conventional model-based method shows that the MADDPG algorithm has higher computational efficiency, which is feasible for real-world applications.
Deep reinforcement learning (DRL) approaches have been used in various application areas to improve efficiency, optimization, or automation. However, very little is known about how the DRL algorithms make decisions and what features affect their performance. Using a case study of a DRL based Heating, Ventilation and Air Conditioning (HVAC) optimization methodology, we demonstrate how we can address these challenges by applying interpretability tools and systematically exploring the model inputs for better understanding the DRL behaviour and decision making process. We developed a methodology for interpretable reinforcement learning and evaluated our approach in real-world house located in Knoxville, TN. Our findings explain the reasoning behind DRL-based optimization decisions under different circumstances which has been discussed and confirmed by the experts in the field.
An ultra-low-strain deformation microstructure was revealed for the first time non-destructively in the bulk interior of an annealed laminated Ti-Al composite-in the "fully recrystallized" Al layer-by a synchrotron-based micro-diffraction technique, namely differential aperture X-ray microscopy (DAXM), through real space mapping with a very high angular resolution (0.01 degrees). This ultra-low-strain deformation microstructure was found to result from the thermal stress, induced during cooling after annealing, due to the different coefficients of thermal expansion for the Ti and Al layers. The annealed sample was further tensile deformed to a strain of 1.66% and followed by in situ DAXM and analyzed by various misorientation mapping methods. The results pointed to the important effects of the initial microstructure and the interface constraint, as well as the grain size and crystal orientation, on the plastic deformation. A gradient in dislocation density from the layer interface to the center was found in the Al layer of the annealed sample, and this gradient increased slightly during tensile deformation. The variation of the dislocation density was further discussed based on the activation and interaction of dislocations in grains of different sizes and orientations during plastic deformation. The findings of this study provided valuable insights in understanding the constraint effect of the laminated metal composite and the design of novel composite materials. (c) 2020 Acta Materialia Inc. Published by Elsevier Ltd. All rights reserved.
The Artificial Intelligence (AI) development described herein uses model-free Deep Reinforcement Learning (DRL) to minimize energy cost during residential heating, ventilation, and air conditioning (HVAC) operation. Building cooling loads and HVAC operation are difficult to accurately model due to complexity, lack of measurements and data, and model specific performance, so online machine learning is used to allow for real-time readjustment in performance. Energy costs for the multi-zone cooling unit shown in this work are minimized by scheduling on/off commands around dynamic prices. By taking advantage of precooling events that take place when the price is low, the agent is able to reduce operational cost without violating user comfort. The DRL controller was tested in simulation where the learner achieved a 43.89% cost reduction when compared to traditional, fixed-setpoint operation. The system is now ready for the next phase of testing in a live, real-time home environment.
In this short communication, a data-driven deep reinforcement learning (deep RL) method is applied to minimize HVAC users’ energy consumption costs while maintaining users’ comfort. The applied deep RL method's efficiency is enhanced by conducting multi-task learning that can achieve an economic control strategy for a multi-zone residential HVAC system in both cooling and heating scenarios. The applied multi-task deep RL method is compared with a rule-based benchmark case and a single-task deep deterministic policy gradient algorithm to verify its effective and generalized application in optimizing HVAC operation.
Residential heating, ventilation, and air conditioning (HVAC) has been considered as an important demand response resource. However, the optimization of residential HVAC control is no trivial task due to the complexity of the thermal dynamic models of buildings and uncertainty associated with both occupant-driven heat loads and weather forecasts. In this paper, we apply a novel model-free deep reinforcement learning (RL) method, known as the deep deterministic policy gradient (DDPG), to generate an optimal control strategy for a multi-zone residential HVAC system with the goal of minimizing energy consumption cost while maintaining the users’ comfort. Here, the applied deep RL-based method learns through continuous interaction with a simulated building environment and without referring to any prior model knowledge. Simulation results show that compared with the state-of-art deep Q network (DQN), the DDPG-based HVAC control strategy can reduce the energy consumption cost by 15% and reduce the comfort violation by 79%; and when compared with a rule-based HVAC control strategy, the comfort violation can be reduced by 98%. In addition, experiments with different building models and retail price models demonstrate that the well-trained DDPG-based HVAC control strategy has high generalization and adaptability to unseen environments, which indicates its practicability for real-world implementation.