Search NASA⌕ Search

SEARCH · Search NASA

Results for “Q learning”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 127 records · Page 7

Exploring the Structural, Dynamic, and Functional Properties of Metal–Organic Frameworks through Molecular Modeling

This review spotlights the role of atomic-level modeling in research on metal-organic frameworks (MOFs), especially the key methodologies of density functional theory (DFT), Monte Carlo (MC) simulations, and molecular dynamics (MD) simulations. The discussion focuses on how periodic and cluster-based DFT calculations can provide novel insights into MOF properties, with a focus on predicting structural transformations, understanding thermodynamic properties and catalysis, and providing information or properties that are fed into classical simulations such as force field parameters or partial charges. Classical simulation methods, highlighting force field selection, databases of MOFs for high-throughput screening, and the synergistic nature of MC and MD simulations, are described. By predicting equilibrium thermodynamic and dynamic properties, these methods offer a wide perspective on MOF behavior and mechanisms. Additionally, the incorporation of machine learning (ML) techniques into quantum and classical simulations is discussed. These methods can enhance accuracy, expedite simulation setup, reduce computational costs, as well as predict key parameters, optimize geometries, and estimate MOF stability. By charting the growth and promise of computational research in the MOF field, the aim is to provide insights and recommendations to facilitate the incorporation of computational modeling more broadly into MOF research.

36 MATERIALS SCIENCE↗

Lesson Plan Prototype for International Space Station's Interactive Video Education Events

The outreach and education components of the International Space Station Program are creating a number of materials, programs, and activities that educate and inform various groups as to the implementation and purposes of the International Space Station. One of the strategies for disseminating this information to K-12 students involves an electronic class room using state of the art video conferencing technology. K-12 classrooms are able to visit the JSC, via an electronic field trip. Students interact with outreach personnel as they are taken on a tour of ISS mockups. Currently these events can be generally characterized as: Being limited to a one shot events, providing only one opportunity for students to view the ISS mockups; Using a "one to many" mode of communications; Using a transmissive, lecture based method of presenting information; Having student interactions limited to Q&A during the live event; Making limited use of media; and Lacking any formal, performance based, demonstration of learning on the part of students. My project involved developing interactive lessons for K-12 students (specifically 7th grade) that will reflect a 2nd generation design for electronic field trips. The goal of this design will be to create electronic field trips that will: Conform to national education standards; More fully utilize existing information resources; Integrate media into field trip presentations; Make support media accessible to both presenters and students; Challenge students to actively participate in field trip related activities; and Provide students with opportunities to demonstrate learning

Zigon, Thomas↗

Design and Analysis of Convolutional Neural Network for RF Signal Modulation Classification for In-Orbit Deployment

To effectively transmit data to and from satellites requires a complex and robust RF communication system. Commonly, several different types of signal modulations may be required to maximize satellite efficiency depending on a variety of unexpected channel impairments. We propose a neural network algorithm capable of learning these RF signal modulations using a supervised learning technique designed for low power, high-efficiency in-orbit deployment. The work presented demonstrates a convolutional neural network (CNN) capable of learning and recognizing a set of modulation schemes commonly used to transmit RF information. We are capable of recognizing the modulation scheme from the I and Q data channels directly, with no preprocessing or data conversion required other than breaking the incoming signal into a set of uniform normalized samples. We perform a network design and size analysis, showing that reasonably high accuracy can be obtained using networks with a relatively low number of trainable parameters. Given that a user of a system such as this may wish to receive a signal using a modulation scheme that the network has not previously learned, we demonstrate that transfer learning can learn new modulation schemes by retraining only the fully connected layers in the CNN. Thus, this type of network would excel in outer space deployment using high-efficiency transfer learning hardware. Modulation recognition can be performed through rapid feedforward computation, and the CNN training process is significantly simplified when learning new modulations is required.

CNN↗

A perfect amalgam - the combination of company in-house training with an academic online certificate program

A well-trained and qualified workforce is an essential element in the modern day work place of the nuclear industry. For generations, from the guilds of the Middle Age through to the early Industrial Age, people learned their crafts through hands-on training provided at the hands of masters. Today, though there is an ever-pressing need for mentoring in the workplace, the fast-paced environment and lack of time to develop specific in-house training programs can create hindrances to both formal and informal learning in the work place. One way to overcome this gap is through private company and academic collaboration. In 2012, the University of Nevada, Las Vegas (UNLV) developed a Nuclear Regulatory Commission sponsored Nuclear Criticality Safety Certificate Program. In collaboration with URENCO USA, an online version of the program was also developed to assist the URENCO criticality workforce in the nuclear criticality safety (NCS) qualification process to further their education. This paper looks into the differences between formal and informal learning and how factors, which shape informal learning, are used to compliment formal learning in the work environment. It further discusses how a collaborative relationship between private and academic partners can assist a company in maintaining a high level of expertise, while assisting with a company's goals of fostering, sustaining and strengthening the nuclear safety culture. (authors)

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗

Cryogenic performance of a cryocooler-cooled superconducting undulator

A cryocooler-cooled superconducting undulator has been installed and operated with beam at the Advanced Photon Source (APS) at Argonne National Laboratory (ANL). The device consists of a dual-core 42-pole magnet structure that is cooled to 4.2 K with a system of four cryocoolers operating in a zero-boil-off configuration. This effort represents the culmination of a development program to establish concept feasibility and evaluate cryostat design and cryocooler-based refrigeration. Cryostat performance is described including cool-down/warm-up, steady-state operation, cooling margin, and the impact of beam during operation in the APS storage ring. Plans for future devices with longer magnets, which will incorporate lessons learned from the development program, are also discussed.

43 PARTICLE ACCELERATORS↗

Comparative analysis of model-free and model-based HVAC control for residential demand response

In this paper, we present a comparative analysis of model-free reinforcement learning (RL) and model predictive control (MPC) approaches for intelligent control of heating, ventilation, and air-conditioning (HVAC). Deep-Q-network (DQN) is used as a candidate for model-free RL algorithm. The two control strategies were developed for residential demand-response (DR) HVAC system. We considered MPC as our golden standard to compare DQN's performance. The question we tried to answer through this work was, What % of MPC's performance can be achieved by model-free RL approach for intelligent HVAC control?. Based on our test result, RL achieved an average of ≈ 62% daily cost saving of MPC. Considering the pure optimization and model-based nature of MPC methods, the RL showed very promising performance. We believe that the interpretations derived from this comparative analysis provide useful insights to choose from various DR approaches and further enhance the performance of the RL-based methods for building energy managements.

Kurte, Kuldeep↗

Characterizing the contribution of bioaerosols diversity from complex aerosol particle samples.

Current global change scenarios expect to significantly increase the contribution of pollen to the total organic aerosol budget. In addition, pollen can burst into smaller fragments that are highly efficient as cloud condensation nuclei. However, current climate models still do not incorporate the effects of these particles due to detection and quantification challenges in complex aerosol mixtures. Chemical biomarkers (i.e. fructose) have commonly been used to trace pollen but this approximation can generate false positive results given the large number of shared compounds between different bioaerosols. A previous pilot EMSL experiment performed on pure pollen samples suggested that using a set of metabolites as a fingerprint of bioaerosols could serve to substantially increase detection accuracy over single biomarker approaches. In this proposal we developed novel bioinformatic strategies to characterize and quantify complex pollen mixtures present in the atmosphere. For that, we used diverse machine learning algorithms to analyze metabolic fingerprints from complex mixtures of different pollen acquired with high-resolution mass spectrometry (Orbitrap Q-Exactive). Our preliminary results demonstrated the potential to accurately discern and identify at specific relative abundances different pollen species in complex mixtures. Our research will lead to a significant advancement in the atmospheric chemistry and provide much needed data to improve the accuracy of climate models.

54 ENVIRONMENTAL SCIENCES↗

The evolution of precipitate crystal structures in an Al-Mg-Si(-Cu) alloy studied by a combined HAADF-STEM and SPED approach

Highlights: • Scanning electron diffraction and machine learning complement lattice imaging • Developed approach gives increased objectivity in precipitate phase analysis • Precipitate phase fractions and solute content obtained with superior statistics • 0.01 at.% Cu changes the Al-Mg-Si system precipitation during overageing • Q’-phase eventually dominates precipitate structures and shows varying Cu-occupancy - Abstract: This work presents a detailed investigation into the effect of a low Cu addition (0.01 at.%) on precipitation in an Al-0.80Mg-0.85Si alloy during ageing. The precipitate crystal structures were assessed by scanning transmission electron microscopy combined with a novel scanning precession electron diffraction approach, which includes machine learning. The combination of techniques enabled evaluation of the atomic arrangement within individual precipitates, as well as an improved estimate of precipitate phase fractions at each ageing condition, through analysis of a statistically significant number of precipitates. Based on the obtained results, the total amount of solute atoms locked inside precipitates could be approximated. It was shown that even with a Cu content close to impurity levels, the Al-Mg-Si system precipitation was significantly affected with overageing. The principal change was due to a gradually increasing phase fraction of the Cu-containing Q′-phase, which eventually was seen to dominate the precipitate structures. The structural overtake could be explained based on a continuous formation of the thermally stable Q′-phase, with Cu atomic columns incorporating less Cu than what could potentially be accommodated.

36 MATERIALS SCIENCE↗

Machine learning using host/guest energy histograms to predict adsorption in metal–organic frameworks: Application to short alkanes and Xe/Kr mixtures

A machine learning (ML) methodology that uses a histogram of interaction energies has been applied to predict gas adsorption in metal–organic frameworks (MOFs) using results from atomistic grand canonical Monte Carlo (GCMC) simulations as training and test data. In this work, the method is first extended to binary mixtures of spherical species, in particular, Xe and Kr. In addition, it is shown that single-component adsorption of ethane and propane can be predicted in good agreement with GCMC simulation using a histogram of the adsorption energies felt by a methyl probe in conjunction with the random forest ML method. Here, the results for propane can be improved by including a small number of MOF textural properties as descriptors. We also discuss the most significant features, which provides physical insight into the most beneficial adsorption energy sites for a given application.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Deep Reinforcement Learning Based Volt-VAR Optimization in Smart Distribution Systems

This paper develops a model-free volt-VAR optimization (VVO) algorithm via multi-agent deep reinforcement learning (DRL) in unbalanced distribution systems. This method is novel since we cast the VVO problem in distribution networks to an intelligent deep Q-network (DQN) framework, which avoids solving a specific optimization model directly when facing time-varying operating conditions in the systems. We consider statuses/ratios of switchable capacitors, voltage regulators, and smart inverters installed at distributed generators as the action variables of the agents. A delicately designed reward function guides these agents to interact with the distribution system, in the direction of reinforcing voltage regulation and power loss reduction simultaneously. The forward-backward sweep method for radial three-phase distribution systems provides accurate power flow results within a few iterations to the DRL environment. The proposed method realizes the dual goals for VVO. We test this algorithm on the unbalanced IEEE 13-bus and 123-bus systems. Numerical simulations validate the excellent performance of this method in voltage regulation and power loss reduction.

41 EE - Solar Energy Technologies Office (EE-4S)↗

Meeting Global Health Needs via Infectious Disease Forecasting: Development of a Reliable Data-Driven Framework

Infectious diseases (IDs) have a significant detrimental impact on global health. Timely and accurate ID forecasting can result in more informed implementation of control measures and prevention policies. To meet the operational decision-making needs of real-world circumstances, we aimed to build a standardized, reliable, and trustworthy ID forecasting pipeline and visualization dashboard that is generalizable across a wide range of modeling techniques, IDs, and global locations. We forecasted 6 diverse, zoonotic diseases (brucellosis, campylobacteriosis, Middle East respiratory syndrome, Q fever, tick-borne encephalitis, and tularemia) across 4 continents and 8 countries. We included a wide range of statistical, machine learning, and deep learning models (n=9) and trained them on a multitude of features (average n=2326) within the One Health landscape, including demography, landscape, climate, and socioeconomic factors. The pipeline and dashboard were created in consideration of crucial operational metrics—prediction accuracy, computational efficiency, spatiotemporal generalizability, uncertainty quantification, and interpretability—which are essential to strategic data-driven decisions. While no single best model was suitable for all disease, region, and country combinations, our ensemble technique selects the best-performing model for each given scenario to achieve the closest prediction. For new or emerging diseases in a region, the ensemble model can predict how the disease may behave in the new region using a pretrained model from a similar region with a history of that disease. The data visualization dashboard provides a clean interface of important analytical metrics, such as ID temporal patterns, forecasts, prediction uncertainties, and model feature importance across all geographic locations and disease combinations. As the need for real-time, operational ID forecasting capabilities increases, this standardized and automated platform for data collection, analysis, and reporting is a major step forward in enabling evidence-based public health decisions and policies for the prevention and mitigation of future ID outbreaks.

60 APPLIED LIFE SCIENCES↗

A Virtualized Computing Platform For Fusion Control Systems

The National Ignition Facility (NIF) at the Lawrence Livermore National Laboratory is a stadium-sized facility that contains a 192-beam, 1.8-Megajoule, 500-Terawatt, ultraviolet laser system together with a 10-meter diameter target chamber with room for multiple experimental diagnostics. NIF is the world's largest and most energetic laser experimental system, providing a scientific center to study inertial confinement fusion (ICF) and matter at extreme energy densities and pressures. NIF's laser beams are designed to compress fusion targets to conditions required for thermonuclear burn, liberating more energy than required to initiate the fusion reactions. 2,500 servers, 400 network devices and 700 terabytes of networked attached storage provide the foundation for NIF's Integrated Computer Control System (ICCS) and Experimental Data Archive. This talk discusses the rationale & benefits for server virtualization in the context of an operational experimental facility, the requirements discovery process used by the NIF teams to establish evaluation criteria for virtualization alternatives, the processes and procedures defined to enable virtualization of servers in a timeframe that did not delay the execution of experimental campaigns and the lessons the NIF teams learned along the way. The virtualization architecture ultimately selected for ICCS is based on the Open Source Xen computing platform and 802.1Q open networking standards. The specific server and network configurations needed to ensure performance and high availability of the control system infrastructure will be discussed.

42 ENGINEERING↗

A Cyber-Physical System for Freeway Ramp Meter Signal Control Using Deep Reinforcement Learning in a Connected Environment

Freeway bottlenecks such as on-ramp merging areas account for about 40% of recurring freeway congestion. It is generally agreed that building more roads and adding more lanes to existing infrastructure does not solve the congestion problem, and so dynamic traffic control measures offer a more cost-effective alternative. Ramp meters, traffic signal devices that regulate traffic flow entering freeways, are among the most effective measures to mitigate congestion at on-ramp merging areas on freeways. The confluence of deep reinforcement learning (RL) and connectivity provides a possible solution to advance ramp meter signal control. Deep RL is a group of machine-learning methods that enables an agent learning from the environment to improve its performance. In this study, three deep RL methods-proximal policy optimization (PPO), Ape-X deep Q-network (DQN), and asynchronous advantage actor-critic agents (A3C)-are explored for ramp meter signal control to maximize vehicle speed and traffic throughput, as well as to minimize energy consumption and emissions at freeway on-ramp merging areas in a connected environment. The low computational requirement and scalability of deep RL for deployment make it a powerful optimization tool for time-sensitive applications such as ramp meter signal control. The results of this study show that deep RL methods yield superior performance to both a fixed-time controller and ALINE A, a state-of-the-art feedback controller.

ADVANCED PROPULSION SYSTEMS,MATHEMATICS AND COMPUT↗

Machine Learning Characterization of Alfvénic and Sub-Alfvénic Chirping and Correlation With Fast-Ion Loss at NSTX

Abrupt large events in the Alfvenic and sub-Alfvenic frequency bands in tokamaks are typically correlated with increased fast-ion loss. Here, machine learning is used to speed up the laborious process of characterizing the behavior of magnetic perturbations from corresponding frequency spectrograms that are typically identified by humans. The analysis allows for comparison between different mode character (such as quiescent, fixed frequency, and chirping, avalanching) and plasma parameters obtained from the TRANSP code, such as the ratio of the neutral beam injection (NBI) velocity and the Alfven velocity (v_inj./v_A), the q-profile, and the ratio of the neutral beam beta and the total plasma beta (beta_beam,i / beta). In agreement with the previous work by Fredrickson et al., we find a correlation between beta_beam,i and mode character. In addition, previously unknown correlations are found between moments of the spectrograms and mode character. Character transition from quiescent to nonquiescent behavior for magnetic fluctuations in the 50200-kHz frequency band is observed along the boundary v_phi ~ (1/4)(v_inj. - 3v_A), where v_phi is the rotation velocity.

machine learning↗

Implementation of AI/DEEP learning disruption predictor into a plasma control system

Abstract This paper reports on advances in the state‐of‐the‐art deep learning disruption prediction models based on the Fusion Recurrent Neural Network (FRNN) originally introduced in a 2019 NATURE publication [ https://doi.org/10.1038/s41586‐019‐1116‐4 ]. In particular, the predictor now features not only the “disruption score,” as an indicator of the probability of an imminent disruption, but also a “sensitivity score” in real time to indicate the underlying reasons for the imminent disruption. This adds valuable physics interpretability for the deep learning model and can provide helpful guidance for control actuators now implemented into a modern plasma control system (PCS). The advance is a significant step forward in moving from modern deep learning disruption prediction to real‐time control and brings novel AI‐enabled capabilities relevant for application to the future burning plasma ITER system. Our analyses use large amounts of data from JET and DIII‐D vetted in the earlier NATURE publication. In addition to “when” a shot is predicted to disrupt, this paper addresses reasons “why” by carrying out sensitivity studies. FRNN is accordingly extended to use more channels of information, including measured DIII‐D signals such as (i) the “n1rms” signal that is correlated with the n = 1 modes with finite frequency, including neoclassical tearing mode and sawtooth dynamics; (ii) the bolometer data indicative of plasma impurity control; and (iii) “q‐min”—the minimum value of the safety factor relevant to the key physics of kink modes. The additional channels and interpretability features expand the ability of the deep learning FRNN software to provide information about disruption subcategories as well as more precise and direct guidance for the actuators in a PCS.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗