SEARCH · Search NASA

Results for “reward engineering”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records

All sources NASA NTRS DOE OSTI

DOE OSTI2021

Deep reinforcement learning for dynamic control of fuel injection timing in multi-pulse compression ignition engines

Conventional compression-ignition (CI) engines have long offered high thermal efficiencies and torque across a wide range of loads, but often require extensive exhaust gas treatment that decreases efficiency to meet ever-increasing emissions regulations. One strategy to decrease emissions is to split the fuel injection into a series of smaller injections. In this paper, we explore a new way of discovering optimal control strategies for the next generation of CI engines using deep reinforcement learning (DRL). We outline a DRL procedure to maximize the weighted reward of engine work while minimizing end-of-cycle NO x emissions. Through the procedure outlined in this paper, we show that the DRL agent is able to reduce NO x emissions threefold while only decreasing network by 2%. We demonstrate the use of transfer learning (TL) across hierarchies of physical models to accelerate the learning process, making this approach feasible for a range of control problems within this space. This paper presents a framework and demonstration for using DRL to design control systems in technology areas such as multi-pulse engine control where a hierarchy of models combined with multi-objective rewards are used for optimal operation.

33 ADVANCED PROPULSION SYSTEMS↗

DOE OSTI2021

Exploring electron beam induced atomic assembly via reinforcement learning in a molecular dynamics environment

We report atom-by-atom assembly of functional materials and devices is perceived as one of the ultimate targets of nanotechnology. Recently it has been shown that the beam of a scanning transmission electron microscope can be used for targeted manipulation of individual atoms. However, the process is highly dynamic in nature rendering control difficult. One possible solution is to instead train artificial agents to perform the atomic manipulation in an automated manner without need for human intervention. As a first step to realizing this goal, we explore how artificial agents can be trained for atomic manipulation in a simplified molecular dynamics environment of graphene with Si dopants, using reinforcement learning. We find that it is possible to engineer the reward function of the agent in such a way as to encourage formation of local clusters of dopants under different constraints. This study shows the potential for reinforcement learning in nanoscale fabrication, and crucially, that the dynamics learned by agents encode specific elements of important physics that can be learned.

77 NANOSCIENCE AND NANOTECHNOLOGY↗

DOE OSTI2024

Integrating an Ensemble Reward System into an Off-Policy Reinforcement Learning Algorithm for the Economic Dispatch of Small Modular Reactor-Based Energy Systems

Nuclear Integrated Energy Systems (NIES) have emerged as a comprehensive solution for navigating the changing energy landscape. They combine nuclear power plants with renewable energy sources, storage systems, and smart grid technologies to optimize energy production, distribution, and consumption across sectors, improving efficiency, reliability, and sustainability while addressing challenges associated with variability. The integration of Small Modular Reactors (SMRs) in NIES offers significant benefits over traditional nuclear facilities, although transferring involves overcoming legal and operational barriers, particularly in economic dispatch. This study proposes a novel off-policy Reinforcement Learning (RL) approach with an ensemble reward system to optimize economic dispatch for nuclear-powered generation companies equipped with an SMR, demonstrating superior accuracy and efficiency when compared to conventional methods and emphasizing RL’s potential to improve NIES profitability and sustainability. Finally, the research attempts to demonstrate the viability of implementing the proposed integrated RL approach in spot energy markets to maximize profits for nuclear-driven generation companies, establishing NIES’ profitability over competitors that rely on fossil fuel-based generation units to meet baseload requirements.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗

DOE OSTI2024

Physics-based reward driven image analysis in microscopy

The rise of electron microscopy has expanded our ability to acquire nanometer and atomically resolved images of complex materials. The resulting vast datasets are typically analyzed by human operators, an intrinsically challenging process due to the multiple possible analysis steps and the corresponding need to build and optimize complex analysis workflows. We present a methodology based on the concept of a Reward Function coupled with Bayesian Optimization, to optimize image analysis workflows dynamically. The Reward Function is engineered to closely align with the experimental objectives and broader context and is quantifiable upon completion of the analysis. Here, cross-section, high-angle annular dark field (HAADF) images of ion-irradiated (Y, Dy)Ba 2 Cu 3 O 7–δ thin-films were used as a model system. The reward functions were formed based on the expected materials density and atomic spacings and used to drive multi-objective optimization of the classical Laplacian-of-Gaussian (LoG) method. These results can be benchmarked against the DCNN segmentation. This optimized LoG* compares favorably against DCNN in the presence of the additional noise. We further extend the reward function approach towards the identification of partially-disordered regions, creating a physics-driven reward function and action space of high-dimensional clustering. We pose that with correct definition, the reward function approach allows real-time optimization of complex analysis workflows at much higher speeds and lower computational costs than classical DCNN-based inference, ensuring the attainment of results that are both precise and aligned with the human-defined objectives.

47 OTHER INSTRUMENTATION↗

NASA NTRS2007

My Rewarding Summer Research Experience at NASA

My summer research experience at the Kennedy Space Center has been a truly rewarding one. As an electrical engineering student at the University of South Florida, I was blessed with a beneficial opportunity to gain valuable knowledge in my career, and also apply it through working at NASA. One of my inspirations in becoming an engineer is to work at NASA someday, and I was very excited and honored to have this opportunity. My goal in this internship was to strengthen my preparation in becoming an engineer by learning new material, acquiring skills by practicing what I learned, and discovering the expectations of engineering work at NASA. Through this summer research, I was able to learn new computer programs and perform various tasks that gave me experience and skills as an engineer. My primary job was to conduct work on the Constellation Test article, which is a simulation model of the Crew Launch Vehicle (CLV) tanking system. This is a prototype of a launch facility and an Ares I Vehicle, which God willing will transport astronauts to the moon. Construction of the CLV is in progress and a test launch is anticipated for 2010. Moreover, the Test Article serves as a demonstration too, training test bed, and may be expanded for new simulation of launch system elements, which could be applied to real life operations. The test article is operated and run by a Programmable Logic Controller (PLC), which is a digital computer that is used to control all forms of machinery such as those in manufacturing buildings and other industries. PLCs are different than other computers because of the physical protection they have against damaging environmental conditions that would destroy other computers. Also, PLCs are equipped with lots of input and output connections that allow extensive amounts of commands to be executed, which would normally require many computers to do. Therefore, PLCs are small, rugged, and extremely powerful tools that may continue to be employed at NASA. Furthermore, in order to conduct productive work on the Test Article, I needed to learn the computer program called RS Logics 5000.

Aviles, Andres↗

NASA NTRS1990

Lunar astronomical observatories - Design studies

The best location in the inner solar system for the grand observatories of the 21st century may be the moon. A multidisciplinary team including university students and faculty in engineering, astronomy, physics, and geology, and engineers from industry is investigating the moon as a site for astronomical observatories and is doing conceptual and preliminary designs for these future observatories. Studies encompass lunar facilities for radio astronomy and astronomy at optical, ultraviolet, and infrared wavelengths of the electromagnetic spectrum. Although there are significant engineering challenges in design and construction on the moon, the rewards for astronomy can be great, such as detection and study of earth-like planets orbiting nearby stars, and the task for engineers promises to stimulate advances in analysis and design, materials and structures, automation and robotics, foundations, and controls. Fabricating structures in the reduced-gravity environment of the moon will be easier than in the zero-gravity environment of earth orbit, as Apollo and space-shuttle missions have revealed. Construction of observatories on the moon can be adapted from techniques developed on the earth, with the advantage that the moon's weaker gravitational pull makes it possible to build larger devices than are practical on earth.

Johnson, Stewart W.↗

DOE OSTI2022

Reinforcement learning for adaptive maintenance policy optimization under imperfect knowledge of the system degradation model and partial observability of system states

Maintenance policy optimization usually is faced with challenges that arise from an imperfect knowledge of system degradation models and from the partial observability of system degradation states. Here, this paper proposes a reinforcement learning method to address these two challenges for a class of maintenance problems with Markov degradation processes. The reinforcement learning approach consists of a learning component and a planning component. Using sequentially collected observations, at each step of decision-making the learning component improves the knowledge of system degradation in terms of the probability distributions of the transition rates based on sequential Bayesian inference. Using the updated transition rates, at each step of decision-making the maintenance policy optimization problem is then formulated as a partially observable Markov decision problem, and the planning component computes the optimal maintenance policy that maximizes the expected cumulative reward. The proposed method is illustrated using a numerical example with repair and inspection maintenance actions. The result shows that as more observations are collected, the learning component progressively learns the true system degradation process, and the planning component adjusts the optimal maintenance policy accordingly as well, which leads to increased reward.

42 ENGINEERING↗

DOE OSTI2023

Matching diverse feedstocks to conversion processes for the future bioeconomy

A wide variety of wasted or underutilized organic feedstocks can be leveraged to build a sustainable bioeconomy, ranging from crop residues to food processor residues and municipal wastes. Leveraging these feedstocks is both high-risk and high-reward. Converting mixed, variable, and/or highly contaminated feedstocks can pose engineering and economic challenges. However, converting these materials to fuels and chemicals can divert waste from landfills, reduce fugitive methane emissions, and enable more responsible forest management to reduce the frequency and severity of wildfires. Historically, low-value components, including ash and lignin, are poised to become valuable coproducts capable of supplementing cement and valuable chemicals. Here, we evaluate the challenges and opportunities associated with converting a range of feedstocks to renewable fuels and chemicals.

09 BIOMASS FUELS↗

DOE OSTI2020

Batteries Annual Progress Report (FY2019)

The Vehicle Technologies Office (VTO) of the Department of Energy (DOE) conducts research and development (R&D) on advanced transportation technologies that would reduce the nation’s use of imported oil and would also lead to reductions in harmful emissions. Technologies supported by VTO include electric drive components such as advanced energy storage devices (primarily batteries), power electronics and electric drive motors, advanced structural materials, energy efficient mobility systems, advanced combustion engines, and fuels. VTO is focused on funding early-stage high-reward/high-risk research to improve critical components needed for more fuel efficient (and cleaner-operating) vehicles. One of the major VTO objectives is to enable U.S. innovators to rapidly develop the next generation of technologies that achieve the cost, range, and charging infrastructure necessary for the widespread adoption of plug-in electric vehicles (PEVs). An important prerequisite for the electrification of the nation’s light duty transportation sector is development of more cost-effective, longer lasting, and more abuse-tolerant PEV batteries. One of the ultimate goals of this research, consistent with the current vehicle electrification trend, is an EV which can provide the full driving performance, convenience, and price of an internal combustion engine (ICE) vehicle. To achieve this, VTO has established the following overarching goal (Source: FY2021 Congressional Budget Justification1): …identify new battery chemistry and cell technologies with the potential to reduce the cost of electric vehicle battery packs by more than half, to less than $100/kWh (ultimate goal is $60/kWh battery cell cost), increase range to 300 miles, and decrease charge time to 15 minutes or less by 2028. VTO works with key U.S. automakers through the United States Council for Automotive Research (USCAR) – an umbrella organization for collaborative research consisting of Fiat Chrysler Automobiles (FCA), the Ford Motor Company, and General Motors. Collaboration with automakers through the partnership known as U.S. Driving Research and Innovation for Vehicle Efficiency and Energy Sustainability (U.S. DRIVE) attempts to enhance the relevance and the success potential of its research portfolio. VTO competitively selects projects for funding through funding opportunity announcements (FOAs). Directly-funded work at the national laboratories are awarded competitively through a lab-call process. During the past year, VTO continued R&D in support of PEVs. Stakeholders for VTO R&D include universities, national laboratories, other government agencies and industry (including automakers, battery manufacturers, material suppliers, component developers, private research firms, and small businesses). This document summarizes the progress of VTO battery R&D projects supported during the fiscal year 2019 (FY 2019).

25 ENERGY STORAGE↗

NASA NTRS2004

A Summer Research Program of NASA/Faculty Fellowships at the Jet Propulsion Laboratory

Albee, Arden↗

NASA NTRS2004

A Summer Research Program of NASA/Faculty Fellowships at the Jet Propulsion Laboratory

The NASA Faculty Fellowship Program (NFFP) is designed to give college and university faculty members a rewarding personal as well as enriching professional experience. Fellowships are awarded to engineering and science faculty for work on collaborative research projects of mutual interest to the fellow and his or her JPL host colleague. The Jet Propulsion Laboratory (JPL) and the California Institute of Technology (Caltech) have participated in the NASA Faculty Fellowship Program for more than 25 years. Administrative offices are maintained both at the Caltech Campus and at JPL; however, most of the activity takes place at JPL. The Campus handles all fiscal matters. The duration of the program is ten continuous weeks. Fellows are required to conduct their research on-site. To be eligible to participate in the program, fellows must be a U.S. citizen and hold a teaching or research appointment at a U.S. university or college. The American Society of Engineering Education (ASEE) contracts with NASA and manages program recruitment. Over the past several years, we have made attempts to increase the diversity of the participants in the NFFP Program. A great deal of attention has been given to candidates from minority-serving institutions. There were approximately 100 applicants for the 34 positions in 2002. JPL was the first-choice location for more than half of them. Faculty from 16 minority-serving institutions participated as well as four women. The summer began with an orientation meeting that included introduction of key program personnel, and introduction of the fellows to each other. During this welcome, the fellows were briefed on their obligations to the program and to their JPL colleagues. They were also given a short historical perspective on JPL and its relationship to Caltech and NASA. All fellows received a package, which included information on administrative procedures, roster of fellows, seminar program, housing questionnaire, directions to JPL, maps of the local area, and a copy of the JPL Universe (a JPL newsletter). A calendar of events for the 2002 NFFP Program was designed to expose the fellows to the full range of JPL activities, seminars, tours, and trips to NASA Dryden, Goldstone, and Palomar Observatory. Weekly brown-bag lunches were also scheduled. The lunches provided a time for airing problems that may have arisen during the previous week, soliciting suggestions for program enhancement, announcements, and general socializing. Professor and Mrs. Albee also hosted the annual Summer Faculty Welcome Party at their home. During their ten-week tenure at JPL, the visiting faculty carried out projects in a wide variety of JPL's science, engineering, and technology disciplines, including communication, planetary science, materials research, reliability and quality assurance, astronomy, guidance and control, and micro-sensors. At the end of the NFFP Program, all fellows were required to complete a one-page summary of their summer s work. This was in addition to any documentation required by their host organization. Distribution of the final paycheck was dependent upon submission of this one-page summary and completion of NASA's NFFP evaluation in the EdCATS system. Fellows were also asked to complete a questionnaire for JPL, which enables the program administrators to make any appropriate changes to make the program more beneficial and effective for all involved. The 2002 NFFP Program at JPUCaltech was considered unanimously highly successful by both fellows and JPL colleagues. It provided a significant experience to most faculty members and fresh ideas to JPL researchers. Each year, suggestions for improvement include expansion of the program, longer terms, larger stipends, funds to support graduate students, and funds to continue collaborative research. The NASA Faculty Fellowship Program continues to occupy a significant place in JPL programs and serves to strengthen the ties between NASA, JPL, Caltech, and t academic community. This program is an important part of NASA's commitment to education. No inventions or patents were created during this program.

Albee, Arden↗

NASA NTRS2003

Theory of Collective Intelligence

In this chapter an analysis of the behavior of an arbitrary (perhaps massive) collective of computational processes in terms of an associated "world" utility function is presented We concentrate on the situation where each process in the collective can be viewed as though it were striving to maximize its own private utility function. For such situations the central design issue is how to initialize/update the collective's structure, and in particular the private utility functions, so as to induce the overall collective to behave in a way that has large values of the world utility. Traditional "team game" approaches to this problem simply set each private utility function equal to the world utility function. The "Collective Intelligence" (COIN) framework is a semi-formal set of heuristics that recently have been used to construct private utility. functions that in many experiments have resulted in world utility values up to orders of magnitude superior to that ensuing from use of the team game utility. In this paper we introduce a formal mathematics for analyzing and designing collectives. We also use this mathematics to suggest new private utilities that should outperform the COIN heuristics in certain kinds of domains. In accompanying work we use that mathematics to explain previous experimental results concerning the superiority of COIN heuristics. In that accompanying work we also use the mathematics to make numerical predictions, some of which we then test. In this way these two papers establish the study of collectives as a proper science, involving theory, explanation of old experiments, prediction concerning new experiments, and engineering insights.

Nash equilibrium↗

DOE OSTI2025

Entanglement engineering of optomechanical systems by reinforcement learning

Entanglement is fundamental to quantum information science and technology, yet controlling and manipulating entanglement—so-called entanglement engineering—for arbitrary quantum systems remains a formidable challenge. There are two difficulties: the fragility of quantum entanglement and its experimental characterization. We develop a model-free deep reinforcement-learning (RL) approach to entanglement engineering, in which feedback control together with weak continuous measurement and partial state observation is exploited to generate and maintain desired entanglement. We employ quantum optomechanical systems with linear or nonlinear photon–phonon interactions to demonstrate the workings of our machine-learning-based entanglement engineering protocol. In particular, the RL agent sequentially interacts with one or multiple parallel quantum optomechanical environments, collects trajectories, and updates the policy to maximize the accumulated reward to create and stabilize quantum entanglement over an arbitrary amount of time. The machine-learning-based model-free control principle is applicable to the entanglement engineering of experimental quantum systems in general.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2022

The Necessity of International Particle Physics Opportunities for American Education

This Snowmass2021 Contributed Paper addresses the role of the Particle Physics community in creating and fostering international connections in American education. It describes the pressing need to introduce students and faculty to the challenges and rewards of international collaboration, not only to develop the next generation of scientists and engineers for particle physics, but to maintain and build U.S. leadership on an increasingly competitive world stage. We present and assess current efforts in education and public engagement with an eye toward identifying those activities in need of change or increased resources to improve audience reach and program efficacy. We also consider possible new activities that might improve upon or complement existing programs, with the common goal of providing all U.S. students with the opportunity to benefit from a quality international scientific experience.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

DOE OSTI2021

Center for direct catalytic conversion of biomass to biofuels (Final Technical Report)

Lignocellulosic biomass has only one-third the energy density of crude oil and lacks petroleum’s versatility as a feedstock for fuels and chemicals. Since 2009, the Center for direct Catalytic Conversion of Biomass to Biofuels (C3Bio) has recognized the potential of chemical catalysis and fast-pyrolysis to overcome such limitations by transforming the main components of biomass (cellulose, xylan, and lignin) from grasses and trees directly to liquid hydrocarbons and aromatic co-products. In 2014, C3Bio proposed to develop critical systems-level understanding of how biomass structural complexity at molecular, nanoscale, and mesoscale levels impacts the yields and selectivities of desired products from catalytic and pyrolytic transformations. Our long-term goal was to gain unprecedented control of effective routing of carbon: we aimed to specify both the structures within, and the reaction products from, lignocellulosic biomass. Enabled by the EFRC high-risk, high-reward approach to grand challenge science, our interdisciplinary team of plant biologists, chemists, and chemical engineers disrupted the conventional paradigm of the cellulosic biorefinery into a new future of “no carbon left behind” - the full utilization of carbon from plant cell walls in energy-dense fuels.

09 BIOMASS FUELS↗

NASA NTRS2013

JPL Community View on Challenges and Rewards of MBSE

No abstract available

SysML↗

NASA NTRS2007

Hubble 2006: Science Year in Review

The 10 science articles selected for this years annual science report exemplify the range of Hubble research from the Solar System, across our Milky Way, and on to distant galaxies. The objects of study include a new feature on Jupiter, binaries in the Kuiper Belt, Cepheid variable stars, the Orion Nebula, distant transiting planets, lensing galaxies, active galactic nuclei, red-and-dead galaxies, and galactic outflows and jets. Each narrative strives to construct the readers understanding of the topics and issues, and to place the latest research in historical, as well as scientific, context. These essays reveal trends in the practice of astronomy. More powerful computers are permitting astronomers to study ever larger data sets, enabling the discovery of subtle effects and rare objects. (Two investigations created mosaic images that are among the largest produced to date.) Multiwavelength data sets from ground-based telescopes, as well as other great observatories Spitzer and Chandraare increasingly important for holistic interpretations of Hubble results. This yearbook also presents profiles of 12 individuals who work with Hubble, or Hubble data, on a daily basis. They are representative of the many students, scientists, engineers, and other professions who are proudly associated with Hubble. Their stories collectively communicate the excitement and reward of careers related to space science and technology.

Hubble space telescope↗

NASA NTRS2008

Designing the Ares I Crew Launch Vehicle Upper Stage Element and Integrating the Stack at NASA's Marshall Space Flight Center

Fielding an integrated launch vehicle system entails many challenges, not the least of which is the fact that it has been over 30 years since the United States has developed a human-rated vehicle - the venerable Space Shuttle. Over time, whole generations of rocket scientists have passed through the aerospace community without the opportunity to perform such exacting, demanding, and rewarding work. However, with almost 50 years of experience leading the design, development, and end-to-end systems engineering and integration of complex launch vehicles, NASA's Marshall Space Flight Center offers the in-house talent - both junior- and senior-level personnel - to shape a new national asset to meet the requirements for safe, reliable, and affordable space exploration solutions.' These personnel are housed primarily in Marshall's Engineering Directorate and are matrixed into the programs and projects that reside at the rocket center. Fortunately, many Apollo era and Shuttle engineers, as well as those who gained valuable hands-on experience in the 1990s by conducting technology demonstrator projects such as the Delta-Clipper Experimental Advanced, X-33, X-34, and X-37, as well as the short-lived Orbital Space Plane, work closely with industry partners to advance the nation's strategic capability for human access to space. Currently, only three spacefaring nations have this distinction, including the United States, Russia, and, more recently, China. The U.S. National Space Policy of2006 directs that NASA provide the means to travel to space, and the NASA Appropriations Act of2005 provided the initial funding to begin in earnest to replace the Shuttle after the International Space Station construction is complete in 20 IO? These and other strategic goals and objectives are documented in NASA's 2006 Strategic Plan.3 In 2005, a team of NASA aerospace experts conducted the Exploration Systems Architecture Study, which recommended a two-vehicle approach to America's next space transportation system for missions to the International Space Station in the next decade and to explore the Moon and establish an outpost around the 2020 timeframe.4 Based on this extensive study, NASA selected the Ares I crew launch vehicle configuration and the heavy-lift Ares V cargo launch vehicle (fig 1). This paper will give an overview of NASA's approach to integrating the Ares I vehicle stack using capabilities and assets that are resident in Marshall's Engineering Directorate, working in partnership with other NASA Centers and the U.S. aerospace industry. It also will provide top-level details on the progress of the in-house design of the Ares I vehicle's upper stage element.

Lyles, Garry↗