Search NASA⌕ Search

SEARCH · Search NASA

Results for “algorithm testing”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 559 records · Page 31
All sourcesNASA NTRSDOE OSTI
NASA NTRSDate not supplied

Predicting Airport Runway Configurations for Decision-Support Using Supervised Learning

One of the most challenging tasks for air traffic controllers is runway configuration management (RCM). It deals with the optimal selection of runways to operate on (for arrivals and departures) based on traffic, surface wind speed, wind direction, other environmental variables, noise constraints, and several other airport-specific factors. It affects the efficiency of the National Airspace System (NAS) and both surface and airspace operations can benefit from better understanding future runway configurations. In this paper, we present a comprehensive implementation of predictive models for runway configuration estimation from large volumes of historical data. Specifically, operational data from two full years (2018 and 2019) is collected, analyzed, and fused together to build the data product used in this work. The data set differs from prior work in the field in terms of its scope, resolution, and variety of factors collected and considered. Meteorological data is collected from two different sources – current weather conditions from METAR (Meteorological Terminal Aviation Routine Weather Report) and forecast weather conditions from Localized Aviation MOS Program (LAMP). Operational data from the Federal Aviation Administration (FAA) Aviation System Performance Metrics (ASPM) related to scheduled and actual number of arrivals and departures, average taxi times, etc. are collected. NASA’s Sherlock Data Warehouse is used to identify critical information such as go-arounds, and other events that might impact RCM decision-making. All data is collected and aggregated over 15-minute intervals throughout the two years. This provides a resolution like the timescales that might be necessary for runway configuration management decision-making. A variety of supervised learning algorithms are tested including Support Vector Machine, Random Forest, Gradient Boosting, etc. including tuning of the model hyperparameters. The modeling process is applied and presented on two representative U.S. airports – Charlotte Douglas International Airport (KCLT) and Denver International Airport (KDEN). The two airports present different levels of complexity in terms of the total number of configurations used and provide a balanced perspective on the generalizability of the developed approach to other airports in the NAS. Initial results are promising (F1 score of 0.91 at KCLT and 0.83 at KDEN) for data in the test set. The final paper will contain a comprehensive comparison between different models and model building strategies as well as further refined results. Most important predictors for each airport will be identified along with a discussion and recommendations on adapting the framework to other scenarios.

Tejas G Puranik↗
NASA NTRSDate not supplied

Discovery and Analysis of Rare High-Impact Failure Modes using Adversarial RL-Informed Sampling

Adaptive learning agents have tremendous potential to handle critical tasks currently performed by humans. Unfortunately, due to their complexity, it can be difficult to verify that these learning agents do not have critical failure modes. Standard verification and validation methods often do not apply directly to learning agents and Monte Carlo methods have difficulty covering even a small fraction of the state space, especially in multiagent systems or over long time horizons. To overcome this difficulty, we demonstrate an adaptive stress-testing method based on reinforcement learning of correlations that raise the probability of failure. This approach has three key properties: (1) it is able to find rare failure modes with far greater sample efficiency than Monte Carlo methods, (2) it can estimate the true probability of a failure mode despite the inherent bias in the learning method, and (3) it is capable of learning and resampling compact representations of multimodal failure spaces. These properties are important in practice as we need to find disparate failure modes while accounting for their actual relevance. This is a significant advantage over traditional adaptive stress testing methods that give abstract likelihoods of particular failure instances, but cannot estimate the probability of a broader failure mode. We test our algorithm on a simple problem from the aviation domain where an autonomous aircraft lands in gusty wind conditions. The results suggest that we can find failure modes with far fewer samples than the Monte Carlo approach and simultaneously estimate the probability of failure.

reinforcement learning↗
NASA NTRSDate not supplied

CyanoSCape: Freshwater Phytoplankton and Floating Aquatic Vegetation Biodiversity

In Southern Africa, the impacts of anthropogenic activities on biodiversity and ecosystem services are exacerbated by the climate crisis. Rapid land use change and the lack of emphasis on environmentally sustainable agricultural practices has hindered hydrological processes and compromised riverine and aquatic ecosystems. This poses obvious risks to natural/indigenous aquatic biodiversity and long-term ecosystem sustainability. Phytoplankton serve as the foundation of the freshwater food web with zooplankton as consumers, which feed fish, invertebrates, and so on up the food chain that comprises the biodiversity of the freshwater system that serves as habitat for biodiversity as well. The diversity of phytoplankton (microscopic organisms) includes photosynthesizing bacteria (cyanobacteria), plant-like diatoms, dinoflagellates, and green algae. Nutrient run-off from agricultural fertilizers and urban overflows, warm temperatures, abundant light availability and compromised hydrological systems provide an ideal environment for cyanobacteria to flourish. Increased prevalence of cyanobacteria, due to both natural and anthropogenic causes, can incur significant effects on the biodiversity of the overall phytoplankton assemblage. These bloom forming algae can significantly outcompete other phytoplankton classes in warmer and eutrophic waters where they are quick to dominate the freshwater system. Eutrophication and toxic cyanobacteria blooms (cyanoHABs) in the inland waters of the Greater Cape Floristic Region (GCFR) incur significant effects on the biodiversity of the overall phytoplankton assemblage and provide a favorable environment forthe overgrowth of floating aquatic vegetation (FAV), which is often invasive and associated with reduced aquatic biodiversity. The algal biodiversity of the GCFR’s freshwater systems is not well characterized. Hyperspectral optical observations are expected to facilitate the improvement of current phytoplankton functional type retrievals significantly, as the sensitivity is sufficient that the distinctive, fine spectral features of different phytoplankton groups can be detected. This will enable testing emerging algorithms and inform the development of new algorithms for use with upcoming hyperspectral satellite missions in this decade. Innovations in optical sensor sensitivity and next generation machine learning capabilities considerably enhance the potential for accurate and rapid detection of phytoplankton, namely the presence, extent, and diversity of cyanobacteria present in cyanoHABs and additionally, invasive FAV. Upcoming hyperspectral satellite missions such as NASA’s Surface Biology and Geology (SBG), Plankton, Aerosol, Cloud, ocean Ecosystem (PACE), and the European Space Agency’s Copernicus Hyperspectral Imaging Mission (CHIME) will provide imagery with unprecedented spectral and spatial resolution which will further enable the discovery of linkages between the seasonality and dynamics of HABs and FAV. The overarching goal of this project is to utilize hyperspectral data, with recently developed and next-generation algorithms, to determine the biodiversity of freshwater systems phytoplankton assemblage with emphasis on genus level distinction, as well as monitor the prevalence and diversity of FAV.

CyanoSCape↗
NASA NTRSDate not supplied

Discovery and Analysis of Rare High-Impact Failure Modes using Adversarial RL-Informed Sampling

Adaptive learning agents have tremendous potential to handle critical tasks currently performed by humans. Unfortunately, due to their complexity, it can be difficult to verify that these learning agents do not have critical failure modes. Standard verification and validation methods often do not apply directly to learning agents and Monte Carlo methods have difficulty covering even a small fraction of the state space, especially in multiagent systems or over long time horizons. To overcome this difficulty, we demonstrate an adaptive stress-testing method based on reinforcement learning of correlations that raise the probability of failure. This approach has three key properties: (1) it is able to find rare failure modes with far greater sample efficiency than Monte Carlo methods, (2) it can estimate the true probability of a failure mode despite the inherent bias in the learning method, and (3) it is capable of learning and resampling compact representations of multimodal failure spaces. These properties are important in practice as we need to find disparate failure modes while accounting for their actual relevance. This is a significant advantage over traditional adaptive stress testing methods that give abstract likelihoods of particular failure instances, but cannot estimate the probability of a broader failure mode. We test our algorithm on a simple problem from the aviation domain where an autonomous aircraft lands in gusty wind conditions. The results suggest that we can find failure modes with far fewer samples than the Monte Carlo approach and simultaneously estimate the probability of failure.

Validation↗
NASA NTRSDate not supplied

Discovery and Analysis of Rare High-Impact Failure Modes using Adversarial RL-Informed Sampling

Adaptive learning agents have tremendous potential to handle critical tasks currently performed by humans. Unfortunately, due to their complexity, it can be difficult to verify that these learning agents do not have critical failure modes. Standard verification and validation methods often do not apply directly to learning agents and Monte Carlo methods have difficulty covering even a small fraction of the state space, especially in multiagent systems or over long time horizons. To overcome this difficulty, we demonstrate an adaptive stress-testing method based on reinforcement learning of correlations that raise the probability of failure. This approach has three key properties: (1) it is able to find rare failure modes with far greater sample efficiency than Monte Carlo methods, (2) it can estimate the true probability of a failure mode despite the inherent bias in the learning method, and (3) it is capable of learning and resampling compact representations of multimodal failure spaces. These properties are important in practice as we need to find disparate failure modes while accounting for their actual relevance. This is a significant advantage over traditional adaptive stress testing methods that give abstract likelihoods of particular failure instances, but cannot estimate the probability of a broader failure mode. We test our algorithm on a simple problem from the aviation domain where an autonomous aircraft lands in gusty wind conditions. The results suggest that we can find failure modes with far fewer samples than the Monte Carlo approach and simultaneously estimate the probability of failure.

Validation↗
NASA NTRSDate not supplied

Predicting Airport Runway Configurations for Decision-Support Using Supervised Learning

One of the most challenging tasks for air traffic controllers is runway configuration management (RCM). It deals with the optimal selection of runways to operate on (for arrivals and departures) based on current and forecast of traffic, surface wind speed, wind direction, other environmental variables, noise constraints, and several other airport-specific factors. In this paper, a methodology using supervised learning is developed to build a predictive model for RCM decision-support from large volumes of historical data. Data from two full years (2018 and 2019) related to current and forecast weather, demand/capacity, etc. is collected, analyzed, and fused together. A variety of supervised learning algorithms are tested for predicting runway configuration and hyperparameter tuning is carried out to select the best performing model. The validation process involves two airports of low (Charlotte Douglas International Airport, CLT) and high (Denver International Airport, DEN) complexity of configuration decision-making. The results show significant promise for the two airports with test accuracy of 93% (CLT) and 73% (DEN). The methodology is scalable and generalizable to other airports across the U.S. National Airspace System.

air traffic management↗
NASA NTRSDate not supplied

Predicting Airport Runway Configuration for Decision-Support Using Supervised Learning

One of the most challenging tasks for air traffic controllers is runway configuration management (RCM). It deals with the optimal selection of runways to operate on (for arrivals and departures) based on current and forecast of traffic, surface wind speed, wind direction, other environmental variables, noise constraints, and several other airport-specific factors. In this paper, a methodology using supervised learning is developed to build a predictive model for RCM decision-support from large volumes of historical data. Data from two full years (2018 and 2019) related to current and forecast weather, demand/capacity, etc. is collected, analyzed, and fused together. A variety of supervised learning algorithms are tested for predicting runway configuration and hyperparameter tuning is carried out to select the best performing model. The validation process involves two airports of low (Charlotte Douglas International Airport, CLT) and high (Denver International Airport, DEN) complexity of configuration decision-making. The results show significant promise for the two airports with test accuracy of 93% (CLT) and 73% (DEN). The methodology is scalable and generalizable to other airports across the U.S. National Airspace System.

air traffic management↗
NASA NTRSDate not supplied

Use of TEMPO as a Proxy for Hyperspectral Geostationary Ocean Color Measurements from the GeoXO OCX Instrument: Harnessing Machine Learning and Principal Component Techniques for Atmospheric and Glint Correction

Retrievals of ocean color from space are important for better understanding the ocean ecosystem. The launch of atmospheric geostationary hyperspectral sensors such as TEMPO, provides a unique opportunity to examine the diurnal variability in ocean ecology. While TEMPO does not have as high spatial resolution or full spectral coverage as planned coastal ocean sensors such as the Geosynchronous Littoral Imaging and Monitoring Radiometer (GLIMR) or GeoXO Ocean Color instrument (OCX), its hourly measurements provide coverage of regions such as Lake Erie and the Gulf of Mexico at spatial scales of approximately 5 km. These data can be useful for testing new algorithms. We will apply our newly developed machine learning based atmospheric correction approach for ocean color retrievals to TEMPO data. Our approach begins by decomposing measured radiances from hyperspectral sensors into spectral features that describe the scattering and absorption of the atmosphere as well as the underlying surface reflectance. The coefficients of the principal components are then used to train a neural network to predict ocean color properties derived from collocated MODIS/VIIRS physically-based retrievals. This machine learning approach does not rely on radiative transfer modeling, and the use of MODIS/VIIRS data for training accounts for possible calibration b in hyperspectral data. Previously, we applied our approach using blue and UV wavelengths with the Ozone Monitoring Instrument (OMI) and TROPOspheric Monitoring Instrument (TROPOMI) to show that it can estimate ocean color properties in less-than-ideal conditions such as lightly to moderately clouded conditions as well as sun glint and thus improve the spatial coverage of ocean color measurements. TEMPO provides an opportunity to improve on this approach since it will provide collocated measurements at green and red wavelengths that were not available from OMI and TROPOMI and are important particularly for coastal waters. Additionally, our technique can be applied early in the mission and has potential to demonstrate the value of near real time ocean color products that are important for monitoring of harmful algae blooms and other oceanic phenomena.

Zachary Fasnacht↗
NASA NTRS2024

Summary of Model-based Attitude Control of LUVOIR in Modular Dynamic Analysis (MDA) Simulink Environment

This document overviews the work I completed in the second half of my summer 2018 internship experience in Code 591 at NASA Goddard Space Flight Center. Please see the former memorandum for additional context on the project. The take away point is that LUVOIR A has been modeled as three rigid bodies linked by 1-DOF rotary joints. An LQR attitude controller was designed for precision pointing of the spacecraft with the design requirement that steady state oscillations have amplitude less than a miliarcsecond. The performance of the algorithm was tested in the Modular Dynamic Analysis Simulink library developed by J. Roger Chen at NASA Goddard. While the initial simulations were of rigid bodies, subsequent simulations included flexible body modes on all three bodies of the model. The simulated structural deformations initially destabilized the LQR controller thus requiring a redesign of the K gain matrix. The final simulation demonstrated a stabilizing feedback gain with miliarcsecond precision within 400 minutes. This run used 20 modes on the spacecraft, 9 on the bus, and 100 on the payload. Future recommended work includes the development of a Vibration Isolation and Precision Pointing System (VIPPS) Simulink model. Armed with this model, the system should be modeled as four bodies whose associated flexible files must be generated from the spacecraft through Gimbal 1, Gimbal 1 through Gimbal 2, Gimbal 2 through the VIPPS interface, and the VIPPS interface through the payload.

William Bentz↗
DOE OSTI2019

Hierarchical Management of Distributed Energy Resources Using Chance-Constrained OPF and Extremum Seeking Control: Preprint

Distributed energy resources (DERs) are becoming an important part of distribution systems because of their economic and environmental benefits. Although their inherent intermittency and volatility introduce uncertainties into the system, they have potential to provide controllability to the system under proper coordination. In this paper, we propose a hierarchical control algorithm for distribution systems with DERs so that they have controllability similar to that of a generator bus. The upper level scheduler solves a chance-constrained optimal power flow (OPF) problem to plan the operation of the DERs based on forecasts, and the lower level distributed DER controllers leverage the extremum seeking approach to deliver the planned power at the feeder head. The proposed algorithm is tested on a modified IEEE 13-node feeder, demonstrating its effectiveness.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2019

Hierarchical Management of Distributed Energy Resources Using Chance-Constrained OPF and Extremum Seeking Control

Distributed energy resources (DERs) are becoming an important part of distribution systems, because of their economic and environmental benefits. Although their inherent intermittency and volatility introduce uncertainties into the electric power system, they have the potential to provide controllability to the system under proper coordination. In this paper, we propose a hierarchical control algorithm for distribution systems with DERs so that they have controllability similar to a generator bus. The upper level scheduler solves a chance-constrained optimal power flow (OPF) problem to plan the operation of the DERs based on forecasts, and the lower level distributed DER controllers leverage the extremum seeking approach to deliver the planned power at the feeder head. The proposed algorithm is tested on a modified IEEE 13-node feeder, demonstrating its effectiveness.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2020

How Should Machine Learning Be Successfully Used for Wind Speed Vertical Extrapolation?

An accurate characterization of the wind resource available at hub-height is required for an efficient and bankable wind farm project. However, direct measurement of wind speed at the constantly increasing height of the hub of commercial wind turbines is oftentimes challenging and expensive, so that it is common practice to vertically extrapolate the wind resource from lower and more easily accessible levels. Conventional techniques for wind speed vertical extrapolation include the use of a power law and a logarithmic profile. While simple, the limits in accuracy of these methods have been shown in various studies. Recently, machine learning has been proposed as a new method to vertically extrapolate winds. All the published studies on the topic assess the performance of machine learning techniques in vertically extrapolating the wind resource at the same location where the algorithm has been trained. However, in real-world applications, the wind resource is measured at the instrument location, but it then needs to be extrapolated at hub height at the location of the wind turbines within the find farm. To be able to fully recommend the use of machine learning techniques over the simple power law and logarithmic law, the spatial variability of the performance improvements of the machine learning approaches needs to be assessed. Here, we propose a round-robin validation of a machine learning-based method for wind speed extrapolation. We use 20 months of observations at four locations spanning a 100 km wide region at the Southern Great Plains (SGP) atmospheric observatory, in north-central Oklahoma. At each location, we train a random forest to predict 30-min average wind speed at 143 m AGL. We use as input features lidar wind speed at 65 m AGL, time of day, sonic anemometer wind speed at 4 m AGL, turbulent kinetic energy, and Obukhov length. First, we perform a same-site comparison of the performance of the proposed random forest against the conventional techniques for wind speed extrapolation (namely power law and logarithmic profile, with widely accepted stability corrections). We find that the random forest outperforms the power law in vertically extrapolating wind speed in all the considered stability regimes, with a 33% reduction in MAE for stable conditions, and a 31% reduction in unstable conditions. Similar results are found when comparing predictions of extrapolated winds from the logarithmic profile and the random forest with the observed values. Next, we propose a round-robin validation, to use the random forest trained at each site to extrapolate wind speed at the remaining three sites. We find that the performance of the random forest approach degrades when the algorithm is tested at a site different than the training one. However, even under those circumstances, the machine learning-based approach still outperforms the conventional techniques for wind speed extrapolation, with, on average, a reduction in mean absolute error between 15 and 20% over the conventional methods, with the largest benefits obtained under stable conditions.

Monte Carlo↗
DOE OSTI2021

An Orthogonal Recursive Bisection (ORB) Based Time Advancement Algorithm for CFD-DEM Solvers

The time integration of the granular phase in coupled computational fluid dynamics (CFD) – discrete element method (DEM) simulations presents a unique computational challenge brought about by the large variations in particle collisional time scales. Particles in the dilute regions of the computational domain can be advanced with large time steps while dense regions require much smaller time increments. However, the time step size in most solvers is globally set as the limit for accuracy and stability imposed by the collisions and is typically orders of magnitude less than that required away from collisions. This work addresses this precise issue and provides a strategy to avoid the use of a global conservative small time step size for the entire set of particles.A novel time stepping algorithm for CFD-DEM solvers using a partitioning approach using orthogonal recursive bisection (ORB) that allows for variable time steps among particles is described and its computational performance is compared against baseline explicit methods, typically used in several CFD-DEM solvers. ORB has advantages of being relatively quick and easy to update incrementally and has the required heuristic behavior (i.e., it will split the region in half with a cluster on each side) when groups of particles are well separated (clustered). The algorithm presented in this work uses a local time stepping approach to resolve collisional time scales for subsets of particles that are present at the leaves of the ORB, thereby resulting in substantial reduction of computational cost. The parallel implementation of this method where a ``knapsack” algorithm is used in tandem with ORB for effective load-balancing is also presented, where a best possible partitioning is obtained based on number of particles and local time-stepping costs. The algorithm is tested against benchmark problems with varying particle distributions that include fluidized bed and riser flow scenarios. Preliminary results indicate that the approach is 2-3X faster than traditional explicit methods for problems that involve both dense and dilute regions, while maintaining the same level of accuracy.

adaptive timestepping↗
DOE OSTI2021

Material transfers detection with seismic observations

We are exploring the use of data from a seismic network around a research nuclear reactor and isotope production facility at Oak Ridge National Laboratory to study activity patterns related to the transfer of nuclear material. A sensor network with eight seismometers was installed around the High Flux Isotope Reactor and the Radiochemical Engineering Development Center and started operating in October 2019. These data are used to detect and characterize operational events around facility. We are using those data to extract signals related to the movement of vehicles involved in the transport of nuclear materials (e.g., transportation of reactor fuel, targets, and product isotopes). Large vehicles produce mechanical energy that can be observed as seismic signals, either as the result of sound emanating from the vehicle engine or by the generation of surface waves as a response of the ground to the load of the vehicles. In particular, the station located near the entrance gate of the facility displays a clear seismic signature as vehicles cross a metal platform on the ground. This signal is characterized by sharp energy bursts that correspond to the number of axles on the vehicle. We developed and are testing an algorithm to identify sequences of energy bursts to count and characterize vehicles of different sizes. We are also exploring the use of seismic polarization analysis to measure the degree of polarization of the observed signals. These two techniques for detection and characterization of vehicles used for nuclear material transfers will be validated with seismic data from a targeted collection using different vehicles and routes similar to the ones used in real-life scenarios at the High Flux Isotope Reactor.

Marcillo, Omar↗
DOE OSTI2022

Scaling Building Energy Audits through Machine Learning Methods on Novel Drone Image Data

Building energy audits are time-consuming and labor-intensive. This paper describes a new method using machine learning (ML) techniques on novel data sources (drone images) to improve the identification of building characteristics and retrofit opportunities, and thereby reduce the effort for audits. The new ML method includes: (1) Building footprint extraction using line extraction, polygonization, and polygon-merging, (2) Building envelope extraction using PIX4d modeling software to reconstruct a building 3D model, (3) Visualization tool for viewing images from the 3D model, (4) Window-to-wall ratio (WWR) using state-of-art deep neural network semantic segmentation, (5) Envelope thermal anomaly detection using an unsupervised machine learning clustering algorithm, and (6) Rooftop energy equipment detection based on an object detection algorithm. The testing of this method involved a comparison of additional ML-generated information overlaid on current ‘state-of-practice’ audit and remote assessment baselines using evaluation metrics: labor time and associated cost, marginal benefits of using ML-generated information in workflows for audits and remote assessments, integration potential with existing processes and tools, and replicability/scalability of the method. In two test buildings in California that had comprehensive drawings and meter data available, the ML method effectively generated a building footprint, envelope, rooftop equipment, WWR, and locations of envelope thermal anomalies. Projected target segments of the ML method are sites with minimal drawings and energy data, and underserved sectors such as multistoried housing, disadvantaged communities, and schools for which the ML method can enable identification of building asset characteristics and prioritization of envelope retrofits and decentralized energy equipment retrofits.

Singh, Reshma↗
DOE OSTI2023

Efficient Network Partitioning: Application for Decentralized State Estimation in Power Distribution Grids: Preprint

Increase in the proliferation of DERs requires real-time situational awareness for efficient grid operations. State estimation plays an important role for real time control and management of the power grid. As the sensing infrastructure grows, aggregating and handling high volumes of data at a centralized location is extremely difficult. To address this challenge, this paper first proposes a novel and efficient hierarchical spectral clustering-based network partition algorithm followed by a decentralized compressive sensing (DCS) based state estimation. The applicability of the proposed network partitioning algorithm is tested on IEEE-123 bus, IEEE-8500 node, and a 6204-node distribution network. The results shows that the proposed approach efficiently divides the network into multiple sub-networks with the minimum edge connections among the neighbors. Then, we perform DCS-based state estimation on the 6204-node distribution network after dividing the network into 18 optimal partitions. Simulation results show that DCS-based state estimation recovers the system states with high accuracy and low complexity.

ADMM↗