Search NASA⌕ Search

SEARCH · Search NASA

Results for “distributed algorithms”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 577 records · Page 32
All sourcesNASA NTRSDOE OSTI
NASA NTRS1994

Optimal pattern distributions in Rete-based production systems

Since its introduction into the AI community in the early 1980's, the Rete algorithm has been widely used. This algorithm has formed the basis for many AI tools, including NASA's CLIPS. One drawback of Rete-based implementation, however, is that the network structures used internally by the Rete algorithm make it sensitive to the arrangement of individual patterns within rules. Thus while rules may be more or less arbitrarily placed within source files, the distribution of individual patterns within these rules can significantly affect the overall system performance. Some heuristics have been proposed to optimize pattern placement, however, these suggestions can be conflicting. This paper describes a systematic effort to measure the effect of pattern distribution on production system performance. An overview of the Rete algorithm is presented to provide context. A description of the methods used to explore the pattern ordering problem area are presented, using internal production system metrics such as the number of partial matches, and coarse-grained operating system data such as memory usage and time. The results of this study should be of interest to those developing and optimizing software for Rete-based production systems.

Scott, Stephen L.↗
NASA NTRSDate not supplied

Shock-Layer Radiation Insights Available Through Flowfield Property Binning

This paper introduces a method for assessing similarity between available shock-layer radiation measurements and the radiative heating to a vehicle of interest. This method will be referred to as the flowfield property binning (FPB) approach. The challenge in developing this approach is that radiometer measurements from a surface normal line of sight, observed radiation measurements of the entire shock-layer from a distance, or measured shock-tube nonequilibrium profiles are notably different geometrically than the radiative flux to a vehicle surface. Therefore, the relative contributions of various regions of the shock-layer to the measurement, and how these relative contributions compare with the radiative heating to a vehicle of interest, are unclear. By tracking the radiation contribution from each flowfield location, and then grouping this contribution based on two local flowfield properties, these geometric differences are removed. This provide a two dimensional group, or bin, distribution that represents the radiation environment in the same format for various types of radiation simulations, such as observed radiation or radiative flux to a vehicle surface. In the limits of optically-thin radiation transport and thermochemical equilibrium flow, the choice of pressure and temperature as the binning flowfield properties is shown to provide a bin distribution that uniquely defines the radiation simulation. This concept is found to be sufficient for establishing similarity between two cases even when the optically-thin radiation transport and thermochemical equilibrium assumptions are not satisfied. A simple metric of comparison between bin distributions from different cases is provided by the histogram intersection algorithm. This algorithm results in a histogram intersection factor (H) between 0 and 1, which represents the fraction of the radiation that is similar between the two cases. Therefore, if H evaluated between bin distributions for two completely different radiation problems is near 1, then they represent essentially the same radiation problem. This means that they depend on the same weighting of various radiation modeling parameters evaluated at the same pressure and temperature. If one of these cases agrees well with a radiation measurement, then the matching bin distributions provide justification for applying this experimental validation to the other case. Even for H values as low as 0.2, the measurement case could be considered relevant if there are no other useful measurements. This approach allows observed radiation measurements to directly inform radiative heating simulations, which allows NASA’s past and future investments in observed radiation measurements to directly inform the radiative heating uncertainty or margins for future vehicle design. This FPB approach is also used to determine shock-tube conditions for informing backshell radiative heating simulations, where the normal shock velocity and Lagrangian time from the shock is used for the binning variables instead of pressure and temperature.

Shock Layer Radiation↗
NASA NTRS1999

In Pursuit of Abundance Anomalies and the FIP Effect in Late-Type Stellar Coronae

Key EUVE and ASCA data have been retrieved from their respective archives. New software has been written in the IDL language to carry out data analysis and to interface with the relevant atomic physics databases. During the analysis of ASCA spectra, it was found that the abundances of elements other than Fe could not be constrained very well, and Fe abundances were not constrained unless the underlying emission measure distribution model was reasonably well-determined. Consequently, the study has concentrated on the quantity Fe/H. A method has been developed as a means of deriving Fe/H based on fitting the continuum to EUVE spectra, thereby using the Fe lines to determine the Fe abundance. A Monte Carlo Markov Chain algorithm was also developed to determine the emission measure distribution based on observed spectral lines. This is the first application of this type of monte carlo approach to this scientific problem. This work has resulted in three scientific publications, one of which published, one of which is now ready for submission and the other of which is still in preparation:

Drake, Jeremy↗
DOE OSTI2025

Decentralized Distributed Proximal Policy Optimization (DD-PPO) for High Performance Computing Scheduling on Multi-User Systems

Resource allocation in High Performance Computing (HPC) environments presents a complex and multifaceted challenge for job scheduling algorithms. Beyond the efficient allocation of system resources, schedulers must account for and optimize multiple performance metrics, including job wait time and system throughput. Traditional heuristic-based scheduling algorithms increasingly struggle and lack the efficiency needed to meet the demands and address the complexity and scale of modern HPC systems. Consequently, recent research efforts have focused on leveraging advancements in Artificial Intelligence (AI) and Deep Learning (DL), particularly Reinforcement Learning (RL), to develop more adaptable and intelligent scheduling strategies. Previous RL-based scheduling approaches have explored a range of algorithms, from Deep Q-Networks (DQN) to Proximal Policy Optimization (PPO), and more recently, hybrid methods that integrate Graph Neural Networks (GNNs) with RL techniques. However, a common limitation across these methods is their reliance on relatively small datasets, with few methods being evaluated using large-scale, multi-million-job trace datasets representative of real-world HPC workloads. Moreover, existing RL schedulers face scalability issues due to centralized policy updates, which hinder training efficiency and performance when applied to large datasets. This study introduces a novel RL-based scheduler utilizing Decentralized Distributed Proximal Policy Optimization (DD-PPO) algorithm, which supports large-scale distributed training across multiple workers without requiring parameter synchronization at every step. By eliminating reliance on centralized updates to a shared policy, the DD-PPO scheduler enhances scalability, training efficiency, and sample utilization. Experimental validation using a large real-world dataset containing over 11.5 million job traces collected from petascale HPC systems over six years assesses the influence of dataset scale on training effectiveness and compares DD-PPO performance to traditional and advanced scheduling approaches. The experimental results demonstrate improved scheduling performance in comparison to both heuristic-based schedulers and existing RL-based scheduling algorithms.

AI↗
NASA NTRS2005

A Comparative Study of Probability Collectives Based Multi-agent Systems and Genetic Algorithms

We compare Genetic Algorithms (GA's) with Probability Collectives (PC), a new framework for distributed optimization and control. In contrast to GA's, PC-based methods do not update populations of solutions. Instead they update an explicitly parameterized probability distribution p over the space of solutions. That updating of p arises as the optimization of a functional of p. The functional is chosen so that any p that optimizes it should be p peaked about good solutions. The PC approach works in both continuous and discrete problems. It does not suffer from the resolution limitation of the finite bit length encoding of parameters into GA alleles. It also has deep connections with both game theory and statistical physics. We review the PC approach using its motivation as the information theoretic formulation of bounded rationality for multi-agent systems. It is then compared with GA's on a diverse set of problems. To handle high dimensional surfaces, in the PC method investigated here p is restricted to a product distribution. Each distribution in that product is controlled by a separate agent. The test functions were selected for their difficulty using either traditional gradient descent or genetic algorithms. On those functions the PC-based approach significantly outperforms traditional GA's in both rate of descent, trapping in false minima, and long term optimization.

Huang, Chien-Feng↗
NASA NTRS2009

Trajectory-Oriented Approach to Managing Traffic Complexity: Trajectory Flexibility Metrics and Algorithms and Preliminary Complexity Impact Assessment

This document describes exploratory research on a distributed, trajectory oriented approach for traffic complexity management. The approach is to manage traffic complexity based on preserving trajectory flexibility and minimizing constraints. In particular, the document presents metrics for trajectory flexibility; a method for estimating these metrics based on discrete time and degree of freedom assumptions; a planning algorithm using these metrics to preserve flexibility; and preliminary experiments testing the impact of preserving trajectory flexibility on traffic complexity. The document also describes an early demonstration capability of the trajectory flexibility preservation function in the NASA Autonomous Operations Planner (AOP) platform.

Idris, Husni↗
DOE OSTI2024

A remote sensing algorithm for vertically resolved cloud condensation nuclei number concentrations from airborne and spaceborne lidar observations

Cloud condensation nuclei (CCN) are mediators of aerosol–cloud interactions (ACIs), contributing to the largest uncertainties in the understandings of global climate change. We present a novel remote-sensing-based algorithm that quantifies the vertically resolved CCN number concentrations (N CCN ) using aerosol optical properties measured by a multiwavelength lidar. The algorithm considers five distinct aerosol subtypes with bimodal size distributions. The inversion used the lookup tables developed in this study, based on the observations from the Aerosol Robotic Network, to efficiently retrieve optimal particle size distributions from lidar measurements. The method derives dry aerosol optical properties by implementing hygroscopic enhancement factors in lidar measurements. The retrieved optically equivalent particle size distributions and aerosol-type-dependent particle composition are utilized to calculate critical diameters using κ-Köhler theory and N CCN at six supersaturations ranging from 0.07 % to 1.0 %. Sensitivity analyses indicate that uncertainties in extinction coefficients and relative humidity greatly influence the retrieval error in N CCN . The potential of this algorithm is further evaluated by retrieving N CCN using airborne lidar from the NASA ObseRvations of Aerosols above CLouds and their intEractionS (ORACLES) campaign and is validated against simultaneous measurements from the CCN counter. The independent validation with robust correlation demonstrates promising results. Furthermore, the N CCN has been retrieved for the first time using a proposed algorithm from spaceborne lidar – Cloud-Aerosol Lidar with Orthogonal Polarization (CALIOP) – measurements. The application of this new capability demonstrates the potential for constructing a 3D CCN climatology at a global scale, which helps to better quantify ACI effects and thus reduce the uncertainty in aerosol climate forcing.

54 ENVIRONMENTAL SCIENCES↗
NASA NTRS1998

Parallel Implicit Algorithms for CFD

The main goal of this project was efficient distributed parallel and workstation cluster implementations of Newton-Krylov-Schwarz (NKS) solvers for implicit Computational Fluid Dynamics (CFD.) "Newton" refers to a quadratically convergent nonlinear iteration using gradient information based on the true residual, "Krylov" to an inner linear iteration that accesses the Jacobian matrix only through highly parallelizable sparse matrix-vector products, and "Schwarz" to a domain decomposition form of preconditioning the inner Krylov iterations with primarily neighbor-only exchange of data between the processors. Prior experience has established that Newton-Krylov methods are competitive solvers in the CFD context and that Krylov-Schwarz methods port well to distributed memory computers. The combination of the techniques into Newton-Krylov-Schwarz was implemented on 2D and 3D unstructured Euler codes on the parallel testbeds that used to be at LaRC and on several other parallel computers operated by other agencies or made available by the vendors. Early implementations were made directly in Massively Parallel Integration (MPI) with parallel solvers we adapted from legacy NASA codes and enhanced for full NKS functionality. Later implementations were made in the framework of the PETSC library from Argonne National Laboratory, which now includes pseudo-transient continuation Newton-Krylov-Schwarz solver capability (as a result of demands we made upon PETSC during our early porting experiences). A secondary project pursued with funding from this contract was parallel implicit solvers in acoustics, specifically in the Helmholtz formulation. A 2D acoustic inverse problem has been solved in parallel within the PETSC framework.

Keyes, David E.↗
NASA NTRS2018

Observing the Full Spectrum of the Rain Drop Size Distribution

GPM DSD retrievals exhibit inconsistencies between GV, DPR and Combined algorithm retrievals. Development of positive bias in convective Dm rain DSD noted, and strongest in KuPR retrieval. Associated epsilons are too low and result in markedly reduced convective rain rates (a current issue in the retrievals). Source may be NUBF. Issues with the large end of the DSD not withstanding, on the small end of the DSD, combined MPS and 2DVD measurements fit with generalized gamma functions exhibit strong potential for representing the entire spectrum of the DSD and subsequently the whole rain rate spectrum.

Hydrology↗
DOE OSTI2025

Scalable edge clustering of dynamic graphs via weighted line graphs

Timestamped relational datasets consisting of records (or connections) between pairs of entities are ubiquitous in network science. For applications like peer-to-peer communication, email, various social network interactions, and computer network security, it is useful to organize these records into groups based on how and when they are occurring. Weighted line graphs offer a natural way to model how records are related in such datasets but for large real-world graph topologies, building and utilizing the line graph is prohibitively expensive. Here, we present the framework to cluster the edges of a dynamic graph via the associated line graph that contains two major contributions. The first is a method to work with the line graph implicitly and the second is a distributed scale implementation of an agglomerative hierarchical graph clustering algorithm. We outline a novel hierarchical dynamic graph edge clustering approach that efficiently breaks massive relational datasets into small sets of edges containing events at various timescales. This is in stark contrast to traditional graph clustering algorithms that prioritize highly connected (clique-like) community structures. Our approach relies on constructing a sufficient subgraph of a weighted line graph and applying a hierarchical agglomerative clustering. This approach is related to scalable techniques from spatial clustering, nonlinear-dimension reduction, topological data analysis, and draws particular inspiration from HDBSCAN. As an edge clustering, this method yields an overlapping node clustering. Our algorithm is parallelizable and we demonstrate efficient clustering of a billion-scale, real-world dynamic graph into small edge sets that correlate in topology and time. The entire clustering process for a graph with tens of billions of edges takes just a few minutes of run time on 256 nodes of a distributed compute environment. We argue how the output of the edge clustering is useful for a multitude of data visualization and powerful machine learning tasks, both involving the original massive dynamic graph data and metadata associated with the nodes and edges. Finally, we describe how this approach can be extended to dynamic hypergraphs and dynamic graphs/hypergraphs with unstructured data living on vertices and edges.

Data Analysis↗
NASA NTRS1999

Solar Microflare with BATSE

Our work on detecting and cataloging solar microflares using an automated method is illustrated in the accompanying figure. The figure represents the solar microflare distribution during the period of April 1991 to November 1992, the height of solar activity after the launch of The Compton Gamma Ray Observatory (CGRO). It also shows the distribution extending below the distribution obtained at Goddard Space Flight Center (GSFC) by manual means. We have implemented significant refinements in the search algorithm. The algorithm in its simplest form searches for transient events and based upon the distribution of the signal among the different Burst and Transient Source Experiment (BATSE) detectors, we can assign it to be of solar origin if the signal distribution conforms to what one expects from a burst or transient from that direction. One of the major problems in the earlier effort was to search for microflares and large flares simultaneously. The requirement for a dynamic range of almost 10 (exp 4) resulted in ambiguous identifications at the low side of the distribution. We have since restricted the search to events with peak count rates under 2000 s (exp -1). Larger events are easily identified in the manual search, so we have chosen not to duplicate that work. The second problem was that missing counts existed below channel 0 in the Burst and Transient Source Experiment Large Area Detector data (BATSE LAD). These have been recovered and are now included in the search process. This provides data below 20 keV, and as we get closer to the thermal part of the spectrum, it provides greater sensitivity. The third problem was that too many BATSE detector were used in the search. Detectors with pointing directions far from the Sun, although detecting the event, had poorly known responses. Detectors greater than approximately 60 deg. off the Sun are no longer included in the search process. By reducing the systematic errors with the large off-axis detectors we can conduct more rigorous statistical tests of a candidate event to ascertain whether it originated from the solar direction. We have reprocessed the period in the early mission that covers solar maximum and constructed the microflare distribution shown in the figure. The results of the automated search start to deviate from the manual search results below about 1000 s (exp -1). Not only do we now have this distribution but we have a database of solar microflares that was used to construct the distribution. This database contains the signal at higher energy channels as well as that in channel zero (and below). From this one can, using software at GSFC, construct a photon spectrum for some of the larger microflares. It can also be used in other solar studies, especially those that correlate the X-ray flux with emission at other wavelengths. With some additional effort we hope to integrate this database into the corresponding one residing at the Solar Data Analysis Center at GSFC. The entire CGRO mission's data can now be reprocessed to obtain the microflare distribution at all phases of the solar cycle. This work is in progress. The results of this work will be presented in forthcoming scientific workshops and conferences.

Ryan, James M.↗
DOE OSTI2024

Situational awareness-enhancing community-level load mapping with opportunistic machine learning

Motivated by present and forthcoming challenges in the adoption and integration of distributed renewable energy, we develop a machine learning (ML) approach that builds short-fuse mappings connecting the occasionally-unobservable true load in one target community with information-rich signals collected from relatively more instrumented reference communities. Our setting is inspired by and tailored to target communities with significant unobservable behind-the-meter solar generation, where true load (a relatively well-behaved quantity of interest to grid operators) is hard to discern during daytime due to insufficient instrumentation and/or privacy reasons, but that can be related to reference communities with low unobservable distributed variable generation or with sufficient instrumentation. The developed mapping, herein realized with Support Vector Machine regression, is built using nighttime data from all communities, when their distributed generation is low or zero. Our ML algorithm opportunistically learns to correlate signals of interest and then is operationally used the next day to shed light into target community load evolution. The mapping is subsequently rebuilt, rolling its short-fuse scope perpetually forward in time. Here, we demonstrate the efficacy of our approach on nine synthetically generated topologies and associated timeseries stemming from real-world data, on which we observe cumulative error performance that yields lower than 10% and 15% daily-averaged mean absolute percentage errors in target community load estimation on more than about 75% and 90% of days, respectively, in multiple yearly evaluations that shed light on long-term performance also under seasonal and one-off effects. The proposed ML-powered methodology can offer grid operators much-improved visibility into a previously obscure space and can also serve as an additional source of information in broader, multi-modal solar disaggregation solutions.

14 SOLAR ENERGY↗
NASA NTRS2004

The Development of Two Science Investigator-led Processing Systems (SIPS) for NASA's Earth Observation System (EOS)

In 2001, NASA Goddard Space Flight Center's Laboratory for Terrestrial Physics started the construction of a science Investigator-led Processing System (SIPS) for processing data from the Ozone Monitoring Instrument (OMI) which will launch on the Aura platform in mid 2004. The Ozone Monitoring Instrument (OMI) is a contribution of the Netherlands Agency for Aerospace Programs (NIVR) in collaboration with the Finnish Meteorological Institute (FMI) to the Earth Observing System (EOS) Aura mission. It will continue the Total Ozone Monitoring System (TOMS) record for total ozone and other atmospheric parameters related to ozone chemistry and climate. OMI measurements will be highly synergistic with the other instruments on the EOS Aura platform. The LTP previously developed the Moderate Resolution Imaging Spectrometer (MODIS) Data Processing System (MODAPS), which has been in full operations since the launches of the Terra and Aqua spacecrafts in December, 1999 and May, 2002 respectively. During that time, it has continually evolved to better support the needs of the MODIS team. We now run multiple instances of the system managing faster than real time reprocessings of the data as well as continuing forward processing. The new OMI Data Processing System (OMIDAPS) was adapted from the MODAPS. It will ingest raw data from the satellite ground station and process it to produce calibrated, geolocated higher level data products. These data products will be transmitted to the Goddard Distributed Active Archive Center (GDAAC) instance of the Earth Observing System (EOS) Data and Information System (EOSDIS) for long term archive and distribution to the public. The OMIDAPS will also provide data distribution to the OMI Science Team for quality assessment, algorithm improvement, calibration, etc. We have taken advantage of lessons learned from the MODIS experience and software already developed for MODIS. We made some changes in the hardware system organization, database and software to adapt the system for OMI. We replaced the fundamental database system, Sybase, with an Open Source RDBMS called PostgreSQL, and based the entire OMIDAPS on a cluster of Linux based commodity computers rather than the large SGI servers that MODAPS uses. Rather than relying on a central I/O server host, the new system distributes its data archive among multiple server hosts in the cluster. OMI is also customizing the graphical user interfaces and reporting structure to more closely meet the needs of the OMI Science Team. Prior to 2003, simulated OMI data and the science algorithms were not ready for production testing. We initially constructed a prototype system and tested using a 25 year dataset of Total Ozone Mapping Spectrometer (TOMS) and Solar Backscatter Ultraviolet Instrument (SBUV) data. This prototype system provided a platform to support the adaptation of the algorithms for OMI, and provided reprocessing of the historical data aiding in its analysis. In a recent reanalysis of the TOMS data, the OMIDAPS processed 108,000 full orbits of data through 4 processing steps per orbit, producing about 800,000 files (400 GiB) of level 2 and greater data files. More recently we have installed two instances of the OMIDAPS for integration and testing of OM1 science processes as they get delivered from the Science Team. A Test instance of the OMIDAPS has also supported a series of "Interface Confidence Tests" (ICTs) and End-to-End Ground System tests to ensure the launch readiness of the system. This paper will discuss the high-level hardware, software, and database organization of the OMIDAPS and how it builds on the MODAPS heritage system. It will also provide an overview of the testing and implementation of the production OMIDAPS.

Tilmes, Curt↗
DOE OSTI2024

Joint Management and Optimization of Residential Natural Gas and Electricity Distribution Networks Coupled via Fuel Cells

The interesting properties of natural gas as well as the growing electric power demand worldwide have led to increasing attention to natural-gas-based distributed generation applications in electric distribution systems. This paper goes over the interdependency between a residential natural gas network and an electric distribution network that are coupled via fuel cells. The modeling of the gas network is introduced first, and then the algorithm for gas flow study is presented. The optimal placement and sizing of fuel cell based distributed generation systems are formulated to minimize the losses in both the gas and electric distribution networks, subject to their model constraints. In addition to this, in order to capture the probabilistic nature of the optimization problem under study, the K-means clustering algorithm is applied to the gas and electricity demands to determine hourly load states and their corresponding probabilities. Furthermore, simulation studies are carried out on an integrated system consisting of the IEEE 69-bus distribution feeder and a radial 27-node natural gas network to verify the developed optimization model and the proposed method.

24 POWER TRANSMISSION AND DISTRIBUTION↗
NASA NTRS1990

Use of microwave satellite data to study variations in rainfall over the Indian Ocean

The University of Wisconsin Space Science and Engineering Center mapped rainfall over the Indian Ocean using a newly developed Scanning Multichannel Microwave Radiometer (SMMR) rain-retrieval algorithm. The short-range objective was to characterize the distribution and variability of Indian Ocean rainfall on seasonal and annual scales. In the long-range, the objective is to clarify differences between land and marine regimes of monsoon rain. Researchers developed a semi-empirical algorithm for retrieving Indian Ocean rainfall. Tools for this development have come from radiative transfer and cloud liquid water models. Where possible, ground truth information from available radars was used in development and testing. SMMR rainfalls were also compared with Indian Ocean gauge rainfalls. Final Indian Ocean maps were produced for months, seasons, and years and interpreted in terms of historical analysis over the sub-continent.

Hinton, Barry B.↗
NASA NTRS2001

Ground-Based Passive Microwave Remote Sensing Observations of Soil Moisture at S and L Band with Insight into Measurement Accuracy

Accurate estimates of spatially heterogeneous algorithm variables and parameters are required in determining the spatial distribution of soil moisture using radiometer data from aircraft and satellites. A ground-based experiment in passive microwave remote sensing of soil moisture was conducted in Huntsville, Alabama from July 1-14, 1996 to study retrieval algorithms and their sensitivity to variable and parameter specification. With high temporal frequency observations at S and L band, we were able to observe large scale moisture changes following irrigation and rainfall events, as well as diurnal behavior of surface moisture among three plots, one bare, one covered with short grass and another covered with alfalfa. The L band emitting depth was determined to be on the order of 0-3 or 0-5 cm below 0.30 cubic centimeter/cubic centimeter with an indication of a shallower emitting depth at higher moisture values. Surface moisture behavior was less apparent on the vegetated plots than it was on the bare plot because there was less moisture gradient and because of difficulty in determining vegetation water content and estimating the vegetation b parameter. Discrepancies between remotely sensed and gravimetric, soil moisture estimates on the vegetated plots point to an incomplete understanding of the requirements needed to correct for the effects of vegetation attenuation. Quantifying the uncertainty in moisture estimates is vital if applications are to utilize remotely-sensed soil moisture data. Computations based only on the real part of the complex dielectric constant and/or an alternative dielectric mixing model contribute a relatively insignificant amount of uncertainty to estimates of soil moisture. Rather, the retrieval algorithm is much more sensitive to soil properties, surface roughness and biomass.

Laymon, Charles A.↗
DOE OSTI2024

Efficient First-Order Algorithms for Large-Scale, Non-Smooth Maximum Entropy Models with Application to Wildfire Science

Maximum entropy (MaxEnt) models are a class of statistical models that use the maximum entropy principle to estimate probability distributions from data. Due to the size of modern data sets, MaxEnt models need efficient optimization algorithms to scale well for big data applications. State-of-the-art algorithms for MaxEnt models, however, were not originally designed to handle big data sets; these algorithms either rely on technical devices that may yield unreliable numerical results, scale poorly, or require smoothness assumptions that many practical MaxEnt models lack. In this paper, we present novel optimization algorithms that overcome the shortcomings of state-of-the-art algorithms for training large-scale, non-smooth MaxEnt models. Our proposed first-order algorithms leverage the Kullback–Leibler divergence to train large-scale and non-smooth MaxEnt models efficiently. For MaxEnt models with discrete probability distribution of n elements built from samples, each containing m features, the stepsize parameter estimation and iterations in our algorithms scale on the order of O(mn) operations and can be trivially parallelized. Moreover, the strong ℓ1 convexity of the Kullback–Leibler divergence allows for larger stepsize parameters, thereby speeding up the convergence rate of our algorithms. To illustrate the efficiency of our novel algorithms, we consider the problem of estimating probabilities of fire occurrences as a function of ecological features in the Western US MTBS-Interagency wildfire data set. Our numerical results show that our algorithms outperform the state of the art by one order of magnitude and yield results that agree with physical models of wildfire occurrence and previous statistical analyses of wildfire drivers.

Physics↗
NASA NTRS2016

Design, Development, and Testing of a UAV Hardware-in-the-Loop Testbed for Aviation and Airspace Prognostics Research

The airspace is becoming more and more complicated, and will continue to do so in the future with the integration of Unmanned Aerial Vehicles (UAVs), autonomy, spacecraft, other forms of aviation technology into the airspace. The new technology and complexity increases the importance and difficulty of safety assurance. Additionally, testing new technologies on complex aviation systems & systems of systems can be very difficult, expensive, and sometimes unsafe in real life scenarios. Prognostic methodology provides an estimate of the health and risks of a component, vehicle, or airspace and knowledge of how that will change over time. That measure is especially useful in safety determination, mission planning, and maintenance scheduling. The developed testbed will be used to validate prediction algorithms for the real-time safety monitoring of the National Airspace System (NAS) and the prediction of unsafe events. The framework injects flight related anomalies related to ground systems, routing, airport congestion, etc. to test and verify algorithms for NAS safety. In our research work, we develop a live, distributed, hardware-in-the-loop testbed for aviation and airspace prognostics along with exploring further research possibilities to verify and validate future algorithms for NAS safety. The testbed integrates virtual aircraft using the X-Plane simulator and X-PlaneConnect toolbox, UAVs using onboard sensors and cellular communications, and hardware in the loop components. In addition, the testbed includes an additional research framework to support and simplify future research activities. It enables safe, accurate, and inexpensive experimentation and research into airspace and vehicle prognosis that would not have been possible otherwise. This paper describes the design, development, and testing of this system. Software reliability, safety and latency are some of the critical design considerations in development of the testbed. Integration of HITL elements in the development phases and veri cation/ validation are key elements to this report.

HITL↗