Search NASASearch

SEARCH · Search NASA

Results for “Streaming Algorithms”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 109 records · Page 6

Using Visual Programming to Simulate, Test, and Display a Telemetry Stream

Advantages of using visual programming to create, modify, test and display a telemetry stream are presented. Commercial visual programming software is being used to test new algorithms as part of the ground support for the Galileo spacecraft Test Bed.

visual programming real-time data display text-bas

Low-Complexity Adaptive Lossless Compression of Hyperspectral Imagery

A low-complexity, adaptive predictive technique for lossless compression of hyperspectral imagery is described. This technique is designed to be suitable for implementation in hardware such as a field programmable gate array (FPGA); such an implementation could be used for high-speed compression of hyperspectral imagery onboard a spacecraft. The predictive step of the technique makes use of the sign algorithm, which is a relative of the least mean square (LMS) algorithm from the field of low-complexity adaptive filtering. The compressed data stream consists of prediction residuals encoded using a method similar to that of the JPEG-LS lossless image compression standard. Compression results are presented for several datasets including some raw Airborne Visible/ Infrared Imaging Spectrometer (AVIRIS) datasets and raw Atmospheric Infrared Sounder (AIRS) datasets. The compression effectiveness obtained with the technique is competitive with that of the best of previously described techniques with similar complexity.

lossless data compression

Attitude Estimation Signal Processing: A First Report on Possible Algorithms and Their Utility

In this brief effort, time has been of the essence. The data had to be acquired from APL/Lincoln Labs, stored, and sorted out to obtain the pertinent streams. This has been a significant part of this effort and hardware and software problems have been addressed with the appropriate solutions to accomplish this part of the task. Passed this, some basic and important algorithms are utilized to improve the performance of the attitude estimation systems. These algorithms are an essential part of the signal processing for the attitude estimation problem as they are utilized to reduce the amount of the additive/multiplicative noise that in general may or may not change its structure and probability density function, pdf, in time. These algorithms are not currently utilized in the processing of the data, at least, we are not aware of their use in this attitude estimation problem. Some of these algorithms, like the variable thresholding, are new conjectures, but one would expect that someone somewhere must have utilized this kind of scheme before. The variable thresholding idea is a straightforward scheme to use in case of a slowly varying pdf, or statistical moments of the unwanted random process. The algorithms here are kept simple but yet effective for processing the data and removing the unwanted noise. For the most part, these algorithms can be arranged so that their consecutive and orderly execution would complement the preceding algorithm and improve the overall performance of the signal processing chain.

Riasati, Vahid R.

Computation of viscous transonic flow over porous airfoils

The viscous effects on transonic flow past an airfoil which contains a shallow cavity beneath a porous surface are studied numerically. The porous region occupies a small portion of the total airfoil surface, and is located near the shock. Both an interactive boundary layer (IBL) algorithm and a thin-layer Navier-Stokes (TLNS) algorithm have been modified for use in studying the outer flow, whereas a stream-function formulation has been used to model the inner flow in the small cavity. The coupling procedure at the porous surface is based on Darcy's law and on the assumption of a constant total presusre in the cavity. In addition, a modified Baldwin-Lomax turbulence model is used to consider the transpired turbulent boundary layer in the TLNS approach, and the Cebeci-Smith turbulence model is used in the IBL approach. According to the present analysis, a porous surface can reduce the wave drag appreciably, but it can also increase viscous losses. As has been observed experimentally, the numerical results indicate that the total drag is reduced at higher Mach numbers and increased at lower Mach numbers when the angles of attack are small. Furthermore, the streamline patterns of passive-shock and boundary-layer interaction are revealed in this study.

Chow, Chuen-Yen

On the uncertainty in single molecule fluorescent lifetime and energy emission measurements

Time-correlated single photon counting has recently been combined with mode-locked picosecond pulsed excitation to measure the fluorescent lifetimes and energy emissions of single molecules in a flow stream. Maximum likelihood (ML) and least square methods agree and are optimal when the number of detected photons is large however, in single molecule fluorescence experiments the number of detected photons can be less than 20, 67% of those can be noise and the detection time is restricted to 10 nanoseconds. Under the assumption that the photon signal and background noise are two independent inhomogeneous poisson processes, we derive the exact joint arrival time probably density of the photons collected in a single counting experiment performed in the presence of background noise. The model obviates the need to bin experimental data for analysis, and makes it possible to analyze formally the effect of background noise on the photon detection experiment using both ML or Bayesian methods. For both methods we derive the joint and marginal probability densities of the fluorescent lifetime and fluorescent emission. the ML and Bayesian methods are compared in an analysis of simulated single molecule fluorescence experiments of Rhodamine 110 using different combinations of expected background nose and expected fluorescence emission. While both the ML or Bayesian procedures perform well for analyzing fluorescence emissions, the Bayesian methods provide more realistic measures of uncertainty in the fluorescent lifetimes. The Bayesian methods would be especially useful for measuring uncertainty in fluorescent lifetime estimates in current single molecule flow stream experiments where the expected fluorescence emission is low. Both the ML and Bayesian algorithms can be automated for applications in molecular biology.

Brown, Emery N.

On the Uncertainty in Single Molecule Fluorescent Lifetime and Energy Emission Measurements

Time-correlated single photon counting has recently been combined with mode-locked picosecond pulsed excitation to measure the fluorescent lifetimes and energy emissions of single molecules in a flow stream. Maximum likelihood (ML) and least squares methods agree and are optimal when the number of detected photons is large, however, in single molecule fluorescence experiments the number of detected photons can be less than 20, 67 percent of those can be noise, and the detection time is restricted to 10 nanoseconds. Under the assumption that the photon signal and background noise are two independent inhomogeneous Poisson processes, we derive the exact joint arrival time probability density of the photons collected in a single counting experiment performed in the presence of background noise. The model obviates the need to bin experimental data for analysis, and makes it possible to analyze formally the effect of background noise on the photon detection experiment using both ML or Bayesian methods. For both methods we derive the joint and marginal probability densities of the fluorescent lifetime and fluorescent emission. The ML and Bayesian methods are compared in an analysis of simulated single molecule fluorescence experiments of Rhodamine 110 using different combinations of expected background noise and expected fluorescence emission. While both the ML or Bayesian procedures perform well for analyzing fluorescence emissions, the Bayesian methods provide more realistic measures of uncertainty in the fluorescent lifetimes. The Bayesian methods would be especially useful for measuring uncertainty in fluorescent lifetime estimates in current single molecule flow stream experiments where the expected fluorescence emission is low. Both the ML and Bayesian algorithms can be automated for applications in molecular biology.

Brown, Emery N.

Level 1 Processing of MODIS Direct Broadcast Data at the GSFC DAAC

The GSFC DAAC is working to test and package the MODIS Level 1 Processing software for Aqua Direct Broadcast data. This entails the same code base, but different lookup tables for Aqua and Terra. However, the most significant change is the use of ancillary attitude and ephemeris files instead of orbit/attitude information within the science data stream (as with Terra). In addition, we are working on Linux: ports of the algorithms, which could eventually enable processing on PC clusters. Finally, the GSFC DAAC is also working with the GSFC Direct Readout laboratory to ingest Level 0 data from the GSFC DB antenna into the main DAAC, enabling level 1 production in near real time in support of applications users, such as the Synergy project. The mechanism developed for this could conceivably be extended to other participating stations.

Lynnes, Christopher

Web-based Visualization and Analytics of Petascale Data: Equity as a Tide that Lifts All Boats

Scientists generate petabytes of data daily to help uncover environmental trends or behaviors that are hard to predict. For example, understanding climate simulations based on the long-term average of temperature, precipitation, and other environmental variables is essential to predicting and establishing root causes of future undesirable scenarios and assessing possible mitigation strategies. While supercomputer centers provide a powerful infrastructure for generating petabytes of simulation output, accessing and analyzing these datasets interactively remains challenging on multiple fronts. This paper presents an approach to managing, visualizing, and analyzing petabytes of data within a browser on equipment ranging from the top NASA supercomputer to commodity hardware like a laptop. Our novel data fabric abstraction layer allows user-friendly querying of scientific information while hid-ing the complexities of dealing with file systems or cloud services.We also optimize network utilization while streaming from petas-cale repositories through state-of-the-art progressive compression algorithms. Based on this abstraction, we provide customizable dashboards that can be accessed from any device with any inter-net connection, enabling interactive visual analysis of vast amounts of data to a wide range of users - from top scientists with access to leadership-class computing environments to undergraduate students of disadvantaged backgrounds from minority-serving institutions. We focus on NASA’s use of petascale climate datasets as an example of particular societal impact and, therefore, a case where achieving equity in science participation is critical. We validate our approach by improving the ability of climate scientists to visually explore their data via two fully interactive dashboards. We further validate our approach by deploying the dashboards and simplified training materials in the classroom at a minority-serving institution.These dashboards, released in simplified form to the general public, contribute significantly to a broader push to democratize the access and use of climate data.

Data visualization

The Collection 6 'dark-target' MODIS Aerosol Products

Aerosol retrieval algorithms are applied to Moderate resolution Imaging Spectroradiometer (MODIS) sensors on both Terra and Aqua, creating two streams of decade-plus aerosol information. Products of aerosol optical depth (AOD) and aerosol size are used for many applications, but the primary concern is that these global products are comprehensive and consistent enough for use in climate studies. One of our major customers is the international modeling comparison study known as AEROCOM, which relies on the MODIS data as a benchmark. In order to keep up with the needs of AEROCOM and other MODIS data users, while utilizing new science and tools, we have improved the algorithms and products. The code, and the associated products, will be known as Collection 6 (C6). While not a major overhaul from the previous Collection 5 (C5) version, there are enough changes that there are significant impacts to the products and their interpretation. In its entirety, the C6 algorithm is comprised of three sub-algorithms for retrieving aerosol properties over different surfaces: These include the dark-target DT algorithms to retrieve over (1) ocean and (2) vegetated-dark-soiled land, plus the (3) Deep Blue (DB) algorithm, originally developed to retrieve over desert-arid land. Focusing on the two DT algorithms, we have updated assumptions for central wavelengths, Rayleigh optical depths and gas (H2O, O3, CO2, etc.) absorption corrections, while relaxing the solar zenith angle limit (up to 84) to increase pole-ward coverage. For DT-land, we have updated the cloud mask to allow heavy smoke retrievals, fine-tuned the assignments for aerosol type as function of season location, corrected bugs in the Quality Assurance (QA) logic, and added diagnostic parameters such as topographic altitude. For DT-ocean, improvements include a revised cloud mask for thin-cirrus detection, inclusion of wind speed dependence in the retrieval, updates to logic of QA Confidence flag (QAC) assignment, and additions of important diagnostic information. At the same time as we have introduced algorithm changes, we have also accounted for upstream changes including: new instrument calibration, revised land-sea masking, and changed cloud masking. Upstream changes also impact the coverage and global statistics of the retrieved AOD. Although our responsibility is to the DT code and products, we have also added a product that merges DT and DB product over semi-arid land surfaces to provide a more gap-free dataset, primarily for visualization purposes. Preliminary validation shows that compared to surface-based sunphotometer data, the C6, Level 2 (along swath) DT-products compare at least as well as those from C5. C6 will include new diagnostic information about clouds in the aerosol field, including an aerosol cloud mask at 500 m resolution, and calculations of the distance to the nearest cloud from clear pixels. Finally, we have revised the strategy for aggregating and averaging the Level 2 (swath) data to become Level 3 (gridded) data. All together, the changes to the DT algorithms will result in reduced global AOD (by 0.02) over ocean and increased AOD (by 0.02) over land, along with changes in spatial coverage. Changes in calibration will have more impact to Terras time series, especially over land. This will result in a significant reduction in artificial differences in the Terra and Aqua datasets, and will stabilize the MODIS data as a target for AEROCOM studie

Aerosol retrieval algorithms

Viscous flow simulations in VTOL aerodynamics

The critical issues in viscous flow simulations, such as boundary-layer separation, entrainment, turbulence modeling, and compressibility, are discussed with regard to the ground effects problem for vertical-takeoff-and-landing (VTOL) aircraft. A simulation of the two-dimensional incompressible lift jet in ground proximity is based on solution of the Reynolds-averaged Navier-Stokes equations and a turbulence-model equation which are written in stream function-vorticity form and are solved using Hoffman's augmented-central-difference algorithm. The resulting equations and their shortcomings are discussed when the technique is extended to two-dimensional compressible and three-dimensional incompressible flows.

Bower, W. W.

Systolic VLSI Reed-Solomon Decoder

Decoder for digital communications provides high-speed, pipelined ReedSolomon (RS) error-correction decoding of data streams. Principal new feature of proposed decoder is modification of Euclid greatest-common-divisor algorithm to avoid need for time-consuming computations of inverse of certain Galois-field quantities. Decoder architecture suitable for implementation on very-large-scale integrated (VLSI) chips with negative-channel metaloxide/silicon circuitry.

Shao, H. M.

Multiprocessing on supercomputers for computational aerodynamics

Very little use is made of multiple processors available on current supercomputers (computers with a theoretical peak performance capability equal to 100 MFLOPs or more) in computational aerodynamics to significantly improve turnaround time. The productivity of a computer user is directly related to this turnaround time. In a time-sharing environment, the improvement in this speed is achieved when multiple processors are used efficiently to execute an algorithm. The concept of multiple instructions and multiple data (MIMD) through multi-tasking is applied via a strategy which requires relatively minor modifications to an existing code for a single processor. Essentially, this approach maps the available memory to multiple processors, exploiting the C-FORTRAN-Unix interface. The existing single processor code is mapped without the need for developing a new algorithm. The procedure for building a code utilizing this approach is automated with the Unix stream editor. As a demonstration of this approach, a Multiple Processor Multiple Grid (MPMG) code is developed. It is capable of using nine processors, and can be easily extended to a larger number of processors. This code solves the three-dimensional, Reynolds averaged, thin-layer and slender-layer Navier-Stokes equations with an implicit, approximately factored and diagonalized method. The solver is applied to generic oblique-wing aircraft problem on a four processor Cray-2 computer. A tricubic interpolation scheme is developed to increase the accuracy of coupling of overlapped grids. For the oblique-wing aircraft problem, a speedup of two in elapsed (turnaround) time is observed in a saturated time-sharing environment.

Yarrow, Maurice

Multi-processing on supercomputers for computational aerodynamics

The MIMD concept is applied, through multitasking, with relatively minor modifications to an existing code for a single processor. This approach maps the available memory to multiple processors, exploiting the C-FORTRAN-Unix interface. An existing single processor algorithm is mapped without the need for developing a new algorithm. The procedure of designing a code utilizing this approach is automated with the Unix stream editor. A Multiple Processor Multiple Grid (MPMG) code is developed as a demonstration of this approach. This code solves the three-dimensional, Reynolds-averaged, thin-layer and slender-layer Navier-Stokes equations with an implicit, approximately factored and diagonalized method. This solver is applied to a generic, oblique-wing aircraft problem on a four-processor computer using one process for data management and nonparallel computations and three processes for pseudotime advance on three different grid systems.

Yarrow, Maurice

Multiprocessing on supercomputers for computational aerodynamics

Little use is made of multiple processors available on current supercomputers (computers with a theoretical peak performance capability equal to 100 MFLOPS or more) to improve turnaround time in computational aerodynamics. The productivity of a computer user is directly related to this turnaround time. In a time-sharing environment, such improvement in this speed is achieved when multiple processors are used efficiently to execute an algorithm. The concept of multiple instructions and multiple data (MIMD) is applied through multitasking via a strategy that requires relatively minor modifications to an existing code for a single processor. This approach maps the available memory to multiple processors, exploiting the C-Fortran-Unix interface. The existing code is mapped without the need for developing a new algorithm. The procedure for building a code utilizing this approach is automated with the Unix stream editor.

Yarrow, Maurice

The computation of inviscid rotational gasdynamic flows using an alternate velocity decompositon

The present investigation has the objective to present a method which can be considered to be a modification of existing potential flow algorithms. The method can be employed for the accurate and efficient solution of rotational gasdynamic flows. The general approach to computing inviscid rotational flows considered makes use of a velocity decomposition discussed by Clebsch (1859). Recent applications of the Clebsch representation in inviscid flows have involved the convection of an externally generated vorticity. Attention is given to the case of a steady, inviscid flow of a perfect gas with a uniform incoming stream. The developed procedure makes it possible to obtain a rapid solution to inviscid, compressible, rotational flows with solution algorithms which are a modification of existing potential flow codes. An application of the procedure to supersonic rotational conical flow is discussed.

Grossman, B.

Contemplating Synergistic Algorithms for the NASA ACE Mission

ACE is a proposed Tier 2 NASA Decadal Survey mission that will focus on clouds, aerosols, and precipitation as well as ocean ecosystems. The primary objective of the clouds component of this mission is to advance our ability to predict changes to the Earth's hydrological cycle and energy balance in response to climate forcings by generating observational constraints on future science questions, especially those associated with the effects of aerosol on clouds and precipitation. ACE will continue and extend the measurement heritage that began with the A-Train and that will continue through Earthcare. ACE planning efforts have identified several data streams that can contribute significantly to characterizing the properties of clouds and precipitation and the physical processes that force these properties. These include dual frequency Doppler radar, high spectral resolution lidar, polarimetric visible imagers, passive microwave and submillimeter wave radiometry. While all these data streams are technologically feasible, their total cost is substantial and likely prohibitive. It is, therefore, necessary to critically evaluate their contributions to the ACE science goals. We have begun developing algorithms to explore this trade space. Specifically, we will describe our early exploratory algorithms that take as input the set of potential ACE-like data streams and evaluate critically to what extent each data stream influences the error in a specific cloud quantity retrieval.

precipitation

On the Advantages of Using Harmonized Landsat Sentinel-2 Data for Monitoring Environmental Change

NASA coordinates the Satellite Needs Working Group, dedicated to identifying, communicating, and addressing Earth observation needs of federal agencies. In 2016, the Harmonized Landsat Sentinel-2 (HLS) dataset was formulated and implemented to fulfill multiple needs. The combination of acquisitions from the Landsat and Sentinel-2 platforms results in a global dataset of surface reflectance with a temporal resolution of two days, while retaining the geometry and 30-meter spatial resolution of Landsat data. This harmonization allows for seamless integration with the 40-year archive of Landsat data. The HLS dataset is now available on the Google Earth Engine, enabling HLS utilization in various algorithms and frameworks essential for monitoring environmental change worldwide. During this presentation, we will demonstrate and discuss the advantages of using HLS data in comparison to using separate streams of Landsat and Sentinel-2 data in existing time series-based frameworks for change monitoring. Specifically, we will explore the application of HLS for continuous monitoring of deforestation using time series-based algorithms traditionally run with Landsat data. Additionally, we will showcase the benefits of HLS data for near real-time monitoring of forest disturbance in tropical regions. These examples underscore the value and utility of the HLS dataset for environmental monitoring and analysis.

Pontus Olofsson

Autonomous Environment-Monitoring Networks

Autonomous environment-monitoring networks (AEMNs) are artificial neural networks that are specialized for recognizing familiarity and, conversely, novelty. Like a biological neural network, an AEMN receives a constant stream of inputs. For purposes of computational implementation, the inputs are vector representations of the information of interest. As long as the most recent input vector is similar to the previous input vectors, no action is taken. Action is taken only when a novel vector is encountered. Whether a given input vector is regarded as novel depends on the previous vectors; hence, the same input vector could be regarded as familiar or novel, depending on the context of previous input vectors. AEMNs have been proposed as means to enable exploratory robots on remote planets to recognize novel features that could merit closer scientific attention. AEMNs could also be useful for processing data from medical instrumentation for automated monitoring or diagnosis. The primary substructure of an AEMN is called a spindle. In its simplest form, a spindle consists of a central vector (C), a scalar (r), and algorithms for changing C and r. The vector C is constructed from all the vectors in a given continuous stream of inputs, such that it is minimally distant from those vectors. The scalar r is the distance between C and the most remote vector in the same set. The construction of a spindle involves four vital parameters: setup size, spindle-population size, and the radii of two novelty boundaries. The setup size is the number of vectors that are taken into account before computing C. The spindle-population size is the total number of input vectors used in constructing the spindle counting both those that arrive before and those that arrive after the computation of C. The novelty-boundary radii are distances from C that partition the neighborhood around C into three concentric regions (see Figure 1). During construction of the spindle, the changing spindle radius is denoted by h. It is the final value of h, reached before beginning construction on the next spindle, that is denoted by r. During construction of a spindle, if a new vector falls between C and the inner boundary, the vector is regarded as completely familiar and no action is taken. If the new vector falls into the region between the inner and outer boundaries, it is considered unusual enough to warrant the adjustment of C and r by use of the aforementioned algorithms, but not unusual enough to be considered novel. If a vector falls outside the outer boundary, it is considered novel, in which case one of several appropriate responses could be initiation of construction of a new spindle.

Hand, Charles