Search NASA⌕ Search

SEARCH · Search NASA

Results for “Streaming Algorithms”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 37 records · Page 2
All sourcesNASA NTRSDOE OSTI
NASA NTRS1990

Measuring and modeling near-surface reflected and emitted radiation fluxes at the FIFE site

Information is presented pertaining to the measurement and estimation of reflected and emitted components of the radiation balance. Information is included about reflectance and transmittance of solar radiation from and through the leaves of some grass and forb prairie species, bidirectional reflectance from a prairie canopy is discussed and measured and estimated fluxes are described of incoming and outgoing longwave and shortwave radiation. Results of the study showed only very small differences in reflectances and transmittances for the adaxial and abaxial surfaces of grass species in the visible and infrared wavebands, but some differences in the infrared wavebands were noted for the forbs. Reflectance from the prairie canopy changed as a function of solar and view zenith angles in the solar principal plane with definite asymmetry about nadir. The surface temperature of prairie canopies was found to vary by as much as 5 C depending on view zenith and azimuth position and on the solar azimuth. Aerodynamic temperature calculated from measured sensible heat fluxes ranged from 0 to 3 C higher than nadir-viewed temperatures. Models were developed to estimate incoming and reflected shortwave radiation from data collected with a Barnes Modular Multiband Radiometer. Several algorithms for estimating incoming longwave radiation were evaluated and compared to actual measures of that parameter. Net radiation was calculated using the estimated components of the shortwave radiation streams, determined from the algorithms developed, and from the longwave radiation streams provided by the Brunt, modified Deacon, and the Stefan-Boltzmann models. Estimates of net radiation were compared to measured values and found to be within the measurement error of the net radiometers used in the study.

Blad, Blaine L.↗
NASA NTRSDate not supplied

Generation of Land Surface Reflectance with Combined Geo-KOMPSAT-2A AMI and Himawari 8 AHI Observations

The latest generation of geostationary satellites has opened a new era of Earth observations with unprecedented spatiotemporal resolution and spectral range. Together with GOES 16/17 ABI, FY4-A AGRI, and Himawari-8 AHI, a new Korean geostationary satellite (Geo-KOMPSAT-2A AMI) has operationally collected a full-disk image in 16 channels every ten minutes since July 2019, allowing diurnal land surface monitoring over a large proportion of Asia and all of Oceania. Retrieving accurate surface reflectance (SR) over land from GK-2A/AMI is a challenging but high priority objective. One of the challenges is the absence of a spectral band in the 2.2 m SWIR range from AMI, which is required by many atmospheric correction algorithms to retrieve atmospheric aerosol properties. To remedy this issue, we adopt a strategy that combines concurrent GK-2A/AMI and Himawari 8/AHI observations in order to derive AMI SR. We have adapted the NASA Multi-Angle Implementation of Atmospheric Correction (MAIAC) algorithm to process the data stream from Himawari 8/AHI. The advantages of the MAIAC algorithm is its capability to exploit the high temporal frequency and varying illumination geometry of the geostationary data for advanced cloud/snow detection, aerosol retrieval, and characterization of surface bidirectional reflectance properties. Leveraging the similarities of spectral bands and the sun-target-sensor geometry between AMI and AHI, we are able to create denser time series of observations and enhanced BRDF samples over most of the spatial coverage of AMI (and AHI). The combined stereo-type observations not only help derive SR for AMI but also enhance retrievals of the corresponding AHI surface products. We evaluate the resulting AMI SR using ground (AERONET) observations and corresponding MODIS products. Further, we discuss potential challenges in utilizing the geostationary satellite data for land surface monitoring.

geostationary satellite↗
NASA NTRS2011

Accelerated Adaptive MGS Phase Retrieval

The Modified Gerchberg-Saxton (MGS) algorithm is an image-based wavefront-sensing method that can turn any science instrument focal plane into a wavefront sensor. MGS characterizes optical systems by estimating the wavefront errors in the exit pupil using only intensity images of a star or other point source of light. This innovative implementation of MGS significantly accelerates the MGS phase retrieval algorithm by using stream-processing hardware on conventional graphics cards. Stream processing is a relatively new, yet powerful, paradigm to allow parallel processing of certain applications that apply single instructions to multiple data (SIMD). These stream processors are designed specifically to support large-scale parallel computing on a single graphics chip. Computationally intensive algorithms, such as the Fast Fourier Transform (FFT), are particularly well suited for this computing environment. This high-speed version of MGS exploits commercially available hardware to accomplish the same objective in a fraction of the original time. The exploit involves performing matrix calculations in nVidia graphic cards. The graphical processor unit (GPU) is hardware that is specialized for computationally intensive, highly parallel computation. From the software perspective, a parallel programming model is used, called CUDA, to transparently scale multicore parallelism in hardware. This technology gives computationally intensive applications access to the processing power of the nVidia GPUs through a C/C++ programming interface. The AAMGS (Accelerated Adaptive MGS) software takes advantage of these advanced technologies, to accelerate the optical phase error characterization. With a single PC that contains four nVidia GTX-280 graphic cards, the new implementation can process four images simultaneously to produce a JWST (James Webb Space Telescope) wavefront measurement 60 times faster than the previous code.

Lam, Raymond K.↗
NASA NTRS2014

Snowfall Rate Retrieval using NPP ATMS Passive Microwave Measurements

Passive microwave measurements at certain high frequencies are sensitive to the scattering effect of snow particles and can be utilized to retrieve snowfall properties. Some of the microwave sensors with snowfall sensitive channels are Advanced Microwave Sounding Unit (AMSU), Microwave Humidity Sounder (MHS) and Advance Technology Microwave Sounder (ATMS). ATMS is the follow‐on sensor to AMSU and MHS. Currently, an AMSU and MHS based land snowfall rate (SFR) product is running operationally at NOAA/NESDIS. Based on the AMSU/MHS SFR, an ATMS SFR algorithm has been developed recently. The algorithm performs retrieval in three steps: snowfall detection, retrieval of cloud properties, and estimation of snow particle terminal velocity and snowfall rate. The snowfall detection component utilizes principal component analysis and a logistic regression model. The model employs a combination of temperature and water vapor sounding channels to detect the scattering signal from falling snow and derive the probability of snowfall (Kongoli et al., 2014). In addition, a set of NWP model based filters is also employed to improve the accuracy of snowfall detection. Cloud properties are retrieved using an inversion method with an iteration algorithm and a two‐stream radiative transfer model (Yan et al., 2008). A method developed by Heymsfield and Westbrook (2010) is adopted to calculate snow particle terminal velocity. Finally, snowfall rate is computed by numerically solving a complex integral. The ATMS SFR product is validated against radar and gauge snowfall data and shows that the ATMS algorithm outperforms the AMSU/MHS SFR.

Meng, Huan↗
NASA NTRS1988

Application of acoustic-Doppler current profiler and expendable bathythermograph measurements to the study of the velocity structure and transport of the Gulf Stream

The degree to which Acoustic-Doppler Current Profiler (ADCP) and expendable bathythermograph (XBT) data can provide quantitative measurements of the velocity structure and transport of the Gulf Stream is addressed. An algorithm is used to generate salinity from temperature and depth using an historical Temperature/Salinity relation for the NW Atlantic. Results have been simulated using CTD data and comparing real and pseudo salinity files. Errors are typically less than 2 dynamic cm for the upper 800 m out of a total signal of 80 cm (across the Gulf Stream). When combined with ADCP data for a near-surface reference velocity, transport errors in isopycnal layers are less than about 1 Sv (10 to the 6th power cu m/s), as is the difference in total transport for the upper 800 m between real and pseudo data. The method is capable of measuring the real variability of the Gulf Stream, and when combined with altimeter data, can provide estimates of the geoid slope with oceanic errors of a few parts in 10 to the 8th power over horizontal scales of 500 km.

Joyce, T. M.↗
NASA NTRS1982

A vortex wake capturing method for potential flow calculations

A method is presented for modifying finite difference solutions of the potential equation to include the calculation of non-planar vortex wake features. The approach is an adaptation of Baker's 'cloud in cell' algorithm developed for the stream function-vorticity equations. The vortex wake is tracked in a Lagrangian frame of reference as a group of discrete vortex filaments. These are distributed to the Eulerian mesh system on which the velocity is calculated by a finite difference solution of the potential equation. An artificial viscosity introduced by the finite difference equations removes the singular nature of the vortex filaments. Computed examples are given for the two-dimensional time dependent roll-up of vortex wakes generated by wings with different spanwise loading distributions.

Murman, E. M.↗
NASA NTRS2008

Visualization Component of Vehicle Health Decision Support System

The visualization front-end of a Decision Support System (DSS) also includes an analysis engine linked to vehicle telemetry, and a database of learned models for known behaviors. Because the display is graphical rather than text-based, the summarization it provides has a greater information density on one screen for evaluation by a flight controller.This tool provides a system-level visualization of the state of a vehicle, and drill-down capability for more details and interfaces to separate analysis algorithms and sensor data streams. The system-level view is a 3D rendering of the vehicle, with sensors represented as icons, tied to appropriate positions within the vehicle body and colored to indicate sensor state (e.g., normal, warning, anomalous state, etc.). The sensor data is received via an Information Sharing Protocol (ISP) client that connects to an external server for real-time telemetry. Users can interactively pan, zoom, and rotate this 3D view, as well as select sensors for a detail plot of the associated time series data. Subsets of the plotted data can be selected and sent to an external analysis engine to either search for a similar time series in an historical database, or to detect anomalous events. The system overview and plotting capabilities are completely general in that they can be applied to any vehicle instrumented with a collection of sensors. This visualization component can interface with the ISP for data streams used by NASA s Mission Control Center at Johnson Space Center. In addition, it can connect to, and display results from, separate analysis engine components that identify anomalies or that search for past instances of similar behavior. This software supports NASA's Software, Intelligent Systems, and Modeling element in the Exploration Systems Research and Technology Program by augmenting the capability of human flight controllers to make correct decisions, thus increasing safety and reliability. It was designed specifically as a tool for NASA's flight controllers to monitor the International Space Station and a future Crew Exploration Vehicle.

Jacob, Joseph↗
NASA NTRS2009

Optimal Compression Methods for Floating-point Format Images

We report on the results of a comparison study of different techniques for compressing FITS images that have floating-point (real*4) pixel values. Standard file compression methods like GZIP are generally ineffective in this case (with compression ratios only in the range 1.2 - 1.6), so instead we use a technique of converting the floating-point values into quantized scaled integers which are compressed using the Rice algorithm. The compressed data stream is stored in FITS format using the tiled-image compression convention. This is technically a lossy compression method, since the pixel values are not exactly reproduced, however all the significant photometric and astrometric information content of the image can be preserved while still achieving file compression ratios in the range of 4 to 8. We also show that introducing dithering, or randomization, when assigning the quantized pixel-values can significantly improve the photometric and astrometric precision in the stellar images in the compressed file without adding additional noise. We quantify our results by comparing the stellar magnitudes and positions as measured in the original uncompressed image to those derived from the same image after applying successively greater amounts of compression.

Pence, W. D.↗
NASA NTRS2015

NPP ATMS Snowfall Rate Product

Passive microwave measurements at certain high frequencies are sensitive to the scattering effect of snow particles and can be utilized to retrieve snowfall properties. Some of the microwave sensors with snowfall sensitive channels are Advanced Microwave Sounding Unit (AMSU), Microwave Humidity Sounder (MHS) and Advance Technology Microwave Sounder (ATMS). ATMS is the follow‐on sensor to AMSU and MHS. Currently, an AMSU and MHS based land snowfall rate (SFR) product is running operationally at NOAA/NESDIS. Based on the AMSU/MHS SFR, an ATMS SFR algorithm has been developed recently. The algorithm performs retrieval in three steps: snowfall detection, retrieval of cloud properties, and estimation of snow particle terminal velocity and snowfall rate. The snowfall detection component utilizes principal component analysis and a logistic regression model. The model employs a combination of temperature and water vapor sounding channels to detect the scattering signal from falling snow and derive the probability of snowfall (Kongoli et al., 2015). In addition, a set of NWP model based filters is also employed to improve the accuracy of snowfall detection. Cloud properties are retrieved using an inversion method with an iteration algorithm and a two‐stream radiative transfer model (Yan et al., 2008). A method developed by Heymsfield and Westbrook (2010) is adopted to calculate snow particle terminal velocity. Finally, snowfall rate is computed by numerically solving a complex integral. NCEP CMORPH analysis has shown that integration of ATMS SFR has improved the performance of CMORPH‐Snow. The ATMS SFR product is also being assessed at several NWS Weather Forecast Offices for its usefulness in weather forecast.

Forecasting↗
NASA NTRS2015

Snowfall Rate Retrieval Using Passive Microwave Measurements and Its Applications in Weather Forecast and Hydrology

(AMSU), Microwave Humidity Sounder (MHS) and Advance Technology Microwave Sounder (ATMS). ATMS is the follow‐on sensor to AMSU and MHS. Currently, an AMSU and MHS based land snowfall rate (SFR) product is running operationally at NOAA/NESDIS. Based on the AMSU/MHS SFR, an ATMS SFR algorithm has also been developed. The algorithm performs retrieval in three steps: snowfall detection, retrieval of cloud properties, and estimation of snow particle terminal velocity and snowfall rate. The snowfall detection component utilizes principal component analysis and a logistic regression model. It employs a combination of temperature and water vapor sounding channels to detect the scattering signal from falling snow and derives the probability of snowfall. Cloud properties are retrieved using an inversion method with an iteration algorithm and a two‐stream radiative transfer model. A method adopted to calculate snow particle terminal velocity. Finally, snowfall rate is computed by numerically solving a complex integral. The SFR products are being used mainly in two communities: hydrology and weather forecast. Global blended precipitation products traditionally do not include snowfall derived from satellites because such products were not available operationally in the past. The ATMS and AMSU/MHS SFR now provide the winter precipitation information for these blended precipitation products. Weather forecasters mainly rely on radar and station observations for snowfall forecast. The SFR products can fill in gaps where no conventional snowfall data are available to forecasters. The products can also be used to confirm radar and gauge snowfall data and increase forecasters' confidence in their prediction.

Meng, Huan↗
NASA NTRSDate not supplied

Data Science and the Knowledge Discovery Adventure

This talk will cover the important steps involved in the data science and knowledge discovery process: • Initial fact gathering (interview domain experts, review reports, articles, state-of-the-art) • Identify the problem (prediction, classification, statistical analysis, etc.) • Survey supporting data sources • Understand the data (numerical, categorical, text, sampling rate, data quality issues, etc.) • Selecting relevant features and sources • Acquire the data (set up agreements with the data stewards, APIs to download, etc.) • Merge data sources (temporal, spatial, common key, other ontologies...) • Feature Engineering (non linear domain knowledge or physics-based relationships) • Build data processing pipeline (may need to tap into data stream, develop parallel processing algorithm, federated learning etc.) • Build model and test (tune hyper-parameters, cross validation.) • Analyze/Validate results (do the results make sense. Does it answer the original question). • Deploy/Publish (Monitor and assess benefits)

Data science↗
NASA NTRS2001

Gamma-Weighted Discrete Ordinate Two-Stream Approximation for Computation of Domain Averaged Solar Irradiance

An algorithm is developed for the gamma-weighted discrete ordinate two-stream approximation that computes profiles of domain-averaged shortwave irradiances for horizontally inhomogeneous cloudy atmospheres. The algorithm assumes that frequency distributions of cloud optical depth at unresolved scales can be represented by a gamma distribution though it neglects net horizontal transport of radiation. This algorithm is an alternative to the one used in earlier studies that adopted the adding method. At present, only overcast cloudy layers are permitted.

Kato, S.↗
NASA NTRS2005

A Streaming Language Implementation of the Discontinuous Galerkin Method

We present a Brook streaming language implementation of the 3-D discontinuous Galerkin method for compressible fluid flow on tetrahedral meshes. Efficient implementation of the discontinuous Galerkin method using the streaming model of computation introduces several algorithmic design challenges. Using a cycle-accurate simulator, performance characteristics have been obtained for the Stanford Merrimac stream processor. The current Merrimac design achieves 128 Gflops per chip and the desktop board is populated with 16 chips yielding a peak performance of 2 Teraflops. Total parts cost for the desktop board is less than $20K. Current cycle-accurate simulations for discretizations of the 3-D compressible flow equations yield approximately 40-50% of the peak performance of the Merrimac streaming processor chip. Ongoing work includes the assessment of the performance of the same algorithm on the 2 Teraflop desktop board with a target goal of achieving 1 Teraflop performance.

Barth, Timothy↗
DOE OSTI2026

Large-scale real-time signal processing in physics experiments: the ALICE TPC FPGA pipeline

For LHC Run 3, the ALICE Time Projection Chamber was upgraded to operate in continuous readout mode. Interaction rates of up to 50 kHz in Pb-Pb collisions require real-time processing of more than 3 TB s -1 of raw detector data. This requirement is met by a custom FPGA-based processing pipeline that performs the complete front-end data treatment fully in-stream, including common-mode correction, pedestal subtraction, ion-tail filtering, zero suppression, and dense data packing. A central element of the design is a highly parallel common-mode correction algorithm operating directly on the streaming data. It robustly identifies signal-free readout channels on a time-bin basis and applies pad-dependent scaling to compensate for local variations in capacitive coupling in the GEM readout. In combination with pedestal subtraction and ion-tail filtering, this enables accurate baseline restoration under extreme high-occupancy conditions, preventing signal loss while efficiently suppressing noise prior to zero suppression. The pipeline operates continuously at the full detector bandwidth and reduces the raw input rate of approximately 3 TB s -1 to about 900 GBps for Pb-Pb collisions at the target interaction rate. Overall, it represents a large-scale FPGA-based real-time signal-processing implementation for high-energy physics detector readout.

Digital signal processing (DSP)↗
NASA NTRS2016

Fast and Accurate Hybrid Stream PCRTMSOLAR Radiative Transfer Model for Reflected Solar Spectrum Simulation in the Cloudy Atmosphere

A hybrid stream PCRTM-SOLAR model has been proposed for fast and accurate radiative transfer simulation. It calculates the reflected solar (RS) radiances with a fast coarse way and then, with the help of a pre-saved matrix, transforms the results to obtain the desired high accurate RS spectrum. The methodology has been demonstrated with the hybrid stream discrete ordinate (HSDO) radiative transfer (RT) model. The HSDO method calculates the monochromatic radiances using a 4-stream discrete ordinate method, where only a small number of monochromatic radiances are simulated with both 4-stream and a larger N-stream (N ≥ 16) discrete ordinate RT algorithm. The accuracy of the obtained channel radiance is comparable to the result from N-stream moderate resolution atmospheric transmission version 5 (MODTRAN5). The root-mean-square errors are usually less than 5x10(exp -4) mW/sq cm/sr/cm. The computational speed is three to four-orders of magnitude faster than the medium speed correlated-k option MODTRAN5. This method is very efficient to simulate thousands of RS spectra under multi-layer clouds/aerosols and solar radiation conditions for climate change study and numerical weather prediction applications.

Yang, Qiguang↗
NASA NTRS2004

Downlink Data Multiplexer

A multiplexer/demultiplexer system has been developed to enable the transmission, over a single channel, of four data streams generated by a variety of sources at different (including variable) bit rates. In the original intended application, replicas of this multiplexer/demultiplexer system would be incorporated into the spacecraft-to-ground communication systems of the space shuttles. The multiplexer of each system would be installed in the spacecraft, where it would acquire and process data from such sources as commercial digital camcorders, video tape recorders, and the spacecraft telemetry system. The demultiplexer of each system would be installed in a ground station. Purely terrestrial systems of similar design could be attractive for use in situations in which there are requirements to transmit multiple streams of high-quality video data and possibly other data over single channels. The figure is a block diagram of the multiplexer as configured to process data received via three fiber-optic channels like those of the International Space Station and one electrical-cable channel that conforms to the Institute of Electrical and Electronic Engineers (IEEE) 1394 standard. (This standard consists of specifications of a high-speed serial data interface, the physical layer of which includes a cable known in the art as "FireWire." An IEEE 1394 interface can also transfer power between the components to which it is connected.) The fiber-optic channels carry packet and/or bit-stream signals that conform to the standards of the Consultative Committee for Space Data Systems (CCSDS). The IEEE 1394 interface accepts an isochronous signal like that from a digital camcorder or a video tape recorder. The processing of the four input data streams to combine them into one output stream is governed by a statistical multiplexing algorithm that features a flow-control capability and makes it possible to utilize the transmission channel with nearly 100-percent efficiency. This algorithm allocates the available bandwidth of the transmission channel to the data streams according to a combination of data rates and preassigned priorities. Incoming data streams that demand too much bandwidth are blocked. Bandwidth not needed for a transmission of a given data stream is allocated to other streams as available. Priority is given to the IEEE 1394 stream. In addition to the four incoming data streams, the multiplexer transmits data on the status of the system. An operator can monitor and control the multiplexer via displays and controls on the multiplexer housing. The output of the multiplexer is connected via a coaxial cable with an impedance of 50 Ohms to an interface circuit compatible with the space-shuttle high-speed digital downlink, which operates at a rate of 48 Mb/s.

Holland, Douglas↗
NASA NTRS2004

Value-added Data Services at the Goddard Earth Sciences Data and Information Services Center

The NASA Goddard Earth Sciences Data and Information Services Center (GES DISC), in addition to serving the Earth Science community as one of the major Distributed Active Archives Centers (DAACs), provides much more than just data. Among the value-added services available to general users are subsetting data spatially and/or by parameter, online analysis (to avoid downloading unnecessarily all the data), and assistance in obtaining data from other centers. Services available to data producers and high-volume users include consulting on building new products with standard formats and metadata and construction of data management systems. A particularly useful service is data processing at the DISC (i.e., close to the input data) with the users algorithm. This can take a number of different forms: as a configuration-managed algorithm within the main processing stream; as a stand-alone program next to the on-line data storage; as build-it-yourself code within the Near-Archive Data Mining (NADM) system; or as an on-the-fly analysis with simple algorithms embedded into the web-based tools. Partnerships between the GES DISC and scientists, both producers and users, allow the scientists to concentrate on science, while the GES DISC handles the data management, e.g., formats, integration, and data processing. The existing data management infrastructure at the GES DISC supports a wide spectrum of options: from simple data support to sophisticated on-line analysis tools, producing economies of scale and rapid time-to-deploy. At the same time, such partnerships allow the GES DISC to serve the user community more efficiently and to better prioritize on-line holdings. Several examples of successful partnerships are described in the presentation.

Leptoukh, Gregory G.↗
NASA NTRS2005

NASA Remote Sensing Data in Earth Sciences: Processing, Archiving, Distribution, Applications at the GES DISC

The NASA Goddard Earth Sciences Data and Information Services Center (GES DISC) is one of the major Distributed Active Archive Centers (DAACs) archiving and distributing remote sensing data from the NASA's Earth Observing System. In addition to providing just data, the GES DISC/DAAC has developed various value-adding processing services. A particularly useful service is data processing a t the DISC (i.e., close to the input data) with the users' algorithms. This can take a number of different forms: as a configuration-managed algorithm within the main processing stream; as a stand-alone program next to the on-line data storage; as build-it-yourself code within the Near-Archive Data Mining (NADM) system; or as an on-the-fly analysis with simple algorithms embedded into the web-based tools (to avoid downloading unnecessary all the data). The existing data management infrastructure at the GES DISC supports a wide spectrum of options: from data subsetting data spatially and/or by parameter to sophisticated on-line analysis tools, producing economies of scale and rapid time-to-deploy. Shifting processing and data management burden from users to the GES DISC, allows scientists to concentrate on science, while the GES DISC handles the data management and data processing at a lower cost. Several examples of successful partnerships with scientists in the area of data processing and mining are presented.

Leptoukh, Gregory G.↗