Search NASA⌕ Search

SEARCH · Search NASA

Results for “data streaming”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 271 records · Page 15

Global Ocean Evaporation: How Well Can We Estimate Interannual to Decadal Variability?

Evaporation from the world's oceans constitutes the largest component of the global water balance. It is important not only as the ultimate source of moisture that is tied to the radiative processes determining Earth's energy balance but also to freshwater availability over land, governing habitability of the planet. Here we focus on variability of ocean evaporation on scales from interannual to decadal by appealing to three sources of data: the new MERRA-2 (Modern-Era Retrospective analysis for Research and Applications -2); climate models run with historical sea-surface temperatures, ice and atmospheric constituents (so-called AMIP experiments); and state-of-the-art satellite retrievals from the Seaflux and HOAPS (Hamburg Ocean-Atmosphere Parameters and Fluxes from Satellite) projects. Each of these sources has distinct advantages as well as drawbacks. MERRA-2, like other reanalyses, synthesizes evaporation estimates consistent with observationally constrained physical and dynamical models-but data stream discontinuities are a major problem for interpreting multi-decadal records. The climate models used in data assimilation can also be run with lesser constraints such as with SSTs and sea-ice (i.e. AMIPs) or with additional, minimal observations of surface pressure and marine observations that have longer and less fragmentary observational records. We use the new ERA-20C reanalysis produced by ECMWF embodying the latter methodology. Still, the model physics biases in climate models and the lack of a predicted surface energy balance are of concern. Satellite retrievals and comparisons to ship-based measurements offer the most observationally-based estimates, but sensor inter-calibration, algorithm retrieval assumptions, and short records are dominant issues. Our strategy depends on maximizing the advantages of these combined records. The primary diagnostic tool used here is an analysis of bulk aerodynamic computations produced by these sources and uses a first-order Taylor series analysis of wind speed, SST, near-surface stability and relative humidity variations around climatology to gauge the importance of these components. We find that the MERRA-2 evaporation record is strongly influenced by the availability of wind speed and humidity from passive microwave imagers beginning in the late 1980s as well as by the SST record. The trend over the period 1980 to present is nearly 10%. AMIP or the ERA-20C trends are much smaller. We find that ENSO-related signals involving both wind speed and thermodynamic variability remain the primary signal in the latter and are confirmed by satellite retrievals. We present uncertainty estimates based on the various data sources and discuss the implications for GEWEX water and energy budget science challenges.

Robertson, Franklin R.↗

Navigating Exascale Operational Data Analytics: From Inundation to Insight

In this paper, we address the challenges in achieving sustainable data-driven efficiency by providing a detailed exploration of the end-to-end operational data analytics (ODA) framework that evolved through two generations of supercomputer systems at the Oak Ridge Leadership Computing Facility (OLCF). This framework addresses large data streams ingested from heavily instrumented HPC environment that accumulates multi-terabytes per day. We outline the multifaceted data life cycle across HPC procurement, operations, and research & development, identifying key obstacles and design decisions that shape effective strategies in building and supporting data pipelines end-to-end. By sharing key insights and lessons learned from our experience, we offer recommendations for the HPC community on enabling sustainable operational data analytics and beyond. Our contributions aim to bridge the gap between potential and real benefits of operational data, guiding future efforts towards integrated and sustainable operational intelligence in high-performance computing environments.

Shin, Woong↗

Fine structure in RF spectra of lightning return stroke wave forms

The power spectra of the wide-band (10 Hz to 100 kHz) magnetic-field signals for a number of lightning return strokes measured during a thunderstorm which occurred in Lindau in August, 1984 have been calculated. The RF magnetic field data are obtained with the engineering unit of the Galileo Jupiter Probe lightning experiment. Each return stroke data stream is passed through an adaptive filter designed to whiten its spectrum. The spectra of the magnetic field data definitely show fine structure, with two or three distinct peaks in the spectra of many of the waveforms. A peak at f of about 60-70 kHz is often seen in the power spectra of the waveform time segments preceding and following the rise-to-peak amplitude of the return stroke.

Lanzerotti, L. J.↗

Space telemetry degradation due to Manchester data asymmetry induced carrier tracking phase error

The deleterious effects that the Manchester (or Bi-phi) data asymmetry has on the performance of phase-modulated residual carrier communication systems are analyzed. Expressions for the power spectral density of an asymmetric Manchester data stream, the interference-to-carrier signal power ratio (I/C), and the error probability performance are derived. Since data asymmetry can cause undesired spectral components at the carrier frequency, the I/C ratio is given as a function of both the data asymmetry and the telemetry modulation index. Also presented are the data asymmetry and asymmetry-induced carrier tracking loop and the system bit-error rate to various parameters of the models.

Nguyen, Tien M.↗

Commercial potential of remote sensing data from the Earth observing system

The purpose was to assess the market potential of remote sensing value-added products from the Earth Observing System (EOS) platform. Sensors on the EOS platform were evaluated to determine which qualities and capabilities could be useful to the commercial user. The approach was to investigate past and future satellite data distribution programs. A questionnaire was developed for use in a telephone survey. Based on the results of the survey of companies that add value to remotely sensed data, conversations with the principal investigators in charge of each EOS sensor, a study of past commercial satellite data ventures, and reading from the commercial remote sensing industry literature, three recommendations were developed: develop a strategic plan for commercialization of EOS data, define a procedure for commercial users within the EOS data stream, and develop an Earth Observations Commercial Applications Program-like demonstration program within NASA using EOS simulated data.

Merry, Carolyn J.↗

A digital twin platform for building performance monitoring and optimization: Performance simulation and case studies

Advancements in sensor technology, data analytics, affordable compute, and communication infrastructure have paved the way for Digital Twin technology in optimizing building operations and controls. This study presents the development of an open and interoperable web-based Digital Twin platform for integrating diverse data streams and facilitating effective user interactions. The platform utilizes modern technologies for the web framework and time-series data management, ensuring scalability and responsiveness. The backend supports seamless integration of diverse data sources and emulators, incorporating data from building sensors and meters, external weather Application Programming Interfaces, and advanced EnergyPlus simulation models of the building and its energy systems including the Distributed Energy Resources that are formulated in Functional Mockup Units. A simulation case study was conducted with FlexLab, a test facility on Lawrence Berkeley National Laboratory campus. The case study includes normal operations, Distributed Energy Resource integration, and power outage scenarios, to illustrate the Digital Twin’s ability to provide critical insights into energy performance and thermal resilience. The results demonstrated the platform’s potential as a decision-support tool for optimizing building energy performance and enhancing resilience against extreme weather events. Future work will focus on deploying the Digital Twin platform to a real building for field validation, extending its capabilities to cover more scenarios such as bidirectional Electric Vehicle interactions, and enhancing user engagement.

EnergyPlus↗

Pinch Heat Integration Tool (PIT) v0.9

The pinch heat integration tool (PIT) is a multi-module tool designed for identifying and analyzing thermal energy integration at facilities. It generates pinch and grand composite curves to identify optimal streams for heat exchange and industrial heat pumps. Users input heating and cooling stream data via an Excel sheet. The tool comprises four modules: 1. 'Main Pinch' for creating a pinch diagram using user-provided data. 2. 'GCC' for generating a grand composite curve to identify utility optimization opportunities. 3. 'Heat Exchanger' for analyzing stream combinations for heat exchange. 4. 'Heat Pump' for evaluating streams as sources and sinks for heat pumps and calculating the heat pump's coefficient of performance based on the selected refrigerant.

Karki, Unique [Lawrence Berkeley National Laborato↗

Moving NASA Remote Sensing Data to the GIS Environment for Health Studies

There has been an increasing demand by the health community for improved data on many different environmental factors relevant to the links between the environment and disease occurrence and transmission. These data are important for GIS-based monitoring, risk mapping, and surveillance of epidemiological parameters on a large number of different spatial, temporal, and spectral resolutions. Accordingly, NASA is developing new approaches to data collection and distribution in order to improve access to multiple sources of data streams to increase spatial and temporal coverage. Methods are being developed to incorporate different, scalable capabilities to handle multiple data sources by adding, deleting and replacing components as required as well as associated tools for their management. An approach has been to search for innovative solutions focused on the creation, use and manipulation of data stored in many different archives. These include data transformation and combination as well as data and information tools that can assist the public health and science community to use existing and anticipated products in new and flexible ways. This presentation will provide an inventory of geophysical parameters derived from satellite remote sensing sensors that are useful for GIS-based public health studies. The presentation will also discuss the physical and scientific limitations of access to and use of these data for health applications such as resolution and format differences, lack of software interoperability, data access problems. Finally, there will be a summary of the recent steps the NASA program has taken to bring NASA-generated satellite products to a wider range of users in the GIS community.

Vicente, Gilberto A.↗

Improving Geolocation Accuracy of the Advanced Meteorological Imager on the GEO-KOMPSAT-2A

GeoNEX is a collaborative project led by scientists from NASA and many other international institutes to generate Earth monitoring products using data streams from the latest geostationary (GEO) sensors. Its consistent processing and common gridding systems can produce research-quality data products from GEO sensors and leverage GEO-GEO or GEO-LEO (low earth orbit) synergistic uses. Currently, GeoNEX has produced and disseminated L1G (geometrically corrected Level 1 products) from GOES 16/17 ABIs and Himawari-8 AHI, but a new Korean geostationary sensor (Advanced Meteorological Imager, AMI) onboard Geo-KOMPSAT-2A covering a large proportion of Asia and all of Oceania is in development. Our recent efforts on assessing geolocation accuracy in ABI and AHI suggest a nontrivial residual exists in both level 1B data with varying spatiotemporal patterns. The findings urge us to prioritize identifying and correcting geolocation residuals of AMI to generate accurate and consistent GeoNEX top-of-atmosphere (TOA) reflectance products and following processing chains. Here we implement a phase correlation correction approach to a visible band (0.64 μm, 500 m) using landmarks prepared from finer scale digital terrain models. We characterize spatiotemporal patterns (e.g., diurnal & daily) of geolocation residuals of AMI before and after correction. We then assess stability of datasets and quantify impact of unexpected geolocation errors on terrestrial monitoring. The geolocation corrected AMI data are further compared with GeoNEX AHI L1G products which are able to create unique stereo-type observations with AMI through leveraging the similarities of spectral bands and the sun-target-sensor geometry. Further, we discuss challenges in utilizing the GEO-GEO (e.g., AMI & AHI) satellite data for potential applications.

Geostationary Satellites↗

Definition and fabrication of an airborne scatterometer radar signal processor

A hardware/software system which incorporates a microprocessor design and software for the calculation of normalized radar cross section in real time was developed. Interface is provided to decommutate the NASA ADAS data stream for aircraft parameters used in processing and to provide output in the form of strip chart and pcm compatible data recording.

Source record↗

User data management

The primary objective is to identify, develop, and demonstrate key data management technologies to support user access to Space Station data. To accomplish this objective, there are several technical challenges which must be addressed. The first is how to provide routine customer access to high volume, dynamic and distributed data bases. This access will encompass the functions of mission and payload planning and operations, data processing and analysis, and data archive and distribution. Secondly, there must be some analysis of architectures for handling high volume data streams like those expected from the Space Station. This analysis will examine the use of packetized versus non-packetized data formats, modular expansion capabilities, real-time versus non-real-time data processing, and the interfaces and architecture required to support telescience operations. The task will also determine benchmarks of performance capabilities for technology operations, such as varied data base structures, data access procedures, distributed data base design, and data base machines. Information is provided here in outline form.

Source record↗

Do Better Satellite Precipitation Algorithms Improve Landslide Hazard Assessment?

Satellites make it possible to estimate precipitation in near real time. Given the challenges of achieving global coverage by other means, these data are used widely. However, few systems for landslide hazard assessment rely on satellite precipitation estimates. This could be due in part to perceptions of accuracy, although latency, spatial resolution, and other factors may also be important. We test whether recent changes to data streams from the Global Precipitation Measurement mission (GPM) have improved its potential for use in landslide prediction. Specifically, we examine data produced by the Integrated Multi-satellitERetrievals for the GPM (IMERG) algorithm, which was upgraded to version 7 this year. IMERG relies upon other algorithms, including the Goddard Profiling Algorithm (GPROF) and the GPM Combined Radar-Radiometer Algorithm (CORRA). Many changes have been made during the switch from IMERG version 6 to version 7. These include upgrading CORRA and GPROF to version 7, to improve the accuracy of precipitation in frozen, mountainous, and coastal areas. The measured intensity of some storms has been enhanced with a new algorithm, the Scheme for Histogram Adjustment with Ranked Precipitation Estimates in the Neighborhood. Combined with many others, these changes to IMERG should improve its utility for landslide hazard assessment in a variety of contexts. To test this idea, we retrain the global Landslide Hazard Assessment for Situational Awareness (LHASA) model twice—first with data from IMERG version 6B and second with 7B. Since current daily rainfall is the most important variable in determining outcomes predicted by LHASA, it should reflect changes made to that input. First, we grid the landslides at a daily, thirty-arcsecond resolution. This serves as the response variable. At each of these sites current and antecedent rainfall are extracted, along with antecedent snow mass and soil moisture, slope, and PGA. In addition, one million grid cells are selected at random points to represent conditions under which landslides (probably) do not occur. After merging these data, we hold back 20% of the dataset for validation purposes and train a machine-learning model with the rest. We assess both the model’s overall ability to identify landslides and its ability to predict specific large landslide disasters.

Thomas A Stanley↗

Redis-Based Streaming Architecture for Accelerator Beam Instrumentation DAQ Systems

The Fermilab Acceleraor Division, Beam Instrumentation Department, is always adopting modern and current software methodologies for complex DAQ architectures. This paper highlights the Redis Adapter (RA) as the key software component enabling high performance, modular communication between digitizers and distributed control systems by leveraging Redis and containerization. The RA provides a unified, efficient interface between Redis based data streams and consumer systems. In the legacy architecture, digitized data flowed through the custom, UDP based Distributed Data Communication Protocol in the middle layer. In the current system, DDCP remains the ingestion path, while the RA serves as the decoupling layer. The proposed system replaces old VME digitizers with a SOM-based digitizer that communicates with Redis using the RA. The RA acts as both a performance-critical bridge and a protocol-agnostic adapter, ensuring compatibility with legacy control frameworks while enabling future scalability and modularity. This restructuring of the middle layer also helps the system achieve high throughput, reduce latency, and simplify the data path. Finally, we will demonstrate how RA is utilized in our two core products to deliver both legacy compatibility and future flexibility.

Joshi, S. [Fermilab]↗

Extended testing of a general contextual classifier using the massively parallel processor - Preliminary results and test plans

Earlier encouraging test results of a contextual classifier that combines spatial and spectral information employing a general statistical approach are expanded. The earlier results were of limited meaning because they were produced from small (50-by-50 pixel) data sets. An implementation of the contextual classifier on NASA Goddard's Massively Parallel Processor (MPP) is presented; for the first time the MPP makes feasible the testing of the classifier on large data sets (a 12-hour test on a VAX-11/780 minicomputer now takes 5 minutes on the MPP). The MPP is a Single-Instruction, Multiple Data Stream computer, consisting of 16,384 bit serial microprocessors connected in a 128-by-128 mesh array with each element having data transfer connections with its four nearest neighbors so that the MPP is capable of billions of operations per second. Preliminary results are given (with more expected for the conference) and plans are mentioned for extended testing of the contextual classifier on Thematic Mapper data sets.

Tilton, J. C.↗

The Atacama Cosmology Telescope: Data Characterization and Map Making

We present a description of the data reduction and mapmaking pipeline used for the 2008 observing season of the Atacama Cosmology Telescope (ACT). The data presented here at 148 GHz represent 12% or the 90 TB collected by ACT from 2007 to 2010. In 2008 we observed for 136 days, producing a total of 142h of data (11 TB for the 148 GHz band only), with a daily average of 10.5 h of observation. From these, 108.5 h were devoted to 850 sq deg stripe (11.2 h by 9 deg.1) centered on a declination of -52 deg.7, while 175 h were devoted to a 280 square deg stripe (4.5 h by 4 deg.8) centered at the celestial equator. We discuss sources of statistical and systematic noise, calibration, telescope pointing and data selection. Out of 1260 survey hours and 1024 detectors per array, 816 h and 593 effective detectors remain after data selection for this frequency band, yielding a 38 % survey efficiency. The total sensitivity in 2008, determined from the noise level between 5 Hz and 20 Hz in the time-ordered data stream (TOD), is 32 muK square root of s in CMB units. Atmospheric brightness fluctuations constitute the main contaminant in the data and dominate the detector and noise covariance at low frequencies in the TOD. The maps were made by solving the lease squares problem using the Preconditioned Conjugate Gradient method, incorporating the details of the detector and noise correlations. Cross-correlation with WMAP sky maps as well as analysis from simulations reveal the our maps are unbiased at l > 300. This paper accompanies the public release of the 148 GHz southern stripe maps from 2008. The techniques described here will be applied to future maps and data releases.

Duenner, Rolando↗

Suomi NPP VIIRS Prelaunch and On-orbit Geometric Calibration and Characterization

The Visible Infrared Imager Radiometer Suite (VIIRS) sensor was launched 28 October 2011 on the Suomi National Polarorbiting Partnership (SNPP) satellite. VIIRS has 22 spectral bands covering the spectrum between 0.412 m and 12.01 m, including 16 moderate resolution bands (M-bands) with a spatial resolution of 750 m at nadir, 5 imaging resolution bands (I-bands) with a spatial resolution of 375 m at nadir, and 1 day-night band (DNB) with a near-constant 750 m spatial resolution throughout the scan. These bands are located in a visible and near infrared (VisNIR) focal plane assembly (FPA), a short- and mid-wave infrared (SWMWIR) FPA and a long-wave infrared (LWIR) FPA. All bands, except the DNB, are co-registered for proper environmental data records (EDRs) retrievals. Observations from VIIRS instrument provide long-term measurements of biogeophysical variables for climate research and polar satellite data stream for the operational communitys use in weather forecasting and disaster relief and other applications. Well Earth-located (geolocated) instrument data is important to retrieving accurate biogeophysical variables. This paper describes prelaunch pointing and alignment measurements, and the two sets of on-orbit correction of geolocation errors, the first of which corrected error from 1,300 m to within 75 m (20 I-band pixel size), and the second of which fine tuned scan angle dependent errors, bringing VIIRS geolocation products to high maturity in one and a half years of the SNPP VIIRS on-orbit operations. Prelaunch calibration and the on-orbit characterization of sensor spatial impulse responses and band-to-band co-registration (BBR) are also described.

Geolocation↗

Preliminary Findings of Inflight Icing Field Test to Support Icing Remote Sensing Technology Assessment

NASA and the National Center for Atmospheric Research have developed an icing remote sensing technology that has demonstrated skill at detecting and classifying icing hazards in a vertical column above an instrumented ground station. This technology has recently been extended to provide volumetric coverage surrounding an airport. Building on the existing vertical pointing system, the new method for providing volumetric coverage will utilize a vertical pointing cloud radar, a multifrequency microwave radiometer with azimuth and elevation pointing, and a NEXRAD radar. The new terminal area icing remote sensing system processes the data streams from these instruments to derive temperature, liquid water content, and cloud droplet size for each examined point in space. These data are then combined to ultimately provide icing hazard classification along defined approach paths into an airport.

remote sensors↗

Analysis of 2015 Winter In-Flight Icing Case Studies with Ground-Based Remote Sensing Systems Compared to In-Situ SLW Sondes

National Aeronautics and Space Administration (NASA) and the National Center for Atmospheric Research (NCAR) have developed an icing remote sensing technology that has demonstrated skill at detecting and classifying icing hazards in a vertical column above an instrumented ground station. This technology has recently been extended to provide volumetric coverage surrounding an airport. Building on the existing vertical pointing system, the new method for providing volumetric coverage utilizes a vertical pointing cloud radar, a multi-frequency microwave radiometer with azimuth and elevation pointing, and a NEXRAD radar. The new terminal area icing remote sensing system processes the data streams from these instruments to derive temperature, liquid water content, and cloud droplet size for each examined point in space. These data are then combined to ultimately provide icing hazard classification along defined approach paths into an airport. To date, statistical comparisons of the vertical profiling technology have been made to Pilot Reports and Icing Forecast Products. With the extension into relatively large area coverage and the output of microphysical properties in addition to icing severity, the use of these comparators is not appropriate and a more rigorous assessment is required. NASA conducted a field campaign during the early months of 2015 to develop a database to enable the assessment of the new terminal area icing remote sensing system and further refinement of terminal area icing weather information technologies in general. In addition to the ground-based remote sensors listed earlier, in-situ icing environment measurements by weather balloons were performed to produce a comprehensive comparison database. Balloon data gathered consisted of temperature, humidity, pressure, super-cooled liquid water content, and 3-D position with time. Comparison data plots of weather balloon and remote measurements, weather balloon flight paths, bulk comparisons of integrated liquid water content and icing cloud extent agreement, and terminal-area hazard displays are presented. Discussions of agreement quality and paths for future development are also included.

aviation meteorology↗