Search NASA⌕ Search

SEARCH · Search NASA

Results for “data reduction”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 469 records · Page 26

The faint limit of the Hubble Space Telescope faint object spectrograph and rejection of the cosmic-ray background

The faintest object which can be observed with the Hubble Space Telescope (HST) Faint Object Spectrograph (FOS) is set by the detector cosmic-ray background and not by object flux. We use data from Beaver and Lyons to show that 48% of the background counts are from cosmic rays which each generate a near instantaneous burst of two or more counts. Setting the FOS threshold parameter REJLIM = 1, which rejects all frames with more than one count increases the ratio of signal-to-dark counts (S/D) by a factor of 1.94, regardless of the frame time or the object signal, because half of the dark counts which arrive in single counts (from either cosmic-ray bursts or thermal dark current photoemission) are rejected at the same rate as the object signal. But the signal-to-noise ratio (SNR proportional to S/square root of S + D) increases by at most a factor of 1.35, and for realistic parameters and taking into account dead time, we expect a gain in SNR of only 1.18. If a diode has failed and now emits noise, no data at all will be recorded. The chance of this occurring is approximately = 10% , and for this reason we do not recommend the use of REJLIM. The two-point correlation function of dark counts per pixel has strong peaks every four pixels, caused by the action of quarter-stepping on the counts from large bursts. The counts from such bursts spread over at least 80-100 diodes, and we show that such bursts can be rejected during data reduction if the data are recorded in RAPID mode with individual exposures of about 35 seconds. The SNR of the spectrum can also be improved by weighting each exposure by its SNR (a function ofthe mean dark count rate at that time), since dark varies by a factor of 2 around an orbit. These two procedures together increase the SNR by a factor of 1.1.3 (a 28% gain in exposure time) in regions of a spectrum where the object is much fainter than the background. We find that the Ly-alpha and O I sky emission lines give at most 4 (counts/array/s/sq arcsec), and typically half this number. With a small frame time and a small entrance aperture, these counts cause REJLIM = 1 to reject only a few percent of the data. We discuss only the HST FOS blue side detector, but the same ideas apply to any photon counter which records bursts of noise, and the calculations resemble those for the loss of signal due to coincidences in photon counters. In the ideal photon counter the dark counts are readily distinguished from photons. One way to do this is to add detectors which records only dark current, so that every cosmic ray produces more than one count and the instrument can be operated in an anti-coincidence mode.

Tytler, David↗

Tools and Data Services from the NASA Earth Satellite Observations for Remote Sensing Commercial Applications

Several commercial applications of remote sensing data, such as water resources management, environmental monitoring, climate prediction, agriculture, forestry, preparation for and migration of extreme weather events, require access to vast amounts of archived high quality data, software tools and services for data manipulation and information extraction. These on the other hand require gaining detailed understanding of the data's internal structure and physical implementation of data reduction, combination and data product production. The time-consuming task must be undertaken before the core investigation can begin and is an especially difficult challenge when science objectives require users to deal with large multi-sensor data sets of different formats, structures, and resolutions.

Vicente, Gilberto↗

Assessment of Technologies for the Space Shuttle External Tank Thermal Protection System and Recommendations for Technology Improvement - Part III: Material Property Characterization, Analysis, and Test Methods

The objective of this report is to contribute to the independent assessment of the Space Shuttle External Tank Foam Material. This report specifically addresses material modeling, characterization testing, data reduction methods, and data pedigree. A brief description of the External Tank foam materials, locations, and standard failure modes is provided to develop suitable background information. A review of mechanics based analysis methods from the open literature is used to provide an assessment of the state-of-the-art in material modeling of closed cell foams. Further, this report assesses the existing material property database and investigates sources of material property variability. The report presents identified deficiencies in testing methods and procedures, recommendations for additional testing as required, identification of near-term improvements that should be pursued, and long-term capabilities or enhancements that should be developed.

Gates, Thomas S.↗

Tools and Data Services from the NASA Earth Satellite Observations for Climate Applications

Climate science and applications require access to vast amounts of archived high quality data, software tools and services for data manipulation and information extraction. These on the other hand require gaining detailed understanding of the data's internal structure and physical implementation to data reduction, combination and data product production. This time-consuming task must be undertaken before the core investigation can begin and is an especially difficult challenge when science objectives require users to deal with large multi-sensor data sets of different formats, structures, and resolutions. In order to address these issues the Goddard Space Flight Center (GSFC) Earth Sciences (GES), Data and Information Service Center (DISC) Distributed Active Archive Center (DAAC) has made great progress in facilitating science and applications research by developing innovative tools and data services applied to the Earth sciences atmospheric and climate data. The GES/DISC/DAAC has successfully implemented and maintained a long-term climate satellite data archive and developed tools and services to a variety of atmospheric science missions including AIRS, AVHRR, MODIS, SeaWiFS, SORCE, TOMS, TOVS, TRMM, and UARS and Aura instruments providing researchers with excellent opportunities to acquire accurate and continuous atmospheric measurements. Since the number of climate science products from these various missions is steadily increasing as a result of more sophisticated sensors and new science algorithms, the main challenge for data centers like the GES/DISC/DAAC is to guide users through the variety of data sets and products, provide tools to visualize and reduce the volume of the data and secure uninterrupted and reliable access to data and related products. This presentation will describe the effort at the GES/DISC/DAAC to build a bridge between multi-sensor data and the effective scientific use of the data, with an emphasis on the heritage satellite observations and science products for climate applications. The intent is to inform users of the existence of this large collection of data and products; suggest starting points for cross-platform science projects and data mining activities and provide data services and tools information. More information about the GES/DISC/DAAC satellite data and products, tools, and services can be found at http://daac.gsfc.nasa.gov.

Vicente, Gilberto A.↗

Effects of data selection on the assimilation of AIRS data

The Atmospheric InfraRed Sounder (AIRS), flying aboard NASA's Earth Observing System (EOS) Aqua satellite with the Advanced Microwave Sounding Unit-A (AMSU-A), has been providing data for use in numerical weather prediction (NWP) and data assimilation systems (DAS) for over three years. The full AIRS data set is currently not transmitted in near-real-time (NRT) to the NWP centers. Instead, data sets with reduced spatial and spectral information are produced and made available in NRT. In this paper, we evaluate the use of different channel selections and error specifications. We achieved significant positive impact from the Aqua AIRS/AMSU-A combination in both hemispheres during our experimental time period of January 2003. The best results were obtained using a set of 156 channels that did not include any in the 6.7micron water vapor band. The latter have a large influence on both temperature and humidity analyses. If observation and background errors are not properly specified, the partitioning of temperature and humidity information from these channels will not be correct, and this can lead to a degradation in forecast skill. We found that changing the specified channel errors had a significant effect on the amount of data that entered into the analysis as a result of quality control thresholds that are related to the errors. However, changing the channel errors within a relatively small window did not significantly impact forecast skill with the 155 channel set. We also examined the effects of different types of spatial data reduction on assimilated data sets and NWP forecast skill. Whether we picked the center or the warmest AIRS pixel in a 3x3 array affected the amount of data ingested by the analysis but had a negligible impact on the forecast skill.

Joiner, Joanna↗

The HEASARC Swift Gamma-Ray Burst Archive: The Pipeline and the Catalog

Since its launch in late 2004, the Swift satellite triggered or observed an average of one gamma-ray burst (GRB) every 3 days, for a total of 771 GRBs by 2012 January. Here, we report the development of a pipeline that semi automatically performs the data-reduction and data-analysis processes for the three instruments on board Swift (BAT, XRT, UVOT). The pipeline is written in Perl, and it uses only HEAsoft tools and can be used to perform the analysis of a majority of the point-like objects (e.g., GRBs, active galactic nuclei, pulsars) observed by Swift. We run the pipeline on the GRBs, and we present a database containing the screened data, the output products, and the results of our ongoing analysis. Furthermore, we created a catalog summarizing some GRB information, collected either by running the pipeline or from the literature. The Perl script, the database, and the catalog are available for downloading and querying at the HEASARC Web site.

BURST ARCHIVE↗

Quality-Aware Data Management for Large Scale Scientific Applications

Increasingly larger scale simulations are generating an unprecedented amount of output data, causing researchers to explore new data staging methods that buffer, use, and/or reduce such data online rather than simply pushing it to disk. Leveraging the capabilities of data staging, this study explores the potential for data reduction via online data compression, first using general compression techniques and then proposing usespecific methods that permit users to define simple data queries that cause only the data identified by those queries to be emitted. Using online methods for code generation and deployment, with such dynamic data queries, end users can precisely identify the quality of information (QoI) of their output data, by explicitly determining what data may be lost vs. retained, in contrast to general-purpose lossy compression methods that do not provide such levels of control. The paper also describes the key elements of a quality-aware data management system (QADMS) for highend machines enabled by this approach. Initial experimental results demonstrate that QADMS can effectively reduce data movement cost and improve the QoS while meeting the QoI constraint stated by users.

Zou, Hongbo↗

Automatic Classification of Biological Targets in a Tidal Channel Using a Multibeam Sonar

Multibeam sonars are widely used for environmental monitoring of fauna at marine renewable energy sites. However, they can rapidly accrue vast volumes of data, which poses a challenge for data processing. Here, using data from a deployment in a tidal channel with peak currents of 1–2 m s –1 , we demonstrate the data-reduction benefits of real-time automatic classification of targets detected and tracked in multibeam sonar data. First, we evaluate classification capabilities for three machine learning algorithms: random forests, support vector machines, and k-nearest neighbors. For each algorithm, a hill-climbing search optimizes a set of hand-engineered attributes that describe tracked targets. Here, the random forest algorithm is found to be most effective—in postprocessing, discriminating between biological and nonbiological targets with a recall rate of 0.97 and a precision of 0.60. In addition, 89% of biological targets are correctly classified as either seals, diving birds, fish schools, or small targets. Model dependence on the volume of training data is evaluated. Second, a real-time implementation of the model is shown to distinguish between biological targets and nonbiological targets with nearly the same performance as in postprocessing. From this, we make general recommendations for implementing real-time classification of biological targets in multibeam sonar data and the transferability of trained models.

16 TIDAL AND WAVE POWER↗

Data-parallel Python for High Energy Physics Analyses

In this paper, we explore features available in Python which are useful for data reduction tasks in High Energy Physics (HEP). Highlevel abstractions in Python are convenient for implementing data reduction tasks. However, in order for such abstractions to be practical, the efficiency of their performance must also be high. Because the data sets we process are typically large, we care about both I/O performance and in-memory processing speed. In particular, we evaluate the use of data-parallel programming, using MPI and numpy, to process a large experimental data set (42 TiB) stored in an HDF5 file. We measure the speed of processing of the data, distinguishing between the time spent reading data and the time spent processing the data in memory, and demonstrate the scalability of both, using up to 1200 KNL nodes (76800 cores) on Cori at NERSC

Paterno, Marc↗

Analysis of Information Content in High-Spectral Resolution Sounders using Subset Selection Analysis

In this paper, we summarize the results of the sensitivity analysis and data reduction carried out to determine the information content of AIRS and IASI channels. The analysis and data reduction was based on the use of subset selection techniques developed in the linear algebra and statistical community to study linear dependencies in high dimensional data sets. We applied the subset selection method to study dependency among channels by studying the dependency among their weighting functions. Also, we applied the technique to study the information provided by the different levels in which the atmosphere is discretized for retrievals and analysis. Results from the method correlate well with intuition in many respects and point out to possible modifications for band selection in sensor design and number and location of levels in the analysis process.

Velez-Reyes, Miguel↗

Integrating ORNL’s HPC and Neutron Facilities with a Performance-Portable CPU/GPU Ecosystem

We explore the development of a performance-portable CPU/GPU ecosystem to integrate two of the US Department of Energy’s (DOE’s) largest scientific instruments, the Oak Ridge Leadership Computing facility and the Spallation Neutron Source (SNS), both of which are housed at Oak Ridge National Laboratory. We select a relevant data reduction workflow use-case to obtain the differential scattering cross-section from data collected by SNS’s CORELLI and TOPAZ instruments. We compare the current CPU-only production implementation using the Garnet Python multiprocess package based on the Mantid C++ framework against our proposed CPU/GPU implementation that uses the LLVM-based, just-in-time Julia scientific language and the JACC.jl performance-portable package. Two proxy apps were developed: (i) an app for extracting relevant Mantid kernels (MDNorm) in C++ and (ii) the Julia MiniVATES.jl miniapp. We present performance results for NVIDIA A100 and AMD MI100 GPUs and AMD EPYC 7513 and 7662 CPUs. The results provide insights for future generations of data reduction software that can embrace performance portability for an integrated research infrastructure across DOE’s experimental and computational facilities.

Hahn, Steven↗

Data Mining of Network Logs

The statement of purpose is to analyze network monitoring logs to support the computer incident response team. Specifically, gain a clear understanding of the Uniform Resource Locator (URL) and its structure, and provide a way to breakdown a URL based on protocol, host name domain name, path, and other attributes. Finally, provide a method to perform data reduction by identifying the different types of advertisements shown on a webpage for incident data analysis. The procedures used for analysis and data reduction will be a computer program which would analyze the URL and identify and advertisement links from the actual content links.

Collazo, Carlimar↗

New analysis indicates no thermal inversion in the atmosphere of HD 209458b

An important focus of exoplanet research is the determination of the atmospheric temperature structure of strongly irradiated gas giant planets, or hot Jupiters. HD 209458b is the prototypical exoplanet for atmospheric thermal inversions, but this assertion does not take into account recently obtained data or newer data reduction techniques. We reexamine this claim by investigating all publicly available Spitzer Space Telescope secondary-eclipse photometric data of HD 209458b and performing a self-consistent analysis. We employ data reduction techniques that minimize stellar centroid variations, apply sophisticated models to known Spitzer systematics, and account for time-correlated noise in the data. We derive new secondary-eclipse depths of 0.119% ± 0.007%, 0.123% ± 0.006%, 0.134% ± 0.035%, and 0.215% ± 0.008% in the 3.6, 4.5, 5.8, and 8.0 μm bandpasses, respectively. We feed these results into a Bayesian atmospheric retrieval analysis and determine that it is unnecessary to invoke a thermal inversion to explain our secondary-eclipse depths. The data are well fitted by a temperature model that decreases monotonically between pressure levels of 1 and 0.01 bars. We conclude that there is no evidence for a thermal inversion in the atmosphere of HD 209458b.

79 ASTRONOMY AND ASTROPHYSICS↗

Goldstone solar system radar

Planning, direction, experimental design, and coordination of data-acquisition and engineering activities in support of all Goldstone planetary radar astronomy were performed. This work demands familiarity with the various components of a planetary radar telescope (transmitter, receiver, antenna, computer hardware and software) as well as knowledge of how the entire system must function as a cohesive unit to meet the particular scientific objectives at hand in a given observation. Support radar data-processing facilities, currently being used for virtually all Goldstone data reduction includes: a VAX 11/780 computer system, an FPS 5210 array processor, terminals, tape drives, and image-display devices, as well as a large body of data-reduction software to accommodate the variety of data-acquisition formats and strategems. Successful 113-cm radar observation of Callisto and the near-Earth asteroid 1981 Midas and Goldstone/VLA radar observations of Saturn's rings were obtained. Quick-look verification programs from data taken with phase-coded cw (i.e., ranging) waveforms, applicable to Venus, the Moon, and small bodies were completed. Definition of scientific and engineering requirements on instrument performance, radar system configuration, and personnel, for all 1988 Goldstone radar investigations was accomplished.

Jurgens, Raymond F.↗

A Peculiar Class of Debris Disks from Herschel/DUNES: A Steep Fall Off in the Far Infrared

Context. The existence of debris disks around old main sequence stars is usually explained by continuous replenishment of small dust grains through collisions from a reservoir of larger objects. Aims. We present photometric data of debris disks around HIP 103389 (HD199260), HIP 100350 (HN Peg, HD206860), and HIP 114948 (HD 219482), obtained in the context of our Herschel Open TIme Key Program DUNES (DUst around NEarby Stars). Methods. We used Herschel/PACS to detect the thermal emission of the three debris disks with a 30 sigma sensitivity of a few mJy at l00 micron and 160 micron. In addition, we obtained Herschel/PACS photometric data at 70 micron for HIP 103389. These observations are complemented by a large variety of optical to far-infrared photometric data. Two different approaches are applied to reduce the Herschel data to investigate the impact of data reduction on the photometry. We fit analytical models to the available spectral energy distribution (SED) data using the fitting method of simulated therma1 annealing as well as a classical grid search method. Results. The SEDs of the three disks potentially exhibit an unusually steep decrease at wavelengths >= 70 micron. We investigate the significance of the peculiar shape of these SEDs and the impact on models of the disks provided it is real. Using grain compositions that have been applied successfully for modeling of many other debris disks, our modeling reveals that such a steep decrease of the SEDs in the long wavelength regime is inconsistent with a power-law exponent of the grain size distribution -3.5 expected from a standard equilibrium collisional cascade. In contrast, a steep grain size distribution or, alternatively an upper grain size in the range of few tens of micrometers are implied. This suggests that a very distinct range of grain sizes would dominate the thermal. emission of such disks. However, we demonstrate that the understanding of the data of faint sources obtained with Herschel is still incomplete and that the significance of our results depends on the version of the data reduction pipeline used. Conclusions. A new mechanism to produce the dust in the presented debris disks, deviations from the conditions required for a standard equilibrium collisional cascade (grain size exponent of -3.5), and/or significantly different dust properties would be necessary to explain the potentially steep SED shape of the three debris disks presented.

Ertel, S.↗

Reduction of Skylab 2 ground winds

A dampometer was designed and fabricated for measuring the amount of damping in the wind tunnel test model. The ground winds data reduction system (GWDRS) was taken to Langley Research Center in support of wind tunnel tests on the Skylab 2 program. Final data reduction of the test results was accomplished by the GWDRS.

Rolfes, M. J.↗