Search NASA⌕ Search

SEARCH · Search NASA

Results for “raw data”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 433 records · Page 24
All sourcesNASA NTRSDOE OSTI
NASA NTRS1995

Bringing "Scientific Expeditions" Into the Schools

Two new technologies, the FASTexpedition and Remote FAST, have been developed that provide remote, 3D, high resolution, dynamic, interactive viewing of scientific data (such as simulations or measurements of fluid dynamics). The FASTexpedition permits one to access scientific data from the World Wide Web, take guided expeditions through the data, and continue with self controlled expeditions through the data. Remote FAST permits collaborators at remote sites to simultaneously view an analysis of scientific data being controlled by one of the collaborators. Control can be transferred between sites. These technologies are now being used for remote collaboration in joint university, industry, and NASA projects in computational fluid dynamics (CFD) and wind tunnel testing. Also, NASA Ames Research Center has initiated a project to make scientific data and guided expeditions through the data available as FASTexpeditions on the World Wide Web for educational purposes. Previously, remote visualiZation of dynamic data was done using video format (transmitting pixel information) such as video conferencing or MPEG movies on the Internet. The concept for this new technology is to send the raw data (e.g., grids, vectors, and scalars) along with viewing scripts over the Internet and have the pixels generated by a visualization tool running on the viewer's local workstation. The visualization tool that is currently used is FAST (Flow Analysis Software Toolkit). The advantages of this new technology over using video format are: 1. The visual is much higher in resolution (1280xl024 pixels with 24 bits of color) than typical video format transmitted over the network. 2. The form of the visualization can be controlled interactively (because the viewer is interactively controlling the visualization tool running on his workstation). 3. A rich variety of guided expeditions through the data can be included easily. 4. A capability is provided for other sites to see a visual analysis of one site as the analysis is interactively performed. Control of the analysis can be passed from site to site. 5. The scenes can be viewed in 3D using stereo vision. 6. The network bandwidth used for the visualization using this new technology is much smaller than when using video format. (The measured peak bandwidth used was 1 Kbit/sec whereas the measured bandwidth for a small video picture was 500 Kbits/sec.)

Watson, Val↗
NASA NTRS2012

Revamping Spacecraft Operational Intelligence with Splunk

So what is Splunk? Instead of giving the technical details, which you can find online, I'll tell you what it did for me. Splunk slapped everything into one place, with one uniform format, and gave me the ability to forget about all these annoying details of where it is, how to parse it, and all that. Instead, I only need to interact with Splunk to find the data I need. This sounds simple and obvious, but it's surprising what you can do once you all of your data is indexed in one place. By having your data organized, querying becomes much easier. Let's say that I want to search telemetry for a sensor_name gtemp_1 h and to return all data that is at most five minutes old. And because Splunk can hook into a real ]time stream, this data will always be up-to-date. Extending the previous example, I can now aggregate all types of data into one view based in time. In this picture, I've got transaction logs, telemetry, and downlinked files all in one page, organized by time. Even though the raw data looks completely than this, I've defined interfaces that transform it into this uniform format. This gives me a more complete picture for the question what was the spacecraft doing at this particular time? And because querying data is simple, I can start with a big block of data and whiddle it down to what I need, rather than hunting around for the individual pieces of data that I need. When we have all the data we need, we can begin widdling down the data with Splunk's Unix-like search syntax. These three examples highlights my trial-and-error attempts to find large temperature changes. I begin by showing the first 5 temperatures, only to find that they're sorted chronologically, rather than from highest temperatures to lowest temperatures. The next line shows sorting temperatures by their values, but I find that that fs not really what I want either. I want to know the delta temperatures between readings. Looking through Splunk's user manual, I find the delta function, which lets me dynamically generate new information to use in my query. With that extra piece of information, I can now return only the telemetry readings where the temperature changed by at least 10. One other useful feature I'll mention is that all of these queries can be run through Splunk's API. So any scripting language you can think of can plug right in and make these queries. This gives us the ability to build a lot of new tools.

operational intelligence↗
DOE OSTI2025

Supporting Data for "On the magnetic contribution of itinerant electrons to neutron diffraction in the topological antiferromagnet CeAlGe"

Contents of this DOI are data for the research paper "On the magnetic contribution of itinerant electrons to neutron diffraction in the topological antiferromagnet CeAlGe" by the authors: V. Pomjakushin et al. The dataset contains raw data from neutron scattering experiments of CeAlGe powder performed at the CNCS spectrometer at Spallation Neutron Source at ORNL.

36 MATERIALS SCIENCE↗
NASA NTRS2005

Public Outreach and Archiving of Data from the High Resolution Stereo Camera Onboard Mars Express: 2004 The First Year

During the first year of operation, corresponding to the calendar year 2004, the HRSC imaging experiment onboard ESA's Mars Express mission recorded 23 Gigabyte of 8-bit compressed raw data. After processing, the amount of data increased to more than 344 Gigabyte of decompressed and radiometrically calibrated scientifically useable image products. Every six months these HRSC Level 2 data are fed into ESA's Planetary Science Archive (PSA) that sends all data also to the Planetary Data System (PDS) to ensure easy availability to the interested user. On their respective web portals, the European Space Agency published in cooperation with the Principal Investigator-Group at Freie Universitat Berlin and the German Space Agency (DLR) almost 40 sets of high-level image scenes and movies for PR needs that have been electronically visited many hundred thousand times.

Koehler, U.↗
DOE OSTI2025

Legacy Survey of Space and Time Data Preview 1: raw dataset type

The Legacy Survey of Space and Time Data Preview 1 (DP1) is the first release of data from the NSF-DOE Vera C. Rubin Observatory. It consists of raw and calibrated single-epoch images, co-adds, difference images, detection catalogs, and other derived data products. DP1 is based on 1792 science-grade optical/near-infrared exposures acquired over 48 distinct nights by the Rubin Commissioning Camera, LSSTComCam, on the Simonyi Survey Telescope at the Summit Facility on Cerro Pachón, Chile during the first on-sky commissioning campaign in late 2024. DP1 covers a total of approximately 15 sq. deg. over seven roughly equally-sized non-contiguous fields, each independently observed in six broad photometric bands, ugrizy, spanning a range of stellar densities and latitudes and overlapping with external reference datasets. This dataset is a subset of the full data release consisting of the raw dataset type. These are unprocessed images from the LSST Commissioning Camera. This release contains 16,125 datasets of this type.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2026

Federated learning for 2D synchrotron x-ray diffractometry: a cross-institutional approach for phase quantification of Ti–6Al–4V alloy

High-energy Two dimensional (2D) synchrotron x-ray diffractometry provides important insights into the atomistic structure and phase evolution of materials, yet traditional analysis methods remain complex, knowledge-intensive, and computationally demanding. Deep-learning models offer a powerful alternative for automating their analysis. Institutions that hold these datasets may be unwilling to share their data due to privacy and security policies, as well as the challenges associated with large-scale data transfer. As a result, models trained on local datasets often perform well only on their own data but exhibit bias and poor generalization across different instruments or facilities. To overcome these limitations, we explore federated learning (FL) for 2D synchrotron diffractograms, enabling collaborative model training without exchanging raw data. In this study, 2D synchrotron diffractograms of Ti–6Al–4V alloy collected from two independent facilities are used to train convolutional neural networks for predicting the β-phase volume fraction. Experimental results show that federated global models significantly outperform locally trained models in terms of generalization and achieve accuracy comparable to centralized trained models. These findings demonstrate the potential of FL to enable secure, cross-institutional collaboration and enhance the scalability of deep-learning-based materials characterization.

36 MATERIALS SCIENCE↗
NASA NTRS1982

The Use of Principal Components for Creating Improved Imagery for Geometric Control Point Selection

A directed principal component (PC) analysis and its transformation was applied to 7-channel thematic mapper simulator (TMS) data and 4-channel LANDSAT multispectral scanner system (MSS) data collected over the city of Lancaster, Pennsylvania, to create improved imagery for geometric control point selection for image to image registration. Nineteen temporally stable geometric control points, such as road interactions and bridges, were selected for a 236 sq km area. The control points were visible on both the TMS and MSS imagery. On the first attempt the corresponding image control points were selected on both data sets without using the principal components transformation. Many of the road intersection locations were visible but the actual road crossings could not be distinguished. As a result, mensuration errors using raw data exceeded the equivalent of two (79 x 79 m) pixels. The application of a guided principal components transformation yielded TMS and MSS single band images showing improved detail in the scene's urban and residential infrastructure. The PC transformed data sets were then utilized for the reselection of geometric control points. By shown greater detail, control points on both the TMS and MSS imagery could be located with greater precision using the PC transformed data.

Imhoff, M. L.↗
NASA NTRS1976

Program on Earth Observation Data Management Systems (EODMS)

An assessment was made of the needs of a group of potential users of satellite remotely sensed data (state, regional, and local agencies) involved in natural resources management in five states, and alternative data management systems to satisfy these needs are outlined. Tasks described include: (1) a comprehensive data needs analysis of state and local users; (2) the design of remote sensing-derivable information products that serve priority state and local data needs; (3) a cost and performance analysis of alternative processing centers for producing these products; (4) an assessment of the impacts of policy, regulation and government structure on implementing large-scale use of remote sensing technology in this community of users; and (5) the elaboration of alternative institutional arrangements for operational Earth Observation Data Management Systems (EODMS). It is concluded that an operational EODMS will be of most use to state, regional, and local agencies if it provides a full range of information services -- from raw data acquisition to interpretation and dissemination of final information products.

Eastwood, L. F., Jr.↗
NASA NTRS1994

TOPEX/POSEIDON tides estimated using a global inverse model

Altimetric data from the TOPEX/POSEIDON mission will be used for studies of global ocean circulation and marine geophysics. However, it is first necessary to remove the ocean tides, which are aliased in the raw data. The tides are constrained by the two distinct types of information: the hydrodynamic equations which the tidal fields of elevations and velocities must satisfy, and direct observational data from tide gauges and satellite altimetry. Here we develop and apply a generalized inverse method, which allows us to combine rationally all of this information into global tidal fields best fitting both the data and the dynamics, in a least squares sense. The resulting inverse solution is a sum of the direct solution to the astronomically forced Laplace tidal equations and a linear combination of the representers for the data functionals. The representer functions (one for each datum) are determined by the dynamical equations, and by our prior estimates of the statistics or errors in these equations. Our major task is a direct numerical calculation of these representers. This task is computationally intensive, but well suited to massively parallel processing. By calculating the representers we reduce the full (infinite dimensional) problem to a relatively low-dimensional problem at the outset, allowing full control over the conditioning and hence the stability of the inverse solution. With the representers calculated we can easily update our model as additional TOPEX/POSEIDON data become available. As an initial illustration we invert harmonic constants from a set of 80 open-ocean tide gauges. We then present a practical scheme for direct inversion of TOPEX/POSEIDON crossover data. We apply this method to 38 cycles of geophysical data records (GDR) data, computing preliminary global estimates of the four principal tidal constituents, M(sub 2), S(sub 2), K(sub 1) and O(sub 1). The inverse solution yields tidal fields which are simultaneously smoother, and in better agreement with altimetric and ground truth data, than previously proposed tidal models. Relative to the 'default' tidal corrections provided with the TOPEX/POSEIDON GDR, the inverse solution reduces crossover difference variances significantly (approximately 20-30%), even though only a small number of free parameters (approximately equal to 1000) are actually fit to the crossover data.

Egbert, Gary D.↗
DOE OSTI2025

Scalable Hybrid Learning Techniques for Scientific Data Compression

Data compression is becoming critical for storing scientific data because many scientific applications need to store large amounts of data and post process this data for scientific discovery. Unlike image and video compression algorithms that limit errors to primary data (PD), scientists require compression techniques that accurately preserve derived quantities of interest (QoIs). Here, this article presents a physics-informed compression technique implemented as an end-to-end, scalable, GPU-based pipeline for data compression that addresses this requirement. Our hybrid compression technique combines machine learning techniques and standard compression methods. Specifically, we combine an autoencoder, an error-bounded lossy compressor to provide guarantees on raw data error, and a constraint satisfaction post-processing step to preserve the QoIs within a minimal error (generally less than floating point error). The effectiveness of the data compression pipeline is demonstrated by compressing nuclear fusion simulation data generated by a large-scale fusion code, XGC, which produces hundreds of terabytes of data in a single day. Our approach works within the ADIOS framework and results in compression by a factor of more than 150 while requiring only a few percent of the computational resources necessary for generating the data, making the overall approach highly effective for practical scenarios.

ITER↗
DOE OSTI2026

QProR: An Efficient Framework for Quantity-of-Interest Based Progressive Retrieval with Guaranteed Error Control

Scientific applications generate an unprecedented volume of data, overwhelming the network and file systems’ bandwidth and posing challenges for efficient and scalable data retrieval and analysis. Progressive data compression offers a promising solution by enabling on-demand retrieval at reduced size. However, existing progressive methods either fail to bound the errors in essential quantities of interest (QoIs) derived from raw data or suffer from suboptimal retrieval efficiency. In this work, we propose QProR, an efficient QoI-based progressive framework that optimizes progressive retrieval for target QoIs. Our key contributions include: (1) a systematic framework that integrates error-controlled lossy compressors with bitplane encoding while decoupling the two processes for high flexibility and adaptability; (2) a novel weighted bitplane encoding method which incorperates QoI knowledge into data refactoring to enhance retrieval efficiency; (3) an optimized retrieval strategy that accounts for the varying impacts of different variables on multivariate QoIs; (4) comprehensive evaluations using six real-world datasets from multiple scientific applications and thorough comparisons against state of the arts. Experimental results demonstrate that QProR achieves up to 80.38% reduction in the retrieval size under the same requested QoI error tolerance, when compared with the best-performing existing methods. When transferring 384 GB of scientific data to remote sites, QProR delivers up to 1.68 × speedup in the end-to-end data transfer performance.

Li, Wenbo [University of Kentucky]↗
DOE OSTI2025

Arctic shrub and Eriophorum leaf and root decomposition, northern Alaska, 2017-2018

This data package contains litter decomposition data collected from 170 plots of rapidly expanding shrub genera (Alnus, Betula, and Salix) and a widespread sedge (Eriophorum vaginatum) along a latitudinal and temperature gradient in northern Alaska. These data were produced from a litter bag experiment that took place from July 2017 to July 2018 and include mass loss and nitrogen loss decomposition metrics for both leaf and root litters. These raw data support a submitted manuscript that examines the variability in decomposition between shrub and graminoid leaf and root litters across a 1-year experiment across the graminoid-dominated Arctic tundra and reveals how deciduous shrub expansion affects litter decomposition in tundra ecosystems. Data are presented by site (n=5) and patch (shrub or sedge plot) in csv files. The site and plot location data and environmental measurement data are provided in Fraterrigo and Chen (2020). Additional methods regarding plot distribution and environmental measurements are in Chen et al. (2020) and Fraterrigo et al. (2024).

54 ENVIRONMENTAL SCIENCES↗
NASA NTRS1994

Flight experience with lightweight, low-power miniaturized instrumentation systems

Engineers at the NASA Dryden Flight Research Facility (NASA-Dryden) have conducted two flight research programs with lightweight, low-power miniaturized instrumentation systems built around commercial data loggers. One program quantified the performance of a radio-controlled model airplane. The other program was a laminar boundary-layer transition experiment on a manned sailplane. The purpose of this article is to report NASA-Dryden personnel's flight experience with the miniaturized instrumentation systems used on these two programs. This article will describe the data loggers, the sensors, and the hardware and software developed to complete the systems. It also describes how the systems were used and covers the challenges encountered to make them work. Examples of raw data and derived results will be shown as well. For some flight research applications where miniaturized instrumentation is a requirement, the authors conclude that commercially available data loggers and sensors are viable alternatives. In fact, the data loggers and sensors make it possible to gather research-quality data in a timely and cost-effective manner.

LOW POWER MINIATURIZED INSTRUM↗
NASA NTRS2000

Generating Accurate Urban Area Maps from Nighttime Satellite (DMSP/OLS) Data

There has been an increasing interest by the international research community to use the nighttime acquired "city-lights" data sets collected by the US Defense Meteorological Satellite Program's Operational Linescan system to study issues relative to urbanization. Many researchers are interested in using these data to estimate human demographic parameters over large areas and then characterize the interactions between urban development , natural ecosystems, and other aspects of the human enterprise. Many of these attempts rely on an ability to accurately identify urbanized area. However, beyond the simple determination of the loci of human activity, using these data to generate accurate estimates of urbanized area can be problematic. Sensor blooming and registration error can cause large overestimates of urban land based on a simple measure of lit area from the raw data. We discuss these issues, show results of an attempt to do a historical urban growth model in Egypt, and then describe a few basic processing techniques that use geo-spatial analysis to threshold the DMSP data to accurately estimate urbanized areas. Algorithm results are shown for the United States and an application to use the data to estimate the impact of urban sprawl on sustainable agriculture in the US and China is described.

Imhoff, Marc↗
NASA NTRS1996

Fast 3D Net Expeditions: Tools for Effective Scientific Collaboration on the World Wide Web

Two new technologies, the FASTexpedition and Remote FAST, have been developed that provide remote, 3D (three dimensional), high resolution, dynamic, interactive viewing of scientific data. The FASTexpedition permits one to access scientific data from the World Wide Web, take guided expeditions through the data, and continue with self controlled expeditions through the data. Remote FAST permits collaborators at remote sites to simultaneously view an analysis of scientific data being controlled by one of the collaborators. Control can be transferred between sites. These technologies are now being used for remote collaboration in joint university, industry, and NASA projects. Also, NASA Ames Research Center has initiated a project to make scientific data and guided expeditions through the data available as FASTexpeditions on the World Wide Web for educational purposes. Previously, remote visualization of dynamic data was done using video format (transmitting pixel information) such as video conferencing or MPEG (Motion Picture Expert Group) movies on the Internet. The concept for this new technology is to send the raw data (e.g., grids, vectors, and scalars) along with viewing scripts over the Internet and have the pixels generated by a visualization tool running on the viewers local workstation. The visualization tool that is currently used is FAST (Flow Analysis Software Toolkit). The advantages of this new technology over using video format are: (1) The visual is much higher in resolution (1280x1024 pixels with 24 bits of color) than typical video format transmitted over the network. (2) The form of the visualization can be controlled interactively (because the viewer is interactively controlling the visualization tool running on his workstation). (3) A rich variety of guided expeditions through the data can be included easily. (4) A capability is provided for other sites to see a visual analysis of one site as the analysis is interactively performed. Control of the analysis can be passed from site to site. (5) The scenes can be viewed in 3D using stereo vision. (6) The network bandwidth for the visualization using this new technology is much smaller than when using video format. (The measured peak bandwidth used was 1 Kbit/sec whereas the measured bandwidth for a small video picture was 500 Kbits/sec.) This talk will illustrate the use of these new technologies and present a proposal for using these technologies to improve science education.

Watson, Val↗
NASA NTRS2004

Multi-Sensor Distributive On-Line Processing, Visualization, and Analysis Infrastructure for an Agricultural Information System at the NASA Goddard Earth Sciences DAAC

The Goddard Space Flight Center Earth Sciences Data and Information Services Center (GES DISC) Distributed Active Center (DAAC) is developing an Agricultural Information System (AIS), evolved from an existing TRMM On-line Visualization and Analysis System precipitation and other satellite data products and services. AIS outputs will be ,integrated into existing operational decision support system for global crop monitoring, such as that of the U.N. World Food Program. The ability to use the raw data stored in the GES DAAC archives is highly dependent on having a detailed understanding of the data's internal structure and physical implementation. To gain this understanding is a time-consuming process and not a productive investment of the user's time. This is an especially difficult challenge when users need to deal with multi-sensor data that usually are of different structures and resolutions. The AIS has taken a major step towards meeting this challenge by incorporating an underlying infrastructure, called the GES-DISC Interactive Online Visualization and Analysis Infrastructure or "Giovanni," that integrates various components to support web interfaces that ,allow users to perform interactive analysis on-line without downloading any data. Several instances of the Giovanni-based interface have been or are being created to serve users of TRMM precipitation, MODIS aerosol, and SeaWiFS ocean color data, as well as agricultural applications users. Giovanni-based interfaces are simple to use but powerful. The user selects geophysical ,parameters, area of interest, and time period; and the system generates an output ,on screen in a matter of seconds.

Teng, William↗
DOE OSTI2026

Intelligent experiments through real-time AI: Fast Data Processing and Autonomous Detector Control for sPHENIX and future EIC detectors (Phase-I)

With an ever increasing demand for high precision data from modern detectors for discovery science and precision measurements, all major high energy nuclear and particle experiments, current and future, are facing the challenge on how to deal with the large volume of raw data generated from sophisticated state-of-the-art detectors in high rate collisions. These goals need to be balanced with available hardware and cost limits on DAQ (Data AcQuisition system) bandwidth and offline computing resources to capture, store and process the signal events. Two prototypical examples are the upcoming sPHENIX experiment, the DOE next generation heavy ion physics experiment at the Relativistic Heavy Ion Collider at BNL, and the future EIC experiments that are planned to be online circa 2030.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗