Search NASA⌕ Search

SEARCH · Search NASA

Results for “data processing steps”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 127 records · Page 7

A Framework for Propagation of Uncertainties in the Kepler Data Analysis Pipeline

The Kepler space telescope is designed to detect Earth-like planets around Sun-like stars using transit photometry by simultaneously observing 100,000 stellar targets nearly continuously over a three and a half year period. The 96-megapixel focal plane consists of 42 charge-coupled devices (CCD) each containing two 1024 x 1100 pixel arrays. Cross-correlations between calibrated pixels are introduced by common calibrations performed on each CCD requiring downstream data products access to the calibrated pixel covariance matrix in order to properly estimate uncertainties. The prohibitively large covariance matrices corresponding to the ~75,000 calibrated pixels per CCD preclude calculating and storing the covariance in standard lock-step fashion. We present a novel framework used to implement standard propagation of uncertainties (POU) in the Kepler Science Operations Center (SOC) data processing pipeline. The POU framework captures the variance of the raw pixel data and the kernel of each subsequent calibration transformation allowing the full covariance matrix of any subset of calibrated pixels to be recalled on-the-fly at any step in the calibration process. Singular value decomposition (SVD) is used to compress and low-pass filter the raw uncertainty data as well as any data dependent kernels. The combination of POU framework and SVD compression provide downstream consumers of the calibrated pixel data access to the full covariance matrix of any subset of the calibrated pixels traceable to pixel level measurement uncertainties without having to store, retrieve and operate on prohibitively large covariance matrices. We describe the POU Framework and SVD compression scheme and its implementation in the Kepler SOC pipeline.

Clarke, Bruce D.↗

NASA's EOSDIS, Trust and Certification

NASA's Earth Observing System Data and Information System (EOSDIS) has been in operation since August 1994, managing most of NASA's Earth science data from satellites, airborne sensors, filed campaigns and other activities. Having been designated by the Federal Government as a project responsible for production, archiving and distribution of these data through its Distributed Active Archive Centers (DAACs), the Earth Science Data and Information System Project (ESDIS) is responsible for EOSDIS, and is legally bound by the Office of Management and Budgets circular A-130, the Federal Records Act. It must follow the regulations of the National Institute of Standards and Technologies (NIST) and National Archive and Records Administration (NARA). It must also follow the NASA Procedural Requirement 7120.5 (NASA Space Flight Program and Project Management). All these ensure that the data centers managed by ESDIS are trustworthy from the point of view of efficient and effective operations as well as preservation of valuable data from NASA's missions. Additional factors contributing to this trust are an extensive set of internal and external reviews throughout the history of EOSDIS starting in the early 1990s. Many of these reviews have involved external groups of scientific and technological experts. Also, independent annual surveys of user satisfaction that measure and publish the American Customer Satisfaction Index (ACSI), where EOSDIS has scored consistently high marks since 2004, provide an additional measure of trustworthiness. In addition, through an effort initiated in 2012 at the request of NASA HQ, the ESDIS Project and 10 of 12 DAACs have been certified by the International Council for Science (ICSU) World Data System (WDS) and are members of the ICSUWDS. This presentation addresses questions such as pros and cons of the certification process, key outcomes and next steps regarding certification. Recently, the ICSUWDS and Data Seal of Approval (DSA) organizations merged their Core Trustworthy Data Repositories Requirements and require that members be recertified every three years. Given the rigor with which NASA manages the ESDIS Project and the DAACs, the recertification through WDSDSA, while involving some additional work, is a relatively simple process.

Certification↗

Meeting SPD-41a Requirements for BPS

In April 2021 BPS Formed the Science Data Management Work Group. This Poster describes the process and forward steps.

BPS Science Data Management↗

U.S. data processing for the IRAS project

The JPL's Scientific Data Analysis System (SDAS), which will process IRAS data and produce a catalogue of perhaps a million infrared sources in the sky, as well as other information for astronomical records, is described. The purposes of SDAS are discussed, and the major SDAS processors are shown in block diagram. The catalogue processing is addressed, mentioning the basic processing steps which will be applied to raw detector data. Signal reconstruction and conversion to astrophysical units, source detection, source confirmation, data management, and survey data products are considered in detail.

Duxbury, J. H.↗

Updating the Finite Element Model of the Aerostructures Test Wing using Ground Vibration Test Data

Improved and/or accelerated decision making is a crucial step during flutter certification processes. Unfortunately, most finite element structural dynamics models have uncertainties associated with model validity. Tuning the finite element model using measured data to minimize the model uncertainties is a challenging task in the area of structural dynamics. The model tuning process requires not only satisfactory correlations between analytical and experimental results, but also the retention of the mass and stiffness properties of the structures. Minimizing the difference between analytical and experimental results is a type of optimization problem. By utilizing the multidisciplinary design, analysis, and optimization (MDAO) tool in order to optimize the objective function and constraints; the mass properties, the natural frequencies, and the mode shapes can be matched to the target data to retain the mass matrix orthogonality. This approach has been applied to minimize the model uncertainties for the structural dynamics model of the Aerostructures Test Wing (ATW), which was designed and tested at the National Aeronautics and Space Administration (NASA) Dryden Flight Research Center (DFRC) (Edwards, California). This study has shown that natural frequencies and corresponding mode shapes from the updated finite element model have excellent agreement with corresponding measured data.

Lung, Shun-fat↗

Updating the Finite Element Model of the Aerostructures Test Wing Using Ground Vibration Test Data

Improved and/or accelerated decision making is a crucial step during flutter certification processes. Unfortunately, most finite element structural dynamics models have uncertainties associated with model validity. Tuning the finite element model using measured data to minimize the model uncertainties is a challenging task in the area of structural dynamics. The model tuning process requires not only satisfactory correlations between analytical and experimental results, but also the retention of the mass and stiffness properties of the structures. Minimizing the difference between analytical and experimental results is a type of optimization problem. By utilizing the multidisciplinary design, analysis, and optimization (MDAO) tool in order to optimize the objective function and constraints; the mass properties, the natural frequencies, and the mode shapes can be matched to the target data to retain the mass matrix orthogonality. This approach has been applied to minimize the model uncertainties for the structural dynamics model of the aerostructures test wing (ATW), which was designed and tested at the National Aeronautics and Space Administration Dryden Flight Research Center (Edwards, California). This study has shown that natural frequencies and corresponding mode shapes from the updated finite element model have excellent agreement with corresponding measured data.

Lung, Shun-Fat↗

JSC Toxicology Web Site

Data on chemical and biological materials to be flown in the pressurized volumes of habitable spacecraft, including the International Space Station (ISS), are needed by JSC toxicologists to assess the toxicity and assign hazard levels. This document defines submission schedules and establishes requirements for the types and format of these data. JSC 27472 Rev A is a major revision of JSC 25607, "Requirements for Submission of Test Sample-Materials Data for Shuttle Payload Safety Evaluations", dated October 1994, which was subsequently re-issued (September 1996) with a new document number, JSC 27472, but with the same title and date and no revisions. The revisions in the present document have been necessitated by the recent introduction of a two-step process (described in this document) for verification of data for flight materials and by the anticipated needs of the ISS. The requirements -for data submission apply to items which contain liquids, gases, gels, greases, powders/ particulates, radioisotopes, or biological materials and are located in the habitable pressurized volume of ISS or U.S. operated spacecraft. These include, but are not limited to, science payloads, government furnished equipment (GFE), risk mitigation experiments (RmEs), development test objectives (DTOs), detailed supplementary objectives (DSOs), life science experiments, and medical studies.

Garcia, Hector D.↗

Spaceborne Gravity Gradiometers

The current status of gravity gradiometers and technology that could be available in the 1990's for the GRAVSAT-B mission are assessed. Problems associated with sensors, testing, spacecraft, and data processing are explored as well as critical steps, schedule, and cost factors in the development plan.

Wells, W. C.↗

Interferometry theory for the block 2 processor

Presented is the interferometry theory for the Block 2 processor, including a high-level functional description and a discussion of data structure. The analysis covers the major processing steps: cross-correlation, fringe counter-rotation, transformation to the frequency domain, phase calibration, bandwidth synthesis, and extraction of the observables of amplitude, phase, phase rate, and delay. Also included are analyses for fractional bitshift correction, station clock error, ionosphere correction, and effective frequencies for the observables.

Thomas, J. B.↗

High Accuracy Evaluation of the Finite Fourier Transform Using Sampled Data

Many system identification and signal processing procedures can be done advantageously in the frequency domain. A required preliminary step for this approach is the transformation of sampled time domain data into the frequency domain. The analytical tool used for this transformation is the finite Fourier transform. Inaccuracy in the transformation can degrade system identification and signal processing results. This work presents a method for evaluating the finite Fourier transform using cubic interpolation of sampled time domain data for high accuracy, and the chirp Zeta-transform for arbitrary frequency resolution. The accuracy of the technique is demonstrated in example cases where the transformation can be evaluated analytically. Arbitrary frequency resolution is shown to be important for capturing details of the data in the frequency domain. The technique is demonstrated using flight test data from a longitudinal maneuver of the F-18 High Alpha Research Vehicle.

Morelli, Eugene A.↗

Using Docker Containers to Extend Reproducibility Architecture for the NASA Earth Exchange (NEX)

NASA Earth Exchange (NEX) is a data, supercomputing and knowledge collaboratory that houses NASA satellite, climate and ancillary data where a focused community can come together to address large-scale challenges in Earth sciences. As NEX has been growing into a petabyte-size platform for analysis, experiments and data production, it has been increasingly important to enable users to easily retrace their steps, identify what datasets were produced by which process chains, and give them ability to readily reproduce their results. This can be a tedious and difficult task even for a small project, but is almost impossible on large processing pipelines. We have developed an initial reproducibility and knowledge capture solution for the NEX, however, if users want to move the code to another system, whether it is their home institution cluster, laptop or the cloud, they have to find, build and install all the required dependencies that would run their code. This can be a very tedious and tricky process and is a big impediment to moving code to data and reproducibility outside the original system. The NEX team has tried to assist users who wanted to move their code into OpenNEX on Amazon cloud by creating custom virtual machines with all the software and dependencies installed, but this, while solving some of the issues, creates a new bottleneck that requires the NEX team to be involved with any new request, updates to virtual machines and general maintenance support. In this presentation, we will describe a solution that integrates NEX and Docker to bridge the gap in code-to-data migration. The core of the solution is saemi-automatic conversion of science codes, tools and services that are already tracked and described in the NEX provenance system, to Docker - an open-source Linux container software. Docker is available on most computer platforms, easy to install and capable of seamlessly creating and/or executing any application packaged in the appropriate format. We believe this is an important step towards seamless process deployment in heterogeneous environments that will enhance community access to NASA data and tools in a scalable way, promote software reuse, and improve reproducibility of scientific results.

earth exchange↗

Real-Time Visualization of an HPF-based CFD Simulation

Current time-dependent CFD simulations produce very large multi-dimensional data sets at each time step. The visual analysis of computational results are traditionally performed by post processing the static data on graphics workstations. We present results from an alternate approach in which we analyze the simulation data in situ on each processing node at the time of simulation. The locally analyzed results, usually more economical and in a reduced form, are then combined and sent back for visualization on a graphics workstation.

Kremenetsky, Mark↗

Processing Satellite Images on Tertiary Storage: A Study of the Impact of Tile Size on Performance

Before raw data from a satellite can be used by an Earth scientist, it must first undergo a number of processing steps including basic processing, cleansing, and geo-registration. Processing actually expands the volume of data collected by a factor of 2 or 3 and the original data is never deleted. Thus processing and storage requirements can exceed 2 terrabytes/day. Once processed data is ready for analysis, a series of algorithms (typically developed by the Earth scientists) is applied to a large number of images in a data set. The focus of this paper is how best to handle such images stored on tape using the following assumptions: (1) all images of interest to a scientist are stored on a single tape, (2) images are accessed and processed in the order that they are stored on tape, and (3) the analysis requires access to only a portion of each image and not the entire image.

Yu, JieBing↗

Resource Prospector Instrumentation for Lunar Volatiles Prospecting, Sample Acquisition and Processing

Data gathered from lunar missions within the last two decades have significantly enhanced our understanding of the volatile resources available on the lunar surface, specifically focusing on the polar regions. Several orbiting missions such as Clementine and Lunar Prospector have suggested the presence of volatile ices and enhanced hydrogen concentrations in the permanently shadowed regions of the moon. The Lunar Crater Observation and Sensing Satellite (LCROSS) mission was the first to provide direct measurement of water ice in a permanently shadowed region. These missions with other orbiting assets have laid the groundwork for the next step in the exploration of the lunar surface; providing ground truth data of the volatiles by mapping the distribution and processing lunar regolith for resource extraction. This next step is the robotic mission Resource Prospector (RP). Resource Prospector is a lunar mission to investigate 'strategic knowledge gaps' (SKGs) for in-situ resource utilization (ISRU). The mission is proposed to land in the lunar south pole near a permanently shadowed crater. The landing site will be determined by the science team with input from broader international community as being near traversable landscape that has a high potential of containing elevated concentrations of volatiles such as water while maximizing mission duration. A rover will host the Regolith & Environment Science and Oxygen & Lunar Volatile Extraction (RESOLVE) payload for resource mapping and processing. The science instruments on the payload include a 1-meter drill, neutron spectrometer, a near infrared spectrometer, an operations camera, and a reactor with a gas chromatograph-mass spectrometer for volatile analysis. After the RP lander safely delivers the rover to the lunar surface, the science team will guide the rover team on the first traverse plan. The neutron spectrometer (NS) and near infrared (NIR) spectrometer instruments will be used as prospecting tools to guide the traverse path. The NS will map the water-equivalent hydrogen concentration as low as 0.5% by weight to an 80 centimeter depth as the rover traverses the lunar landscape. The NIR spectrometer will measure surficial H2O/OH as well as general mineralogy. When the prospecting instruments identify a potential volatile-rich area during the course of a traverse, the prospect is then mapped out and the most promising location identified. An augering drill capable of sampling to a depth of 100 centimeters will excavate regolith for analysis. A quick assay of the drill cuttings will be made using an operations camera and NIR spectrometer. With the water depth confirmed by this first auguring activity, a regolith sample may be extracted for processing. The drill will deliver the regolith sample to a crucible that will be sealed and heated. Evolved volatiles will be measured by a gas chromatograph-mass spectrometer and the water will be captured and photographed. RP is a solar powered mission, which given the polar location translates to a relatively short mission duration on the order of 4-15 days. This short mission duration drives the concept of operations, instrumentation, and data analysis towards critical real time analysis and decision support. Previous payload field tests have increased the fidelity of the hardware, software, and mission operations. Current activities include a mission level field test to optimize interfaces between the payload and rover as well as better understand the interaction of the science and rover teams during the mission timeline. This paper will include the current status of the science instruments on the payload as well as the integrated field test occurring in fall of 2015. The concept of operations will be discussed, including the real time science and engineering decision-making process based on the critical data from the instrumentation. The path to flight will be discussed with the approach to this ambitious low cost mission.

ISRU↗

Resource Prospector Instrumentation for Lunar Volatiles Prospecting, Sample Acquisition and Processing

Data gathered from lunar missions within the last two decades have significantly enhanced our understanding of the volatile resources available on the lunar surface, specifically focusing on the polar regions. Several orbiting missions such as Clementine and Lunar Prospector have suggested the presence of volatile ices and enhanced hydrogen concentrations in the permanently shadowed regions of the moon. The Lunar Crater Observation and Sensing Satellite (LCROSS) mission was the first to provide direct measurement of water ice in a permanently shadowed region. These missions with other orbiting assets have laid the groundwork for the next step in the exploration of the lunar surface; providing ground truth data of the volatiles by mapping the distribution and processing lunar regolith for resource extraction. This next step is the robotic mission Resource Prospector (RP).Resource Prospector is a lunar mission to investigate strategic knowledge gaps (SKGs) for in-situ resource utilization (ISRU). The mission is proposed to land in the lunar south pole near a permanently shadowed crater. The landing site will be determined by the science team with input from broader international community as being near traversable landscape that has a high potential of containing elevated concentrations of volatiles such as water while maximizing mission duration. A rover will host the Regolith Environment Science and Oxygen Lunar Volatile Extraction (RESOLVE) payload for resource mapping and processing. The science instruments on the payload include a 1-meter drill, neutron spectrometer, a near infrared spectrometer, an operations camera, and a reactor with a gas chromatograph-mass spectrometer for volatile analysis.

Lunar Crater↗

ISCCP reduced resolution satellite radiance data

The International Satellite Cloud Climatology Project (ISCCP) is the first active project of the World Climate Research Program. It is a multinational data collection project focused on collecting a data set that will improve the ability to predict and/or simulate the radiative effects of clouds on climate. For specified cloud parameters, the goals are to archieve values for 3-hour periods over the whole globe for 5 years at 30 km resolution. The task of collecting and processing radiance data from both geosynchronous and polar orbiting satellites began in July 1983. A diagram was shown illustrating the flow of data from the transmitting satellites to the various receiving institutions that handle it. The various stages of processing were then explained in detail, emphasizing Level B3-normalized, reformatted, reduced raw satellite data. The reduction of data by sampling is an essential step in the flow. By the time the ISCCP data reaches the Global Processing Center at Goddard Institute for Space Studies (GISS), the volume has been reduced by a factor of 1000. The Pilot Climate Data System (PLDS) will provide access to the ISCCP data set. It should prove to be one of the cleanest satellite data sets because it will have been through three filters--that of the operational agency, the Global Processing Center, and the PCDS. The ISCCP data set also includes other correlative data sets delivered in compatible format.

Rossow, W.↗

Determining the Optimal Number of Clusters with the Clustergram

Cluster analysis aids research in many different fields, from business to biology to aerospace. It consists of using statistical techniques to group objects in large sets of data into meaningful classes. However, this process of ordering data points presents much uncertainty because it involves several steps, many of which are subject to researcher judgment as well as inconsistencies depending on the specific data type and research goals. These steps include the method used to cluster the data, the variables on which the cluster analysis will be operating, the number of resulting clusters, and parts of the interpretation process. In most cases, the number of clusters must be guessed or estimated before employing the clustering method. Many remedies have been proposed, but none is unassailable and certainly not for all data types. Thus, the aim of current research for better techniques of determining the number of clusters is generally confined to demonstrating that the new technique excels other methods in performance for several disparate data types. Our research makes use of a new cluster-number-determination technique based on the clustergram: a graph that shows how the number of objects in the cluster and the cluster mean (the ordinate) change with the number of clusters (the abscissa). We use the features of the clustergram to make the best determination of the cluster-number.

Fluegemann, Joseph K.↗

TRMM Common Microphysics Products: A Tool for Evaluating Spaceborne Precipitation Retrieval Algorithms

A customized product for analysis of microphysics data collected from aircraft during field campaigns in support of the TRMM program is described. These Common Microphysics Products (CMP's) are designed to aid in evaluation of TRMM spaceborne precipitation retrieval algorithms. Information needed for this purpose (e.g., particle size spectra and habit, liquid and ice water content) was derived using a common processing strategy on the wide variety of microphysical instruments and raw native data formats employed in the field campaigns. The CMP's are organized into an ASCII structure to allow easy access to the data for those less familiar with and without the tools to accomplish microphysical data processing. Detailed examples of the CMP show its potential and some of its limitations. This approach may be a first step toward developing a generalized microphysics format and an associated community-oriented, non-proprietary software package for microphysics data processing, initiatives that would likely broaden community access to and use of microphysics datasets.

Kingsmill, David E.↗