Search NASA⌕ Search

SEARCH · Search NASA

Results for “data pipelines”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 451 records · Page 25
All sourcesNASA NTRSDOE OSTI
NASA NTRS2017

H31G-1596: DeepSAT's CloudCNN: A Deep Neural Network for Rapid Cloud Detection from Geostationary Satellites

Cloud and cloud shadow detection has important applications in weather and climate studies. It is even more crucial when we introduce geostationary satellites into the field of terrestrial remote sensing. With the challenges associated with data acquired in very high frequency (10-15 mins per scan), the ability to derive an accurate cloud shadow mask from geostationary satellite data is critical. The key to the success for most of the existing algorithms depends on spatially and temporally varying thresholds,which better capture local atmospheric and surface effects.However, the selection of proper threshold is difficult and may lead to erroneous results. In this work, we propose a deep neural network based approach called CloudCNN to classify cloudshadow from Himawari-8 AHI and GOES-16 ABI multispectral data. DeepSAT's CloudCNN consists of an encoderdecoder based architecture for binary-class pixel wise segmentation. We train CloudCNN on multi-GPU Nvidia Devbox cluster, and deploy the prediction pipeline on NASA Earth Exchange (NEX) Pleiades supercomputer. We achieved an overall accuracy of 93.29% on test samples. Since, the predictions take only a few seconds to segment a full multispectral GOES-16 or Himawari-8 Full Disk image, the developed framework can be used for real-time cloud detection, cyclone detection, or extreme weather event predictions.

GOES-1↗
NASA NTRS1979

VLSI system for synthetic aperture radar /SAR/ processing

The paper discusses an SAR problem based on actual requirements set forth by NASA for a spaceborne application. The requirements for high resolution and high quality necessitate a data sampling rate of 7.5 MHz. For each data value 1,025 4-bit complex multiply + add operations are needed, which is equivalent to 7.7 GHz complex multiply + add operation rate. Since this rate is much too high for general purpose systems, a special-purpose device was sought. This paper discusses two architectures based on parallel operation of 1,025 identical cells, each of which is capable of performing arithmetic, storage, and several control operations. The operation rate in each device is only 7.5 MHz, which is quite manageable, especially with the help of a substantial degree of pipelining. A computational-mathematical analysis is used as a primary tool for evaluating the design and some of its tradeoffs. Two different approaches are discussed and compared; both are based on having 1,025 identical cells working in parallel, but differ in their dual approaches to the flow of data.

Cohen, D.↗
NASA NTRSDate not supplied

A Pipeline for Assessing the Quality of Rna-Seq Datasets in GeneLab

Transcriptome profiling by RNA sequencing (RNA-seq) is a powerful approach to identify gene expression changes in organisms exposed to unique environments such as spaceflight. One of the challenges of evaluating RNA-seq data both within and across different space-relevant studies is the ability to control for technical differences, including the use of different library preparation kits, sequencing platforms, RNA yield, and person-to-person variation. To help address this issue, the National Institute of Standards and Technology (NIST, nist.gov) initiated a consortium, at the request of industry and academia, to develop a set of controls for gene expression measurements. The result was a set of 92 unlabeled, polyadenylated transcripts that range from 250 – 2,000 nucleotides in length to mimic natural eukaryotic mRNAs. These External RNA Controls Consortium (ERCC) genes can be used in any RNA-seq experiment, by adding known concentrations of the ERCC genes to samples after RNA extraction, to offer a standard measurement for data comparison. At NASA GeneLab, we employ these controls as part of our standard operating procedures for every in-house RNA-seq study to assess the limit of detection, dynamic range, and power of differential expression analysis both within and across experiments. Here we will discuss the use, benefits, and limitations of ERCC genes and other types of controls, such as universal RNA references, to generate quality control information for RNA-seq studies conducted at GeneLab.

GeneLab↗
NASA NTRS1990

Collection Of Software For Computer Graphics

Ames Research Graphics System (ARCGRAPH) collection of software libraries and software utilities assisting researchers in generating, manipulating, and visualizing graphical data. Defines metafile format containing device-independent graphical data. File format used with various computer-graphics-manipulation and -animation software packages at Ames, including SURF (COSMIC Program ARC-12381) and GAS (COSMIC Program ARC-12379). Consists of two-stage "pipeline" used to put out graphical primitives. ARCGRAPH libraries developed on VAX computer running VMS.

Hibbard, Eric A.↗
NASA NTRS1990

Balancing Loads Among Parallel Data Processors

Heuristic algorithm minimizes amount of memory used by multiprocessor system. Distributes load of many identical, short computations among multiple parallel digital data processors, each of which has its own (local) memory. Each processor operates on distinct and independent set of data in larger shared memory. As integral part of load-balancing scheme, total amount of space used in shared memory minimized. Possible applications include artificial neural networks or image processors for which "pipeline" and vector methods of load balancing inappropriate.

Baffes, Paul Thomas↗
NASA NTRS2023

Data-constrained Solar Modeling with GX Simulator

To facilitate the study of solarflares and active regions, we have created a modeling framework, the freelydistributed GX Simulator IDL package, that combines 3D magnetic and plasma structures with thermal andnonthermal models of the chromosphere, transition region, and corona. Its object-based modular architecture,which runs on Windows, Mac, and Unix/Linux platforms, offers the ability to either import 3D density andtemperature distribution models, or to assign numerically defined coronal or chromospheric temperatures anddensities, or their distributions, to each individual voxel. GX Simulator can apply parametric heating modelsinvolving average properties of the magneticfield lines crossing a given voxel, as well as compute and investigatethe spatial and spectral properties of radio,(sub)millimeter, EUV, and X-ray emissions calculated from the model,and quantitatively compare them with observations. The package includes a fully automatic model productionpipeline that, based on minimal users input, downloads the required SDO/HMI vector magneticfield data,performs potential or nonlinear force-freefield extrapolations, populates the magneticfield skeleton withparameterized heated plasma coronal models that assume either steady-state or impulsive plasma heating, andgenerates non-LTE density and temperature distribution models of the chromosphere that are constrained byphotospheric measurements. The standardized models produced by this pipeline may be further customizedthrough specialized IDL scripts, or a set of interactive tools provided by the graphical user interface. Here, wedescribe the GX Simulator framework and its applications.

Solar active regions↗
NASA NTRS2020

Atacama Cosmology Telescope: Component-separated maps of CMB temperature and the thermal Sunyaev-Zel'dovich effect

Optimal analyses of many signals in the cosmic microwave background (CMB) require map-level extraction of individual components in the microwave sky, rather than measurements at the power spectrum level alone. To date, nearly all map-level component separation in CMB analyses has been performed exclusively using satellite data. In this paper, we implement a component separation method based on the internal linear combination (ILC) approach which we have designed to optimally account for the anisotropic noise (in the 2D Fourier domain) often found in ground-based CMB experiments. Using this method, we combine multi-frequency data from the Planck satellite and the Atacama Cosmology Telescope Polarimeter (ACTPol) to construct the first wide area (≈2100 sq. deg.), arcminute-resolution component-separated maps of the CMB temperature anisotropy and the thermal Sunyaev-Zel'dovich (tSZ) effect sourced by the inverse-Compton scattering of CMB photons off hot, ionized gas. Our ILC pipeline allows for explicit deprojection of various contaminating signals, including a modified blackbody approximation of the cosmic infrared background (CIB) spectral energy distribution. The cleaned CMB maps will be a useful resource for CMB lensing reconstruction, kinematic SZ cross-correlations, and primordial non-Gaussianity studies. The tSZ maps will be used to study the pressure pro les of galaxies, groups, and clusters through cross-correlations with halo catalogs, with dust contamination controlled via CIB deprojection. The data products described in this paper are available on LAMBDA.

Mathew S. Madhavacheril↗
NASA NTRS2001

ROSAT Science Data Center

This report provides a summary of the Smithsonian Astrophysical Observatory (SAO) ROSAT SCIENCE DATA CENTER (RSDC) activities for the recent years of our contract. Details have already been reported in the monthly reports. The SAO was responsible for the High Resolution Imager (HRI) detector on ROSAT. We also provided and supported the HRI standard analysis software used in the pipeline processing (SASS). Working with our colleagues at the Max Planck in Garching Germany (MPE), we fixed bugs and provided enhancements. The last major effort in this area was the port from VMS/VAX to VMS/ALPHA architecture. In 1998, a timing bug was found in the HRI standard processing system which degraded the positional accuracy because events accessed incorrect aspect solutions. The bug was fixed and we developed off-line correction routines and provided them to the community. The Post Reduction Off-line Software (PROS) package was developed by SAO and runs in the IRAF environment. Although in recent years PROS was not a contractual responsibility of the RSDC, we continued to maintain the system and provided new capabilities such as the ability to deal with simulated AXAF data in preparation for the NASA call for proposals for Chandra. Our most recent activities in this area included the debugging necessary for newer versions of IRAF which broke some of our software. At SAO we have an operating version of PROS and hope to release a patch even though almost all functionality that was lost was subsequently recovered via an IRAF patch (i.e. most of our problems were caused by an IRAF bug).

Murray, Stephen↗
NASA NTRS2020

Automated Cardiovascular Pathology Assessment using Semantic Segmentation and Ensemble Learning

Cardiac magnetic resonance imaging provides high spatial resolution, enabling improved extraction of important functional and morphological features for cardiovascular disease staging. Segmentation of ventricular cavities and myocardium in cardiac cine sequencing provides a basis to quantify cardiac measures such as ejection fraction. A method is presented that curtails the expense and observer bias of manual cardiac evaluation by combining semantic segmentation and disease classification into a fully automatic processing pipeline. The initial processing element consists of a robust dilated convolutional neural network architecture for voxel-wise segmentation of the myocardium and ventricular cavities. The resulting comprehensive volumetric feature matrix captures diagnostic clinical procedure data and is utilized by the final processing element to model a cardiac pathology classifier. Our approach evaluated anonymized cardiac images from a training data set of 100 patients (4 pathology groups, 1 healthy group, 20 patients per group) examined at the University Hospital of Dijon. The top average Dice index scores achieved were 0.940, 0.886, 0.849 for structure segmentation of the left ventricle (LV), myocardium and right ventricle (RV) respectively. A 5-ary pathology classification accuracy of 90% was recorded on an independent test set using the trained model. Performance results demonstrate potential for advanced machine learning methods to deliver accurate, efficient and reproducible cardiac pathological assessment.

Semantic Segmentation↗
NASA NTRSDate not supplied

Reanalysis of Rat Data from Spacelab Life Sciences 2 (SLS-2) to Reveal Research Gaps in Spaceflight Data

Using and analyzing the legacy data obtained in space life sciences missions has the potential to provide researchers a complete picture of the molecular changes associated with space without further experimentation. This project’s objective is to extract, filter, organize, and analyze all Rattus norvegicus data and metadata obtained from Columbia’s Spacelab Life Sciences 2 (SLS-2, STS-58) mission to explore the ways that we can compile information from model organisms, in our case rats, to create a reliable model to understand biological mechanisms in response to these space flight changes. By reusing rare space legacy data coupled with data analysis techniques, we can combine individual preexisting datasets with current ones to gain new, comprehensive insights about the effects of spaceflight on our bodies. Our methods can also lead to the creation of a standardized pipeline that could be applied to other space life science datasets for analysis. In this review, every biological experiment conducted on rats in the SLS-2 Mission was studied with our pipeline to create a new biological library and model that could be used by scientists from around the world to make novel discoveries and develop new hypotheses from this priceless information without the limitation of the costs of spaceflight experimentation.

rats↗
NASA NTRS1990

On recursive least-squares filtering algorithms and implementations

In many real-time signal processing applications, fast and numerically stable algorithms for solving least-squares problems are necessary and important. In particular, under non-stationary conditions, these algorithms must be able to adapt themselves to reflect the changes in the system and take appropriate adjustments to achieve optimum performances. Among existing algorithms, the QR-decomposition (QRD)-based recursive least-squares (RLS) methods have been shown to be useful and effective for adaptive signal processing. In order to increase the speed of processing and achieve high throughput rate, many algorithms are being vectorized and/or pipelined to facilitate high degrees of parallelism. A time-recursive formulation of RLS filtering employing block QRD will be considered first. Several methods, including a new non-continuous windowing scheme based on selectively rejecting contaminated data, were investigated for adaptive processing. Based on systolic triarrays, many other forms of systolic arrays are shown to be capable of implementing different algorithms. Various updating and downdating systolic algorithms and architectures for RLS filtering are examined and compared in details, which include Householder reflector, Gram-Schmidt procedure, and Givens rotation. A unified approach encompassing existing square-root-free algorithms is also proposed. For the sinusoidal spectrum estimation problem, a judicious method of separating the noise from the signal is of great interest. Various truncated QR methods are proposed for this purpose and compared to the truncated SVD method. Computer simulations provided for detailed comparisons show the effectiveness of these methods. This thesis deals with fundamental issues of numerical stability, computational efficiency, adaptivity, and VLSI implementation for the RLS filtering problems. In all, various new and modified algorithms and architectures are proposed and analyzed; the significance of any of the new method depends crucially on specific application.

Hsieh, Shih-Fu↗
NASA NTRS1978

Multiprocessing tradeoffs and the wind-tunnel simulation problem

The architecture tradeoff issue is discussed, concentrating on an oversimplified version of the multiprocessing aspect. A degree of symmetric multiprocessing is unavoidable; the choice is either complete symmetric multiprocessing or a number of identical pipelines. The processing elements can be geared to do either plane-multiprocessing, or line-multiprocessing. Other important design choices are the number notation, word length, main memory size, cache memory size, and different means to implement data transport.

Chen, T. C.↗
NASA NTRS1997

Calibrating the WFPC2 astrometry for MDS

The Hubble Space Telescope (HST) Medium Deep Survey has optimized an automated procedure to associate and stack WFPC2 data. The coordinates listed in HST WFPC2 image and/or jitter file headers are often found to be insufficient to stack images for cosmic ray removal. We discuss results from software developed for the HST Medium Deep Survey (MDS) pipeline to evaluate these shifts by cross-correlation of the images. We will also discuss the distortion of the pre-cool-down WFPC2 field and attempts to derive absolute coordinates for MDS WFPC2 observations using the USNO-Al.0 half billion object catalog.

Ratnatunga, Kavan U.↗
NASA NTRS2016

The TESS Science Processing Operations Center

The Transiting Exoplanet Survey Satellite (TESS) will conduct a search for Earth’s closest cousins starting in late 2017. TESS will discover approx.1,000 small planets and measure the masses of at least 50 of these small worlds. The Science Processing Operations Center (SPOC) is being developed based on the Kepler science pipeline and will generate calibrated pixels and light curves on the NAS Pleiades supercomputer. The SPOC will search for periodic transit events and generate validation products for the transit-like features in the light curves. All TESS SPOC data products will be archived to the Mikulski Archive for Space Telescopes.

TESS↗
NASA NTRSDate not supplied

Seven Years of Bright Meteor Data from the NASA All Sky Fireball Network

The NASA All Sky Fireball Network is a network consisting of 18 all-sky meteor cameras across the continental US. We present a data release of 33,660 bright meteors collected between 2013 and 2019. The released data consists of trajectory, orbit, radiant, shower association, and brightness. We discuss the camera hardware and software used to make the observations, the characteristics of the data, and known error sources. The camera hardware consists of Watec 902H2 Ultimate CCD cameras with a fish-eye lens giving an all-sky field of view. The data is complete until an absolute meteor magnitude of approximately -4, but meteors down to a magnitude of -1 are routinely observed. The cameras are spread across the continental United States in five subnetworks, with each network consisting of two to five cameras. The cameras report to a central server where the data is processed each morning. The network is modeled after the University of Western Ontario’s Southern Ontario Meteor Network. The cameras are running All-sky and Guided Real-time Detection (ASGARD) meteor detection software and analysis pipeline.

Aaron Kingery↗
NASA NTRS2022

Robust Vision-based Multi-spacecraft Guidance Navigation and Control using CNN-based Pose Estimation

In this paper, we present an end-to-end simulation framework for tracking an uncooperative Target spacecraft in Low Earth Orbit using a CubeSat-class Ego spacecraft outfitted with a camera. Currently, capturing high-fidelity realistic images in space for this scenario is difficult and exorbitantly expensive. Therefore, we developed a framework to simulate the spacecraft orbits in Basilisk software and generate high-fidelity realistic images of spacecraft in Unreal Engine, including the effects from Sun, Earth, Moon and stars. The Ego spacecraft uses cameras to capture images of the uncooperative Target and estimates its position and attitude using a CNN based 6DOF pose estimation pipeline, eliminating need for large SWAP-C(Size, Weight, Power and Cost) sensors like LIDAR or reliance on inter-spacecraft communication, This CNN, which is motivated by ESA’s Pose Estimation challenge of 2019, is trained using simulated data from our end-to-end simulation framework. We compare the performance of two distinct CNNbased algorithms for pose estimation along a nominal trajectory. In presence of non-Gaussian modeling uncertainties, the statedependent estimation error is characterized with a quadratic upper-bound. The quadratically-bounded error can be used by a robust controller to maneuver

Rahmani, Amir↗
NASA NTRS2009

Multi-Mission Automated Task Invocation Subsystem

Multi-Mission Automated Task Invocation Subsystem (MATIS) is software that establishes a distributed data-processing framework for automated generation of instrument data products from a spacecraft mission. Each mission may set up a set of MATIS servers for processing its data products. MATIS embodies lessons learned in experience with prior instrument- data-product-generation software. MATIS is an event-driven workflow manager that interprets project-specific, user-defined rules for managing processes. It executes programs in response to specific events under specific conditions according to the rules. Because requirements of different missions are too diverse to be satisfied by one program, MATIS accommodates plug-in programs. MATIS is flexible in that users can control such processing parameters as how many pipelines to run and on which computing machines to run them. MATIS has a fail-safe capability. At each step, MATIS captures and retains pertinent information needed to complete the step and start the next step. In the event of a restart, this information is retrieved so that processing can be resumed appropriately. At this writing, it is planned to develop a graphical user interface (GUI) for monitoring and controlling a product generation engine in MATIS. The GUI would enable users to schedule multiple processes and manage the data products produced in the processes. Although MATIS was initially designed for instrument data product generation,

Cheng, Cecilia S.↗
NASA NTRS1974

Applications of ERTS-1 imagery to terrestrial and marine environmental analyses in Alaska

ERTS-1 imagery provides a means of distinguishing and monitoring estuarine surface water circulation patterns and changes in the relative sediment load of discharging rivers on a regional basis. It also will aid local fishing industries by augmenting currently available hydrologic and navigation charts. The interpretation of geologic and vegetation features resulted in preparation of improved surficial geology, vegetation and permafrost terrain maps at a scale of 1:1 million utilizing ERTS-1 band 7 imagery. This information will be further utilized in a route and site selection study for the Nome to Kobuk Road in central Alaska. Large river icings along the proposed Alaska pipeline route have been monitored. Sea ice deformation and drift northeast of Point Barrow, Alaska has been measured and shorefast ice accumulation and ablation along the west coast of Alaska is being mapped for the spring and early summer seasons. These data will be used for route and site selection, regional environmental analysis, identification and inventory of natural resources, land use planning, and in land use regulation and management.

Anderson, D. M.↗