Search NASA⌕ Search

SEARCH · Search NASA

Results for “data reduction pipelines”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 73 records · Page 4

K2P{sup 2}—A photometry pipeline for the K2 mission

With the loss of a second reaction wheel, resulting in the inability to point continuously and stably at the same field of view, the NASA Kepler satellite recently entered a new mode of observation known as the K2 mission. The data from this redesigned mission present a specific challenge; the targets systematically drift in position on an ∼6 hr timescale, inducing a significant instrumental signal in the photometric time series—this greatly impacts the ability to detect planetary signals and perform asteroseismic analysis. Here we detail our version of a reduction pipeline for K2 target pixel data, which automatically defines masks for all targets in a given frame; extracts the target’s flux and position time series; corrects the time series based on the apparent movement on the CCD (either in 1D or 2D), combined with the correction of instrumental and/or planetary signals via the Kepler Asteroseismic Science Operations Center (KASOC) filter, thus rendering the time series ready for asteroseismic analysis; computes power spectra for all targets; and identifies potential contaminations between targets. From a test of our pipeline on a sample of targets from the K2 campaign 0, the recovery of data for multiple targets increases the amount of potential light curves by a factor of ⩾10. Our pipeline could be applied to the upcoming TESS and PLATO 2.0 missions.

79 ASTRONOMY AND ASTROPHYSICS↗

GROWTH on S190426c: Real-time Search for a Counterpart to the Probable Neutron Star–Black Hole Merger using an Automated Difference Imaging Pipeline for DECam

The discovery of a transient kilonova following the gravitational-wave (GW) event GW170817 highlighted the critical need for coordinated rapid and wide-field observations, inference, and follow-up across the electromagnetic spectrum. In the southern hemisphere, the Dark Energy Camera (DECam) on the Blanco 4 m telescope is well suited to this task, as it is able to cover wide fields quickly while still achieving the depths required to find kilonovae like the one accompanying GW170817 to ~500 Mpc, the binary neutron star (NS) horizon distance for current generation of LIGO/Virgo collaboration (LVC) interferometers. Here, as part of the multi-facility follow-up by the Global Relay of Observatories Watching Transients Happen collaboration, we describe the observations and automated data movement, data reduction, candidate discovery, and vetting pipeline of our target-of-opportunity DECam observations of S190426c, the first possible NS-black hole merger detected in GWs. Starting 7.5 hr after S190426c, over 11.28 hr of observations, we imaged an area of 525 deg 2 ( r band) and 437 deg 2 ( z band); this was 16.3% of the total original localization probability, and nearly all of the probability visible from the southern hemisphere. The machine-learning-based pipeline was optimized for fast turnaround, delivering transients for human vetting within 17 minutes, on average, of shutter closure. We reported nine promising counterpart candidates 2.5 hr before the end of our observations. One hour after our data-taking ended (roughly 20 hr after the announcement of S190426c), LVC released a refined skymap that reduced the probability coverage of our observations to 8.0%, demonstrating a critical need for localization updates on shorter (~hour) timescales. Our observations yielded no detection of a bona fide counterpart to m z = 21.7 and m r = 22.2 at the 5σ level of significance, consistent with the refined LVC positioning. Here, we view these observations and rapid inferencing as an important real-world test for this novel end-to-end wide-field pipeline.

79 ASTRONOMY AND ASTROPHYSICS↗

Query-Driven Visualization and Analysis

This report focuses on an approach to high performance visualization and analysis, termed query-driven visualization and analysis (QDV). QDV aims to reduce the amount of data that needs to be processed by the visualization, analysis, and rendering pipelines. The goal of the data reduction process is to separate out data that is "scientifically interesting'' and to focus visualization, analysis, and rendering on that interesting subset. The premise is that for any given visualization or analysis task, the data subset of interest is much smaller than the larger, complete data set. This strategy---extracting smaller data subsets of interest and focusing of the visualization processing on these subsets---is complementary to the approach of increasing the capacity of the visualization, analysis, and rendering pipelines through parallelism. This report discusses the fundamental concepts in QDV, their relationship to different stages in the visualization and analysis pipelines, and presents QDV's application to problems in diverse areas, ranging from forensic cybersecurity to high energy physics.

97 MATHEMATICS AND COMPUTING↗

The HEASARC Swift Gamma-Ray Burst Archive: The Pipeline and the Catalog

Since its launch in late 2004, the Swift satellite triggered or observed an average of one gamma-ray burst (GRB) every 3 days, for a total of 771 GRBs by 2012 January. Here, we report the development of a pipeline that semi automatically performs the data-reduction and data-analysis processes for the three instruments on board Swift (BAT, XRT, UVOT). The pipeline is written in Perl, and it uses only HEAsoft tools and can be used to perform the analysis of a majority of the point-like objects (e.g., GRBs, active galactic nuclei, pulsars) observed by Swift. We run the pipeline on the GRBs, and we present a database containing the screened data, the output products, and the results of our ongoing analysis. Furthermore, we created a catalog summarizing some GRB information, collected either by running the pipeline or from the literature. The Perl script, the database, and the catalog are available for downloading and querying at the HEASARC Web site.

BURST ARCHIVE↗

Cheetah : software for high-throughput reduction and analysis of serial femtosecond X-ray diffraction data

The emerging technique of serial X-ray diffraction, in which diffraction data are collected from samples flowing across a pulsed X-ray source at repetition rates of 100 Hz or higher, has necessitated the development of new software in order to handle the large data volumes produced. Sorting of data according to different criteria and rapid filtering of events to retain only diffraction patterns of interest results in significant reductions in data volume, thereby simplifying subsequent data analysis and management tasks. Meanwhile the generation of reduced data in the form of virtual powder patterns, radial stacks, histograms and other meta data creates data set summaries for analysis and overall experiment evaluation. Rapid data reduction early in the analysis pipeline is proving to be an essential first step in serial imaging experiments, prompting the authors to make the tool described in this article available to the general community. Originally developed for experiments at X-ray free-electron lasers, the software is based on a modular facility-independent library to promote portability between different experiments and is available under version 3 or later of the GNU General Public License.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

DRAGraces: A Pipeline for the GRACES High-resolution Spectrograph at Gemini

This paper describes the software Data Reduction and Analysis for GRACES (DRAGraces), which is a pipeline reducing spectra from Gemini Remote Access to the CFHT ESPaDOnS Spectrograph (GRACES) at the Gemini North Telescope. The code is written in the IDL language. It is designed to find all the GRACES frames in a given directory, automatically determine the list of bias, flat, arc, and science frames, and perform the whole reduction and extraction within a few minutes. We compare the output from DRAGraces with that of the Open source Pipeline for ESPaDOnS Reduction and Analysis (OPERA), a pipeline developed at the Canada–France–Hawaii Telescope (CFHT) that also can extract GRACES spectra. Both pipelines were developed completely independently, yet they give very similar extracted spectra. They both have their advantages and disadvantages. For instance, DRAGraces is more straightforward and easy to use and is less likely to be derailed by a parameter that needs to be tweaked, while OPERA offers a more careful extraction that can be significantly superior when the highest resolution is required and when the signal-to-noise ratio is low. One should compare both before deciding which one to use for their science. Yet, both pipelines deliver a fairly comparable resolution power (R ~ 52.8k and 36.6k for DRAGraces and R ~ 58k and 40k for OPERA in high- and low-resolution spectral modes, respectively), wavelength solution, and signal-to-noise ratio per resolution element.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

A machine-learning-driven data labeling pipeline for scientific analysis in MLExchange

This study introduces a novel labeling pipeline to accelerate the labeling process of scientific data sets by using artificial intelligence (AI)-guided tagging techniques. This pipeline includes a set of interconnected web-based graphical user interfaces (GUIs), where Data Clinic and MLCoach enable the preparation of machine learning (ML) models for data reduction and classification, respectively, while Label Maker is used for label assignment. Throughout this pipeline, data can be accessed through a direct connection to a file system or through Tiled for access through Hypertext Transfer Protocol (HTTP). Our experimental results present three use cases where this labeling pipeline has been instrumental for the study of large X-ray scattering data sets in the area of pattern recognition, the remote analysis of resonant soft X-ray scattering data and the fine-tuning process of foundation models. These use cases highlight the labeling capabilities of this pipeline, including the ability to label large data sets in a short period of time, to perform remote data analysis while minimizing data movement and to enhance the fine-tuning process of complex ML models with human involvement.

Chavez, Tanny (ORCID:0000000193172896)↗

Second Season QUIET Observations: Measurements of the CMB Polarization Power Spectrum at 95 GHz

The Q/U Imaging ExperimenT (QUIET) has observed the cosmic microwave background (CMB) at 43 and 95GHz. The 43-GHz results have been published in QUIET Collaboration et al. (2011), and here we report the measurement of CMB polarization power spectra using the 95-GHz data. This data set comprises 5337 hours of observations recorded by an array of 84 polarized coherent receivers with a total array sensitivity of 87 {mu}K{radical}s. Four low-foreground fields were observed, covering a total of {approx} 1000 square degrees with an effective angular resolution of 12'.8, allowing for constraints on primordial gravitational waves and high-signal-to-noise measurements of the E-modes across three acoustic peaks. The data reduction was performed using two independent analysis pipelines, one based on a pseudo-C{ell} (PCL) cross-correlation approach, and the other on a maximum-likelihood (ML) approach. All data selection criteria and filters were modified until a predefined set of null tests had been satisfied before inspecting any non-null power spectrum. The results derived by the two pipelines are in good agreement. We characterize the EE, EB and BB power spectra between {ell} = 25 and 975 and find that the EE spectrum is consistent with {Lambda}CDM, while the BB power spectrum is consistent with zero. Based on these measurements, we constrain the tensor-to-scalar ratio to r = 1.1{sup +0.9}{sub -0.8} (r < 2.8 at 95% C.L.) as derived by the ML pipeline, and r = {sup +0.9}{sub -0.8} (r < 2.7 at 95% C.L.) as derived by the PCL pipeline. In one of the fields, we find a correlation with the dust component of the Planck Sky Model, though the corresponding excess power is small compared to statistical errors. Finally, we derive limits on all known systematic errors, and demonstrate that these correspond to a tensor-to-scalar ratio smaller than r = 0.01, the lowest level yet reported in the literature.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

SECOND SEASON QUIET OBSERVATIONS: MEASUREMENTS OF THE COSMIC MICROWAVE BACKGROUND POLARIZATION POWER SPECTRUM AT 95 GHz

The Q/U Imaging ExperimenT (QUIET) has observed the cosmic microwave background (CMB) at 43 and 95 GHz. The 43 GHz results have been published in a previous paper, and here we report the measurement of CMB polarization power spectra using the 95 GHz data. This data set comprises 5337 hr of observations recorded by an array of 84 polarized coherent receivers with a total array sensitivity of 87 {mu}K{radical}s. Four low-foreground fields were observed, covering a total of {approx}1000 deg{sup 2} with an effective angular resolution of 12.'8, allowing for constraints on primordial gravitational waves and high signal-to-noise measurements of the E-modes across three acoustic peaks. The data reduction was performed using two independent analysis pipelines, one based on a pseudo-C {sub l} (PCL) cross-correlation approach, and the other on a maximum-likelihood (ML) approach. All data selection criteria and filters were modified until a predefined set of null tests had been satisfied before inspecting any non-null power spectrum. The results derived by the two pipelines are in good agreement. We characterize the EE, EB, and BB power spectra between l = 25 and 975 and find that the EE spectrum is consistent with {Lambda}CDM, while the BB power spectrum is consistent with zero. Based on these measurements, we constrain the tensor-to-scalar ratio to r = 1.1{sup +0.9} {sub -0.8} (r < 2.8 at 95% C.L.) as derived by the ML pipeline, and r = 1.2{sup +0.9} {sub -0.8} (r < 2.7 at 95% C.L.) as derived by the PCL pipeline. In one of the fields, we find a correlation with the dust component of the Planck Sky Model, though the corresponding excess power is small compared to statistical errors. Finally, we derive limits on all known systematic errors, and demonstrate that these correspond to a tensor-to-scalar ratio smaller than r = 0.01, the lowest level yet reported in the literature.

79 ASTRONOMY AND ASTROPHYSICS↗

Research on the DESI Experiment at Southern Methodist University

The SMU observational cosmology group in 2017-2019 continued its work in preparing DESI for operations slated to start in 2020. There are three major developments: a) a real-time spectroscopic data reduction and monitoring software needed for DESI Operations, b) components of spectroscopic simulation needed to support DESI science, and c) techniques and procedures to estimate spectroscopic observational systematics on BAO measurements for emission line galaxies (ELGs). We successfully developed a fast, flexible spectroscopic pipeline incorporating a comprehensive set of data quality algorithms. This included substantial development of data reduction algorithms, as well as the infrastructure to build pipelines for different data types necessary for calibration and science. Effort on data quality included supporting real-time monitoring during operations and long-term data quality needs for analysis. During this period, the SMU team contributed to development of DESI’s 1D spectroscopic simulation solutions, with a flexible pipeline that also mimicked the impact of the offline pipeline. Further work to upgrade the fast, parametric simulation, QuickCat, was also performed. We have also developed an initial methodology for the forward modeling of spectroscopic systematic uncertainties on ELG BAO measurements resulting from observing conditions. Kehoe also contributed to commissioning and science requirements documents, and served in difference service roles during the reporting period.

79 ASTRONOMY AND ASTROPHYSICS↗

A SURVEY OF ALKALI LINE ABSORPTION IN EXOPLANETARY ATMOSPHERES

We obtained over 90 hr of spectroscopic observations of four exoplanetary systems with the Hobby-Eberly Telescope. Observations were taken in transit and out of transit, and we analyzed the differenced spectra-i.e., the transmission spectra-to inspect it for absorption at the wavelengths of the neutral sodium (Na I) doublet at {lambda}{lambda}5889, 5895 and neutral potassium (K I) at {lambda}7698. We used the transmission spectrum at Ca I {lambda}6122-which shows strong stellar absorption but is not an alkali metal resonance line that we expect to show significant absorption in these atmospheres-as a control line to examine our measurements for systematic errors. We use an empirical Monte Carlo method to quantify these systematic errors. In a reanalysis of the same data set using a reduction and analysis pipeline that was derived independently, we confirm the previously seen Na I absorption in HD 189733b at a level of (- 5.26 {+-} 1.69) Multiplication-Sign 10{sup -4} (the average value over a 12 A integration band to be consistent with previous authors). Additionally, we tentatively confirm the Na I absorption seen in HD 209458b (independently by multiple authors) at a level of (- 2.63 {+-} 0.81) Multiplication-Sign 10{sup -4}, though the interpretation is less clear. Furthermore, we find Na I absorption of (- 3.16 {+-} 2.06) Multiplication-Sign 10{sup -4} at <3{sigma} in HD 149026b; features apparent in the transmission spectrum are consistent with real absorption and indicate this may be a good target for future observations to confirm. No other results (Na I in HD 147506b and Ca I and K I in all four targets) are significant to {>=}3{sigma}, although we observe some features that we argue are primarily artifacts.

79 ASTRONOMY AND ASTROPHYSICS↗

The Second Data Release of the Beijing–Arizona Sky Survey

This paper describes the second data release (DR2) of the Beijing-Arizona Sky Survey (BASS). BASS is an imaging survey covering a 5400 deg 2 footprint in the g and r bands using the 2.3 m Bok telescope. DR2 includes the observations through 2017 July obtained by BASS and by the Mayall z-band Legacy Survey (MzLS), which used the 4 m Mayall telescope to observe the same footprint. BASS and MzLS have completed 72% and 76% of their observations. The two surveys will be served for the spectroscopic targeting of the upcoming Dark Energy Spectroscopic Instrument. Both BASS and MzLS data are reduced by the same pipeline. We have updated the basic data reduction and photometric calibrations in DR2. In particular, source detections are performed on stacked images, and photometric measurements are co-added from single-epoch images based on these sources. The median 5σ point-source depths after Galactic extinction corrections are 24.05, 23.61, and 23.10 mag for the g, r, and z bands, respectively. The DR2 data products include stacked images, co-added catalogs, and single-epoch images and catalogs. The BASS website (http://batc.bao.ac.cn/BASS/) provides detailed information and links to download the data.

79 ASTRONOMY AND ASTROPHYSICS↗

The Updated DESGW Processing Pipeline for the Third LIGO/VIRGO Observing Run

The DESGW group seeks to identify electromagnetic counterparts of gravitational wave events seen by the LIGO-VIRGO network, such as those expected from binary neutron star mergers or neutron star- black hole mergers. DESGW was active throughout the first two LIGO observing seasons, following up several binary black hole mergers and the first binary neutron star merger, GW170817. We describe the modifications to the observing strategy generation and image processing pipeline between the second (ending in August 2017) and third (beginning in April 2019) LIGO observing seasons. The modifications include a more robust observing strategy generator, further parallelization of the image reduction software and difference imaging processing pipeline, data transfer streamlining, and a public webpage listing identified counterpart candidates that updates in real time. Taken together, the additional parallelization steps enable us to identify potential electromagnetic counterpa rts within fully calibrated search images in less than one hour, compared to the 3-5 hours it would typically take during the first two seasons. These performance improvements are critical to the entire EM followup community, as rapid identification (or rejection) of candidates enables detailed spectroscopic followup by multiple instruments as soon as possible, leading to more information about the environment immediately following such gravitational wave events.

79 ASTRONOMY AND ASTROPHYSICS↗

The updated DESGW processing pipeline for the third LIGO/VIRGO observing run

The DESGW group seeks to identify electromagnetic counterpartsof gravitational wave events seen by the LIGO-VIRGO network, such as thoseexpected from binary neutron star mergers or neutron star- black hole mergers.DESGW was active throughout the first two LIGO observing seasons, followingup several binary black hole mergers and the first binary neutron star merger,GW170817. We describe the modifications to the observing strategy generationand image processing pipeline between the second (ending in August 2017)and third (beginning in April 2019) LIGO observing seasons. The modifica-tions include a more robust observing strategy generator, further parallelizationof the image reduction software and dierence imaging processing pipeline,data transfer streamlining, and a web page listing identified counterpart candi-dates that updates in real time. Taken together, the additional parallelizationsteps enable us to identify potential electromagnetic counterparts within fullycalibrated search images in less than one hour, compared to the 3-5 hours itwould typically take during the first two seasons. These performance improve-ments are critical to the entire EM followup community, as rapid identification(or rejection) of candidates enables detailed spectroscopic followup by multipleinstruments as soon as possible, leading to more information about the environ-ment immediately following such gravitational wave events.

Herner, Kenneth R.↗

Quantifying impact reduction due to avoidance, minimization and restoration for a natural gas pipeline in the Peruvian Andes

We present monitoring methods and quantitative biodiversity data to document components of the mitigation hierarchy. We estimated avoidance, minimization, restoration and impact reduction in quality hectares for the 25 m wide right of way of a 408 km natural gas buried pipeline that crosses 14 Ecological Landscape Units (ELUs) in the tropical Andes of Peru. We found that applying the mitigation hierarchy as part of a comprehensive biodiversity action plan substantially reduced impacts on biodiversity in all habitats studied. Avoidance and right of way minimization contributed to significant impact reduction. We quantified impact reduction during construction and operation on the right of way of the pipeline over a five-year period and found that restoration was the greatest contributor to reducing impacts. We documented that most ELUs have a positive restoration trajectory. We also documented how monitoring over large scale spatial scales, in combination with site-specific monitoring, generated data for management to determine restoration priorities and impact mitigation. A biodiversity action plan that incorporated the mitigation hierarchy and a science-based biodiversity monitoring and assessment program contributed to biodiversity management of the project and played an important role in minimizing and managing impacts. - Highlights: •Quantitative data for mitigation hierarchy components are presented. •Application of the mitigation hierarchy reduced impacts on biodiversity. •A biodiversity monitoring program provided data to reduce impacts.

03 NATURAL GAS↗

The Fifteenth Data Release of the Sloan Digital Sky Surveys: First Release of MaNGA-derived Quantities, Data Visualization Tools, and Stellar Library

Twenty years have passed since first light for the Sloan Digital Sky Survey (SDSS). Here, we release data taken by the fourth phase of SDSS (SDSS-IV) across its first three years of operation (2014 July-2017 July). This is the third data release for SDSS-IV, and the 15th from SDSS (Data Release Fifteen; DR15). New data come from MaNGA—we release 4824 data cubes, as well as the first stellar spectra in the MaNGA Stellar Library (MaStar), the first set of survey-supported analysis products (e.g., stellar and gas kinematics, emission-line and other maps) from the MaNGA Data Analysis Pipeline, and a new data visualization and access tool we call "Marvin." The next data release, DR16, will include new data from both APOGEE-2 and eBOSS; those surveys release no new data here, but we document updates and corrections to their data processing pipelines. The release is cumulative; it also includes the most recent reductions and calibrations of all data taken by SDSS since first light. In this paper, we describe the location and format of the data and tools and cite technical references describing how it was obtained and processed. The SDSS website (www.sdss.org) has also been updated, providing links to data downloads, tutorials, and examples of data use. Although SDSS-IV will continue to collect astronomical data until 2020, and will be followed by SDSS-V (2020–2025), we end this paper by describing plans to ensure the sustainability of the SDSS data archive for many years beyond the collection of data.

79 ASTRONOMY AND ASTROPHYSICS↗

RESULTS OF THE LICK OBSERVATORY SUPERNOVA SEARCH FOLLOW-UP PHOTOMETRY PROGRAM: BVRI LIGHT CURVES OF 165 TYPE Ia SUPERNOVAE

We present BVRI light curves of 165 Type Ia supernovae (SNe Ia) from the Lick Observatory Supernova Search follow-up photometry program from 1998 through 2008. Our light curves are typically well sampled (cadence of 3-4 days) with an average of 21 photometry epochs. We describe our monitoring campaign and the photometry reduction pipeline that we have developed. Comparing our data set to that of Hicken et al., with which we have 69 overlapping supernovae (SNe), we find that as an ensemble the photometry is consistent, with only small overall systematic differences, although individual SNe may differ by as much as 0.1 mag, and occasionally even more. Such disagreement in specific cases can have significant implications for combining future large data sets. We present an analysis of our light curves which includes template fits of light-curve shape parameters useful for calibrating SNe Ia as distance indicators. Assuming the B - V color of SNe Ia at 35 days past maximum light can be presented as the convolution of an intrinsic Gaussian component and a decaying exponential attributed to host-galaxy reddening, we derive an intrinsic scatter of {sigma} = 0.076 {+-} 0.019 mag, consistent with the Lira-Phillips law. This is the first of two papers, the second of which will present a cosmological analysis of the data presented herein.

79 ASTRONOMY AND ASTROPHYSICS↗

Determination of Erosion/Corrosion Rates in Hanford Tank Farms Radioactive Waste Transfer System Pipelines

The twenty-eight double-shell underground radioactive waste storage tanks at the U. S. Department of Energy’s Hanford Site near Richland, WA are interconnected by the Waste Transfer System network of buried steel encased pipelines and pipe jumpers in below-grade pits. The pipeline material is stainless steel or carbon steel in 51 mm to 152 mm (2 in. to 6 in.) sizes. The pipelines carry slurries ranging up to 20 volume percent solids and supernatants with less than one volume percent solids at velocities necessary to prevent settling. The pipelines, installed between 1976 and 2011, were originally intended to last until the 2028 completion of the double-shell tank storage mission. The mission has been subsequently extended. In 2010 the Tank Operating Contractor began a systematic evaluation of the Waste Transfer System pipeline conditions applying guidelines from API 579-1/ASME FFS-1 (2007), Fitness-For-Service. Between 2010 and 2014 Fitness-for-Service examinations of the Waste Transfer System pipeline materials, sizes, and components were completed. In parallel, waste throughput histories were prepared allowing side-by-side pipeline wall thinning rate comparisons between carbon and stainless steel, slurries and supernatants and throughput volumes. The work showed that for transfer volumes up to 6.1E+05 m 3 (161 million gallons), the highest throughput of any pipeline segment examined, there has been no detectable wall thinning in either stainless or carbon steel pipeline material regardless of waste fluid characteristics or throughput. The paper describes the field and laboratory evaluation methods used for the Fitness-for-Service examinations, the results of the examinations, and the data reduction methodologies used to support Hanford Waste Transfer System pipeline wall thinning conclusions.

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗