Search NASA⌕ Search

Engineering topics

Scargle, Jeffrey D.

Publications and source records attributed to Scargle, Jeffrey D..

At least 19 records

Characterizing the Gamma-Ray Variability of the Brightest Flat Spectrum Radio Quasars Observed with the Fermi LAT

Almost 10 yr of γ-ray observations with the Fermi Large Area Telescope have revealed extreme γ-ray outbursts from flat spectrum radio quasars (FSRQs), temporarily making these objects the brightest γ-ray emitters in the sky. Yet, the location and mechanisms of the γ-ray emission remain elusive. We characterize long-term γ-ray variability and the brightest γ-ray flares of six FSRQs. Consecutively zooming in on the brightest flares, which we identify in an objective way through Bayesian blocks and a hill-climbing algorithm, we find variability on subhour timescales and as short as minutes for two sources in our sample (3C 279 and CTA 102) and weak evidence for variability at timescales less than the Fermi satellite's orbit of 95 minutes for PKS 1510–089 and 3C 454.3. This indicates extremely compact emission regions in the jet. We do not find any signs of γ-ray absorption in the broad-line region (BLR), which indicates that γ-rays are produced at distances greater than hundreds of gravitational radii from the central black hole. This is further supported by a cross-correlation analysis between γ-ray and radio/millimeter light curves, which is consistent with γ-ray production at the same location as the millimeter core for 3C 273, CTA 102, and 3C 454.3. The inferred locations of the γ-ray production zones are still consistent with the observed decay times of the brightest flares if the decay is caused by external Compton scattering with BLR photons. Yet, the minute-scale variability is challenging to explain in such scenarios.

79 ASTRONOMY AND ASTROPHYSICS↗

Time and Measurement Days

Questions in data analysis involving the concepts of time and measurement are often pushed into the background or reserved for a philosophical discussion. Some examples are: a) Is causality a consequence of the laws of physics, or can the arrow of time be reversed? b) Can we determine the arrow of time of an event? c) Do we need the continuum hypothesis for the underlying function in any measurement process? d) Can we say anything about the analyticity of the underlying process of an event? e) Would it be valid to model a non-analytical process as function of time? f) What are the implications of all these questions for classical Fourier techniques? However, in the age of big data gathered either from space missions supplying ultra-precise long time series, or e.g. LIGO data from the ground, the moment to bring these questions to the foreground seems arrived. The limitations of our understanding of some fundamental processes is emphasized by the lack of solution for problems open for more than 2 decades, such as the non-detection of solar g-modes, or the modal identification of main sequence stellar pulsators like delta Scuti stars. Flicker noise or 1/f noise, for example, attributed in solar-like stars to granulation, is analyzed mostly only to apply noise reduction techniques, neither considering the classical problem of 1/f noise that was introduced a 100 years ago, nor taking into account ergodic or non-ergodic solutions that make inapplicable spectral analysis techniques in practice. This topic was discussed by Nicholas W. Watkins during the ITISE meeting held in Granada in 2016. There he presented preliminary results of his research on Mandelbrot's related work. We reproduce here his quotation of Mandelbrot (1999) "There is a sharp contrast between a highly anomalous ("non-white") noise that proceeds in ordinary clock time and a noise whose principal anomaly is that it is restricted to fractal time", suggesting a connection with the above proposed topics that could be phrased as the following additional questions:a) Is self-organized criticality (SOC) frequent in astrophysical phenomena? b) Could all fractals in nature be considered stochastic? c) Could we establish mathematical/physical relationships between chaotic and fractal behaviors in time series? d) Could the differences between fractals and chaos in terms of analyticity be used to understand the residuals of the fitting of stellar light curves? In this meeting we would like to approximate these problems from a holistic and multidisciplinary perspective, taking into account not only technical issues but also the deeper implications. In particular the concept of connectivity (introduced in Pascual-Granado et al. A&A, 2015) could be used to implement, within the framework of ARMA processes, an "arrow of time" (see attached document), and so studying the possible implications in the concept of time as envisaged by Watkins.

data analysis↗

Structure in the 3D Galaxy Distribution. III. Fourier Transforming the Universe: Phase and Power Spectra

We demonstrate the effectiveness of a relatively straightforward analysis of the complex 3D Fourier transform of galaxy coordinates derived from redshift surveys. Numerical demonstrations of this approach are carried out on a volume-limited sample of the Sloan Digital Sky Survey redshift survey. The direct unbinned transform yields a complex 3D data cube quite similar to that from the Fast Fourier Transform of finely binned galaxy positions. In both cases, deconvolution of the sampling window function yields estimates of the true transform. Simple power spectrum estimates from these transforms are roughly consistent with those using more elaborate methods. The complex Fourier transform characterizes spatial distributional properties beyond the power spectrum in a manner different from (and we argue is more easily interpreted than) the conventional multipoint hierarchy. We identify some threads of modern large-scale inference methodology that will presumably yield detections in new wider and deeper surveys.

79 ASTRONOMY AND ASTROPHYSICS↗

Chromospheric Variability: Analysis of 36 years of Time Series from the National Solar Observatory/Sacramento Peak Ca II K-line Monitoring Program

Analysis of more than 36 years of time series of seven parameters measured in the NSO/AFRL/Sac Peak K-line monitoring program elucidates five elucidates five components of the variation: (1) the solar cycle (period approx. 11 years), (2) quasi-periodic variations (periods approx 100 days), (3) a broad band stochastic process (wide range of periods), (4) rotational modulation, and (5) random observational errors. Correlation and power spectrum analyses elucidate periodic and aperiodic variation of the chromospheric parameters. Time-frequency analysis illuminates periodic and quasi periodic signals, details of frequency modulation due to differential rotation, and in particular elucidates the rather complex harmonic structure (1) and (2) at time scales in the range approx 0.1 - 10 years. These results using only full-disk data further suggest that similar analyses will be useful at detecting and characterizing differential rotation in stars from stellar light-curves such as those being produced by NASA's Kepler observatory. Component (3) consists of variations over a range of timescales, in the manner of a 1/f random noise process. A timedependent Wilson-Bappu effect appears to be present in the solar cycle variations (1), but not in the stochastic process (3). Component (4) characterizes differential rotation of the active regions, and (5) is of course not characteristic of solar variability, but the fact that the observational errors are quite small greatly facilitates the analysis of the other components. The recent data suggest that the current cycle is starting late and may be relatively weak. The data analyzed in this paper can be found at the National Solar Observatory web site http://nsosp.nso.edu/cak_mon/, or by file transfer protocol at ftp://ftp.nso.edu/idl/cak.parameters.

Solar physics↗

SOLAR CYCLE VARIABILITY AND SURFACE DIFFERENTIAL ROTATION FROM Ca II K-LINE TIME SERIES DATA

Analysis of over 36 yr of time series data from the NSO/AFRL/Sac Peak K-line monitoring program elucidates 5 components of the variation of the 7 measured chromospheric parameters: (a) the solar cycle (period {approx} 11 yr), (b) quasi-periodic variations (periods {approx} 100 days), (c) a broadband stochastic process (wide range of periods), (d) rotational modulation, and (e) random observational errors, independent of (a)-(d). Correlation and power spectrum analyses elucidate periodic and aperiodic variation of these parameters. Time-frequency analysis illuminates periodic and quasi-periodic signals, details of frequency modulation due to differential rotation, and in particular elucidates the rather complex harmonic structure (a) and (b) at timescales in the range {approx}0.1-10 yr. These results using only full-disk data suggest that similar analyses will be useful for detecting and characterizing differential rotation in stars from stellar light curves such as those being produced by NASA's Kepler observatory. Component (c) consists of variations over a range of timescales, in the manner of a 1/f random process with a power-law slope index that varies in a systematic way. A time-dependent Wilson-Bappu effect appears to be present in the solar cycle variations (a), but not in the more rapid variations of the stochastic process (c). Component (d) characterizes differential rotation of the active regions. Component (e) is of course not characteristic of solar variability, but the fact that the observational errors are quite small greatly facilitates the analysis of the other components. The data analyzed in this paper can be found at the National Solar Observatory Web site http://nsosp.nso.edu/cak{sub m}on/, or by file transfer protocol at ftp://ftp.nso.edu/idl/cak.parameters.

79 ASTRONOMY AND ASTROPHYSICS↗

Alanna Conners and the Origins of Principled Data Analysis

Alanna was one of the most important pioneers in the development of not just sophisticated algorithms for analyzing astronomical data but more importantly an overall viewpoint emphasizing the use of statistically sound principles in place of blind application of cook-book recipes, or black boxes. I will outline some of the threads of this viewpoint, emphasizing time series data, with a focus on the importance of these developments for the Age of Digital Astronomy that we are entering.

age of digital astronomy↗

STUDIES IN ASTRONOMICAL TIME SERIES ANALYSIS. VI. BAYESIAN BLOCK REPRESENTATIONS

This paper addresses the problem of detecting and characterizing local variability in time series and other forms of sequential data. The goal is to identify and characterize statistically significant variations, at the same time suppressing the inevitable corrupting observational errors. We present a simple nonparametric modeling technique and an algorithm implementing it-an improved and generalized version of Bayesian Blocks-that finds the optimal segmentation of the data in the observation interval. The structure of the algorithm allows it to be used in either a real-time trigger mode, or a retrospective mode. Maximum likelihood or marginal posterior functions to measure model fitness are presented for events, binned counts, and measurements at arbitrary times with known error distributions. Problems addressed include those connected with data gaps, variable exposure, extension to piecewise linear and piecewise exponential representations, multivariate time series data, analysis of variance, data on the circle, other data modes, and dispersed data. Simulations provide evidence that the detection efficiency for weak signals is close to a theoretical asymptotic limit derived by Arias-Castro et al. In the spirit of Reproducible Research all of the code and data necessary to reproduce all of the figures in this paper are included as supplementary material.

79 ASTRONOMY AND ASTROPHYSICS↗

Studies in Astronomical Time Series Analysis. VI. Bayesian Block Representations

This paper addresses the problem of detecting and characterizing local variability in time series and other forms of sequential data. The goal is to identify and characterize statistically significant variations, at the same time suppressing the inevitable corrupting observational errors. We present a simple nonparametric modeling technique and an algorithm implementing it-an improved and generalized version of Bayesian Blocks [Scargle 1998]-that finds the optimal segmentation of the data in the observation interval. The structure of the algorithm allows it to be used in either a real-time trigger mode, or a retrospective mode. Maximum likelihood or marginal posterior functions to measure model fitness are presented for events, binned counts, and measurements at arbitrary times with known error distributions. Problems addressed include those connected with data gaps, variable exposure, extension to piece- wise linear and piecewise exponential representations, multivariate time series data, analysis of variance, data on the circle, other data modes, and dispersed data. Simulations provide evidence that the detection efficiency for weak signals is close to a theoretical asymptotic limit derived by [Arias-Castro, Donoho and Huo 2003]. In the spirit of Reproducible Research [Donoho et al. (2008)] all of the code and data necessary to reproduce all of the figures in this paper are included as auxiliary material.

signal detection↗

Heterogeneity in Short Gamma-Ray Bursts

We analyze the Swift/BAT sample of short gamma-ray bursts, using an objective Bayesian Block procedure to extract temporal descriptors of the bursts' initial pulse complexes (IPCs). The sample comprises 12 and 41 bursts with and without extended emission (EE) components, respectively. IPCs of non-EE bursts are dominated by single pulse structures, while EE bursts tend to have two or more pulse structures. The medians of characteristic timescales - durations, pulse structure widths, and peak intervals - for EE bursts are factors of approx 2-3 longer than for non-EE bursts. A trend previously reported by Hakkila and colleagues unifying long and short bursts - the anti-correlation of pulse intensity and width - continues in the two short burst groups, with non-EE bursts extending to more intense, narrower pulses. In addition we find that preceding and succeeding pulse intensities are anti-correlated with pulse interval. We also examine the short burst X-ray afterglows as observed by the Swift/XRT. The median flux of the initial XRT detections for EE bursts (approx 6 X 10(exp -10) erg / sq cm/ s) is approx > 20 x brighter than for non-EE bursts, and the median X-ray afterglow duration for EE bursts (approx 60,000 s) is approx 30 x longer than for non-EE bursts. The tendency for EE bursts toward longer prompt-emission timescales and higher initial X-ray afterglow fluxes implies larger energy injections powering the afterglows. The longer-lasting X-ray afterglows of EE bursts may suggest that a significant fraction explode into more dense environments than non-EE bursts, or that the sometimes-dominant EE component efficiently p()wers the afterglow. Combined, these results favor different progenitors for EE and non-EE short bursts.

Norris, Jay P.↗

Influence of Coronal Abundance Variations

The PI of this project was Jeff Scargle of NASA/Ames. Co-I's were Alma Connors of Eureka Scientific/Wellesley, and myself. Part of the work was subcontracted to Eureka Scientific via SAO, with Vinay Kashyap as PI. This project was originally assigned grant number NCC2-1206, and was later changed to NCC2-1350 for administrative reasons. The goal of the project was to obtain, derive, and develop statistical and data analysis tools that would be of use in the analyses of high-resolution, high-sensitivity data that are becoming available with new instruments. This is envisioned as a cross-disciplinary effort with a number of "collaborators" including some at SA0 (Aneta Siemiginowska, Peter Freeman) and at the Harvard Statistics department (David van Dyk, Rostislav Protassov, Xiao-li Meng, Epaminondas Sourlas, et al). We have developed a new tool to reliably measure the metallicities of thermal plasma. It is unfeasible to obtain high-resolution grating spectra for most stars, and one must make the best possible determination based on lower-resolution, CCD-type spectra. It has been noticed that most analyses of such spectra have resulted in measured metallicities that were significantly lower than when compared with analyses of high- resolution grating data where available (see, e.g., Brickhouse et al., 2000, ApJ 530,387). Such results have led to the proposal of the existence of so-called Metal Abundance Deficient, or "MAD" stars (e.g., Drake, J.J., 1996, Cool Stars 9, ASP Conf.Ser. 109, 203). We however find that much of these analyses may be systematically underestimating the metallicities, and using a newly developed method to correctly treat the low-counts regime at the high-energy tail of the stellar spectra (van Dyk et al. 2001, ApJ 548,224), have found that the metallicities of these stars are generally comparable to their photospheric values. The results were reported at the AAS (Sourlas, Yu, van Dyk, Kashyap, and Drake, 2000, BAAS 196, v32, #54.02), and at the conference on Statistical Challenges in Modem Astronomy (Sourlas, van Dyk, Kashyap, Drake, and Pease, 2003, SCMA 111, Eds. E.D.Feigelson, G.J.Babu, New York:Springer, p489-490). We also described the limitations of one of the most egregiously misused and misapplied statistical tests in astrophysical literature, the F-test for verifying model components (Protassov, van Dyk, Connors, Kashyap, and Siemiginowska, 2002, ApJ, 571,545). Indeed, a search through the ApJ archives turned up 170 papers in the 5 previous years that used the F-test explicitly in some form or the other, and with the vast majority of them not using it correctly! Indeed, looking at just 4 issues of the ApJ in 2001, we found 13 instances of its use, of which nine were demonstrably incorrect. Clearly, it is difficult to understate the importance of this issue. We also worked on speeding up Bayes Blocks and Sparse Bayes Blocks algorithms to make them more tractable for large searches. We also supported staistics students and postdocs in both explicit physics- model-based (spectra with tens of thousands of atomic lines) and "model-free" -- i.e. non-parametric or semi-parametric -- algorithms. Work on using more of the latter is just beginning; while using multi-scale methods for Poisson imaging has come to hition. In fact, "An Image Restoration Technique with Error Estimates", by D. Esch, A. Connors, M. Karovska, and D. van Dyk, was published by ApJ (Esch et a1.2004, ApJ, 610, 1213). The code has been delivered to M. Karovska for CXC; and is available for beta-testing upon request. The other large project we worked on was on the self-consistent modeling of logN-logs curves in the Poisson limit. logN-logs curves are a fundamental tool in the study of source populations, luminosity functions, and cosmological parameters. However, their determination is hampered by statistical effects such as the Eddington bias, incompleteness due to detection efficiency, faint source flux fluctuations, etc. We have develed a new and powerful method using the full Poisson machinery that allows us to model the logN-logs distribution of X-ray sources in a self-consistent manner. Because we properly account for all the above statistical effects, our modeling is valid over the full range of the data, and not just for strong sources, as is normally done. Using a Bayesian approach and modeling the fluxes with known functional forms such as simple or broken power-laws, and conditioning the expected photon counts on the fluxes, the background contamination, effective area, detector vignetting, and detection probability, we can delve deeply into the low counts regime and extend the usefulness of medium sensitivity surveys such as ChAMP by orders of magnitude. The built-in flexibility of the algorithm also allows a simultaneous analysis of multiple datasets. We have applied this analysis to a set a Chandra observations (Sourlas, Kashyap, Zezas, van Dyk, 2004, HEAD #8, #16.32)

Scargle, Jeffrey D.↗

Discovery of Planetary Systems With SIM

We are witnessing the birth of a new observational science: the discovery and characterization of extrasolar planetary systems. In the past five years, over 70 extrasolar planets have been discovered by precision Doppler surveys, most by members of this SIM team. We are using the data base of information gleaned from our Doppler survey to choose the best targets for a new SIN planet search. In the same way that our Doppler database now serves SIM, our team will return a reconnaissance database to focus Terrestrial Planet Finder (TPF) into a more productive, efficient mission.

Marcy, Geoffrey W.↗

Advanced Statistical and Data Analysis Tools for Astrophysics

The goal of the project is to obtain, derive, and develop statistical and data analysis tools that would be of use in the analyses of high-resolution, high-sensitivity data that are becoming available with new instruments. This is envisioned as a cross-disciplinary effort with a number of collaborators.

Kashyap, V.↗

Publication Bias ( The "File-Drawer Problem") in Scientific Inference

Publication bias arises whenever the probability that a study is published depends on the statistical significance of its results. This bias, often called the file-drawer effect since the unpublished results are imagined to be tucked away in researchers' file cabinets, is potentially a severe impediment to combining the statistical results of studies collected from the literature. With almost any reasonable quantitative model for publication bias, only a small number of studies lost in the file-drawer will produce a significant bias. This result contradicts the well known Fail Safe File Drawer (FSFD) method for setting limits on the potential harm of publication bias, widely used in social, medical and psychic research. This method incorrectly treats the file drawer as unbiased, and almost always miss-estimates the seriousness of publication bias. A large body of not only psychic research, but medical and social science studies, has mistakenly relied on this method to validate claimed discoveries. Statistical combination can be trusted only if it is known with certainty that all studies that have been carried out are included. Such certainty is virtually impossible to achieve in literature surveys.

Scargle, Jeffrey D.↗

Bayesian Blocks: A New Method to Analyze Photon Counting Data

A Bayesian analysis of photon-counting data leads to a new time-domain algorithm for detecting localized structures (bursts), revealing pulse shapes, and generally characterizing intensity variations. The raw counting data -- time-tag events (TTE), time-to-spill (TTS) data, or binned counts -- is converted to a maximum likelihood segmentation of the observation into time intervals during which the photon arrival rate is perceptibly constant -- i.e. has a fixed intensity without statistically significant variations. The resulting structures, Bayesian Blocks, can be thought of as bins with arbitrary spacing determined by the data. The method itself sets no lower limit to the time scale on which variability can be detected. We have applied the method to RXTE data on Cyg X-1, yielding information on this source's short-time-scale variability.

Scargle, Jeffrey D.↗

Statistical Challenges in Modern Astronomy II

Wavelets are the central idea of a broad framework for thinking about displaying, and analyzing data - not just a new specialized technique. They are localized basis functions that provide an extremely flexible and efficient representation of time series. Current developments in the theory and application of wavelets are yielding powerful multiresolution techniques for the analysis of time series and image data. Wavelet methods can effect noise removal. characterization of stochastic behavior, depiction of power spectrum time-variations, and data compression - for time series data, image data, and data of higher dimensions. I review published applications of wavelets in astronomy.

Scargle, Jeffrey D.↗

The Dripping Handrail Model: Transient Chaos in Accretion Systems

We define and study a simple dynamical model for accretion systems, the "dripping handrail" (DHR). The time evolution of this spatially extended system is a mixture of periodic and apparently random (but actually deterministic) behavior. The nature of this mixture depends on the values of its physical parameters - the accretion rate, diffusion coefficient, and density threshold. The aperiodic component is a special kind of deterministic chaos called transient chaos. The model can simultaneously exhibit both the quasiperiodic oscillations (QPO) and very low frequency noise (VLFN) that characterize the power spectra of fluctuations of several classes of accretion systems in astronomy. For this reason, our model may be relevant to many such astrophysical systems, including binary stars with accretion onto a compact object - white dwarf, neutron star, or black hole - as well as active galactic nuclei. We describe the systematics of the DHR's temporal behavior, by exploring its physical parameter space using several diagnostics: power spectra, wavelet "scalegrams," and Lyapunov exponents. In addition, we note that for large accretion rates the DHR has periodic modes; the effective pulse shapes for these modes - evaluated by folding the time series at the known period - bear a resemblance to the similarly- determined shapes for some x-ray pulsars. The pulsing observed in some of these systems may be such periodic-mode accretion, and not due to pure rotation as in the standard pulsar model.

Young, Karl↗

Dripping handrails and the quasi-periodic oscillations of the AM Herculis objects

AM Her objects exhibit periodic, quasi-periodic, and aperiodic variability on timescales ranging from seconds to years. Here, we investigate a process for the production of aperiodic and quasi-periodic accretion rate fluctuations. We consider the nonlinear dynamical model known as the dripping handrail (DHR). The DHR, basically a model for certain types of spatially extended systems and loosely based on water condensing on and dripping off a handrail, has recently been used as a model for the quasi-periodic oscillations (QPO) and very low frequency noise of the low-mass X-ray binary Sco X-1. Here, we show that (1) the DHR is a robust QPO generation process in that it leads to QPO production under a wide range of conditions and assumptions; (2) the phenomenology of the DHR is consistent with the observed aperiodic and quasi-periodic varibility of the AM Her QPO source VV Pup over timescales ranging from 16 ms to 20 s; and (3) a single DHR model can produce both broadband QPOs and features with quality Q greater than 20 as observed in several AM Her QPO sources.

Steiman-Cameron, Thomas Y.↗

The quasi-periodic oscillations and very low frequency noise of Scorpius X-1 as transient chaos - A dripping handrail?

We present evidence that the quasi-periodic oscillations (QPO) and very low frequency noise (VLFN) characteristic of many accretion sources are different aspects of the same physical process. We analyzed a long, high time resolution EXOSAT observation of the low-mass X-ray binary (LMXB) Sco X-1. The X-ray luminosity varies stochastically on time scales from milliseconds to hours. The nature of this variability - as quantified with both power spectrum analysis and a new wavelet technique, the scalegram - agrees well with the dripping handrail accretion model, a simple dynamical system which exhibits transient chaos. In this model both the QPO and VLFN are produced by radiation from blobs with a wide size distribution, resulting from accretion and subsequent diffusion of hot gas, the density of which is limited by an unspecified instability to lie below a threshold.

Scargle, Jeffrey D.↗