Test of hypotheses and estimation of the correlation coefficient using quantiles, ii
Sample quantiles for data compression of space telemetry
DATA COMPRESSOR↗
SEARCH · Search NASA
Results for “quantile sampling”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Tests of hypotheses and estimation of the correlation coefficient using quantiles i
Sampled data for data compression of space telemetry
DATA COMPRESSOR↗
Spline approximation of quantile functions
The study reported here explored the development and utility of a spline representation of the sample quantile function of a continuous probability distribution in providing a functional description of a random sample and a method of generating random variables. With a spline representation, the random samples are generated by transforming a sample of uniform random variables to the interval of interest. This is useful, for example, in simulation studies in which a random sample represents the only known information about the distribution. The spline formulation considered here consists of a linear combination of cubic basis splines (B-splines) fit in a least squares sense to the sample quantile function using equally spaced knots. The following discussion is presented in five parts. The first section highlights major results realized from the study. The second section further details the results obtained. The methodology used is described in the third section, followed by a brief discussion of previous research on quantile functions. Finally, the results of the study are evaluated.
Schiess, J. R.↗
Scale Mixtures of Frechet Distributions as Asymptotic Approximations of Extreme Precipitation
This paper is a further development of the results of [20] where, based on the negative binomial model for the duration of wet periods measured in days [16], an asymptotic approximation was proposed for the distribution of the maximum daily precipitation volume within a wet period. This approximation has the form of a scale mixture of the Fr´echet distribution with the gamma mixing distribution and coincides with the distribution of a positive power of a random variable having the Snedecor–Fisher distribution. Here we extend this result to the mth extremes, m ∈ ℕ, and sample quantiles. The proof of this result is based on the representation of the negative binomial distribution as a mixed geometric (and hence, mixed Poisson) distribution [17] and limit theorems for extreme order statistics in samples with random sizes having mixed Poisson distributions [10]. Some analytic properties of the obtained limit distribution are described. In particular, it is demonstrated that under certain conditions the limit distribution of the maximum precipitation is mixed exponential and hence, is infinitely divisible. It is shown that under the same conditions this limit distribution can be represented as a scale mixture of stable or Weibull or Pareto or folded normal laws. The corresponding product representations for the limit random variable can be used for its computer simulation. The results of fitting this distribution to real data are presented illustrating high adequacy of the proposed model. It is also shown that the limit distribution of sample quantiles is the well-known Student distribution. Several methods are proposed for the estimation of the parameters of the asymptotic distributions. The obtained mixture representations for the limit laws and the corresponding asymptotic approximations provide better insight into the nature of mixed probability (“Bayesian”) models.
97 MATHEMATICS AND COMPUTING↗
Approximating Photo- z PDFs for Large Surveys
Modern galaxy surveys produce redshift probability density functions (PDFs) in addition to traditional photometric redshift (photo-z) point estimates. However, the storage of photo-z PDFs may present a challenge with increasingly large catalogs, as we face a trade-off between the accuracy of subsequent science measurements and the limitation of finite storage resources. This work presents qp, a Python package for manipulating parameterizations of one-dimensional PDFs, as suitable for photo-z PDF compression. We use qp to investigate the performance of three simple PDF storage formats (quantiles, samples, and step functions) as a function of the number of stored parameters on two realistic mock data sets, representative of upcoming surveys with different data qualities. We propose some best practices for choosing a photo-z PDF approximation scheme and demonstrate the approach on a science case using performance metrics on both ensembles of individual photo-z PDFs and an estimator of the overall redshift distribution function. We show that both the properties of the set of PDFs we wish to approximate and the fidelity metric(s) chosen affect the optimal parameterization. Additionally, we find that quantiles and samples outperform step functions, and we encourage further consideration of these formats for PDF approximation.
79 ASTRONOMY AND ASTROPHYSICS↗
Trigger Detection for Adaptive Scientific Workflows Using Percentile Sampling
The increasing complexity of both scientific simulations and high-performance computing system architectures are driving the need for adaptive workflows, in which the composition and execution of computational and data manipulation steps dynamically depend on the evolutionary state of the simulation itself. Consider, for example, the frequency of data storage. Critical phases of the simulation should be captured with high frequency and with high fidelity for postanalysis; however, we cannot afford to retain the same frequency for the full simulation due to the high cost of data movement. We can instead look for triggers, indicators that the simulation will be entering a critical phase, and adapt the workflow accordingly. In this paper, we present a methodology for detecting triggers and demonstrate its use in the context of direct numerical simulations of turbulent combustion using S3D. We show that chemical explosive mode analysis (CEMA) can be used to devise a noise-tolerant indicator for rapid increase in heat release. However, exhaustive computation of CEMA values dominates the total simulation, and thus is prohibitively expensive. To overcome this computational bottleneck, we propose a quantile sampling approach. Our sampling-based algorithm comes with provable error/confidence bounds, as a function of the number of samples. Most importantly, the number of samples is independent of the problem size, and thus our proposed sampling algorithm offers perfect scalability. Our experiments on homogeneous charge compression ignition and reactivity controlled compression ignition simulations show that the proposed method can detect rapid increases in heat release, and its computational overhead is negligible. Our results will be used to make dynamic workflow decisions regarding data storage and mesh resolution in future combustion simulations.
97 MATHEMATICS AND COMPUTING↗
Efficient Sampling Allocation Procedures for Optimal Quantile Selection
We propose a dynamic sampling allocation and selection paradigm for finding the alternative with the optimal quantile in a Bayesian framework. Myopic allocation policies (MAPs), analogous to existing methods in classic ranking and selection for selecting the alternative with the optimal mean, and computationally efficient selection policies are derived for selecting the alternative with the optimal quantile. Under certain conditions, we prove that the proposed MAPs and selection procedures are consistent, which means that the best quantile would be eventually correctly selected as the sample size goes to infinity. Numerical experiments demonstrate that the proposed schemes can significantly improve the performance.
Computer Science↗
Spline methods for approximating quantile functions and generating random samples
Two cubic spline formulations are presented for representing the quantile function (inverse cumulative distribution function) of a random sample of data. Both B-spline and rational spline approximations are compared with analytic representations of the quantile function. It is also shown how these representations can be used to generate random samples for use in simulation studies. Comparisons are made on samples generated from known distributions and a sample of experimental data. The spline representations are more accurate for multimodal and skewed samples and to require much less time to generate samples than the analytic representation.
Schiess, J. R.↗
Quantile-specific heritability of plasma fibrinogen concentrations
Fibrinogen is a moderately heritable blood protein showing different genetic effects by sex, race, smoking status, pollution exposure, and disease status. These interactions may be explained in part by “quantile-dependent expressivity”, where the effect size of a genetic variant depends upon whether the phenotype (e.g. plasma fibrinogen concentration) is high or low relative to its distribution. Determine whether fibrinogen heritability (h 2 ) is quantile-specific, and whether quantile-specific h 2 could account for fibrinogen gene-environment interactions. Plasma fibrinogen concentrations from 5689 offspring-parent pairs and 1932 sibships from the Framingham Heart Study were analyzed. Quantile-specific heritability from offspring-parent (β OP , h 2 = 2β OP /(1+r spouse )) and full-sib regression slopes (β FS , h 2 = {(1+8r spouse β FS ) 0.05 –1}/(2r spouse )) were robustly estimated by quantile regression with nonparametric significance assigned from 1000 bootstrap samples. Quantile-specific h 2 (±SE) increased with increasing percentiles of the offspring’s age- and sex-adjusted fibrinogen distribution when estimated from β OP (P trend = 5.5x10 -6 ): 0.30±0.05 at the 10 th , 0.37±0.04 at the 25 th , 0.48±0.05 at the 50 th , 0.61±0.06 at the 75 th , and 0.65±0.08 at the 90 th percentile, and when estimated from β FS (P trend = 0.008): 0.28±0.04 at the 10 th , 0.31±0.04 at the 25 th , 0.36±0.03 at the 50 th , 0.41±0.05 at the 75 th , and 0.50±0.06 at the 90 th percentile. The larger genetic effect at higher average fibrinogen concentrations may contribute to fibrinogen’s greater heritability in women than men and in Blacks than Whites, and greater increase from smoking and air pollution for the FGB -455G>A A-allele. It may also explain greater fibrinogen differences between: 1) FGB -455G>A genotypes during acute phase reactions than usual conditions, 2) GTSM1 and IL-6 -572C>G genotypes in smokers than nonsmokers, 3) FGB -148C>T genotypes in untreated than treated diabetics, and LPL PvuII genotypes in macroalbuminuric than normoalbuminuric patients. Fibrinogen heritability is quantile specific, which may explain or contribute to its gene-environment interactions. The analyses do not disprove the traditional gene-environment interpretations of these examples, rather quantile-dependent expressivity provides an alternative explanation that warrants consideration.
59 BASIC BIOLOGICAL SCIENCES↗
Quantile-dependent expressivity of plasma adiponectin concentrations may explain its sex-specific heritability, gene-environment interactions, and genotype-specific response to postprandial lipemia
“Quantile-dependent expressivity” occurs when the effect size of a genetic variant depends upon whether the phenotype (e.g. adiponectin) is high or low relative to its distribution. We have previously shown that the heritability ( h 2 ) of adiposity, lipoproteins, postprandial lipemia, pulmonary function, and coffee and alcohol consumption are quantile-specific. Whether adiponectin heritability is quantile specific remains to be determined. Plasma adiponectin concentrations from 4,182 offspring-parent pairs and 1,662 sibships from the Framingham Heart Study were analyzed. Quantile-specific heritability from offspring-parent (β OP , h 2 = 2 β OP /(1 + r spouse )) and full-sib regression slopes ( β FS , h 2 = {(1 + 8r spouse β FS ) 0.05 -1}/(2r spouse )) were robustly estimated by quantile regression with nonparametric significance assigned from 1,000 bootstrap samples. Quantile-specific h 2 (± SE) increased with increasing percentiles of the offspring’s age- and sex-adjusted adiponectin distribution when estimated from β OP ( P trend = 2.2 × 10 -6 ): 0.30 ± 0.03 at the 10th, 0.33 ± 0.04 at the 25th, 0.43 ± 0.04 at the 50th, 0.55 ± 0.05 at the 75th, and 0.57 ± 0.08 at the 90th percentile, and when estimated from β FS (P trend = 7.6 × 10 -7 ): 0.42 ± 0.03 at the 10th, 0.44 ± 0.04 at the 25th, 0.56 ± 0.05 at the 50th, 0.73 ± 0.08 at the 75th, and 0.79 ± 0.11 at the 90th percentile. Consistent with quantile-dependent expressivity, adiponectin’s: (1) heritability was greater in women in accordance with their higher adiponection concentrations; (2) relationships to ADIPOQ polymorphisms were modified by adiposity in accordance with its adiponectin-lowering effect; (3) response to rosiglitazone was predicted by the 45T> G ADIPOQ polymorphism; (4) difference by ADIPOQ haplotypes increased linearly with increasing postprandial adiponectin concentrations. Adiponectin heritability is quantile dependent, which may explain sex-specific heritability, gene-environment and gene-drug interactions, and postprandial response by haplotypes.
Postprandial↗
Practical rare event sampling for extreme mesoscale weather
Extreme mesoscale weather, including tropical cyclones, squall lines, and floods, can be enormously damaging and yet challenging to simulate; hence, there is a pressing need for more efficient simulation strategies. Here, we present a new rare event sampling algorithm called quantile diffusion Monte Carlo (quantile DMC). Quantile DMC is a simple-to-use algorithm that can sample extreme tail behavior for a wide class of processes. We demonstrate the advantages of quantile DMC compared to other sampling methods and discuss practical aspects of implementing quantile DMC. To test the feasibility of quantile DMC for extreme mesoscale weather, we sample extremely intense realizations of two historical tropical cyclones, 2010 Hurricane Earl and 2015 Hurricane Joaquin. Our results demonstrate quantile DMC’s potential to provide low-variance extreme weather statistics while highlighting the work that is necessary for quantile DMC to attain greater efficiency in future applications.
71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
Rare Event Sampling Improves Mercury Instability Statistics
Due to the chaotic nature of planetary dynamics, there is a non-zero probability that Mercury's orbit will become unstable in the future. Previous efforts have estimated the probability of this happening between 3 and 5 billion years in the future using a large number of direct numerical simulations with an N-body code, but were not able to obtain accurate estimates before 3 billion years in the future because Mercury instability events are too rare. In this paper we use a new rare-event sampling technique, Quantile Diffusion Monte Carlo (QDMC), to estimate that the probability of a Mercury instability event in the next 2 billion years is approximately 10 –4 in the REBOUND N-body code. We show that QDMC provides unbiased probability estimates at a computational cost of up to 100 times less than direct numerical simulation. QDMC is easy to implement and could be applied to many problems in planetary dynamics in which it is necessary to estimate the probability of a rare event.
79 ASTRONOMY AND ASTROPHYSICS↗
Using Quantile Comparisons to Classify Environmental Samples
Explore the source record for details and available documents.
Weber, Charles F.↗
Quantile-Dependent Expressivity of Serum Uric Acid Concentrations
“Quantile-dependent expressivity” occurs when the effect size of a genetic variant depends upon whether the phenotype (e.g., serum uric acid) is high or low relative to its distribution. Analyses were performed to test whether serum uric acid heritability is quantile-specific and whether this could explain some reported gene-environment interactions. Methods. Serum uric acid concentrations were analyzed from 2151 sibships and 12,068 offspring-parent pairs from the Framingham Heart Study. Quantile-specific heritability from offspring-parent regression slopes (β OP , h 2 = 2β OP /(1 + r spouse )) and full-sib regression slopes (β FS ,h 2 = {(1 + 8r spouse β FS ) 0.5 - 1}/(2r spouse ) was robustly estimated by quantile regression with nonparametric significance assigned from 1000 bootstrap samples. Results. Quantile-specific h 2 (±SE) increased with increasing percentiles of the offspring’s sex- and age-adjusted uric acid distribution when estimated from β OP (P trend = 0.001): 0.34 ± 0.03 at the 10th, 0.36 ± 0.03 at the 25th, 0.41 ± 0.03 at the 50th, 0.46 ± 0.04 at the 75th, and 0.49 ± 0.05 at the 90th percentile and when estimated from β FS (P trend = 0.006). This is consistent with the larger genetic effect size of (1) the SLC2A9 rs11722228 polymorphism in gout patients vs. controls, (2) the ABCG2 rs2231142 polymorphism in men vs. women, (3) the SLC2A9 rs13113918 polymorphism in obese patients prior to bariatric surgery vs. two-year postsurgery following 29 kg weight loss, (4) the ABCG2 rs6855911 polymorphism in obese vs. nonobese women, and (5) the LRP2 rs2544390 polymorphism in heavier drinkers vs. abstainers. Quantile-dependent expressivity may also explain the larger genetic effect size of an SLC2A9/PKD2/ABCG2 haplotype for high vs. low intakes of alcohol, chicken, or processed meats. Conclusions. Heritability of serum uric acid concentrations is quantile-specific.
59 BASIC BIOLOGICAL SCIENCES↗
Quantile-dependent expressivity of serum C-reactive protein concentrations in family sets
“Quantile-dependent expressivity” occurs when the effect size of a genetic variant depends upon whether the phenotype (e.g., C-reactive protein, CRP) is high or low relative to its distribution. We have previously shown that the heritabilities ( h 2 ) of coffee and alcohol consumption, postprandial lipemia, lipoproteins, leptin, adiponectin, adiposity, and pulmonary function are quantile-specific. Whether CRP heritability is quantile-specific is currently unknown. Methods Serum CRP concentrations from 2,036 sibships and 6,144 offspring-parent pairs were analyzed from the Framingham Heart Study. Quantile-specific heritability from full-sib (β FS , h 2 ={(1 + 8r spouse β FS ) 0.5 - 1}/(2r spouse )) and offspring-parent regression slopes (β OP , h 2 = 2β OP /(1 + r spouse )) were estimated robustly by quantile regression with nonparametric significance determined from 1,000 bootstrap samples. Results Quantile-specific h 2 (±SE) increased with increasing percentiles of the offspring’s age- and sex-adjusted CRP distribution when estimated from β OP ( P trend = 0.0004): 0.02 ± 0.01 at the 10th, 0.04 ± 0.01 at the 25th, 0.10 ± 0.02 at the 50th, 0.20 ± 0.05 at the 75th, and 0.33 ± 0.10 at the 90th percentile, and when estimated from β FS ( P trend = 0.0008): 0.03±0.01 at the 10th, 0.06 ± 0.02 at the 25th, 0.14 ± 0.03 at the 50th, 0.24 ± 0.05 at the 75th, and 0.53 ± 0.21 at the 90th percentile. Conclusion Heritability of serum CRP concentration is quantile-specific, which may explain or contribute to the inflated CRP differences between CRP (rs1130864, rs1205, rs1800947, rs2794521, rs3091244), FGB (rs1800787), IL-6 (rs1800795, rs1800796), IL6R (rs8192284), TNF- α (rs1800629) and APOE genotypes following CABG surgery, stroke, TIA, curative esophagectomy, intensive periodontal therapy, or acute exercise; during acute coronary syndrome or Staphylococcus aureus bacteremia; or in patients with chronic rheumatoid arthritis, diabetes, peripheral arterial disease, ankylosing spondylitis, obesity or inflammatory bowel disease or who smoke.
59 BASIC BIOLOGICAL SCIENCES↗
Nonparametric procedures for selecting a subset containing the population with the largest alpha-quantile.
Nonparametric method for subset selection containing population with largest alpha quantile, discussing population sampling
Rizvi, M. H.↗
An Empirical Quantile Estimation Approach for Chance-Constrained Nonlinear Optimization Problems
We investigate an empirical quantile estimation approach to solve chance-constrained nonlinear optimization problems. Our approach is based on the reformulation of the chance constraint as an equivalent quantile constraint to provide stronger signals on the gradient. In this approach, the value of the quantile function is estimated empirically from samples drawn from the random parameters, and the gradient of the quantile function is estimated via a finite-difference approximation on top of the quantile-function-value estimation. We establish a convergence theory of this approach within the framework of an augmented Lagrangian method for solving general nonlinear constrained optimization problems. The foundation of the convergence analysis is a concentration property of the empirical quantile process, and the analysis is divided based on whether or not the quantile function is differentiable. In contrast to the sampling-and-smoothing approach used in the literature, the method developed in this paper does not involve any smoothing function and hence the quantile-function gradient approximation is easier to implement and there are less accuracy-control parameters to tune. Furthermore, we demonstrate the effectiveness of this approach and compare it with a smoothing method for the quantile-gradient estimation. Numerical investigation shows that the two approaches are competitive for certain problem instances.
Applied Probability↗
Implicit quantile networks for emulation in jet physics
Abstract The ability to model and sample from conditional densities is important in many physics applications. Implicit quantile networks (IQN) have been successfully applied to this task in domains outside physics. In this work, we illustrate the potential of IQNs as components of emulators using the simulation of jets as an example. Specifically, we use an IQN to map jets described by their 4-momenta at the generation level to jets at the event reconstruction level. The conditional densities emulated by our model closely match those generated by Delphes , while also enabling faster jet simulation.
Kronheim, Braden (ORCID:0000000307040972)↗