Search NASA⌕ Search

SEARCH · Search NASA

Results for “Statistical Algorithms”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 109 records · Page 6
All sourcesNASA NTRSDOE OSTI
DOE OSTI2022

OperonSEQer: A set of machine-learning algorithms with threshold voting for detection of operon pairs using short-read RNA-sequencing data

Operon prediction in prokaryotes is critical not only for understanding the regulation of endogenous gene expression, but also for exogenous targeting of genes using newly developed tools such as CRISPR-based gene modulation. A number of methods have used transcriptomics data to predict operons, based on the premise that contiguous genes in an operon will be expressed at similar levels. While promising results have been observed using these methods, most of them do not address uncertainty caused by technical variability between experiments, which is especially relevant when the amount of data available is small. In addition, many existing methods do not provide the flexibility to determine the stringency with which genes should be evaluated for being in an operon pair. We present OperonSEQer, a set of machine learning algorithms that uses the statistic and p-value from a non-parametric analysis of variance test (Kruskal-Wallis) to determine the likelihood that two adjacent genes are expressed from the same RNA molecule. We implement a voting system to allow users to choose the stringency of operon calls depending on whether your priority is high recall or high specificity. In addition, we provide the code so that users can retrain the algorithm and re-establish hyperparameters based on any data they choose, allowing for this method to be expanded as additional data is generated. We show that our approach detects operon pairs that are missed by current methods by comparing our predictions to publicly available long-read sequencing data. OperonSEQer therefore improves on existing methods in terms of accuracy, flexibility, and adaptability.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2018

A Locally Adapting Technique for Edge Detection using Image Segmentation

Rapid growth in the field of quantitative digital image analysis is paving the way for scientific researchers to make precise measurements about objects in an image. To compute quantities from an image such as the density of compressed materials or the velocity of a shockwave, object boundaries must first be determined. Images containing regions that each have a spatial trend in intensity are of particular interest here. For edge detection, we present a supervised, statistical image segmentation method that incorporates spatial information to locate boundaries between regions with overlapping intensity histograms, specifically for images where the regions are known but precise boundary locations are unknown. The segmentation of a pixel is determined by comparing its intensity to distributions from nearby pixel intensities, and a gradient of the segmented image indicates edge locations. Because of the statistical nature of the algorithm, we use maximum likelihood estimation to quantify uncertainty about each boundary. We demonstrate the success of this algorithm at locating boundaries and providing uncertainty bands on a radiograph of a multicomponent cylinder and on an optical image of a laser-induced shockwave.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

G-Mapper: Learning a Cover in the Mapper Construction

The Mapper algorithm is a visualization technique in topological data analysis (TDA) that outputs a graph reflecting the structure of a given dataset. However, the Mapper algorithm requires tuning several parameters in order to generate a “nice” Mapper graph. This paper focuses on selecting the cover parameter. We present an algorithm that optimizes the cover of a Mapper graph by splitting a cover repeatedly according to a statistical test for normality. Our algorithm is based on G-means clustering, which searches for the optimal number of clusters in 𝑘-means by iteratively applying the Anderson–Darling test. Our splitting procedure employs a Gaussian mixture model to carefully choose the cover according to the distribution of the given data. In conclusion, experiments for synthetic and real-world datasets demonstrate that our algorithm generates covers so that the Mapper graphs retain the essence of the datasets, while also running significantly faster than a previous iterative method.

G-means clustering↗
DOE OSTI2023

Advanced data analysis in inertial confinement fusion and high energy density physics

Bayesian analysis enables flexible and rigorous definition of statistical model assumptions with well-characterized propagation of uncertainties and resulting inferences for single-shot, repeated, or even cross-platform data. This approach has a strong history of application to a variety of problems in physical sciences ranging from inference of particle mass from multi-source high-energy particle data to analysis of black-hole characteristics from gravitational wave observations. The recent adoption of Bayesian statistics for analysis and design of high-energy density physics (HEDP) and inertial confinement fusion (ICF) experiments has provided invaluable gains in expert understanding and experiment performance. In this Review, we discuss the basic theory and practical application of the Bayesian statistics framework. We highlight a variety of studies from the HEDP and ICF literature, demonstrating the power of this technique. Due to the computational complexity of multi-physics models needed to analyze HEDP and ICF experiments, Bayesian inference is often not computationally tractable. Two sections are devoted to a review of statistical approximations, efficient inference algorithms, and data-driven methods, such as deep-learning and dimensionality reduction, which play a significant role in enabling use of the Bayesian framework. We provide additional discussion of various applications of Bayesian and machine learning methods that appear to be sparse in the HEDP and ICF literature constituting possible next steps for the community. We conclude by highlighting community needs, the resolution of which will improve trust in data-driven methods that have proven critical for accelerating the design and discovery cycle in many application areas.

47 OTHER INSTRUMENTATION↗
DOE OSTI2023

Modelling climate analogue regions for a central European city

In this study, we describe a methodology to derive climate analogue cities for spatially highly resolved future climate scenarios. For the computation, a reduced and in hindsight bias-adjusted EURO-CORDEX EUR-11 dataset is used based on two climate scenarios (RCP4.5 and RCP8.5). A total of 389 European cities are processed by the algorithm, which uses five statistical climate variables (2-m air temperature average and amplitude, precipitation sum and amplitude, correlation between 2-m air temperature average and precipitation sum). Additionally, extreme weather events (hot days, summer days, tropical nights, extreme precipitation events) are calculated for further comparison and validation. Finding an appropriate analogue permits a more accurate derivation and depiction of necessary climate adaptation efforts and therefore assist decision-making in city planning. As an example of our method, we searched for plausible climate twins for the mid-sized city of Aachen (Germany) at the end of the twenty-first century. Our results show that the French city of Dijon is highly likely to become Aachen’s climate twin by the end of the century for RCP4.5. As for the scenario RCP8.5, no clear European analogue city could be determined, indicating that the city might enter a novel climate. The nearest match suggests the cities of Florence and Prato in Tuscany. However, considering climate indices, the encompassing region of the French–Spanish city triangle Bordeaux–Toulouse–Bilbao is a better fit. The developed algorithm can be applied to any of the cities included in the dataset.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2024

Measurement of top-quark pair production in association with charm quarks in proton–proton collisions at √s = 13 TeV with the ATLAS detector

Inclusive cross-sections or top-quark pair production in association with charm quarks are measured with proton-proton collision data at a center-of-mass energy of 13 TeV corresponding to an integrated luminosity of 140 fb -1 , collected with the ATLAS experiment at LHC between 2015 and 2018. The measurements are performed by requiring one or two charged leptons (electrons and muons), two b-tagged jets, and at least one additional jet in the final state. A custom flavor-tagging algorithm is employed for the simultaneous identification of b-jets and c-jets. In a fiducial phase space that replicates the acceptance of the ATLAS detector, the cross-sections for $t\bar{t}$ + ≥ 2c and $t\bar{t}$ + 1c production are measured to be $1.28^{+0.27}_{-0.24}$ pb and $6.4^{+1.0}_{-0.9}$ pb, respectively. The measurements are primarily limited by uncertainties in the modeling of inclusive $t\bar{t}$ and $t\bar{t}$ + $b\bar{b}$ production, in the calibration of the flavor-tagging algorithm, and by data statistics. Cross-section predictions from various $t\bar{t}$ simulations are largely consistent with the measured cross-section values, though all underpredict the observed values by 0.5 to 2.0 standard deviations. In a phase-space volume without requirements on the $t\bar{t}$ decay products and the jet multiplicity, the cross-section ratios of $t\bar{t}$ + ≥ 2c and $t\bar{t}$ + 1c to total $t\bar{t}$ + jets production are determined to be (1.23 ± 0.25)% and (8.8 ± 1.3)%.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2023

Novel ammonia recovery from anaerobic digestion by integrating biogas stripping and gypsum absorption

Recovering ammonia from anaerobic digestion (AD) effluent offers the opportunity to produce a renewablefertilizer and reduce environmental impact. A novel process developed for such an application was evaluatedduring this study. The feature of the process is integrating biogas stripping and gypsum (CaSO 4 •2H 2 O) absorption. The result revealed that the lower CO 2 content and the absorption temperature contributed the higherefficiency of ammonia recovery. The gypsum solution recovered around 95% of the stripped ammonia when thestripping gas contained 10% CO 2 and it rose to 100% when the gas contained more than 30% CO 2 . Furthermore,a statistical model and genetic algorithm were applied to estimate the total ammonia recovery and optimize theoperating parameters. The experimental and modeling results indicated that integrating biogas stripping withgypsum absorption can effectively harvest ammonia nitrogen and reduce CO 2 emissions while producing(NH 4 ) 2 SO 4 fertilizer from AD effluent.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2022

What drives the scatter of local star-forming galaxies in the BPT diagrams? A Machine Learning based analysis

ABSTRACT We investigate which physical properties are most predictive of the position of local star forming galaxies on the BPT diagrams, by means of different Machine Learning (ML) algorithms. Exploiting the large statistics from the Sloan Digital Sky Survey (SDSS), we define a framework in which the deviation of star-forming galaxies from their median sequence can be described in terms of the relative variations in a variety of observational parameters. We train artificial neural networks (ANN) and random forest (RF) trees to predict whether galaxies are offset above or below the sequence (via classification), and to estimate the exact magnitude of the offset itself (via regression). We find, with high significance, that parameters primarily associated to variations in the nitrogen-over-oxygen abundance ratio (N/O) are the most predictive for the [N ii]-BPT diagram, whereas properties related to star formation (like variations in SFR or EW(H α)) perform better in the [S ii]-BPT diagram. We interpret the former as a reflection of the N/O–O/H relationship for local galaxies, while the latter as primarily tracing the variation in the effective size of the S+ emitting region, which directly impacts the [S ii] emission lines. This analysis paves the way to assess to what extent the physics shaping local BPT diagrams is also responsible for the offsets seen in high redshift galaxies or, instead, whether a different framework or even different mechanisms need to be invoked.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2022

Sample-Efficient Adaptive Calibration of Quantum Networks Using Bayesian Optimization

All physical systems employed for quantum information tasks must act as unbiased carriers of encoded quantum states. Ensuring such indistinguishability of information carriers is a major challenge in many quantum information applications, including advanced quantum communication protocols. For photons, the workhorses of quantum communication networks, it is difficult to obtain and maintain their indistinguishability because of environment-induced transformations and loss imparted by communication channels, especially in noisy scenarios. Conventional strategies to mitigate these transformations often require hardware or software overhead that is restrictive (e.g., adding noise), infeasible (e.g., on a satellite), or time-consuming for deployed networks. In this work we propose and develop resource-efficient Bayesian optimization techniques to rapidly and adaptively calibrate the indistinguishability of individual photons for quantum networks using only information derived from their measurement. To experimentally validate our approach, we demonstrate the optimization of Hong-Ou-Mandel interference between two photons-a central task in quantum networking- finding rapid, efficient, and reliable convergence towards maximal photon indistinguishability in the presence of high loss and shot noise. We expect our resource-optimized and experimentally friendly methodology will allow fast and reliable calibration of indistinguishable quanta, a necessary task in distributed quantum computing, communications, and sensing, as well as for fundamental investigations.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2023

Optimal Power Flow With State Estimation in the Loop for Distribution Networks

Here in this article, we propose a framework for running optimal control-estimation synthesis in distribution networks. Our approach combines a primal-dual gradient-based optimal power flow solver with a state estimation feedback loop based on a limited set of sensors for system monitoring, instead of assuming exact knowledge of all states. The estimation algorithm reduces uncertainty on unmeasured grid states based on certain online state measurements and noisy "pseudomeasurements." We analyze the convergence of the proposed algorithm and quantify the statistical estimation errors based on a weighted least-squares estimator. The numerical results on a 4521-node network demonstrate that this approach can scale to extremely large networks and provide robustness to both large pseudomeasurement variability and inherent sensor measurement noise.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2021

Rare Events via Cross-Entropy Population Monte Carlo

Rare events are events that happen with very low frequency. Estimating rare event probabilities using Monte Carlo techniques is computationally expensive, often to the point of intractability, and special methods are required. Importance sampling (IS) is a well known technique that uses a proposal distribution in place of a target distribution to lower the variance of the estimator. Key to the success of IS methods is the choice of a proposal distribution, or the parameters governing the distribution. Adaptive importance sampling improves the parameters of a family or population of proposal distributions iteratively through trials. We present a novel cross-entropy population Monte Carlo algorithm, which adapts the parameters of proposals through the cross-entropy method. The proposed method stands apart from previous work in that we are not optimizing a mixture distribution. Instead, we leverage deterministic mixture weights and optimize the distributions individually through a reinterpretation of the typical derivation of the cross-entropy method. Demonstrations on rare event examples show that the algorithm can outperform existing resampling based population Monte Carlo methods, especially for higher-dimensional problems. Finally, we also demonstrate efficacy on a conjunction analysis problem.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

Data-Driven Day-Ahead PV Estimation Using Autoencoder-LSTM and Persistence Model

Inherent variability in photovoltaic (PV) and associated impacts on power systems is a challenging problem for both the PV owners and the grid operators. Existing statistical and machine learning algorithms typically work well for weather conditions similar to historical data. Furthermore, uncertain weather conditions pose a great challenge to the estimation accuracy of the estimation models. With the enhanced integration of intelligent electronic devices and the realization of associated automation in the power grid, renewable energy data is becoming more accessible, which can be utilized by deep learning models and improve the PV power generation estimation accuracy. In this paper, a hybrid deep learning model driven by external weather data is proposed to do day-ahead PV output forecasting at 15-minute-interval. The proposed model is motivated by the recent advancement of Long-Short-Term-Memory (LSTM) networks and AutoEncoder (AE), which estimates uncertainties in sequence while making the prediction for complex weather conditions. Meanwhile, the persistence model (PM) is used to predict continuous sunny weather conditions. The forecasting result is validated with data from multiple locations

42 ENGINEERING↗
DOE OSTI2017

Dependency graph for code analysis on emerging architectures

Direct acyclic dependency (DAG) graph is becoming the standard for modern multi-physics codes.The ideal DAG is the true block-scheme of a multi-physics code. Therefore, it is the convenient object for insitu analysis of the cost of computations and algorithmic bottlenecks related to statistical frequent data motion and dymanical machine state.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Real-Time Health Monitoring for Gas Turbine Components Using Online Learning and High-Dimensional Data

Capital-intensive turbomachinery, such as gas turbines and combined cycle plants, are constantly being monitored for performance anomalies, faults, and physical degradation. Although these power-generating assets are equipped with hundreds of sensors, existing monitoring tools can only handle moderate-sized data. As a result, only a handful of aggregate metrics are used to monitor machine health. At the same time, developing advanced tools suitable for large datasets have been restricted by the lack of appropriate data. The objective of this proposal was to demonstrate a Big Data analytics framework for fault detection and diagnosis in gas turbine applications. We develop a predictive analytics framework methodology guided by these experimental data, industrial data from our collaborators, and physics-based models with engineering domain knowledge. Our analytics framework consists of four key components (1) a data curation process that addresses data storage, data quality assessments, and integrity checks, (2) a feature engineering component that utilizes statistical methods and transformation algorithms guided by physics-based models to extract high-fidelity fault features that can be leveraged for fault detection and classifying fault severities, (3) a Machine Learning-based fault detection and diagnostics algorithms for detecting operational and hardware faults in the combustion and the turbines section. We utilize two industry-class gas turbine component test rigs to generate first of its kind data for critical gas turbine faults with varying severity levels. Advanced gas turbine test facilities will be interrogated using state-of-the-art instrumentation techniques to build fault signatures and data trends for key combustor and turbine faults. Data generated from a combustor test rig (Georgia Tech) and a turbine test rig (Penn State) during both normal operation and with seeded faults serve as the basis for the Big Data sets. The test conditions in the two test facilities include common, critical events that occur in the operation. Utilizing the combustor test rig, we examine two common combustor faults: lean blowout and centerbody degradation. For the turbine section we develop analytic models for monitoring cooling faults in the gas turbine

03 NATURAL GAS↗
DOE OSTI2022

EASTERN NORTH ATLANTIC (ENA) CONDENSATION PARTICLE COUNTER (CPC) with the AEROSOL MASK (ENA-AM) for the YEAR 2017

The Eastern North Atlantic (ENA) central facility is periodically affected by episodes of local aerosols. High-concentration aerosol events associated with potential local aerosol sources are sampled by the Concentration Particle Counter (CPC) and can be observed in the high-time-resolution data. In an effort to remove aerosol data that is not regionally representative from the CPC data, an aerosol mask (ENA-AM) has been previously developed and validated during the ACE-ENA campaing (Gallo et al., 2020). The ENA-AM is a standard deviation algorithm based on the statistically different behavior of adjacent data points used to determine periods when the measurements are impacted by local aerosols. This file contain one-minute time resolution submicron aerosol particle concentration datasets from the CPC for the year 2017, and the corresponding ENA-AM datasets to remove datapoints affected by local aerosol sources.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2022

EASTERN NORTH ATLANTIC (ENA) CONDENSATION PARTICLE COUNTER (CPC) with the AEROSOL MASK (ENA-AM) for the YEAR 2018

The Eastern North Atlantic (ENA) central facility is periodically affected by episodes of local aerosols. High-concentration aerosol events associated with potential local aerosol sources are sampled by the Concentration Particle Counter (CPC) and can be observed in the high-time-resolution data. In an effort to remove aerosol data that is not regionally representative from the CPC data, an aerosol mask (ENA-AM) has been previously developed and validated during the ACE-ENA campaing (Gallo et al., 2020). The ENA-AM is a standard deviation algorithm based on the statistically different behavior of adjacent data points used to determine periods when the measurements are impacted by local aerosols. This file contain one-minute time resolution submicron aerosol particle concentration datasets from the CPC for the year 2018, and the corresponding ENA-AM datasets to remove datapoints affected by local aerosol sources.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2022

EASTERN NORTH ATLANTIC (ENA) CONDENSATION PARTICLE COUNTER (CPC) with the AEROSOL MASK (ENA-AM) for the YEAR 2016

The Eastern North Atlantic (ENA) central facility is periodically affected by episodes of local aerosols. High-concentration aerosol events associated with potential local aerosol sources are sampled by the concentration particle counter (CPC) and can be observed in the high-time-resolution data. In an effort to remove aerosol data that is not regionally representative from the CPC data, an aerosol mask (ENA-AM) has been previously developed and validated during the ACE-ENA campaign (Gallo et al. 2020). The ENA-AM is a standard deviation algorithm based on the statistically different behavior of adjacent data points used to determine periods when the measurements are impacted by local aerosols. This file contains one-minute-time-resolution submicron aerosol particle concentration data sets from the CPC for the year 2016, and the corresponding ENA-AM data sets to remove data points affected by local aerosol sources.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2021

Real-Time Health Monitoring for Gas Turbine Components Using Online Learning and High-Dimensional Data (Final Report)

Capital-intensive turbomachinery, such as gas turbines and combined cycle plants, are constantly being monitored for performance anomalies, faults, and physical degradation. Although these power-generating assets are equipped with hundreds of sensors, existing monitoring tools can only handle moderate-sized data. As a result, only a handful of aggregate metrics are used to monitor machine health. At the same time, developing advanced tools suitable for large datasets have been restricted by the lack of appropriate data. The objective of this proposal was to demonstrate a Big Data analytics framework for fault detection and diagnosis in gas turbine applications. We develop a predictive analytics framework methodology guided by these experimental data, industrial data from our collaborators, and physics-based models with engineering domain knowledge. Our analytics framework consists of four key components: (1) a data curation process that addresses data storage, data quality assessments, and integrity checks, (2) a feature engineering component that utilizes statistical methods and transformation algorithms guided by physics-based models to extract high-fidelity fault features that can be leveraged for fault detection and classifying fault severities, (3) a Machine Learning-based fault detection and diagnostics algorithms for detecting operational and hardware faults in the combustion and the turbines section. We utilize two industry-class gas turbine component test rigs to generate first of its kind data for critical gas turbine faults with varying severity levels. Advanced gas turbine test facilities will be interrogated using state-of-the-art instrumentation techniques to build fault signatures and data trends for key combustor and turbine faults. Data generated from a combustor test rig (Georgia Tech) and a turbine test rig (Penn State) during both normal operation and with seeded faults serve as the basis for the Big Data sets. The test conditions in the two test facilities include common, critical events that occur in the operation. Utilizing the combustor test rig, we examine two common combustor faults: lean blowout and centerbody degradation. For the turbine section we develop analytic models for monitoring cooling faults in the gas turbine.

20 FOSSIL-FUELED POWER PLANTS↗