Search NASA⌕ Search

SEARCH · Search NASA

Results for “sampling algorithms”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,315 records · Page 73

Measurement range of phase retrieval in optical surface and wavefront metrology

Phase retrieval employs very simple data collection hardware and iterative algorithms to determine the phase of an optical field. We have derived limitations on phase retrieval, as applied to optical surface and wavefront metrology, in terms of the speed of beam (i.e., f-number or numerical aperture) and amount of aberration using arguments based on sampling theory and geometrical optics. These limitations suggest methodologies for expanding these ranges by increasing the complexity of the measurement arrangement, the phase-retrieval algorithm, or both. We have simulated one of these methods where a surface is measured at unusual conjugates.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Cosmological parameters from CMB maps without likelihood approximation

We propose an efficient Bayesian Markov chain Monte Carlo (MCMC) algorithm for estimating cosmological parameters from cosmic microwave background (CMB) data without the use of likelihood approximations. It builds on a previously developed Gibbs sampling framework that allows for exploration of the joint CMB sky signal and power spectrum posterior, P(s,C{sub ℓ}|d), and addresses a long-standing problem of efficient parameter estimation simultaneously in regimes of high and low signal-to-noise ratio. To achieve this, our new algorithm introduces a joint Markov chain move in which both the signal map and power spectrum are synchronously modified, by rescaling the map according to the proposed power spectrum before evaluating the Metropolis–Hastings accept probability. Such a move was already introduced by Jewell et al., who used it to explore low signal-to-noise posteriors. However, they also found that the same algorithm is inefficient in the high signal-to-noise regime, since a brute-force rescaling operation does not account for phase information. This problem is mitigated in the new algorithm by subtracting the Wiener filter mean field from the proposed map prior to rescaling, leaving high signal-to-noise information invariant in the joint step, and effectively only rescaling the low signal-to-noise component. To explore the full posterior, the new joint move is then interleaved with a standard conditional Gibbs move for the sky map. We apply our new algorithm to simplified simulations for which we can evaluate the exact posterior to study both its accuracy and its performance, and find good agreement with the exact posterior; marginal means agree to ≲0.006σ and standard deviations to better than ∼3%. The Markov chain correlation length is of the same order of magnitude as those obtained by other standard samplers in the field.

79 ASTRONOMY AND ASTROPHYSICS↗

The Dark Energy Survey: Cosmology Results with ∼1500 New High-redshift Type Ia Supernovae Using the Full 5 yr Data Set

Abstract We present cosmological constraints from the sample of Type Ia supernovae (SNe Ia) discovered and measured during the full 5 yr of the Dark Energy Survey (DES) SN program. In contrast to most previous cosmological samples, in which SNe are classified based on their spectra, we classify the DES SNe using a machine learning algorithm applied to their light curves in four photometric bands. Spectroscopic redshifts are acquired from a dedicated follow-up survey of the host galaxies. After accounting for the likelihood of each SN being an SN Ia, we find 1635 DES SNe in the redshift range 0.10 < z < 1.13 that pass quality selection criteria sufficient to constrain cosmological parameters. This quintuples the number of high-quality z > 0.5 SNe compared to the previous leading compilation of Pantheon+ and results in the tightest cosmological constraints achieved by any SN data set to date. To derive cosmological constraints, we combine the DES SN data with a high-quality external low-redshift sample consisting of 194 SNe Ia spanning 0.025 < z < 0.10. Using SN data alone and including systematic uncertainties, we find Ω M = 0.352 ± 0.017 in flat ΛCDM. SN data alone now require acceleration ( q 0 < 0 in ΛCDM) with over 5 σ confidence. We find ( Ω M , w ) = ( 0.264 − 0.096 + 0.074 , − 0.80 − 0.16 + 0.14 ) in flat w CDM. For flat w 0 w a CDM, we find ( Ω M , w 0 , w a ) = ( 0.495 − 0.043 + 0.033 , − 0.36 − 0.30 + 0.36 , − 8.8 − 4.5 + 3.7 ) , consistent with a constant equation of state to within ∼2 σ . Including Planck cosmic microwave background, Sloan Digital Sky Survey baryon acoustic oscillation, and DES 3 × 2pt data gives (Ω M , w ) = (0.321 ± 0.007, −0.941 ± 0.026). In all cases, dark energy is consistent with a cosmological constant to within ∼2 σ . Systematic errors on cosmological parameters are subdominant compared to statistical errors; these results thus pave the way for future photometrically classified SN analyses.

79 ASTRONOMY AND ASTROPHYSICS↗

Neural Learning Based Blind Source Separation for Detection of Material Defects in Pulsed Thermography Images

In this paper, we introduce a neural learning-based approach to blind source separation for detection of material flaws in pulsed thermography (PT) images. This approach can be used to detect internal defects (pores) in metallic Additively Manufactured (AM) materials. Such defects occur in high-strength alloys produced with direct laser sintering AM method for nuclear energy applications. Pulsed thermal imaging system utilizes a high intensity flash lamp to rapidly heat surface of sample, a high sensitivity infrared camera to capture data of surface temperature variations. The data cube obtained with PT (stack of surface temperature images at different times) can be analyzed with image processing algorithms to detect material defects. Compared with conventional nondestructive evaluation (NDE) methods, such as digital radiography and ultrasonic testing, the PT pulsed infrared thermal detection technique has advantages of one-sided non-contact measurements, fast processing of large sample areas captured in one image. Detection of small material defects requires finding features in thermal images, which have signal contrast levels approaching sensitivity limit of IR camera. In this study, an optimized Neural Learning based Blind Source Separation (NLBSS) algorithm, including Principal Component Analysis (PCA), and Independent Component Analysis (ICA) is demonstrated to automatically extract principal temporal and spatial features of thermography frames to enhance flaw detection. By using the NLBSS algorithm, material internal defects can be automatically detected. Furthermore, this processing approach compensates for experimental thermal imaging artifacts, such as noise and uneven heating. By merging artificial intelligence with the photothermic, the NDE system detects internal calibrated defects of various sizes and depths in AM nuclear-grade metallic alloys.

Zhang, Xin↗

A DIRECT MEASUREMENT OF HIERARCHICAL GROWTH IN GALAXY GROUPS SINCE z {approx} 1

We present the first measurement of the evolution of the galaxy group stellar mass function (GrSMF) to redshift z {approx}> 1 and low masses (M{sub *} > 10{sup 12} M{sub Sun }). Our results are based on early data from the Carnegie-Spitzer-IMACS Survey, utilizing low-resolution spectra and broadband optical/near-IR photometry to measure redshifts for a 3.6 {mu}m selected sample of 37,000 galaxies over a 5.3 deg{sup 2} area to z {approx} 1.2. Employing a standard friends-of-friends algorithm for all galaxies more massive than log M{sub *}/M{sub Sun} = 10.5, we find a total of {approx}4000 groups. Correcting for spectroscopic incompleteness (including slit collisions), we build cumulative stellar mass functions for these groups in redshift bins at z > 0.35, comparing to the z = 0 and z > 0 mass functions from various group and cluster samples. Our derived mass functions match up well with z > 0.35 X-ray-selected clusters, and strong evolution is evident at all masses over the past 8 Gyr. Given the already low level of star formation activity in galaxies at these masses, we therefore attribute most of the observed growth in the GrSMF to group-group and group-galaxy mergers, in accordance with qualitative notions of hierarchical structure formation. Given the factor of 3-10 increase in the number density of groups and clusters with M{sub *} > 10{sup 12} M{sub Sun} since z = 1 and the strong anticorrelation between star formation activity and environmental density during this epoch, this late-time growth in group-sized halos may therefore be an important contributor to the structural and star formation evolution of massive galaxies over the past 8 Gyr.

79 ASTRONOMY AND ASTROPHYSICS↗

Learning Stochastic Parametric Differentiable Predictive Control Policies

We present a scalable unsupervised learning-based method for obtaining explicit control policies for model predictive control problems for stochastic linear systems with additive uncertainties subject to nonlinear chance constraints. We call the proposed method stochastic parametric differentiable predictive control (SP-DPC), which extends the recently proposed deterministic DPC policy optimization algorithm. We formulate the SP-DPC as a deterministic approximation to the stochastic parametric constrained optimal control problem via independent sampling of the problem's parameters and uncertainties. This formulation allows us to directly compute the policy gradients via automatic differentiation of the problem's value function, evaluated over sampled parameters and uncertainties. In particular, the computed expectation of the problem's value function is backpropagated through the finite-time closed-loop system rollouts parametrized by a known nominal system dynamics model and neural control policy. We also provide theoretical probabilistic guarantees on closed-loop stability and chance constraints satisfaction for systems controlled by learned neural policies. We demonstrate the computational efficiency and scalability of the proposed policy optimization algorithm in three numerical examples, including systems with a large number of states or subject to nonlinear constraints.

Drgona, Jan↗

Sample preparation and biomass determination of SRF model mixture using cryogenic milling and the adapted balance method

Highlights: • An alternative sample comminution procedure for SRF is tested. • Proof of principle is shown on a SRF model mixture. • The biogenic content of the SRF is analyzed with the adapted balance method. • The novel method combines combustion analysis and a data reconciliation algorithm. • Factors for the variance of the analysis results are statistically quantified. - Abstract: The biogenic fraction of a simple solid recovered fuel (SRF) mixture (80 wt% printer paper/20 wt% high density polyethylene) is analyzed with the in-house developed adapted balance method (aBM). This fairly new approach is a combination of combustion elemental analysis (CHNS) and a data reconciliation algorithm based on successive linearisation for evaluation of the analysis results. This method shows a great potential as an alternative way to determine the biomass content in SRF. However, the employed analytical technique (CHNS elemental analysis) restricts the probed sample mass to low amounts in the range of a few hundred milligrams. This requires sample comminution to small grain sizes (<200 μm) to generate representative SRF specimen. This is not easily accomplished for certain material mixtures (e.g. SRF with rubber content) by conventional means of sample size reduction. This paper presents a proof of principle investigation of the sample preparation and analysis of an SRF model mixture with the use of cryogenic impact milling (final sample comminution) and the adapted balance method (determination of biomass content). The so derived sample preparation methodology (cutting mills and cryogenic impact milling) shows a better performance in accuracy and precision for the determination of the biomass content than one solely based on cutting mills. The results for the determination of the biogenic fraction are within 1–5% of the data obtained by the reference methods, selective dissolution method (SDM) and {sup 14}C-method ({sup 14}C-M)

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗

Stochastic derivative-free optimization using a trust region framework

This study presents a trust region algorithm to minimize a function f when one has access only to noise-corrupted function values f¯. The model-based algorithm dynamically adjusts its step length, taking larger steps when the model and function agree and smaller steps when the model is less accurate. The method does not require the user to specify a fixed pattern of points used to build local models and does not repeatedly sample points. If f is sufficiently smooth and the noise is independent and identically distributed with mean zero and finite variance, we prove that our algorithm produces iterates such that the corresponding function gradients converge in probability to zero. As a result, we present a prototype of our algorithm that, while simplistic in its management of previously evaluated points, solves benchmark problems in fewer function evaluations than do existing stochastic approximation methods.

97 MATHEMATICS AND COMPUTING↗

Fast and low-dose computed laminography using compressive sensing based technique

Computed laminography (CL) is well known for inspecting microstructures in the materials, weldments and soldering defects in high density packed components or multilayer printed circuit boards. The overload problem on x-ray tube and gross failure of the radio-sensitive electronics devices during a scan are among important issues in CL which needs to be addressed. The sparse-view CL can be one of the viable option to overcome such issues. In this work a numerical aluminum welding phantom was simulated to collect sparsely sampled projection data at only 40 views using a conventional CL scanning scheme i.e. oblique scan. A compressive-sensing inspired total-variation (TV) minimization algorithm was utilized to reconstruct the images. It is found that the images reconstructed using sparse view data are visually comparable with the images reconstructed using full scan data set i.e. at 360 views on regular interval. We have quantitatively confirmed that tiny structures such as copper and tungsten slags, and copper flakes in the reconstructed images from sparsely sampled data are comparable with the corresponding structure present in the fully sampled data case. A blurring effect can be seen near the edges of few pores at the bottom of the reconstructed images from sparsely sampled data, despite the overall image quality is reasonable for fast and low-dose NDT.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Bayesian analysis of energy and count rate data for detection of low count rate radioactive sources

We propose a radiation detection system which generates its own discrete sampling distribution based on past measurements of background. The advantage to this approach is that it can take into account variations in background with respect to time, location, energy spectra, detector-specific characteristics (i.e. different efficiencies at different count rates and energies), etc. This would therefore be a 'machine learning' approach, in which the algorithm updates and improves its characterization of background over time. The system would have a 'learning mode,' in which it measures and analyzes background count rates, and a 'detection mode,' in which it compares measurements from an unknown source against its unique background distribution. By characterizing and accounting for variations in the background, general purpose radiation detectors can be improved with little or no increase in cost. The statistical and computational techniques to perform this kind of analysis have already been developed. The necessary signal analysis can be accomplished using existing Bayesian algorithms which account for multiple channels, multiple detectors, and multiple time intervals. Furthermore, Bayesian machine-learning techniques have already been developed which, with trivial modifications, can generate appropriate decision thresholds based on the comparison of new measurements against a nonparametric sampling distribution. (authors)

97 MATHEMATICS AND COMPUTING↗

Recent Developments in Fast Neutron Detection and Multiplicity Counting with Liquid Scintillator

For many years, LLNL researchers have been developing time-correlated neutron detection techniques and algorithms for applications such as Arms Control, Threat Detection and Nuclear Material Assay. Many of the techniques have been developed specifically for the relatively low efficiency (a few percent) attainable by detector systems limited to man-portability. Historically, thermal neutron detectors (mainly {sup 3}He) were used, taking advantage of the high thermal neutron interaction cross sections. More recently, we have been investigating the use of fast neutron detection with liquid scintillators, inorganic crystals, and in the near future, pulse-shape discriminating plastics that respond over 1000 times faster (ns versus tens of {mu}s) than thermal neutron detectors. Fast neutron detection offers considerable advantages since the inherent ns production timescales of spontaneous fission and neutron-induced fission are preserved and measured instead of being lost by thermalization required for thermal neutron detectors. We are now applying fast neutron technology to the safeguards regime in the form of fast portable digital electronics as well as faster and less hazardous scintillator formulations. Faster detector response times and sensitivity to neutron momentum show promise for measuring, differentiating, and assaying samples that have modest to very high count rates, as well as mixed fission sources like Cm and Pu. We report on measured results with our existing liquid scintillator array and progress on the design of a nuclear material assay system that incorporates fast neutron detection, including the surprising result that fast liquid scintillator detectors become competitive and even surpass the precision of {sup 3}He-based counters measuring correlated pairs in modest (kg) samples of plutonium.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

Autonomous METLIN-Guided In-source Fragment Annotation for Untargeted Metabolomics

Computational metabolite annotation in untargeted profiling aims at uncovering neutral molecular masses of underlying metabolites and assign those with putative identities. Existing annotation strategies rely on the observation and annotation of adducts to determine metabolite neutral masses. However, a significant fraction of features usually detected in untargeted experiments remains unannotated, which limits our ability to determine neutral molecular masses. Despite the availability of tools to annotate, relatively few of them benefit from the inherent presence of in-source fragments in liquid chromatography-electrospray ionization-mass spectrometry. Here, we introduce a strategy to annotate in-source fragments in untargeted data using low-energy tandem mass spectrometry (MS) spectra from the METLIN library. Our algorithm, MISA (METLIN-guided in-source annotation), compares detected features against low-energy fragments from MS/MS spectra, enabling robust annotation and putative identification of metabolic features based on low-energy spectral matching. The algorithm was evaluated through an annotation analysis of a total of 140 metabolites across three different sets of biological samples analyzed with liquid chromatography-mass spectrometry. Results showed that, in cases where adducts were not formed or detected, MISA was able to uncover neutral molecular masses by in-source fragment matching. MISA was also able to provide putative metabolite identities via two annotation scores. These scores take into account the number of in-source fragments matched and the relative intensity similarity between the experimental data and the reference low-energy MS/MS spectra. Overall, results showed that in-source fragmentation is a highly frequent phenomena that should be considered for comprehensive feature annotation. Thus, combined with adduct annotation, this strategy adds a complementary annotation layer, enabling in-source fragments to be annotated and increasing putative identification confidence. The algorithm is integrated into the XCMS Online platform and is freely available at http://xcmsonline.scripps.edu.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Automatic Calibration and Health Monitoring of Infrastructure Sensors

Smart transportation infrastructure relies on networks of heterogeneous sensors - cameras, radars, and lidars - continuously monitoring traffic conditions. However, executing the initial spatial calibration of multiple sensors and the subsequent health monitoring presents significant operational challenges. Environmental factors, mechanical vibrations, and gradual drift cause spatial misalignment, degrading fusion performance and tracking accuracy. Traditional calibration approaches require manual intervention with specialized targets or survey equipment, resulting in service interruptions and high maintenance costs. This work presents an automated framework for initial calibration and continuous health monitoring without human intervention or service disruption. Our approach addresses two critical problems: (1) detecting when sensors become miscalibrated during operation, and (2) automatically re-establishing spatial alignment using only operational traffic data. The health monitoring component analyzes measurement innovations - differences between sensor observations and predicted object states - to detect systematic biases indicative of calibration drift. By computing bias magnitude, directional consistency, and rejection rates, the system identifies miscalibrations as small as 0.5 meters. Unlike traditional methods requiring known calibration targets, our diagnostic operates continuously on live traffic observations, enabling early detection before fusion quality degrades. The automatic recalibration algorithm leverages overlapping sensor fields-of-view and temporal correlation of vehicle observations. Using graph-based optimization, the system automatically discovers which sensor pairs observe common regions, estimates pairwise spatial transformations using RANSAC-based robust estimation, and jointly optimizes all sensor poses through bundle adjustment. The framework handles practical deployment challenges, including different sensor sampling rates (1-10 Hz), varying installation positions, unknown orientations, and limited overlap regions (>10%). When approximate sensor positions are available from installation surveys (+/-1m accuracy), the algorithm additionally estimates sensor orientations, refining both position and rotation to sub-meter and sub-degree accuracy. We validate the framework on multi-hour traffic datasets from six heterogeneous sensors with sampling rates ranging from 1 Hz to 10 Hz. Results demonstrate successful calibration even with sparse overlap (<20%) and automatic detection of miscalibrations exceeding 0.8 meters. This work enables a "deploy-and-forget" sensor infrastructure that maintains calibration autonomously, reducing maintenance costs while improving tracking accuracy. The techniques generalize beyond transportation to any multi-sensor monitoring application requiring robust spatial alignment, including smart cities, industrial monitoring, and surveillance systems.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Efficient simulation of relativistic fermions via vertex models

We have developed an efficient simulation algorithm for strongly interacting relativistic fermions in two-dimensional field theories based on a formulation as a loop gas. It essentially eliminates critical slowing down by sampling two-point correlation functions and allows simulations directly in the massless limit at the critical point. It generates loop configurations with fluctuating topological boundary conditions enabling one to simulate fermions with arbitrary periodic or antiperiodic boundary conditions. As illustrative examples, the algorithm is applied to the Gross-Neveu model and to the Schwinger model in the strong coupling limit.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Reducing acquisition times in multidimensional NMR with a time-optimized Fourier encoding algorithm

Speeding up the acquisition of multidimensional nuclear magnetic resonance (NMR) spectra is an important topic in contemporary NMR, with central roles in high-throughput investigations and analyses of marginally stable samples. A variety of fast NMR techniques have been developed, including methods based on non-uniform sampling and Hadamard encoding, that overcome the long sampling times inherent to schemes based on fast-Fourier-transform (FFT) methods. Here, we explore the potential of an alternative fast acquisition method that leverages a priori knowledge, to tailor polychromatic pulses and customized time delays for an efficient Fourier encoding of the indirect domain of an NMR experiment. By porting the encoding of the indirect-domain to the excitation process, this strategy avoids potential artifacts associated with non-uniform sampling schemes and uses a minimum number of scans equal to the number of resonances present in the indirect dimension. An added convenience is afforded by the fact that a usual 2D FFT can be used to process the generated data. Acquisitions of 2D heteronuclear correlation NMR spectra on quinine and on the anti-inflammatory drug isobutyl propionic phenolic acid illustrate the new method's performance. This method can be readily automated to deal with complex samples such as those occurring in metabolomics, in in-cell as well as in in vivo NMR applications, where speed and temporal stability are often primary concerns.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

A globally sampled high-resolution hand-labeled validation dataset for evaluating surface water extent maps

Effective monitoring of global water resources is increasingly critical due to climate change and population growth. Advancements in remote sensing technology, specifically in spatial, spectral, and temporal resolutions, are revolutionizing water resource monitoring, leading to more frequent and high-quality surface water extent maps using various techniques such as traditional image processing and machine learning algorithms. However, satellite imagery datasets contain trade-offs that result in inconsistencies in performance, such as disparities in measurement principles between optical (e.g., Sentinel-2) and radar (e.g., Sentinel-1) sensors and differences in spatial and spectral resolutions among optical sensors. Therefore, developing accurate and robust surface water mapping solutions requires independent validations from multiple datasets to identify potential biases within the imagery and algorithms. However, high-quality validation datasets are expensive to build, and few contain information on water resources. For this purpose, we introduce a globally sampled, high-spatial-resolution dataset labeled using 3 m PlanetScope imagery. Our surface water extent dataset comprises 100 images, each with a size of 1024×1024 pixels, which were sampled using a stratified random sampling strategy covering all 14 biomes. We highlighted urban and rural regions, lakes, and rivers, including braided rivers and coastal regions. We evaluated two surface water extent mapping methods using our dataset – Dynamic World, based on Sentinel-2, and the NASA IMPACT model, based on Sentinel-1. Dynamic World achieved a mean intersection over union (IoU) of 72.16 % and F1 score of 79.70 %, while the NASA IMPACT model had a mean IoU of 57.61 % and F1 score of 65.79 %. Performance varied substantially across biomes, highlighting the importance of evaluating models on diverse landscapes to assess their generalizability and robustness. Our dataset can be used to analyze satellite products and methods, providing insights into their advantages and drawbacks. Our dataset offers a unique tool for analyzing satellite products, aiding the development of more accurate and robust surface water monitoring solutions. The dataset can be accessed via https://doi.org/10.25739/03nt-4f29.

54 ENVIRONMENTAL SCIENCES↗

J-PLUS: Support vector machine applied to STAR-GALAXY-QSO classification

Context. In modern astronomy, machine learning has proved to be efficient and effective in mining big data from the newest telescopes. Aims. In this study, we construct a supervised machine-learning algorithm to classify the objects in the Javalambre Photometric Local Universe Survey first data release (J-PLUS DR1). Methods. The sample set is featured with 12-waveband photometry and labeled with spectrum-based catalogs, including Sloan Digital Sky Survey spectroscopic data, the Large Sky Area Multi-Object Fiber Spectroscopic Telescope, and VERONCAT – the Veron Catalog of Quasars & AGN. The performance of the classifier is presented with the applications of blind test validations based on RAdial Velocity Extension, the Kepler Input Catalog, the Two Micron All Sky Survey Redshift Survey, and the UV-bright Quasar Survey. A new algorithm was applied to constrain the potential extrapolation that could decrease the performance of the machine-learning classifier. Results. The accuracies of the classifier are 96.5% in the blind test and 97.0% in training cross-validation. The F1-scores for each class are presented to show the balance between the precision and the recall of the classifier. We also discuss different methods to constrain the potential extrapolation.

79 ASTRONOMY AND ASTROPHYSICS↗

A novel procedure on next generation sequencing data analysis using text mining algorithm

Background: Next-generation sequencing (NGS) technologies have provided researchers with vast possibilities in various biological and biomedical research areas. Efficient data mining strategies are in high demand for large scale comparative and evolutional studies to be performed on the large amounts of data derived from NGS projects. Topic modeling is an active research field in machine learning and has been mainly used as an analytical tool to structure large textual corpora for data mining. Methods: We report a novel procedure to analyse NGS data using topic modeling. It consists of four major procedures: NGS data retrieval, preprocessing, topic modeling, and data mining using Latent Dirichlet Allocation (LDA) topic outputs. The NGS data set of the Salmonella enterica strains were used as a case study to show the workflow of this procedure. The perplexity measurement of the topic numbers and the convergence efficiencies of Gibbs sampling were calculated and discussed for achieving the best result from the proposed procedure. Results: The output topics by LDA algorithms could be treated as features of Salmonella strains to accurately describe the genetic diversity of fliC gene in various serotypes. The results of a two-way hierarchical clustering and data matrix analysis on LDA-derived matrices successfully classified Salmonella serotypes based on the NGS data. The implementation of topic modeling in NGS data analysis procedure provides a new way to elucidate genetic information from NGS data, and identify the gene-phenotype relationships and biomarkers, especially in the era of biological and medical big data. Conclusion: The implementation of topic modeling in NGS data analysis provides a new way to elucidate genetic information from NGS data, and identify the gene-phenotype relationships and biomarkers, especially in the era of biological and medical big data.

59 BASIC BIOLOGICAL SCIENCES↗