Search NASA⌕ Search

SEARCH · Search NASA

Results for “large data”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 793 records · Page 44

Parameter uncertainties for imperfect surrogate models in the low-noise regime

Abstract Bayesian regression determines model parameters by minimizing the expected loss, an upper bound to the true generalization error. However, this loss ignores model form error, or misspecification, meaning parameter uncertainties are significantly underestimated and vanish in the large data limit. As misspecification is the main source of uncertainty for surrogate models of low-noise calculations, such as those arising in atomistic simulation, predictive uncertainties are systematically underestimated. We analyze the true generalization error of misspecified, near-deterministic surrogate models, a regime of broad relevance in science and engineering. We show that posterior parameter distributions must cover every training point to avoid a divergence in the generalization error and design a compatible ansatz which incurs minimal overhead for linear models. The approach is demonstrated on model problems before application to thousand-dimensional datasets in atomistic machine learning. Our efficient misspecification-aware scheme gives accurate prediction and bounding of test errors in terms of parameter uncertainties, allowing this important source of uncertainty to be incorporated in multi-scale computational workflows.

Swinburne, Thomas D. (ORCID:0000000232554257)↗

METAREP: JCVI metagenomics reports—an open source tool for high-performance comparative metagenomics

JCVI Metagenomics Reports (METAREP) is a Web 2.0 application designed to help scientists analyze and compare annotated metagenomics datasets. It utilizes Solr/Lucene, a high performance scalable search engine, to quickly query large data collections. Furthermore, users can use its SQL-like query syntax to filter and refine datasets. METAREP provides graphical summaries for top taxonomic and functional classifications as well as a GO, NCBI Taxonomy and KEGG Pathway Browser. Users can compare absolute and relative counts of multiple datasets at various functional and taxonomic levels. Advanced comparative features comprise statistical tests as well as multidimensional scaling, heatmap and hierarchical clustering plots. Summaries can be exported as tab-delimited files, publication quality plots in PDF format. A data management layer allows collaborative data analysis and result sharing.

59 BASIC BIOLOGICAL SCIENCES↗

Migration depth and residence time of juvenile salmonids in the forebays of hydropower dams prior to passage through turbines or juvenile bypass systems: implications for turbine-passage survival

Little is known about the three-dimensional depth distributions in rivers of individually marked fish that are in close proximity to hydropower facilities. Knowledge of the depth distributions of fish approaching dams can be used to understand how vulnerable fish are to injuries such as barotrauma as they pass through dams. To predict the possibility of barotrauma injury caused by pressure changes during turbine passage, it is necessary to understand fish behaviour relative to acclimation depth in dam forebays as they approach turbines. A guiding study was conducted using high-resolution three-dimensional tracking results of salmonids implanted with Juvenile Salmon Acoustic Telemetry System transmitters to investigate the depth distributions of subyearling and yearling Chinook salmon ( Oncorhynchus tshawytscha ) and juvenile steelhead ( Oncorhynchus mykiss ) passing two dams on the Snake River in Washington State. Multiple approaches were evaluated to describe the depth at which fish were acclimated, and statistical analyses were performed on large data sets extracted from ~28 000 individually tagged fish during 2012 and 2013. Our study identified patterns of depth distributions of juvenile salmonids in forebays prior to passage through turbines or juvenile bypass systems. This research indicates that the median depth at which juvenile salmonids approached turbines ranged from 2.8 to 12.2 m, with the depths varying by species/life history, year, location (which dam) and diel period (between day and night). One of the most enlightening findings was the difference in dam passage associated with the diel period. The amount of time that turbine-passed fish spent in the immediate forebay prior to entering the powerhouse was much lower during the night than during the day. This research will allow scientists to understand turbine-passage survival better and enable them to assess more accurately the effects of dam passage on juvenile salmon survival.

60 APPLIED LIFE SCIENCES↗

Genetic and Functional Diversity Help Explain Pathogenic, Weakly Pathogenic, and Commensal Lifestyles in the Genus Xanthomonas

The genus Xanthomonas has been primarily studied for pathogenic interactions with plants. However, besides host and tissue-specific pathogenic strains, this genus also comprises nonpathogenic strains isolated from a broad range of hosts, sometimes in association with pathogenic strains, and other environments, including rainwater. Based on their incapacity or limited capacity to cause symptoms on the host of isolation, nonpathogenic xanthomonads can be further characterized as commensal and weakly pathogenic. This study aimed to understand the diversity and evolution of nonpathogenic xanthomonads compared to their pathogenic counterparts based on their cooccurrence and phylogenetic relationship and to identify genomic traits that form the basis of a life history framework that groups xanthomonads by ecological strategies. We sequenced genomes of 83 strains spanning the genus phylogeny and identified eight novel species, indicating unexplored diversity. While some nonpathogenic species have experienced a recent loss of a type III secretion system, specifically the hrp2 cluster, we observed an apparent lack of association of the hrp2 cluster with lifestyles of diverse species. We performed association analysis on a large data set of 337 Xanthomonas strains to explain how xanthomonads may have established association with the plants across the continuum of lifestyles from commensals to weak pathogens to pathogens. Presence of distinct transcriptional regulators, distinct nutrient utilization and assimilation genes, transcriptional regulators, and chemotaxis genes may explain lifestyle-specific adaptations of xanthomonads.

59 BASIC BIOLOGICAL SCIENCES↗

Moment tensor event identification for collapses

SUMMARY We introduce a seismic identification method for collapse events using moment tensors (MTs). We start by computing full (six-element) MT solutions for 43 identified collapse events from around the world, and statistically characterizing the population on the MT hypersphere. We then test a large data set of over 1000 full MTs for the western U.S. against the distribution of collapses using a MT-based identification method similarly as used for testing explosions. Known collapses and explosions are readily identified, along with other anomalous events in the Geysers and central California coast. Misidentification rates are determined for various screening angles with optimal misidentification rates between earthquakes and collapses on the order of 3 per cent. The method is demonstrated to be very effective at identifying non-earthquake sources with a 97–98 per cent accuracy. It is likely to be transportable to other regions, and can be used for event identification anywhere full MT solutions are routinely calculated.

58 GEOSCIENCES↗

The weirdest SDSS galaxies: results from an outlier detection algorithm

How can we discover objects we did not know existed within the large data sets that now abound in astronomy? In this paper, we present an outlier detection algorithm that we developed, based on an unsupervised Random Forest. We test the algorithm on more than two million galaxy spectra from the Sloan Digital Sky Survey and examine the 400 galaxies with the highest outlier score. We find objects which have extreme emission line ratios and abnormally strong absorption lines, objects with unusual continua, including extremely reddened galaxies. We find galaxy–galaxy gravitational lenses, double-peaked emission line galaxies and close galaxy pairs. We find galaxies with high ionization lines, galaxies that host supernovae and galaxies with unusual gas kinematics. Only a fraction of the outliers we find were reported by previous studies that used specific and tailored algorithms to find a single class of unusual objects. Our algorithm is general and detects all of these classes, and many more, regardless of what makes them peculiar. Lastly, it can be executed on imaging, time series and other spectroscopic data, operates well with thousands of features, is not sensitive to missing values and is easily parallelizable.

79 ASTRONOMY AND ASTROPHYSICS↗

GUNDAM: a toolkit for fast spatial correlation functions in galaxy surveys

ABSTRACT We describe the capabilities of a new software package to calculate two-point correlation functions (2PCFs) of large galaxy samples. The code can efficiently estimate 3D/projected/angular 2PCFs with a variety of statistical estimators and bootstrap errors, and is intended to provide a complete framework (including calculation, storage, manipulation, and plotting) to perform this type of spatial analysis with large redshift surveys. Gundam implements a very fast skip list/linked list algorithm that efficiently counts galaxy pairs and avoids the computation of unnecessary distances. It is several orders of magnitude faster than a naive pair counter, and matches or even surpass other advanced algorithms. The implementation is also embarrassingly parallel, making full use of multicore processors or large computational clusters when available. The software is designed to be flexible, user friendly and easily extensible, integrating optimized, well-tested packages already available in the astronomy community. Out of the box, it already provides advanced features such as custom weighting schemes, fibre collision corrections and 2D correlations. Gundam will ultimately provide an efficient toolkit to analyse the large-scale structure ‘buried’ in upcoming extremely large data sets generated by future surveys.

Donoso, E.↗

Machine Learning in Infectious Disease for Risk Factor Identification and Hypothesis Generation: Proof of Concept Using Invasive Candidiasis

Machine learning (ML) models can handle large data sets without assuming underlying relationships and can be useful for evaluating disease characteristics, yet they are more commonly used for predicting individual disease risk than for identifying factors at the population level. We offer a proof of concept applying random forest (RF) algorithms to Candida-positive hospital encounters in an electronic health record database of patients in the United States. Candida-positive encounters were extracted from the Cerner HealthFacts database; invasive infections were laboratory-positive sterile site Candida infections. Features included demographics, admission source, care setting, physician specialty, diagnostic and procedure codes, and medications received before the first positive Candida culture. We used RF to assess risk factors for 3 outcomes: any invasive candidiasis (IC) vs non-IC, within-species IC vs non-IC (eg, invasive C. glabrata vs noninvasive C. glabrata), and between-species IC (eg, invasive C. glabrata vs all other IC). Fourteen of 169 (8%) variables were consistently identified as important features in the ML models. When evaluating within-species IC, for example, invasive C. glabrata vs non-invasive C. glabrata, we identified known features like central venous catheters, intensive care unit stay, and gastrointestinal operations. In contrast, important variables for invasive C. glabrata vs all other IC included renal disease and medications like diabetes therapeutics, cholesterol medications, and antiarrhythmics. Known and novel risk factors for IC were identified using ML, demonstrating the hypothesis-generating utility of this approach for infectious disease conditions about which less is known, specifically at the species level or for rarer diseases.

60 APPLIED LIFE SCIENCES↗

Hourly Effect of Pretreatment With IV Antibiotics on Blood Culture Positivity Rate in Emergency Department Patients

Abstract Although it is intuitive that antibiotics administered before obtaining a blood culture would reduce the likelihood of obtaining a positive culture, it is not clear exactly how rapidly and to what extent blood becomes sterile after administration of intravenous (IV) antibiotics. Using a large data set of patients admitted from the UFHealth Shands Adult Emergency Department (ED) between 2012 and 2016 (n = 25 686), we had the opportunity to more closely examine the effect of starting IV antibiotics before vs after obtaining blood cultures. We present data on the effect of pretreatment with IV antibiotics for both septic and nonseptic ED patients on the blood culture positivity rate on an hour-by-hour basis, as well as the effects on distribution of species recovered and the impact of antibiotic resistance in empiric treatment with antibiotics.

Rand, Kenneth H.↗

Search for the chiral magnetic effect with isobar collisions at $\sqrt{s_{NN}}$ = 200 GeV by the STAR Collaboration at the BNL Relativistic Heavy Ion Collider

The chiral magnetic effect (CME) is predicted to occur as a consequence of a local violation of P and CP symmetries of the strong interaction amidst a strong electro-magnetic field generated in relativistic heavy-ion collisions. Experimental manifestation of the CME involves a separation of positively and negatively charged hadrons along the direction of the magnetic field. Previous measurements of the CME-sensitive charge-separation observables remain inconclusive because of large background contributions. In order to better control the influence of signal and backgrounds, the STAR Collaboration performed a blind analysis of a large data sample of approximately 3.8 billion isobar collisions of $^{96}_{44}$Ru + $^{96}_{44}$Ru and $^{96}_{40}$Zr + $^{96}_{40}$Zr at $\sqrt{s_{NN}}$ = 200 GeV. Prior to the blind analysis, the CME signatures are predefined as a significant excess of the CME-sensitive observables in Ru + Ru collisions over those in Zr + Zr collisions, owing to a larger magnetic field in the former. Here, a precision down to 0.4% is achieved, as anticipated, in the relative magnitudes of the pertinent observables between the two isobar systems. Observed differences in the multiplicity and flow harmonics at the matching centrality indicate that the magnitude of the CME background is different between the two species. No CME signature that satisfies the predefined criteria has been observed in isobar collisions in this blind analysis.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

Measurement of the Λ hyperon lifetime

A new, more precise measurement of the Λ hyperon lifetime is performed using a large data sample of Pb-Pb collisions at $\sqrt{s_{NN}}$=5.02 TeV with ALICE. The Λ and $\bar{Λ}$ hyperons are reconstructed at midrapidity using their two-body weak decay channel Λ→p+π - and $\bar{Λ}$→$\bar{p}$+π + . The measured value of the Λ lifetime is τ Λ =[261.07±0.37(stat.)±0.72(syst.)] ps. The relative difference between the lifetime of Λ and $\bar{Λ}$, which represents an important test of CPT invariance in the strangeness sector, is also measured. The obtained value (τ Λ -τ $\bar{Λ}$ )/τ Λ =0.0013±0.0028(stat.)±0.0021(syst.) is consistent with zero within the uncertainties. Both measurements of the Λ hyperon lifetime and of the relative difference between τ Λ and τ $\bar{Λ}$ are in agreement with the corresponding world averages of the Particle Data Group and about a factor of three more precise.

43 PARTICLE ACCELERATORS↗

Time-dependent Dalitz plot measurement of CP parameters in B{sup 0}{yields}K{sub S}{sup 0}{pi}{sup +}{pi}{sup -} decays

We present a time-dependent Dalitz plot measurement of CP violation parameters in B{sup 0}{yields}K{sub S}{sup 0}{pi}{sup +}{pi}{sup -} decays. These results are obtained from a large data sample that contains 657x10{sup 6} BB pairs collected at the {upsilon}(4S) resonance with the Belle detector at the KEKB asymmetric-energy e{sup +}e{sup -} collider. For the CP violation parameters, we obtain two consistent solutions that describe the data well. The first of these solutions may be preferred by external information from other measurements. There is no evidence for direct CP violation in B{sup 0}{yields}{rho}{sup 0}(770)K{sub S}{sup 0}, B{sup 0}{yields}f{sub 0}(980)K{sub S}{sup 0} and B{sup 0}{yields}K*{sup +}(892){pi}{sup -}, while measurements of mixing-induced CP violation in B{sup 0}{yields}{rho}{sup 0}(770)K{sub S}{sup 0} and B{sup 0}{yields}f{sub 0}(980)K{sub S}{sup 0} decays are consistent with that of b{yields}ccs decays. We also measured the phase difference between B{sup 0}{yields}K*{sup +}(892){pi}{sup -} and B{sup 0}{yields}K{sup -}(892){pi}{sup +}, which may be used to extract {phi}{sub 3}.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

Magnetic dipole excitations in nuclei: Elementary modes of nucleonic motion

The nucleus is one of the most multifaceted many-body systems in the Universe. It exhibits a multitude of responses depending on the way one ''probes'' it. With increasing technical advancements of beams at the various accelerators and of detection systems the nucleus has, over and over again, surprised us by expressing always new ways of ''organized'' structures and layers of complexity. Nuclear magnetism is one of those fascinating faces of the atomic nucleus discussed in the present review. We shall not just limit ourselves to presenting the by now large data set that has been obtained in the past two decades using various probes, electromagnetic and hadronic alike and that presents ample evidence for a low-lying orbital scissors mode around 3 MeV, albeit fragmented over an energy interval of the order of 1.5 MeV, and higher-lying spin-flip strength in the energy region 5-9 MeV in deformed nuclei nor to the presently discovered evidence for low-lying proton-neutron isovector quadrupole excitations in spherical nuclei. To the contrary, the experimental evidence is put in the perspectives of understanding the atomic nucleus and its various structures of well-organized modes of motion and thus enlarges the discussion to more general fermion and bosonic many-body systems.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

Machine-learning-informed scattering correlation analysis of sheared colloids

We have carried out theoretical analysis, Monte Carlo simulations and machine-learning analysis to quantify microscopic rearrangements of dilute dispersions of spherical colloidal particles from coherent scattering intensity. Both monodisperse and polydisperse dispersions of colloids were created and underwent a rearrangement consisting of an affine simple shear and non-affine rearrangement using the Monte Carlo method. We calculated the coherent scattering intensity of the dispersions and the correlation function of intensity before and after the rearrangement and generated a large data set of angular correlation functions for varying system parameters, including number density, polydispersity, shear strain and non-affine rearrangement. Singular value decomposition of the data set shows the feasibility of machine-learning inversion from the correlation function for the polydispersity, shear strain and non-affine rearrangement using only three parameters. A Gaussian process regressor is then trained on the data set and can retrieve the affine shear strain, non-affine rearrangement and polydispersity with relative errors of 3%, 1% and 6%, respectively. Altogether, our model provides a framework for quantitative studies of both steady and non-steady microscopic dynamics of colloidal dispersions using coherent scattering methods.

Gaussian process regression↗

Fault-Tolerant and Elastic Streaming MapReduce with Decentralized Coordination

The MapReduce programming model, due to its simplicity and scalability, has become an essential tool for processing large data volumes in distributed environments. Recent Stream Processing Systems (SPS) extend this model to provide low-latency analysis of high-velocity continuous data streams. However, integrating MapReduce with streaming poses challenges: first, the runtime variations in data characteristics such as data-rates and key-distribution cause resource overload, that inturn leads to fluctuations in the Quality of the Service (QoS); and second, the stateful reducers, whose state depends on the complete tuple history, necessitates efficient fault-recovery mechanisms to maintain the desired QoS in the presence of resource failures. We propose an integrated streaming MapReduce architecture leveraging the concept of consistent hashing to support runtime elasticity along with locality-aware data and state replication to provide efficient load-balancing with low-overhead fault-tolerance and parallel fault-recovery from multiple simultaneous failures. Our evaluation on a private cloud shows up to 2:8 improvement in peak throughput compared to Apache Storm SPS, and a low recovery latency of 700 -1500 ms from multiple failures.

97 MATHEMATICS AND COMPUTING↗

Preface IEEE LDAV 2023

Join us for the 13th IEEE Symposium on Large Data Analysis and Visualization (IEEE LDAV) on Monday, October 23rd 2023 collocated with IEEE VIS 2023 in Melbourne, Victoria, Australia.

Bremer, Peer-Timo↗

IEEE LDAV 2024 Preface

Join us here for the 14th IEEE Symposium on Large Data Analysis and Visualization (IEEE LDAV) on Sunday, October 13th 2024 collocated with IEEE VIS 2024 in St. Pete Beach, Florida, USA.

Beyer, Johanna↗

Considerations for AMI-Based Operations for Distribution Feeders

More than $5 billion in investments in advanced metering infrastructure (AMI) technologies, AMI deployments, as pervasive secondary network voltage monitoring systems, provide opportunities for utility operations and controls. This paper focuses on the considerations for AMI-based tools and techniques as the industry moves toward operationalizing such large data sets. Phase identification is a first such tool. Numerous distribution network analysis, monitoring, and control applications - including volt/volt-ampere reactive control, state estimation, and distribution automation - require accurate phase connectivity information in the system models. The phase connectivity database maintained by utilities is inaccurate because of a significant amount of missing data, restoration activities, and network reconfiguration. Existing phase identification techniques that estimate phase connectivity work well in distribution feeders that have low or no photovoltaic (PV) generation; however, they fail to identify the phases accurately when considerable PV generation is present. This work addresses the phase identification problem in the presence of high PV generation using statistical analysis methods. Further, insights into the AMI data requirements for this application in terms of data window length and resolution are provided using sensitivity analysis performed on an actual distribution feeder model of San Diego Gas & Electric Company. The results of this study show that the phase connectivity, even in the presence of high PV generation, can be accurately identified using statistical analysis of AMI data of 1 day.

24 POWER TRANSMISSION AND DISTRIBUTION↗