Search NASA⌕ Search

SEARCH · Search NASA

Results for “science data processing”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records

Prospects of federated machine learning in fluid dynamics

Physics-based models have been mainstream in fluid dynamics for developing predictive models. In recent years, machine learning has offered a renaissance to the fluid community due to the rapid developments in data science, processing units, neural network based technologies, and sensor adaptations. So far in many applications in fluid dynamics, machine learning approaches have been mostly focused on a standard process that requires centralizing the training data on a designated machine or in a data center. In this article, we present a federated machine learning approach that enables localized clients to collaboratively learn an aggregated and shared predictive model while keeping all the training data on each edge device. We demonstrate the feasibility and prospects of such a decentralized learning approach with an effort to forge a deep learning surrogate model for reconstructing spatiotemporal fields. Our results indicate that federated machine learning might be a viable tool for designing highly accurate predictive decentralized digital twins relevant to fluid dynamics.

36 MATERIALS SCIENCE↗

LoVoCCS. I. Survey Introduction, Data Processing Pipeline, and Early Science Results

We present the Local Volume Complete Cluster Survey (LoVoCCS; we pronounce it as "low-vox" or "law-vox," with stress on the second syllable), an NSF's National Optical-Infrared Astronomy Research Laboratory survey program that uses the Dark Energy Camera to map the dark matter distribution and galaxy population in 107 nearby (0.03 < z < 0.12) X-ray luminous ([0.1–2.4 keV] L X500 > 10 44 erg s –1 ) galaxy clusters that are not obscured by the Milky Way. The survey will reach Vera C. Rubin Observatory Legacy Survey of Space and Time (LSST) Year 1–2 depth (for galaxies r = 24.5, i = 24.0, signal-to-noise ratio (S/N) > 20; u = 24.7, g = 25.3, z = 23.8, S/N > 10) and conclude in ~2023 (coincident with the beginning of LSST science operations), and will serve as a zeroth-year template for LSST transient studies. We process the data using the LSST Science Pipelines that include state-of-the-art algorithms and analyze the results using our own pipelines, and therefore the catalogs and analysis tools will be compatible with the LSST. We demonstrate the use and performance of our pipeline using three X-ray luminous and observation-time complete LoVoCCS clusters: A3911, A3921, and A85. A3911 and A3921 have not been well studied previously by weak lensing, and we obtain similar lensing analysis results for A85 to previous studies.

79 ASTRONOMY AND ASTROPHYSICS↗

Enhanced Preparation for Intelligent Cybermanufacturing Systems (EPICS)

Opportunities exist for realizing transformative advances in productivity and reductions in energy footprint through ubiquitous sensing in manufacturing environments. Enhanced Preparation for Intelligent Cybermanufacturing Systems (EPICS) is a 21-month (4 academic semesters, plus one summer) experience for graduate students that focuses on scaling the knowledge, understanding and leadership skills in the cyber manufacturing area. Masters students (8/year, 32 total) complete 2-year projects on industrially-driven project topics, rotating to internships in summer semester to work on scoping and implementation at project partners. Students complete academic training in embedded systems, process modeling, data science, and cloud-based systems design. Their projects are targeted toward sensor retrofit, process monitoring, root cause analysis, and sensor fusion.

Advanced Manufacturing↗

Conserving Land–Atmosphere Synthesis Suite (CLASS)

Accurate estimates of terrestrial water and energy cycle components are needed to better understand climate processes and improve models’ ability to simulate future change. Various observational estimates are available for the individual budget terms; however, these typically show inconsistencies when combined in a budget. In this work, a Conserving Land–Atmosphere Synthesis Suite (CLASS) of estimates of simultaneously balanced surface water and energy budget components is developed. Individual CLASS variable datasets, where possible, 1) combine a range of existing variable product estimates, and hence overcome the limitations of estimates from a single source; 2) are observationally constrained with in situ measurements; 3) have uncertainty estimates that are consistent with their agreement with in situ observations; and 4) are consistent with each other by being able to solve the water and energy budgets simultaneously. First, available datasets of a budget variable are merged by implementing a weighting method that accounts both for the ability of datasets to match in situ measurements and the error covariance between datasets. Then, the budget terms are adjusted by applying an objective variational data assimilation technique (DAT) that enforces the simultaneous closure of the surface water and energy budgets linked through the equivalence of evapotranspiration and latent heat. Comparing component estimates before and after applying the DAT against in situ measurements of energy fluxes and streamflow showed that modified estimates agree better with in situ observations across various metrics, but also revealed some inconsistencies between water budget terms in June over the higher latitudes.

54 ENVIRONMENTAL SCIENCES↗

Process mining for healthcare: Characteristics and challenges

Process mining techniques can be used to analyse business processes using the data logged during their execution. These techniques are leveraged in a wide range of domains, including healthcare, where it focuses mainly on the analysis of diagnostic, treatment, and organisational processes. Despite the huge amount of data generated in hospitals by staff and machinery involved in healthcare processes, there is no evidence of a systematic uptake of process mining beyond targeted case studies in a research context. When developing and using process mining in healthcare, distinguishing characteristics of healthcare processes such as their variability and patient-centred focus require targeted attention. Against this background, the Process-Oriented Data Science in Healthcare Alliance has been established to propagate the research and application of techniques targeting the data-driven improvement of healthcare processes. This paper, an initiative of the alliance, presents the distinguishing characteristics of the healthcare domain that need to be considered to successfully use process mining, as well as open challenges that need to be addressed by the community in the future.

59 BASIC BIOLOGICAL SCIENCES↗

Science of the Van Allen Probes Science Operations Centers

The Van Allen Probes mission operations materialized through a distributed model in which operational responsibility was divided between the Mission Operations Center (MOC) and separate instrument specific SOCs. The sole MOC handled all aspects of telemetering and receiving tasks as well as certain scientifically relevant ancillary tasks. Each instrument science team developed individual instrument specific SOCs proficient in unique capabilities in support of science data acquisition, data processing, instrument performance, and tools for the instrument team scientists. In parallel activities, project scientists took on the task of providing a significant modeling tool base usable by the instrument science teams and the larger scientific community. With a mission as complex as Van Allen Probes, scientific inquiry occurred due to constant and significant collaboration between the SOCs and in concert with the project science team. Planned cross-instrument coordinated observations resulted in critical discoveries during the seven-year mission. Instrument cross-calibration activities elucidated a more seamless set of data products. Specific topics include post-launch changes and enhancements to the SOCs, discussion of coordination activities between the SOCs, SOC specific analysis software, modeling software provided by the Van Allen Probes project, and a section on lessons learned. One of the most significant lessons learned was the importance of the original decision to implement individual team SOCs providing timely and well-documented instrument data for the NASA Van Allen Probes Mission scientists and the larger magnetospheric and radiation belt scientific community.

47 OTHER INSTRUMENTATION↗

An Extreme-Preserving Long-Term Gridded Daily Precipitation Data Set for the Conterminous United States

Extreme daily precipitation contributes to flooding that can cause significant economic damages, and so is important to properly capture in gridded meteorological data sets. This work examines precipitation extremes, the mean precipitation on wet days, and fraction of wet days in two widely used gridded data sets over the conterminous U.S. (CONUS). When compared to the underlying station observations, the gridded data show a 27% reduction in annual 1-day maximum precipitation, 25% increase in wet day fraction, 1.5 to 2.5 day increase in mean wet spell length, 30% low bias in 20-year return values of daily precipitation, and 25% decrease in mean precipitation on wet days. It is shown these changes arise primarily from the time-adjustment applied to put the precipitation gauge observations into a uniform time frame, with the gridding process playing a lesser role. A new daily precipitation data set is developed that omits the time-adjustment (as well as extending the gridded data by 7 years) and is shown to perform significantly better in reproducing extreme precipitation metrics. When the new data set is used to force a land surface model, annually averaged 1-day maximum runoff increases 38% compared to the original data, annual mean runoff increases 17%, evapotranspiration drops 2.3%, and fewer wet days leads to a 3.3% increase in estimated solar insolation. Consequently, these changes are large enough to affect portrayals of flood risk and water balance components important for ecological and climate-change applications across the CONUS.

54 ENVIRONMENTAL SCIENCES↗

PeakDecoder enables machine learning-based metabolite annotation and accurate profiling in multidimensional mass spectrometry measurements

Multidimensional measurements using state-of-the-art separations and mass spectrometry provide advantages in untargeted metabolomics analyses for studying biological and environmental bio-chemical processes. However, the lack of rapid analytical methods and robust algorithms for these heterogeneous data has limited its application. Here, we develop and evaluate a sensitive and high-throughput analytical and computational workflow to enable accurate metabolite profiling. Our workflow combines liquid chromatography, ion mobility spectrometry and data-independent acquisition mass spectrometry with PeakDecoder, a machine learning-based algorithm that learns to distinguish true co-elution and co-mobility from raw data and calculates metabolite identification error rates. We apply PeakDecoder for metabolite profiling of various engineered strains of Aspergillus pseudoterreus, Aspergillus niger, Pseudomonas putida and Rhodosporidium toruloides. Results, validated manually and against selected reaction monitoring and gas-chromatography platforms, show that 2683 features could be confidently annotated and quantified across 116 microbial sample runs using a library built from 64 standards.

59 BASIC BIOLOGICAL SCIENCES↗

Molecular Simulations of the Chain Length Dependent Adsorption of C7‐C14 n ‐Alkanes in ZIF‐8

Recent experiments show that the diffusivities of C8 and C10 n-alkanes in ZIF-8 are higher than those of C7 and C9, respectively. We investigated this unusual ‘odd-even' effect by simulating the adsorption of C7-C14 n-alkanes in ZIF-8 using hybrid Monte Carlo molecular simulations. The resultant adsorption isotherms, guest-host energies, isosteric heats, and chain length distributions are analyzed for trends among the n-alkanes. ZIF-8 cages filled with n-alkanes are characterized using a combination of image processing and data science to quantify the differences in packing that occur with chain length. In conclusion, results indicate that packing changes drastically from C7 to C8 and from C9 to C10, which is consistent with the diffusion trends previously reported in experiments.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

ContScout: sensitive detection and removal of contamination from annotated genomes

Contamination of genomes is an increasingly recognized problem affecting several downstream applications, from comparative evolutionary genomics to metagenomics. Here we introduce ContScout, a precise tool for eliminating foreign sequences from annotated genomes. It achieves high specificity and sensitivity on synthetic benchmark data even when the contaminant is a closely related species, outperforms competing tools, and can distinguish horizontal gene transfer from contamination. A screen of 844 eukaryotic genomes for contamination identified bacteria as the most common source, followed by fungi and plants. Furthermore, we show that contaminants in ancestral genome reconstructions lead to erroneous early origins of genes and inflate gene loss rates, leading to a false notion of complex ancestral genomes. Taken together, we offer here a tool for sensitive removal of foreign proteins, identify and remove contaminants from diverse eukaryotic genomes and evaluate their impact on phylogenomic analyses.

59 BASIC BIOLOGICAL SCIENCES↗

Improvement of eukaryotic protein predictions from soil metagenomes

During the last decades, metagenomics has highlighted the diversity of microorganisms from environmental or host-associated samples. Most metagenomics public repositories use annotation pipelines tailored for prokaryotes regardless of the taxonomic origin of contigs. Consequently, eukaryotic contigs with intrinsically different gene features, are not optimally annotated. Using a bioinformatics pipeline, we have filtered 7.9 billion contigs from 6,872 soil metagenomes in the JGI’s IMG/M database to identify eukaryotic contigs. We have re-annotated genes using eukaryote-tailored methods, yielding 8 million eukaryotic proteins and over 300,000 orphan proteins lacking homology in public databases. Comparing the gene predictions we made with initial JGI ones on the same contigs, we confirmed our pipeline improves eukaryotic proteins completeness and contiguity in soil metagenomes. The improved quality of eukaryotic proteins combined with a more comprehensive assignment method yielded more reliable taxonomic annotation. This dataset of eukaryotic soil proteins with improved completeness, quality and taxonomic annotation reliability is of interest for any scientist aiming at studying the composition, biological functions and gene flux in soil communities involving eukaryotes.

54 ENVIRONMENTAL SCIENCES↗

Coassembly and binning of a twenty-year metagenomic time-series from Lake Mendota

Abstract The North Temperate Lakes Long-Term Ecological Research (NTL-LTER) program has been extensively used to improve understanding of how aquatic ecosystems respond to environmental stressors, climate fluctuations, and human activities. Here, we report on the metagenomes of samples collected between 2000 and 2019 from Lake Mendota, a freshwater eutrophic lake within the NTL-LTER site. We utilized the distributed metagenome assembler MetaHipMer to coassemble over 10 terabases (Tbp) of data from 471 individual Illumina-sequenced metagenomes. A total of 95,523,664 contigs were assembled and binned to generate 1,894 non-redundant metagenome-assembled genomes (MAGs) with ≥50% completeness and ≤10% contamination. Phylogenomic analysis revealed that the MAGs were nearly exclusively bacterial, dominated by Pseudomonadota (Proteobacteria, N = 623) and Bacteroidota (N = 321). Nine eukaryotic MAGs were identified by eukCC with six assigned to the phylum Chlorophyta. Additionally, 6,350 high-quality viral sequences were identified by geNomad with the majority classified in the phylum Uroviricota. This expansive coassembled metagenomic dataset provides an unprecedented foundation to advance understanding of microbial communities in freshwater ecosystems and explore temporal ecosystem dynamics.

59 BASIC BIOLOGICAL SCIENCES↗

Standardized phylogenetic and molecular evolutionary analysis applied to species across the microbial tree of life

There is growing interest in reconstructing phylogenies from the copious amounts of genome sequencing projects that target related viral, bacterial or eukaryotic organisms. To facilitate the construction of standardized and robust phylogenies for disparate types of projects, we have developed a complete bioinformatic workflow, with a web-based component to perform phylogenetic and molecular evolutionary (PhaME) analysis from sequencing reads, draft assemblies or completed genomes of closely related organisms. Furthermore, the ability to incorporate raw data, including some metagenomic samples containing a target organism (e.g. from clinical samples with suspected infectious agents), shows promise for the rapid phylogenetic characterization of organisms within complex samples without the need for prior assembly.

59 BASIC BIOLOGICAL SCIENCES↗

Multimodal X-ray nano-spectromicroscopy analysis of chemically heterogeneous systems

Abstract Understanding the nanoscale chemical speciation of heterogeneous systems in their native environment is critical for several disciplines such as life and environmental sciences, biogeochemistry, and materials science. Synchrotron-based X-ray spectromicroscopy tools are widely used to understand the chemistry and morphology of complex material systems owing to their high penetration depth and sensitivity. The multidimensional (4D+) structure of spectromicroscopy data poses visualization and data-reduction challenges. This paper reports the strategies for the visualization and analysis of spectromicroscopy data. We created a new graphical user interface and data analysis platform named XMIDAS (X-ray multimodal image data analysis software) to visualize spectromicroscopy data from both image and spectrum representations. The interactive data analysis toolkit combined conventional analysis methods with well-established machine learning classification algorithms (e.g. nonnegative matrix factorization) for data reduction. The data visualization and analysis methodologies were then defined and optimized using a model particle aggregate with known chemical composition. Nanoprobe-based X-ray fluorescence (nano-XRF) and X-ray absorption near edge structure (nano-XANES) spectromicroscopy techniques were used to probe elemental and chemical state information of the aggregate sample. We illustrated the complete chemical speciation methodology of the model particle by using XMIDAS. Next, we demonstrated the application of this approach in detecting and characterizing nanoparticles associated with alveolar macrophages. Our multimodal approach combining nano-XRF, nano-XANES, and differential phase-contrast imaging efficiently visualizes the chemistry of localized nanostructure with the morphology. We believe that the optimized data-reduction strategies and tool development will facilitate the analysis of complex biological and environmental samples using X-ray spectromicroscopy techniques.

36 MATERIALS SCIENCE↗

Machine learning-driven predictive resource management in complex science workflows

Here, the collaborative efforts of large communities in science experiments, often comprising thousands of global members, reflect a monumental commitment to exploration and discovery. Recently, advanced and complex data processing has gained increasing importance in science experiments. Data processing workflows typically consist of multiple intricate steps, and the precise specification of resource requirements is crucial for each step to allocate optimal resources for effective processing. Estimating resource requirements in advance is challenging due to a wide range of analysis scenarios, varying skill levels among community members, and the continuously increasing spectrum of computing options. One practical approach to mitigate these challenges involves initially processing a subset of each step to measure precise resource utilization from actual processing profiles before completing the entire step. While this two-staged approach enables processing on optimal resources for most of the workflow, it has drawbacks such as initial inaccuracies leading to potential failures and suboptimal resource usage, along with overhead from waiting for initial processing completion, which is critical for fast-turnaround analyses. In this context, our study introduces a novel pipeline of machine learning models within a comprehensive workflow management system, the Production and Distributed Analysis (PanDA) system. These models employ advanced machine learning techniques to predict key resource requirements, overcoming challenges posed by limited upfront knowledge of characteristics at each step. Accurate forecasts of resource requirements enable informed and proactive decision-making in workflow management, enhancing the efficiency of handling diverse, complex workflows across heterogeneous resources.

97 MATHEMATICS AND COMPUTING↗

ExaFEL: extreme-scale real-time data processing for X-ray free electron laser science

ExaFEL is an HPC-capable X-ray Free Electron Laser (XFEL) data analysis software suite for both Serial Femtosecond Crystallography (SFX) and Single Particle Imaging (SPI) developed in collaboration with the Linac Coherent Lightsource (LCLS), Lawrence Berkeley National Laboratory (LBNL) and Los Alamos National Laboratory. ExaFEL supports real-time data analysis via a cross-facility workflow spanning LCLS and HPC centers such as NERSC and OLCF. Our work therefore constitutes initial path-finding for the US Department of Energy's (DOE) Integrated Research Infrastructure (IRI) program. We present the ExaFEL team's 7 years of experience in developing real-time XFEL data analysis software for the DOE's exascale supercomputers. We present our experiences and lessons learned with the Perlmutter and Frontier supercomputers. Furthermore we outline essential data center services (and the implications for institutional policy) required for real-time data analysis. Finally we summarize our software and performance engineering approaches and our experiences with NERSC's Perlmutter and OLCF's Frontier systems. This work is intended to be a practical blueprint for similar efforts in integrating exascale compute resources into other cross-facility workflows.

59 BASIC BIOLOGICAL SCIENCES↗