Search NASA⌕ Search

SEARCH · Search NASA

Results for “High-Throughput Algorithms”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

247 records · Page 14

The Evolution of the Pegasus Workflow Management Software

Since 2001 the Pegasus Workflow Management System has evolved into a robust and scalable system that automates the execution of a number of complex applications running on a variety of heterogeneous, distributed high-throughput, and high-performance computing environments. Pegasus was built on the principle of separation between the workflow description and workflow execution, providing the ability to port and adapt the workflow based on the target execution environment. Through its user-driven research and development it has adapted to the needs of a number of scientific communities, utilizing and developing novel algorithms and software engineering solutions. This paper describes the evolution of Pegasus over time and provides motivations behind the design decisions. The paper concludes with selected lessons learned.

97 MATHEMATICS AND COMPUTING↗

NACHOS, a CubeSat-based high-resolution UV-Visible hyperspectral imager for remote sensing of trace gases: system overview and science objectives

Here, the Nano-satellite Atmospheric Chemistry Hyperspectral Observation System (NACHOS) is a high-throughput (f/2.9), high spectral resolution (~1.3 nm optical resolution, 0.6 nm sampling) Offner-design hyperspectral imager operating in the 300-500 nm spectral region. The 1.5U instrument payload (1U optical system, 0.5U electronics module) is hosted by a 1.5U LANL-designed CubeSat bus to comprise a 3U complete satellite. Spectroscopically similar to NASA’s Ozone Monitoring Instrument (OMI), which provides wide-field global mapping of ozone and other gases at coarse spatial resolution, NACHOS fills the complementary niche of targeted measurements at much higher spatial resolution. With 350 across-track spatial pixels and a 15-degree across-track field of view, NACHOS will provide spectral imaging at roughly 0.4 km per pixel from 500 km altitude. NACHOS incorporates highly streamlined gas-retrieval algorithms for rapid onboard processing, alleviating the need to routinely downlink massive hyperspectral data cubes. We will discuss the instrument design, challenges in achieving mechanical robustness to launch vibration in such a compact instrument, the onboard calibration system, and gas-retrieval data downlink strategy. We will also discuss potential science missions, including monitoring of NO 2 as an easily detected proxy for anthropogenic fossil-fuel greenhouse gases, monitoring lowlevel SO 2 degassing at pre-eruptive volcanoes, H 2 CO from wildfires, and characterization of aerosols. The long-term vision is for a many-satellite constellation that could provide both high spatial resolution and frequent revisits for selected targets of interest. As an initial technology demonstration of this vision, the NACHOS project is currently slated to launch two CubeSats in early 2022.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

Fast 2D Bicephalous Convolutional Autoencoder for Compressing 3D Time Projection Chamber Data

High-energy large-scale particle colliders produce data at high speed in the order of 1 terabytes per second in nuclear physics and petabytes per second in high energy physics. Developing real-time data compression algorithms to reduce such data at high throughput to fit permanent storage has drawn increasing attention. Specifically, at the newly constructed sPHENIX experiment at the Relativistic Heavy Ion Collider (RHIC), a time projection chamber is used as the main tracking detector, which records particle trajectories in a volume of three-dimensional (3D) cylinder. The resulting data are usually very sparse with occupancy around 10.8%. Such sparsity presents a challenge to conventional learning-free lossy compression algorithms, such as SZ, ZFP, and MGARD. The 3D convolutional neural network (CNN)-based approach, Bicephalous Convolutional Autoencoder (BCAE), outperforms traditional methods both in compression rate and reconstruction accuracy. BCAE can also utilize the computation power of graphical processing units suitable for deployment in a modern heterogeneous highperformance computing environment. This work introduces two BCAE variants: BCAE++ and BCAE-2D. BCAE++ achieves a 15% better compression ratio and a 77% better reconstruction accuracy measured in mean absolute error compared with BCAE. BCAE-2D treats the radial direction as the channel dimension of an image, resulting in a 3× speedup in compression throughput. In addition, we demonstrate an unbalanced autoencoder with a larger decoder can improve reconstruction accuracy without significantly sacrificing throughput. Lastly, we observe both the BCAE++ and BCAE-2D can benefit more from using half-precision mode in throughput (76 - 79% increase) without loss in reconstruction accuracy. The source code and links to data and pretrained models can be found at https://github.com/BNL-DAQ-LDRD/NeuralCompression_v2

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

Xi-CAM v1.2.3

Xi-CAM aims to provide a community driven platform for multimodal analysis in synchrotron science. The platform core provides a robust plugin infrastructure for extensibility, allowing continuing development to simply add further functionality. Current modules include tools for characterization with (GI)SAXS, Tomography, and XAS. This will continue to serve as a development base as algorithms for multimodal analysis develop. Seamless remote data access, visualization and analysis are key elements of Xi-CAM, and will become critical to synchrotron data infrastructure as expectations for future data volume and acquisition rates rise with continuously increasing throughputs. The highly interactive design elements of Xi-cam will similarly support a generation of users which depend on immediate data quality feedback during high-throughput or burst acquisition modes.

Pandolfi, Ronald↗

A high-throughput pipeline for designing microarray-based pathogen diagnostic assays

fingerprints for microarray-based pathogen diagnostic assays. The oligonucleotide fingerprints, or DNA microarray probes, are designed for identifying target organisms in environmental or clinical samples. The design process is implemented in a high-performance computing software pipeline that incorporates major algorithmic improvements over a previous version to both reduce computation time and improve specificity assessment. Results: The algorithmic improvements result in significant reduction in runtimes, with the updated pipeline being nearly up to five-times faster than the previous version. The improvements in specificity assessment, based on multiple specificity criteria, result in robust and consistent evaluation of cross-hybridization with nontarget sequences. In addition, the multiple criteria provide finer control on the number of resulting fingerprints, which helps in obtaining a larger number of fingerprints with high specificity. Simulation tests for Francisella tularensis and Yersinia pestis, using a well-established hybridization model to estimate cross-hybridization with nontarget sequences, show that the improved specificity criteria yield a larger number of fingerprints as compared to using a single specificity criterion. Conclusion: The faster runtimes, achieved as the result of algorithmic improvements, are critical for extending the pipeline to process multiple target genomes. The larger numbers of identified fingerprints, obtained by considering broader specificity criteria, are essential for designing probes for hard-to-distinguish target sequences.

59 BASIC BIOLOGICAL SCIENCES↗

Optimal dimensionality selection for independent component analysis of transcriptomic data

Independent component analysis is an unsupervised machine learning algorithm that separates a set of mixed signals into a set of statistically independent source signals. Applied to high-quality gene expression datasets, independent component analysis effectively reveals both the source signals of the transcriptome as co-regulated gene sets, and the activity levels of the underlying regulators across diverse experimental conditions. Two major variables that affect the final gene sets are the diversity of the expression profiles contained in the underlying data, and the user-defined number of independent components, or dimensionality, to compute. Availability of high-quality transcriptomic datasets has grown exponentially as high-throughput technologies have advanced; however, optimal dimensionality selection remains an open question. We computed independent components across a range of dimensionalities for four gene expression datasets with varying dimensions (both in terms of number of genes and number of samples). We computed the correlation between independent components across different dimensionalities to understand how the overall structure evolves as the number of user-defined components increases. We then measured how well the resulting gene clusters reflected known regulatory mechanisms, and developed a set of metrics to assess the accuracy of the decomposition at a given dimension. We found that over-decomposition results in many independent components dominated by a single gene, whereas under-decomposition results in independent components that poorly capture the known regulatory structure. From these results, we developed a new method, called OptICA, for finding the optimal dimensionality that controls for both over- and under-decomposition. Specifically, OptICA selects the highest dimension that produces a low number of components that are dominated by a single gene. We show that OptICA outperforms two previously proposed methods for selecting the number of independent components across four transcriptomic databases of varying sizes. OptICA avoids both over-decomposition and under-decomposition of transcriptomic datasets resulting in the best representation of the organism’s underlying transcriptional regulatory network.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Structural variant analysis of a cancer reference cell line sample using multiple sequencing technologies

The cancer genome is commonly altered with thousands of structural rearrangements including insertions, deletions, translocation, inversions, duplications, and copy number variations. Thus, structural variant (SV) characterization plays a paramount role in cancer target identification, oncology diagnostics, and personalized medicine. As part of the SEQC2 Consortium effort, the present study established and evaluated a consensus SV call set using a breast cancer reference cell line and matched normal control derived from the same donor, which were used in our companion benchmarking studies as reference samples. We systematically investigated somatic SVs in the reference cancer cell line by comparing to a matched normal cell line using multiple NGS platforms including Illumina short-read, 10X Genomics linked reads, PacBio long reads, Oxford Nanopore long reads, and high-throughput chromosome conformation capture (Hi-C). We established a consensus SV call set of a total of 1788 SVs including 717 deletions, 230 duplications, 551 insertions, 133 inversions, 146 translocations, and 11 breakends for the reference cancer cell line. To independently evaluate and cross-validate the accuracy of our consensus SV call set, we used orthogonal methods including PCR-based validation, Affymetrix arrays, Bionano optical mapping, and identification of fusion genes detected from RNA-seq. We evaluated the strengths and weaknesses of each NGS technology for SV determination, and our findings provide an actionable guide to improve cancer genome SV detection sensitivity and accuracy. A high-confidence consensus SV call set was established for the reference cancer cell line. A large subset of the variants identified was validated by multiple orthogonal methods.

59 BASIC BIOLOGICAL SCIENCES↗

The genome-scale metabolic model for the purple non-sulfur bacterium Rhodopseudomonas palustris Bis A53 accurately predicts phenotypes under chemoheterotrophic, chemoautotrophic, photoheterotrophic, and photoautotrophic growth conditions

The purple non-sulfur bacterium Rhodopseudomonas palustris is recognized as a critical microorganism in the nitrogen and carbon cycle and one of the most common members in wastewater treatment communities. This bacterium is metabolically extremely versatile. It is capable of heterotrophic growth under aerobic and anaerobic conditions, but also able to grow photoautotrophically as well as mixotrophically. Therefore R . palustris can adapt to multiple environments and establish commensal relationships with other organisms, expressing various enzymes supporting degradation of amino acids, carbohydrates, nucleotides, and complex polymers. Moreover, R . palustris can degrade a wide range of pollutants under anaerobic conditions, e.g., aromatic compounds such as benzoate and caffeate, enabling it to thrive in chemically contaminated environments. However, many metabolic mechanisms employed by R . palustris to breakdown and assimilate different carbon and nitrogen sources under chemoheterotrophic or photoheterotrophic conditions remain unknown. Systems biology approaches, such as metabolic modeling, have been employed extensively to unravel complex mechanisms of metabolism. Previously, metabolic models have been reconstructed to study selected capabilities of R . palustris under limited experimental conditions. Here, we developed a comprehensive metabolic model (M-model) for R . palustris Bis A53 ( i DT1294) consisting of 2,721 reactions, 2,123 metabolites, and comprising 1,294 genes. We validated the model using high-throughput phenotypic, physiological, and kinetic data, testing over 350 growth conditions. i DT1294 achieved a prediction accuracy of 90% for growth with various carbon and nitrogen sources and close to 80% for assimilation of aromatic compounds. Moreover, the M-model accurately predicts dynamic changes of growth and substrate consumption rates over time under nine chemoheterotrophic conditions and demonstrated high precision in predicting metabolic changes between photoheterotrophic and photoautotrophic conditions. This comprehensive M-model will help to elucidate metabolic processes associated with the assimilation of multiple carbon and nitrogen sources, anoxygenic photosynthesis, aromatic compound degradation, as well as production of molecular hydrogen and polyhydroxybutyrate.

59 BASIC BIOLOGICAL SCIENCES↗

Fast Identification and Removal of Sequence Contamination from Genomic and Metagenomic Datasets

High-throughput sequencing technologies have strongly impacted microbiology, providing a rapid and cost-effective way of generating draft genomes and exploring microbial diversity. However, sequences obtained from impure nucleic acid preparations may contain DNA from sources other than the sample. Those sequence contaminations are a serious concern to the quality of the data used for downstream analysis, causing misassembly of sequence contigs and erroneous conclusions. Therefore, the removal of sequence contaminants is a necessary and required step for all sequencing projects. We developed DeconSeq, a robust framework for the rapid, automated identification and removal of sequence contamination in longer-read datasets (150 bp mean read length). DeconSeq is publicly available as standalone and web-based versions. The results can be exported for subsequent analysis, and the databases used for the web-based version are automatically updated on a regular basis. DeconSeq categorizes possible contamination sequences, eliminates redundant hits with higher similarity to non-contaminant genomes, and provides graphical visualizations of the alignment results and classifications. Using DeconSeq, we conducted an analysis of possible human DNA contamination in 202 previously published microbial and viral metagenomes and found possible contamination in 145 (72%) metagenomes with as high as 64% contaminating sequences. This new framework allows scientists to automatically detect and efficiently remove unwanted sequence contamination from their datasets while eliminating critical limitations of current methods. DeconSeq’s web interface is simple and user-friendly. The standalone version allows offline analysis and integration into existing data processing pipelines. DeconSeq’s results reveal whether the sequencing experiment has succeeded, whether the correct sample was sequenced, and whether the sample contains any sequence contamination from DNA preparation or host. In addition, the analysis of 202 metagenomes demonstrated significant contamination of the non-human associated metagenomes, suggesting that this method is appropriate for screening all metagenomes. DeconSeq is available at http://deconseq.sourceforge.net/.

59 BASIC BIOLOGICAL SCIENCES↗

Scalable Computational Methods for the Analysis of High-Throughput Biological Data

This primary focus of this research project is elucidating genetic regulatory mechanisms that control an organism's responses to low-dose ionizing radiation. Although low doses (at most ten centigrays) are not lethal to humans, they elicit a highly complex physiological response, with the ultimate outcome in terms of risk to human health unknown. The tools of molecular biology and computational science will be harnessed to study coordinated changes in gene expression that orchestrate the mechanisms a cell uses to manage the radiation stimulus. High performance implementations of novel algorithms that exploit the principles of fixed-parameter tractability will be used to extract gene sets suggestive of co-regulation. Genomic mining will be performed to scrutinize, winnow and highlight the most promising gene sets for more detailed investigation. The overall goal is to increase our understanding of the health risks associated with exposures to low levels of radiation.

59 BASIC BIOLOGICAL SCIENCES↗

Developing ML/AI Methods for High-Throughput Characterization of Multiple-Sensor Streams of Tokamak Dynamics for High-Speed Control (Final Report)

This project evaluated and developed new mathematical and algorithmic techniques capable of handling (in real-time) the growing amounts of data generated by modern fusion research. While existing numerical linear algebra (NLA) methods provide the backbone to classical data analysis and algorithms, these methods fundamentally do not port to distributed architectures nor do they allow low-latency data reduction for control. Motivated by the needs for modern fusion reactors, this project explored and implemented new numerical methods to characterize plasma dynamics, respond in real-time to discharge evolution, and to process massive-scale data accurately and rapidly more fully. This project links expertise in multiple-sensor diagnostics of tokamak plasma dynamics from Columbia University’s Plasma Physics Laboratory with expertise in massive-scale data reduction and extreme data control algorithms at Columbia University’s Data Science Institute. This interdisciplinary project (i) applied machine learning methods, (ii) implemented a properly-trained neural-network for very fast processing of high-speed plasma videography, and (ii) developed the applied mathematical methods, based on randomized-NLA (rNLA) routines, for data analysis, reduction, and real-time control. The Columbia University High Beta Tokamak-Extended Pulse (HBT-EP) facility provided data to test new algorithms and partnership with Columbia University's Data Sciences Institute evaluated the broader use of new algorithms for many challenging control applications.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

Determination of saccharides and ethanol from biomass conversion using Raman spectroscopy: Effects of pretreatment and enzyme composition

This dissertation focuses on the development of facile and rapid quantitative Raman spectroscopy measurements for the determination of conversion products in producing bioethanol from corn stover. Raman spectroscopy was chosen to determine glucose, xylose and ethanol in complex hydrolysis and fermentation matrices. Chapter 1 describes the motives and main goals of this work, and includes an introduction to biomass, commonly used pretreatment methods, hydrolysis and fermentation reactions. The principles of Raman spectroscopy, its advantages and applications related to biomass analysis are also illustrated. Chapter 2 and 3 comprise two published or submitted manuscripts, and the thesis concludes with an appendix. In Chapter 2, a Raman spectroscopic protocol is described to study the efficiency of enzymatic hydrolysis of cellulose by measuring the main product in hydrolysate, glucose. Two commonly utilized pretreatment methods were investigated in order to understand their effect on glucose measurements by Raman spectroscopy. Second, a similar method was set up to determine the concentration of ethanol in fermentation broth. Both of these measurements are challenged by the presence of complex matrices. In Chapter 3, a quantitative comparison of pretreatment protocols and the effect of enzyme composition are studied using systematic methods. A multipeak fitting algorithm was developed to analyze spectra of hydrolysate containing two analytes: glucose and xylose. Chapter 4 concludes with a future perspective of this research area. An appendix describes a convenient, rapid spectrophotometric method developed to measure cadmium in water. This method requires relatively low cost instrumentation and can be used in microgravity, such as space shuttles or the International Space Station. This work was performed under the supervision of Professor Marc Porter while at Iowa State University. Research related to producing biofuel from bio-renewable resources, especially bioethanol from biomass, has grown significantly in the past decade due to the high demand and rising costs of fossil fuels. More than 3 percent of the energy consumption in the U.S. is derived from renewable biomass, mostly through industrial heat and steam production by the pulp and paper industry, and electricity generation from municipal solid waste (MSW) and forest industry residues. The utilization of food-based biomass to make fuels has been widely criticized because it may increase food shortages throughout the world and raise the cost of food. Thus, nonfood-based and plentiful lignocellulosic feedstocks, such as corn stover, perennial grass, bagasse, sorghum, wheat/rice straw, herbaceous and woody crops, have great potential to be new bio-renewable sources for energy production. Given that many varieties of biomass are available, there is need for a rapid, simple, high-throughput method to screen the conversion of many plant varieties. The most suitable species for each geographic region must be determined, as well as the optimal stage of harvest, impacts of environmental conditions (temperature, soil, pH, etc.). Various genetically modified plants should be studied in order to establish the desired biomass in bioethanol production. The main screening challenge, however, is the complexity of plant cell wall structures that make reliable and sensitive analysis difficult. To date, one of the most popular methods to produce lignocellulosic ethanol is to perform enzymatic hydrolysis followed by fermentation of the hydrolysate with yeast. There are several vital needs related to the field of chemistry that have been suggested as primary research foci needed to effectively improve lignocellulosic ethanol production. These topics include overcoming the recalcitrance of cellulosic biomass, the pervasiveness of pretreatment, advanced biological processing and better feedstocks. In this thesis, a novel approach using Raman spectroscopy has been developed to address important issues related to bioethanol generation, which will aid the research aimed to solve the topics mentioned above.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Generic, Extensible, Configurable Push-Pull Framework for Large-Scale Science Missions

The push-pull framework was developed in hopes that an infrastructure would be created that could literally connect to any given remote site, and (given a set of restrictions) download files from that remote site based on those restrictions. The Cataloging and Archiving Service (CAS) has recently been re-architected and re-factored in its canonical services, including file management, workflow management, and resource management. Additionally, a generic CAS Crawling Framework was built based on motivation from Apache s open-source search engine project called Nutch. Nutch is an Apache effort to provide search engine services (akin to Google), including crawling, parsing, content analysis, and indexing. It has produced several stable software releases, and is currently used in production services at companies such as Yahoo, and at NASA's Planetary Data System. The CAS Crawling Framework supports many of the Nutch Crawler's generic services, including metadata extraction, crawling, and ingestion. However, one service that was not ported over from Nutch is a generic protocol layer service that allows the Nutch crawler to obtain content using protocol plug-ins that download content using implementations of remote protocols, such as HTTP, FTP, WinNT file system, HTTPS, etc. Such a generic protocol layer would greatly aid in the CAS Crawling Framework, as the layer would allow the framework to generically obtain content (i.e., data products) from remote sites using protocols such as FTP and others. Augmented with this capability, the Orbiting Carbon Observatory (OCO) and NPP (NPOESS Preparatory Project) Sounder PEATE (Product Evaluation and Analysis Tools Elements) would be provided with an infrastructure to support generic FTP-based pull access to remote data products, obviating the need for any specialized software outside of the context of their existing process control systems. This extensible configurable framework was created in Java, and allows the use of different underlying communication middleware (at present, both XMLRPC, and RMI). In addition, the framework is entirely suitable in a multi-mission environment and is supporting both NPP Sounder PEATE and the OCO Mission. Both systems involve tasks such as high-throughput job processing, terabyte-scale data management, and science computing facilities. NPP Sounder PEATE is already using the push-pull framework to accept hundreds of gigabytes of IASI (infrared atmospheric sounding interferometer) data, and is in preparation to accept CRIMS (Cross-track Infrared Microwave Sounding Suite) data. OCO will leverage the framework to download MODIS, CloudSat, and other ancillary data products for use in the high-performance Level 2 Science Algorithm. The National Cancer Institute is also evaluating the framework for use in sharing and disseminating cancer research data through its Early Detection Research Network (EDRN).

Foster, Brian M.↗