Search NASA⌕ Search

SEARCH · Search NASA

Results for “data intensive computing”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 541 records · Page 30
All sourcesNASA NTRSDOE OSTI
DOE OSTI2015

Power and Performance Trade-offs for Space Time Adaptive Processing

Computational efficiency – performance relative to power or energy – is one of the most important concerns when designing RADAR processing systems. This paper analyzes power and performance trade-offs for a typical Space Time Adaptive Processing (STAP) application. We study STAP implementations for CUDA and OpenMP on two computationally efficient architectures, Intel Haswell Core I7-4770TE and NVIDIA Kayla with a GK208 GPU. We analyze the power and performance of STAP’s computationally intensive kernels across the two hardware testbeds. We also show the impact and trade-offs of GPU optimization techniques. We show that data parallelism can be exploited for efficient implementation on the Haswell CPU architecture. The GPU architecture is able to process large size data sets without increase in power requirement. The use of shared memory has a significant impact on the power requirement for the GPU. A balance between the use of shared memory and main memory access leads to an improved performance in a typical STAP application.

Gawande, Nitin A.↗
NASA NTRS1990

Image recovery from edge primitives

A method for extracting edge primitives from Mach-band patterns is presented together with a method for recovering image representations of features outlined by the edge boundaries. The accuracy, stability, and resolution of these representations are assessed. Since these representations are most commonly used in characterizing targets, this method of low-level processing offers new opportunities for computer vision and high data-compressing coding. Two bandpass filters are considered, the spatially invariant Laplacian of Gaussian filter and spatially variant intensity-dependent spatial (IDS) summation. It is shown that the recovery from the IDS bandpass data is particularly advantageous in applications for which robustness to local and temporal variations in illumination is important. It is concluded that the edge primitives extracted from bandpassed images can be an efficient way to store, transmit, and represent images.

Alter-Gartenberg, Rachel↗
DOE OSTI2025

Real-Time FPGA Implementation For Frequency Sweep Interferometry In The LBNF Complex

The Long Baseline Neutrino Facility (LBNF) will deliver a 2.4 MW muon neutrino beam from Fermilab to the Deep Underground Neutrino Experiment (DUNE), requiring unprecedented precision in beamline alignment to achieve DUNE's neutrino oscillation measurement goals. Vertical misalignments of beamline components as small as 0.5 mm can contribute 6-7\% uncertainty in predicted neutrino flux, necessitating sub-0.1 mm alignment monitoring capabilities. The Horn Location Sensor (HLS) system employs frequency sweep interferometry (FSI) in a distributed hydrostatic leveling network to achieve the required precision under harsh radiation conditions up to 5000 kRad/year. Traditional FSI implementations suffer from laser sweep nonlinearities that degrade resolution and require computationally intensive post-processing corrections using gas reference cells. This work presents a real-time FPGA-based implementation of the HLS data acquisition and processing system using a sweep tracker interferometer for dynamic sweep linearization. The system utilizes a PYNQ-Z2 FPGA with programmable logic implementing parallel 16k-point FFT processing across four channels, synchronized by the sweep tracker signal to eliminate post-processing requirements. Spectral performance testing demonstrates significant improvements in peak sharpness compared to traditional fixed-frequency digitization. The FPGA implementation enables real-time displacement monitoring with processing speeds orders of magnitude faster than software-based approaches, essential for the operational requirements of LBNF's eventual distributed sensor network. This advancement in real-time FSI processing directly supports DUNE's precision neutrino physics program by providing the rapid feedback necessary for maintaining stringent beamline alignment tolerances during high-power beam operations.

Rossel, A. Jacob [Fermilab; Unlisted]↗
NASA NTRS2010

Finite Element Model Calibration Approach for Area I-X

Ares I-X is a pathfinder vehicle concept under development by NASA to demonstrate a new class of launch vehicles. Although this vehicle is essentially a shell of what the Ares I vehicle will be, efforts are underway to model and calibrate the analytical models before its maiden flight. Work reported in this document will summarize the model calibration approach used including uncertainty quantification of vehicle responses and the use of non-conventional boundary conditions during component testing. Since finite element modeling is the primary modeling tool, the calibration process uses these models, often developed by different groups, to assess model deficiencies and to update parameters to reconcile test with predictions. Data for two major component tests and the flight vehicle are presented along with the calibration results. For calibration, sensitivity analysis is conducted using Analysis of Variance (ANOVA). To reduce the computational burden associated with ANOVA calculations, response surface models are used in lieu of computationally intensive finite element solutions. From the sensitivity studies, parameter importance is assessed as a function of frequency. In addition, the work presents an approach to evaluate the probability that a parameter set exists to reconcile test with analysis. Comparisons of pretest predictions of frequency response uncertainty bounds with measured data, results from the variance-based sensitivity analysis, and results from component test models with calibrated boundary stiffness models are all presented.

Horta, Lucas G.↗
DOE OSTI2014

A Compute Capable SSD Architecture for Next-Generation Non-volatile Memories

Existing storage technologies (e.g., disks and ash) are failing to cope with the processor and main memory speed and are limiting the overall perfor- mance of many large scale I/O or data-intensive applications. Emerging fast byte-addressable non-volatile memory (NVM) technologies, such as phase-change memory (PCM), spin-transfer torque memory (STTM) and memristor are very promising and are approaching DRAM-like performance with lower power con- sumption and higher density as process technology scales. These new memories are narrowing down the performance gap between the storage and the main mem- ory and are putting forward challenging problems on existing SSD architecture, I/O interface (e.g, SATA, PCIe) and software. This dissertation addresses those challenges and presents a novel SSD architecture called XSSD. XSSD o oads com- putation in storage to exploit fast NVMs and reduce the redundant data tra c across the I/O bus. XSSD o ers a exible RPC-based programming framework that developers can use for application development on SSD without dealing with the complication of the underlying architecture and communication management. We have built a prototype of XSSD on the BEE3 FPGA prototyping system. We implement various data-intensive applications and achieve speedup and energy ef- ciency of 1.5-8.9 and 1.7-10.27 respectively. This dissertation also compares XSSD with previous work on intelligent storage and intelligent memory. The existing ecosystem and these new enabling technologies make this system more viable than earlier ones.

97 MATHEMATICS AND COMPUTING↗
NASA NTRSDate not supplied

NASA’s Moon Trek Portal: New Capabilities Supporting Mission Planning and Engagement

Introduction: NASA’s Moon Trek (https://trek.nasa.gov/moon/) is one of a growing number of interactive, browser-based, online portals for planetary data visualization and analysis produced by NASA’s Solar System Treks Project (SSTP). Moon Trek continues to be enhanced with new data and new capabilities enabling it to facilitate the planning and conducting of upcoming lunar missions by NASA, its commercial partners, and its international partners, as well as advancing its role as a valuable outreach tool. A Comprehensive Online Web Portal: Developed at NASA’s Jet Propulsion Laboratory (JPL) and managed as a project of NASA’s Solar System Exploration Research Virtual Institute (SSERVI) at NASA Ames Research Center, Moon Trek is a browser-based web portal. The portal provides easy-to-use tools for browsing, data layering, data product blending, and feature search among thousands of data products covering topography, mineralogy, elemental abundance, geology, and much more. Visualizations are provided in var-ious map projections, interactive 3D viewing, and in virtual reality. Using an in-house stereo workflow, SSTP is able to produce new NAC-based high-resolution mosaics and DEMs. Diverse Applications for Lunar Exploration: Baseline analytic tools available to all users include dis-tance measurement, elevation profiling, sun angle calculation, and 3D print file generation. More advanced account-level tools allow users to perform more computationally intensive analyses. These include ray-traced lighting analysis for user-specified areas over user-specified time/date ranges and time intervals, electro-static surface potential analysis, subsetting of large data products, slope analysis, and Lunar Laser Ranging geometry calculation. Artificial intelligence (AI) and ma-chine learning (ML) based tools have been implemented for crater detection and hazard analysis, boulder detection and hazard analysis, and rockfall detection. New Tools Facilitating Exploration: Additional, new tools have recently been added and others are in development, offering even greater functionality in con-ducting analyses of potential landing sites and areas of surface operations. The new Line-of-Sight tool facilitates communications planning between locations on the lunar surface, between any given site on the lunar surface and a specified ground station on the Earth, and between a site on the lunar surface and a relay asset in lunar orbit, all taking into account local lunar topography. The new Data Plotter tool provides both tabular and graphical representations of pixel values along a user specified path for a growing number of data products. The new NAC Finder tool will identify and pro-vide access to NAC images that intersect a user-defined path or bounded area. The SSTP development team is looking to leverage the capabilities of its existing AI and ML crater, boulder, and rockfall detection and analysis tools, and extend that technology to a generalized feature detector that can be trained on instances of specific types of landforms and then search the lunar surface for more examples of such features. New traverse planning tools are being developed with use cases in generalized concept studies and specific mission planning in mind. These will facilitate finding optimal traverse paths based on constraints such as slope, lighting, hazard avoidance, and communications. These will be complemented by new traverse visualization capabilities. Users will be able to interactively ride along with a rover, examining 3D views of the terrain while adjusting camera height and viewing angle along with selecting different data layer overlays to drape across the terrain. Engaging the Public: The capabilities being developed for mission planning are being leveraged to further enhance Moon Trek’s proven utility as a valuable public outreach resource. This includes providing multiple lev-els of engagement with different points of entry. At its simplest level, promoting understanding through visualization, media and the public will be able to easily visualize and conduct their own exploration of lunar sites targeted by NASA and its partners. For a more in-depth experience, we are working with our stakeholders to promote understanding through interaction by extend-ing our current landing site and traverse analysis capabilities, making simplified access to these tools available to those who want to explore more deeply key factors in planning a mission through interactive and possibly even gamified experiences. The highest degree of public outreach, focusing on engagement through scientific participation, could be achieved through our work with missions and the NASA Office of the Chief Scientist on Moon Trek’s extension as a tool with specialized capabilities for facilitating citizen science. In such scenarios, participants become members of extended mis-sion science teams, using dedicated and integrated interfaces to analyze mission data to help answer questions key to lunar science and exploration. We are work-ing with NASA’s Office of Communications, museums, planetariums, and the media to help them easily integrate accurate, detailed visualizations of NASA’s lunar destinations and exploration into their content/productions and to engage diverse audiences in diverse venues.

Moon Trek↗
NASA NTRS2010

Finite Element Model Calibration Approach for Ares I-X

Ares I-X is a pathfinder vehicle concept under development by NASA to demonstrate a new class of launch vehicles. Although this vehicle is essentially a shell of what the Ares I vehicle will be, efforts are underway to model and calibrate the analytical models before its maiden flight. Work reported in this document will summarize the model calibration approach used including uncertainty quantification of vehicle responses and the use of nonconventional boundary conditions during component testing. Since finite element modeling is the primary modeling tool, the calibration process uses these models, often developed by different groups, to assess model deficiencies and to update parameters to reconcile test with predictions. Data for two major component tests and the flight vehicle are presented along with the calibration results. For calibration, sensitivity analysis is conducted using Analysis of Variance (ANOVA). To reduce the computational burden associated with ANOVA calculations, response surface models are used in lieu of computationally intensive finite element solutions. From the sensitivity studies, parameter importance is assessed as a function of frequency. In addition, the work presents an approach to evaluate the probability that a parameter set exists to reconcile test with analysis. Comparisons of pre-test predictions of frequency response uncertainty bounds with measured data, results from the variance-based sensitivity analysis, and results from component test models with calibrated boundary stiffness models are all presented.

Horta, Lucas G.↗
DOE OSTI2024

Airborne LiDAR to Improve Canopy Fuels Mapping for Wildfire Modeling

Increasing conflict between wildfire and the built environment has increased the need for more up-to-date and finer resolution canopy fuels data to improve wildfire modeling and associated risk forecasts. The US Forest Service and US Department of the Interior’s LANDFIRE product, which provides 30-m resolution canopy fuels data for the entire US, is one of the most widely used sources of fuels data. However, the last complete mapping effort for LANDFIRE is based on 2016 conditions, and subsequent updates reflect disturbances 1-2 years behind the release year. Airborne systems equipped with Light Detection and Ranging (LiDAR) sensors can be deployed to actively sense canopy structure and estimate canopy fuels data (cover, height, base height, bulk density) at finer resolutions. Canopy base height (CBH) and canopy bulk density (CBD) are difficult to measure both in the field and in LiDAR point clouds. Still, they are important for accurately modeling crown fires, which are often intense and difficult to contain. Additionally, point cloud datasets are large, and calculations require efficient utilization of computational resources. To address these challenges, we are working on an approach that uses openly available National Ecological Observatory Network (NEON) airborne LiDAR data, with calculations processed in the R programming language and parallelized through the lidR package. CBH and CBD are often derived from tree height, diameter at breast height, and species-specific allometries using the Fire and Fuels Extension of the Forest Vegetation Simulator (FFE-FVS). We aim to test if airborne LiDAR can estimate CBH and CBD without the use of empirical equations. Reliable estimates of canopy fuels data directly from airborne LiDAR could streamline quick, fine-resolution updates for use in wildfire behavior models.

54 ENVIRONMENTAL SCIENCES↗
NASA NTRS2021

What it Takes to Compute Highly Accurate Rovibrational Line Lists for Use in Astrochemistry

We review the Best Theory + Reliable High-resolution Experiment (BTRHE) strategy for obtaining highly accurate molecular rovibrational line lists with InfraRed (IR) intensities. The need for highly accurate molecular rovibrational line lists is two-fold: a) assignment of the many rovibrational lines for common stable molecules especially those that exhibit a large amplitude motion, such as NH3, or have a high density of states such as SO2; and b) characterization of the atmospheres of exoplanets which will be one of the main areas of research in astronomy in the coming decades. The first motivation arises due to the need to eliminate lines due to common molecules in an astronomical observation in order to identify lines from new molecules, while the second motivation arises due to the need to obtain accurate molecular opacities in order to characterize the atmosphere of an exoplanet. The BTRHE strategy first consists of using high quality ab initio quantum chemical methods to obtain a global potential energy surface (PES) and dipole moment surface (DMS) that contains the proper physics. The global PES is then refined using a subset of the reliable high-resolution experimental data. The refined PES then gives energy level predictions to an accuracy similar to the reproduction accuracy of the experimental data used in the refinement step in the interpolation region (i.e., within the range of the experimental data used in the refinement step). The accuracy of the energy levels will slowly degrade as they are extrapolated to spectral regions beyond the high-resolution experimental data used in the refinement step. However, because the degradation is slow, the predicted energy levels can be used to assign new high-resolution experiments, and the data from these can then be used in a subsequent refinement step. In this way, the global PES eventually can yield highly accurate energy levels for all desired spectral regions including to very high energies and high J values. We show that IR intensities computed with the BTRHE rovibrational wavefunctions and the DMS can be very accurate provided one has minimized the fitting error of the DMS and tested the completeness of the DMS. Some examples of our work on NH3, CO2, and SO2 are given to highlight the usefulness of the BTRHE strategy and to provide ideas on how to further improve its predictive power in the future. In particular, it is shown how successive refinement steps, once new high-resolution data is available, can lead to PESs that yield highly accurate transition energies to larger spectral regions. The importance of including non-adiabatic corrections to reduce the J-dependence of errors for H-containing molecules is shown with work on NH3. Another very important aspect of the BTRHE approach is the consistency across isotopologues, which allows for highly accurate line lists for any isotopologue once one is obtained for the main isotopologue (which has more high-resolution data available for refinement).

Xinchuan Huang↗
DOE OSTI2020

Turbulence-parameter estimation for current-energy converters using surrogate model optimization

Surrogate models maximize information utility by building predictive models in place of computational or experimentally expensive model runs. Marine hydrokinetic current energy converters require large-domain simulations to estimate array efficiencies and environmental impacts. Meso-scale models typically represent turbines as actuator discs that act as momentum sinks and sources of turbulence and its dissipation. An OpenFOAM model was developed where actuator disc k-ε turbulence was characterized using an approach developed for flows through vegetative canopies. Turbine-wake data from laboratory flume experiments collected at two influent turbulence intensities were used to calibrate parameters in the turbulence-source terms in the k-ε equations. Additionally, parameter influences on longitudinal wake profiles were estimated using Gaussian process regression with subsequent optimization minimizing the objective function within 3.1% of those obtained using the full model representation, but for 74% of the computational cost (far fewer model runs). This framework facilitates more efficient parameterization of the turbulence-source equations using turbine-wake data.

16 TIDAL AND WAVE POWER↗
NASA NTRS1986

Application of laser-velocimetry to the study of the flow around a two-dimensional airfoil

The experimental techniques employed by NASA researchers to obtain a high volume throughput of data in a study of high-lift multielement airfoils using a laser velocimeter (LV) are detailed. The two-dimensional LV system recorded mean velocity, turbulence intensities, Re shear stresses, skewness and kurtosis of the flow. A computer-controlled positioning mount for the LV allowed automated surveys of the flows with a 0.001 in. positioning accuracy. The flow was seeded with oil droplets and additional refinement of the wind tunnel controls permitted rapid speedup at selected areas of the flowfield over specific regions of the airfoil. Surface pressure taps furnished data for selecting regions for more detailed investigation. The data processing capabilities processed 30-40 samples every 10 min in an automated mode. Profiles of the data were computed automatically to furnish graphical displays on demand. Numerous samples of the data displays generated are provided.

Braden, J. A.↗
DOE OSTI2021

Flexible‐rate optoelectronic wireless transmission system at 101 GHz using multiband CAP‐PAM modulation and envelope detection

Abstract In this paper, we have experimentally demonstrated the first flexible‐rate optoelectronic terahertz wireless transmission system at 101 GHz based on multiband CAP‐PAM modulation and cost‐efficient envelope detection. Instead of traditional M‐QAM mapping, PAM‐N mapping is used in CAP modulation with low‐computational complexity. The system performances and optimal capacity in both single‐band case and multiband case are intensively investigated. The results show that the highest data rate for single‐band system is 13.35 Gbit/s with CAP‐PAM‐5 modulation, while the optimum aggregate transmission line rate of 8.49 Gbit/s can be obtained with 4‐sub‐bands CAP‐PAM‐{5,3,2,2} modulation under 7% FEC limit of 3.8 × 10 −3 for four‐user access.

Zhang, Jiao↗
DOE OSTI2020

Machine learning and ligand binding predictions: A review of data, methods, and obstacles

We report that computational predictions of ligand binding is a difficult problem, with more accurate methods being extremely computationally expensive. The use of machine learning for drug binding predictions could possibly leverage the use of biomedical big data in exchange for time-intensive simulations. This paper reviews current trends in the use of machine learning for drug binding predictions, data sources to develop machine learning algorithms, and potential problems that may lead to overfitting and ungeneralizable models. A few popular datasets that can be used to develop virtual high-throughput screening models are characterized using spatial statistics to quantify potential biases. We can see from evaluating some common benchmarks that good performance correlates with models with high-predicted bias scores and models with low bias scores do not have much predictive power. A better understanding of the limits of available data sources and how to fix them will lead to more generalizable models that will lead to novel drug discovery.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2022

Decoding the protein–ligand interactions using parallel graph neural networks

Abstract Protein–ligand interactions (PLIs) are essential for biochemical functionality and their identification is crucial for estimating biophysical properties for rational therapeutic design. Currently, experimental characterization of these properties is the most accurate method, however, this is very time-consuming and labor-intensive. A number of computational methods have been developed in this context but most of the existing PLI prediction heavily depends on 2D protein sequence data. Here, we present a novel parallel graph neural network (GNN) to integrate knowledge representation and reasoning for PLI prediction to perform deep learning guided by expert knowledge and informed by 3D structural data. We develop two distinct GNN architectures: $$\hbox {GNN}_{\mathrm{F}}$$ GNN F is the base implementation that employs distinct featurization to enhance domain-awareness, while $$\hbox {GNN}_{\mathrm{P}}$$ GNN P is a novel implementation that can predict with no prior knowledge of the intermolecular interactions. The comprehensive evaluation demonstrated that GNN can successfully capture the binary interactions between ligand and protein’s 3D structure with 0.979 test accuracy for $$\hbox {GNN}_{\mathrm{F}}$$ GNN F and 0.958 for $$\hbox {GNN}_{\mathrm{P}}$$ GNN P for predicting activity of a protein–ligand complex. These models are further adapted for regression tasks to predict experimental binding affinities and $$\hbox {pIC}_{\mathrm{50}}$$ pIC 50 crucial for compound’s potency and efficacy. We achieve a Pearson correlation coefficient of 0.66 and 0.65 on experimental affinity and 0.50 and 0.51 on $$\hbox {pIC}_{\mathrm{50}}$$ pIC 50 with $$\hbox {GNN}_{\mathrm{F}}$$ GNN F and $$\hbox {GNN}_{\mathrm{P}}$$ GNN P , respectively, outperforming similar 2D sequence based models. Our method can serve as an interpretable and explainable artificial intelligence (AI) tool for predicted activity, potency, and biophysical properties of lead candidates. To this end, we show the utility of $$\hbox {GNN}_{\mathrm{P}}$$ GNN P on SARS-Cov-2 protein targets by screening a large compound library and comparing the prediction with the experimentally measured data.

59 BASIC BIOLOGICAL SCIENCES↗
NASA NTRS2013

Downscaling Satellite Precipitation with Emphasis on Extremes: A Variational 1-Norm Regularization in the Derivative Domain

The increasing availability of precipitation observations from space, e.g., from the Tropical Rainfall Measuring Mission (TRMM) and the forthcoming Global Precipitation Measuring (GPM) Mission, has fueled renewed interest in developing frameworks for downscaling and multi-sensor data fusion that can handle large data sets in computationally efficient ways while optimally reproducing desired properties of the underlying rainfall fields. Of special interest is the reproduction of extreme precipitation intensities and gradients, as these are directly relevant to hazard prediction. In this paper, we present a new formalism for downscaling satellite precipitation observations, which explicitly allows for the preservation of some key geometrical and statistical properties of spatial precipitation. These include sharp intensity gradients (due to high-intensity regions embedded within lower-intensity areas), coherent spatial structures (due to regions of slowly varying rainfall),and thicker-than-Gaussian tails of precipitation gradients and intensities. Specifically, we pose the downscaling problem as a discrete inverse problem and solve it via a regularized variational approach (variational downscaling) where the regularization term is selected to impose the desired smoothness in the solution while allowing for some steep gradients(called 1-norm or total variation regularization). We demonstrate the duality between this geometrically inspired solution and its Bayesian statistical interpretation, which is equivalent to assuming a Laplace prior distribution for the precipitation intensities in the derivative (wavelet) space. When the observation operator is not known, we discuss the effect of its misspecification and explore a previously proposed dictionary-based sparse inverse downscaling methodology to indirectly learn the observation operator from a database of coincidental high- and low-resolution observations. The proposed method and ideas are illustrated in case studies featuring the downscaling of a hurricane precipitation field.

Hurricanes↗
DOE OSTI2019

Highly-scalable, physics-informed GANs for learning solutions of stochastic PDEs

Uncertainty quantification for forward and inverse problems is a central challenge across physical and biomedical disciplines. We address this challenge for the problem of modeling subsurface flow at the Hanford Site by combining stochastic computational models with observational data using physics-informed GAN models. The geographic extent, spatial heterogeneity, and multiple correlation length scales of the Hanford Site require training a computationally intensive GAN model to thousands of dimensions. We develop a highly optimized implementation that scales to {\herogpucount} NVIDIA Volta GPUs. We develop a hierarchical scheme based on a multi-player game-theoretic approach for exploiting domain parallelism, map discriminators and generators to multiple GPUs, and employ efficient communication schemes to ensure training stability and convergence. Our implementation scales to {\heronodecount} nodes on the Summit supercomputer with a \weakscalingeff\% scaling efficiency, achieving peak and sustained half-precision rates of {\heropeakperf} PF/s and {\herosustainedperf} PF/s.

physics informed ML, machine learning, generative ↗
NASA NTRS1986

A computer graphics pilot project - Spacecraft mission support with an interactive graphics workstation

The NASA Computer Graphics Pilot Project was undertaken to enhance the quality control, productivity and efficiency of mission support operations at the Goddard Operations Support Computing Facility. The Project evolved into a set of demonstration programs for graphics intensive simulated control room operations, particularly in connection with the complex space missions that began in the 1980s. Complex mission mean more data. Graphic displays are a means to reduce the probabilities of operator errors. Workstations were selected with 1024 x 768 pixel color displays controlled by a custom VLSI chip coupled to an MC68010 chip running UNIX within a shell that permits operations through the medium of mouse-accessed pulldown window menus. The distributed workstations run off a host NAS 8040 computer. Applications of the system for tracking spacecraft orbits and monitoring Shuttle payload handling illustrate the system capabilities, noting the built-in capabilities of shifting the point of view and rotating and zooming in on three-dimensional views of spacecraft.

Hagedorn, John↗
DOE OSTI2018

Blazing Signature Filter: a library for fast pairwise similarity comparisons

Identifying similarities between datasets is a fundamental task in data mining and has become an integral part of modern scientific investigation. Whether the task is to identify co-expressed genes in large-scale expression surveys or to predict combinations of gene knockouts which would elicit a similar phenotype, the underlying computational task is often a multi-dimensional similarity test. As datasets continue to grow, improvements to the efficiency, sensitivity or specificity of such computation will have broad impacts as it allows scientists to more completely explore the wealth of scientific data. A significant practical drawback of large-scale data mining is the vast majority of pairwise comparisons are unlikely to be relevant, meaning that they do not share a signature of interest. It is therefore essential to efficiently identify these unproductive comparisons as rapidly as possible and exclude them from more time-intensive similarity calculations. The Blazing Signature Filter (BSF) is a highly efficient pairwise similarity algorithm which enables extensive data mining within a reasonable amount of time. The algorithm transforms datasets into binary metrics, allowing it to utilize the computationally efficient bit operators and provide a coarse measure of similarity. As a result, the BSF can scale to high dimensionality and rapidly filter unproductive pairwise comparison. Furthermore, two bioinformatics applications of the tool are presented to demonstrate the ability to scale to billions of pairwise comparisons and the usefulness of this approach.

59 BASIC BIOLOGICAL SCIENCES↗