Search NASA⌕ Search

SEARCH · Search NASA

Results for “data mining system”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 361 records · Page 20
All sourcesNASA NTRSDOE OSTI
DOE OSTI2022

Unsupervised learning from three-component accelerometer data to monitor the spatiotemporal evolution of meso-scale hydraulic fractures

Enhanced geothermal systems can provide a substantial share of the global energy demand. There exist several hurdles in the engineering implementations of such geothermal systems. One such hurdle is the accurate monitoring of the fracture networks created in subsurface through hydraulic stimulation of these systems. Micro seismicity associated with the stimulation is the primary means to locate the event hypocenters for estimating the stimulated rock volume. Existing methods for location the hypocenters are restricted to only the highest amplitude impulsive signals that are simultaneously detected on several sensors. Consequently, a large portion (usually ~99%) of the measurements are left unused. In this paper, an unsupervised manifold-approximation followed by clustering of 3-component accelerometer data is used to analyze the seismicity recorded on a monitoring well. With this method, a larger portion of the measured signal is used for the monitoring of the hydraulic fracture network. We analyze the EGS Collab experiment 1 microseismic data, recorded at the Sanford Underground Research Facility, South Dakota. Using the data from a single three-component accelerometer, the polarization features viz. Azimuth, incidence, rectilinearity, and planarity are used as inputs for the unsupervised manifold approximation followed by clustering. Our study shows that density-based clusters in the projected 3D space correspond to distinct types of hydraulically fractured zones around the injection point. Finally, we show that the temporal evolution of these clusters can be used to track fracture creation and propagation.

58 GEOSCIENCES↗
DOE OSTI2013

The antibody mining toolbox: An open source tool for the rapid analysis of antibody repertoires

In vitro selection has been an essential tool in the development of recombinant antibodies against various antigen targets. Deep sequencing has recently been gaining ground as an alternative and valuable method to analyze such antibody selections. The analysis provides a novel and extremely detailed view of selected antibody populations, and allows the identification of specific antibodies using only sequencing data, potentially eliminating the need for expensive and laborious low-throughput screening methods such as enzyme-linked immunosorbant assay. The high cost and the need for bioinformatics experts and powerful computer clusters, however, have limited the general use of deep sequencing in antibody selections. Here, we describe the AbMining ToolBox, an open source software package for the straightforward analysis of antibody libraries sequenced by the three main next generation sequencing platforms (454, Ion Torrent, MiSeq). The ToolBox is able to identify heavy chain CDR3s as effectively as more computationally intense software, and can be easily adapted to analyze other portions of antibody variable genes, as well as the selection outputs of libraries based on different scaffolds. The software runs on all common operating systems (Microsoft Windows, Mac OS X, Linux), on standard personal computers, and sequence analysis of 1–2 million reads can be accomplished in 10–15 min, a fraction of the time of competing software. Use of the ToolBox will allow the average researcher to incorporate deep sequence analysis into routine selections from antibody display libraries.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2022

Flexible Siting Criteria and Staff Minimization for Micro-Reactors

The economic potential of micro-reactors is vast and underestimated. Commonly-emphasized applications include niche markets such as remote communities, mines and military bases. However, micro-reactors could be used as flexible energy generators also for larger markets, such as mobile and containerized agriculture and manufacturing facilities, district heating, micro-grids for data centers, sea ports, airports and hospitals. The implication is that micro-reactors may have to be deployed also in non-remote locations. Successful implementation of micro-reactors needs a navigable and predictable licensing process, technology-appropriate siting restrictions, risk-informed emergency and safety requirements, and practical operating and maintenance requirements. The primary goal of this project was to develop siting criteria that are tailored to micro-reactors deployable in densely-populated areas, e.g., urban environments. To achieve that goal, we compared the characteristics of the MIT research reactor (MITR) with those of leading micro-reactor concepts (e.g., eVinci, USNC, Aurora), and evaluated whether and how the MITR design basis (e.g., inherent safety features, engineered safety systems, source term, emergency planning and emergency operating procedures) and associated regulations may be applicable to these new micro-reactors as well. What makes MITR a unique analogue in this context is its small power rating (6 MWt) and physical size, mode of operations (24/7 with a somewhat more commercial flavor than typical university reactors), and especially its urban location. Of course significant differences exist, such as mission (power production vs. research) and the reactor design itself. Leveraging the MITR experience, this project was able to generate criteria that will allow micro-reactors to realize their full economic potential as flexible heat and electricity generators for a diverse portfolio of applications in non-remote locations. As such, the outcome of this project might encourage investment in and use of micro-reactors. A second goal of the project was to conceptualize a model of operations for micro-reactors that would minimize the staffing requirements, and thus reduce the cost of electricity and heat generated by these systems. Here too our approach was to systematically review the MITR experience and requirements, as well as survey the innovations in autonomous control technologies and monitoring (e.g., advanced sensors, drones, robotics, AI) that would permit a dramatic reduction in staffing at future micro-reactor installations. The scope of work was expanded after the start date to include also an evaluation of micro-reactor security, using the so-called consequence-based analysis, and the development of a methodology to perform dynamic risk assessment for micro-reactors, using system theory and modeling and simulation.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2016

Large-Scale Public Transcriptomic Data Mining Reveals a Tight Connection between the Transport of Nitrogen and Other Transport Processes in Arabidopsis

Movement of nitrogen to the plant tissues where it is needed for growth is an important contribution to nitrogen use efficiency. However, we have very limited knowledge about the mechanisms of nitrogen transport. Loading of nitrogen into the xylem and/or phloem by transporter proteins is likely important, but there are several families of genes that encode transporters of nitrogenous molecules (collectively referred to as N transporters here), each comprised of many gene members. In this study, we leveraged publicly available microarray data of Arabidopsis to investigate the gene networks of N transporters to elucidate their possible biological roles. First, we showed that tissue-specificity of nitrogen (N) transporters was well reflected among the public microarray data. Then, we built coexpression networks of N transporters, which showed relationships between N transporters and particular aspects of plant metabolism, such as phenylpropanoid biosynthesis and carbohydrate metabolism. Furthermore, genes associated with several biological pathways were found to be tightly coexpressed with N transporters in different tissues. Our coexpression networks provide information at the systems-level that will serve as a resource for future investigation of nitrogen transport systems in plants, including candidate gene clusters that may work together in related biological roles.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2013

Efficient Aho-Corasick String Matching on Emerging Multicore Architectures

String matching algorithms are critical to several scientific fields. Beside text processing and databases, emerging applications such as DNA protein sequence analysis, data mining, information security software, antivirus, ma- chine learning, all exploit string matching algorithms [3]. All these applica- tions usually process large quantity of textual data, require high performance and/or predictable execution times. Among all the string matching algorithms, one of the most studied, especially for text processing and security applica- tions, is the Aho-Corasick algorithm. 1 2 Book title goes here Aho-Corasick is an exact, multi-pattern string matching algorithm which performs the search in a time linearly proportional to the length of the input text independently from pattern set size. However, depending on the imple- mentation, when the number of patterns increase, the memory occupation may raise drastically. In turn, this can lead to significant variability in the performance, due to the memory access times and the caching effects. This is a significant concern for many mission critical applications and modern high performance architectures. For example, security applications such as Network Intrusion Detection Systems (NIDS), must be able to scan network traffic against very large dictionaries in real time. Modern Ethernet links reach up to 10 Gbps, and malicious threats are already well over 1 million, and expo- nentially growing [28]. When performing the search, a NIDS should not slow down the network, or let network packets pass unchecked. Nevertheless, on the current state-of-the-art cache based processors, there may be a large per- formance variability when dealing with big dictionaries and inputs that have different frequencies of matching patterns. In particular, when few patterns are matched and they are all in the cache, the procedure is fast. Instead, when they are not in the cache, often because many patterns are matched and the caches are continuously thrashed, they should be retrieved from the system memory and the procedure is slowed down by the increased latency. Efficient implementations of string matching algorithms have been the fo- cus of several works, targeting Field Programmable Gate Arrays [4, 25, 15, 5], highly multi-threaded solutions like the Cray XMT [34], multicore proces- sors [19] or heterogeneous processors like the Cell Broadband Engine [35, 22]. Recently, several researchers have also started to investigate the use Graphic Processing Units (GPUs) for string matching algorithms in security applica- tions [20, 10, 32, 33]. Most of these approaches mainly focus on reaching high peak performance, or try to optimize the memory occupation, rather than looking at performance stability. However, hardware solutions supports only small dictionary sizes due to lack of memory and are difficult to customize, while platforms such as the Cell/B.E. are very complex to program.

Tumeo, Antonino↗
DOE OSTI2025

Ionospheric Disturbances in GNSS TEC Data: SpaceX Falcon 9 Deorbit Maneuvers Over CONUS in April–May 2024

Traveling ionospheric disturbances (TIDs) driven by a large number of internal and external sources are detectable with dense networks of ground‐based Global Navigation Satellite System (GNSS) receivers' measurements of total electron content (TEC). We present the newly developed System for Rapid Analysis of Ionospheric Dynamics (S‐RAID), providing data and visual GNSS TEC products of TIDs of periods ~2–120 min and horizontal resolution up to tens of kilometers for years 2017–2024. The S‐RAID data reveal myriad natural and anthropogenic TIDs from meteorology and space weather, and from human spaceflight activities. In this report, we focus on new prominent disturbances found during SpaceX second stage deorbit maneuvers in April–May 2024. Signatures include waves emanating from the Falcon's trajectory above California and depletions following its deorbit and passage over Arizona. These findings suggest further opportunities to detect and quantify small‐scale events in the ionosphere as well as to understand the responses of the atmosphere‐ionosphere system to known inputs.

58 GEOSCIENCES↗
DOE OSTI2018

Intelligently-Automated Facilities Expansion with the HEPCloud Decision Engine

The next generation of High Energy Physics experiments are expected to generate exabytes of data---two orders of magnitude greater than the current generation. In order to reliably meet peak demands, facilities must either plan to provision enough resources to cover the forecasted need, or find ways to elastically expand their computational capabilities. Commercial cloud and allocation-based High Performance Computing (HPC) resources both have explicit and implicit costs that must be considered when deciding when to provision these resources, and to choose an appropriate scale. In order to support such provisioning in a manner consistent with organizational business rules and budget constraints, we have developed a modular intelligent decision support system (IDSS) to aid in the automatic provisioning of resources---spanning multiple cloud providers, multiple HPC centers, and grid computing federations.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2018

Machine Learning and Social Media to Mine and Disseminate Big Scientific Data

One of the challenges in supplying the communities with wider access to scientific databases is the need for knowledge of database languages like Structured Query Language (SQL). Although the SQL language has been published in many forms, not everybody is able to write SQL queries. Another challenge is that it might not be practical to make the public aware of the structure of databases. There is a need for novice users to query relational databases using their natural language. To solve this problem, many natural language interfaces to structured databases have been developed. The goal is to provide a more intuitive method for generating database queries and delivering responses. Through social media, which makes it possible to interact with a wide section of the population, and with the help of natural language processing, researchers at the Atmospheric Radiation Measurement (ARM) Data Center at Oak Ridge National Laboratory (ORNL) have developed a concept to enable easy search and retrieval of data from several environmental data centers for the scientific community through social media.Using a machine learning framework that maps natural language text to thousands of datasets, instruments, variables, and data streams, the prototype system would allow users to request data through Twitter and receive a link (via tweet) to applicable data results on the project's search catalog tailored to their key words. This automated identification of relevant data from various petascale archives at ORNL could increase convenience, access, and use of the project's data by the broader community. In this paper we discuss how some data-intensive projects at ORNL are using innovative ways to help in data discovery.

Devarakonda, Ranjeet↗
DOE OSTI2026

Time of Flight Secondary Ion Mass Spectrometry for Characterization of Pt-Coated Porous Transport Layers in PEM Water Electrolyzers

Titanium-based porous transport layers (PTLs) and iridium-based catalyst layers (CLs) are two main components of proton exchange membrane water electrolyzers (PEMWEs). PTLs are typically coated with platinum to minimize interfacial losses and to support long-term operation. Optimizing coatings and the PTL-CL interface requires comprehensive characterization. This study establishes time-of-flight secondary ion mass spectrometry (ToF-SIMS) as a valuable technique for PTL characterization, addressing capabilities and limitations related to PTL morphology. A methodology was developed that uses a Cs + sputter beam for dynamic depth profiling, with data collected in both positive-ion (MCs + ) and negative-ion modes to generate depth profiles, 2D ion maps, and 3D ion reconstructions. ToF-SIMS detected relative differences in platinum-layer thickness between samples; these trends were validated by cross-sectional scanning transmission electron microscope (STEM) measurements and flat-titanium substrate controls. Interfacial oxide layers are identified in both ion modes, with enhanced oxide sensitivity in negative mode. The technique’s high sensitivity enables detection of nanometer-scale coatings and trace impurities within the bulk PTL structure. These results provide a methodological framework for analyzing Pt-coated PTLs, with the potential to extend to other components in PEMWEs and other electrolyzer systems.

36 MATERIALS SCIENCE↗
DOE OSTI2021

A pre-training and self-training approach for biomedical named entity recognition

Named entity recognition (NER) is a key component of many scientific literature mining tasks, such as information retrieval, information extraction, and question answering; however, many modern approaches require large amounts of labeled training data in order to be effective. This severely limits the effectiveness of NER models in applications where expert annotations are difficult and expensive to obtain. In this work, we explore the effectiveness of transfer learning and semi-supervised self-training to improve the performance of NER models in biomedical settings with very limited labeled data (250-2000 labeled samples). We first pre-train a BiLSTM-CRF and a BERT model on a very large general biomedical NER corpus such as MedMentions or Semantic Medline, and then we fine-tune the model on a more specific target NER task that has very limited training data; finally, we apply semi-supervised self-training using unlabeled data to further boost model performance. We show that in NER tasks that focus on common biomedical entity types such as those in the Unified Medical Language System (UMLS), combining transfer learning with self-training enables a NER model such as a BiLSTM-CRF or BERT to obtain similar performance with the same model trained on 3x-8x the amount of labeled data. We further show that our approach can also boost performance in a low-resource application where entities types are more rare and not specifically covered in UMLS.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗
DOE OSTI2020

Application of an Embedded Fracture and Borehole Modeling Approach to the Understanding of EGS Collab Experiment 1

EGS Collab is a series of meso-scale experiments and associated numerical simulation activities being funded by the United States Department of Energy, Geothermal Technologies Office (GTO) to investigate enhanced geothermal system processes under in-situ stress and slightly elevated temperature conditions. This project is designed to provide scientists and engineers with immediate access to impermeable rock at scales larger than possible in the laboratory, but generally smaller than those for commercial production. Immediate access to rock is provided via the existing drifts of the former Homestake Gold Mine, now operated as the Sanford Underground Research Facility in Lead, South Dakota. The objectives of the EGS Collab project are to develop well controlled fracture networks between injector and producer boreholes using normal and shear stimulation for permeability enhancement. The first experimental site was located off the West Access drift on the 4850 Level (4850 feet below ground surface) in phyllite of the Precambrian Poorman formation, and involved the creation of a fracture network comprising a combination of hydraulic and natural fractures. A second experimental site is now being considered near the battery alcove on the 4100 Level in amphibolite and rhyolite of the Yates Unit. Data generated during these experiments will be compared against predictions of a suite of computer codes specifically designed to solve problems involving coupled thermal, hydrological, geomechanical, and geochemical processes. Comparisons between experimental and numerical simulation results will provide code developers with direction for improvements and verification of process models, build confidence in the suite of available numerical tools, and ultimately identify critical future development needs for the geothermal modeling community. Moreover, conducting thorough comparisons of models, modelling approaches, measurement approaches and measured data, via the EGS Collab project, will serve to identify techniques that are most likely to succeed at the Frontier Observatory for Research in Geothermal Energy (FORGE), the GTO’s flagship EGS research effort. Experiment 1 has comprised a series of successful tests, including a long-term chilled-water circulation test, but the testbed has two atypical EGS elements. Active ventilation in the adjacent drift over a 50-year period cooled the testbed rock mass and hydraulic fracturing intersected monitoring boreholes making them conduits for fluid flow. Numerical simulations executed in support of the design of the EGS Collab Experiment 1, computed a radial temperature distribution and stress gradient orthogonal to the drift axis, resulting in the forecast of an oblong hydraulic fracture geometry, extended in the direction of the drift from the stimulation borehole. This paper describes the numerical simulation of long-term chilled-water circulation test with an embedded fracture and borehole modeling approach. The principal objective of the simulation work is show agreement between experimental observations in terms of production fluid temperatures, monitoring borehole temperatures, injection and production pressures, and tracer recoveries using a single conceptual model for the fracture network, using characterization and monitoring data generally available for EGS. A secondary objective is to improve the numerical simulation result comparisons with additional data available from the broader monitoring equipment within the testbed versus deeper, hotter, and more remote EGS. One complicating factor for comparisons between the numerical simulation results and experimental observations, is the Joule-Thomson heating associated with large pressure drops across the fracture network.

numerical simulation, embedded fractures and boreh↗
DOE OSTI2020

Phyllosilicate mineral dissolution upon alkaline treatment under aerobic and anaerobic conditions

The dissolution of phyllosilicate minerals exposed to high-pH environments has been studied to determine the influence of alkaline treatments and variable redox conditions on clay dissolution and the potential formation of new phases via incongruent dissolution phenomena. The present study focused on the dissolution of phyllosilicate minerals (illite, muscovite, and montmorillonite) under anaerobic and aerobic conditions using comparative solutions (sodium hydroxide and ammonium hydroxide) at similar ionic strength in the presence and absence of oxygen. Our batch data show that there is a rapid decrease in aluminum dissolution (< 240 hrs) and slow increase in silica over time (up to 1440 hrs). This trend was particularly evident for montmorillonite which shows the greatest dissolution for ammonium hydroxide, likely due to intercalation of the polyatomic cation ammonium into the mineral’s expandable layers. When comparing alkaline treatments, the strong base, sodium hydroxide, dissolved more of the mica minerals, illite and muscovite, likely due to ion-pairing between the silicate tetrahedra [SiO4]n- and Na+ cations in solution. In addition, redox conditions affect the systems similarly with treatment, although the sodium hydroxide treatment results in greater variability in solution redox conditions. For all investigated phyllosilicates, the calculated aqueous aluminum and silicon ratios over time are significantly different from the minerals’ stoichiometric ratios. As a result, we conclude that incongruent dissolution occurred and suggest formation of secondary precipitates. Understanding the potential for clay mineral alterations from interaction with alkaline solutions has implications for in situ remediation, mining operations, and waste interactions with the subsurface. This research shows that incongruent dissolution is likely and will lead to secondary precipitation which may have long term physical and chemical impacts in the subsurface.

mineral dissolution, phyllosilicate, alkaline, amm↗
DOE OSTI2024

Deep Learning-enhanced Block-Diagram Modeling of Solar Power Systems

Data-driven models of power system inverter-based resources are desired to run simulations faster than with detailed electromagnetic transient models, to hide proprietary design details, to support control system design applications, and to aggregate the effects of distributed energy resources. This paper applies a customized Hammerstein Wiener framework to train block diagram models from thousands of electromagnetic transient simulations or experimental test records. The block diagram models integrate with larger grid simulations as voltagecontrolled current sources or current-controlled voltage sources for several simulators. Guidelines for block architecture and training are presented. Three-phase balanced, three-phase unbalanced, and single-phase examples all achieve an acceptable root mean square error of no more than 0.05 per-unit.

Mcdermott, Thomas E. [Private consulting company]↗
DOE OSTI2026

Hyaloscypha finlandica Metabolome Repository

This repository provides the curated data tables, manuscript figure and table exports, dependency records, and workflow scripts supporting an integrated comparative genomics and untargeted LC-MS/MS metabolomics analysis of Hyaloscypha finlandica strain PMI 746, a root-associated dark septate endophyte of poplar. The repository includes genome-mining summaries from antiSMASH, FunBGCeX, BGC-Prophet, and BiG-SCAPE; processed metabolomics inputs; metabolite annotation evidence; statistical outputs; and publication-facing figures and tables. Raw LC-MS/MS spectra, full genome/protein downloads, and large generated tool outputs are referenced through public archive/accession records and are not stored in Git.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2025

Predicting Initial Trans-Membrane Pressure for Optimized Operations in UF Unit Using Random Forest

With the growing scarcity of freshwater, innovative process design mechanisms like Ultra-filtration(UF) units are increasingly gaining attention among water treatment utilities to address the rising demand. Ensuring reliable water production necessitates efficient resource utilization, minimizing downtime in UF systems. Recent advancements in machine learning (ML) have enabled the development of accurate data-driven models for Model Predictive Control (MPC), often requiring minimal prior knowledge of underlying physical processes. In this study, we present predictive regression models based on Random Forest (RF) and Auto-Regressive (AR) approaches to forecast the initial Trans-Membrane Pressure (TMP) for each filtration cycle in data generated by Direct Potable Reuse (DPR) systems. The proposed RF-based model demonstrates superior performance compared to baseline methods, including historical mean, Last Observation Carried Forward (LOCF), and naïve AR models, across various forecasting horizons in terms of root mean square (RMSE) metric. Accurate prediction of initial TMP is critical for optimizing CCRO operations, as it enables the development of robust modelling frameworks that enhance process efficiency and reliability. The demonstrated efficacy of the RF-based approach highlights its potential as a tool for real-time decision-making in water treatment systems, paving the way for advanced process optimization and sustainable water resource management.

Mukherjee, Subrata [ORNL] (ORCID:0000000309930338)↗
DOE OSTI2024

Electrically Powered High-Salinity Brine Separation Using Dimethyl Ether

Dewatering highly saline aqueous streams, from mining and geothermal leachates to industrial wastewater, is essential for effective resource recovery and safe disposal. Membraneless water extraction (MWE) uses a low-polarity solvent to separate water from concentrated aqueous solutions. In this study, we design a new MWE that uses dimethyl ether (DME) to selectively extract water from high-salinity brines, leveraging the volatility of DME to achieve rapid solvent recovery. By separating water and dissolved salts at a liquid–liquid interface, MWE minimizes the deleterious effects of scaling on vulnerable membrane and heat exchanger surfaces, reducing the need for extensive pretreatment and expensive materials. We begin by developing a computational framework for a multistage counterflow liquid–liquid contactor, which extracts water into DME, coupled with a multistage solvent regenerator that uses vapor compression to efficiently separate the desalinated water from the DME extractant. Excess Gibbs free energy and equation of state frameworks are used to model fluid phase equilibria in water–DME–sodium chloride (NaCl) mixtures, with interaction parameters estimated from experimental data. Incorporating equilibrium calculations into a system-scale computational model, we examine the performance of MWE using DME for the first time. Our analysis demonstrates that MWE can concentrate seawater desalination brine (>1.0 mol NaCl kg –1 ) to zero-liquid discharge salinities, with an energy consumption of under 50 kW h per m 3 of water extracted with a solvent recovery ratio greater than 99.9%. We highlight the importance of staging the vapor compression process to simultaneously minimize energy consumption while enabling brine concentration and product water solvent contamination. Finally, the thermodynamic framework developed here allows for the robust evaluation of new MWE solvents and systems for critical brine concentration and fractional precipitation applications.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2022

ThunderSecure: deploying real-time intrusion detection for 100G research networks by leveraging stream-based features and one-class classification network

Nowadays, data generated by large-scale scientific experiments are on the scale of petabytes per month. These data are transferred through dedicated high-bandwidth networks (40/100G) across distributed sites for processing, storage, and analysis. Like general purpose networks, research networks experience intrusions. However, monitoring anomalies in such high-speed network traffics is challenging given current cyber-infrastructure. Moreover, traditional network intrusion detection systems (NIDS) are signature based. However, anomaly patterns are difficult to define and that rulesets are often not updated frequently enough to reflect the changes of attack behaviors. We present ThunderSecure, a high-throughput, unsupervised learning-based intrusions detection system for 100G research networks. ThunderSecure implements an efficient packet processing and detection pipeline using multi-cores and GPUs. It extracts statistical and temporal features from real-time network data streams and feeds them to a one-class anomaly detection network. A baseline of normal distribution will be created based on the training observation. Testing traffic deviated from the learned profile will be marked as anomalies. We trained ThunderSecure on hundreds of billions of science data packets mirrored from two 100G network connections at Fermi National Accelerator Laboratory. The detection performance was evaluated on traffic captured from the same research network days and weeks after the training with different types of attack flows injected. Results show that ThunderSecure can recognize science data traffic captured long after the training and made nearly certain detection on the segment of the streams where anomalous flows were injected.

100G research network↗
DOE OSTI2016

Immunophenotype Discovery, Hierarchical Organization, and Template-Based Classification of Flow Cytometry Samples

We describe algorithms for discovering immunophenotypes from large collections of flow cytometry samples and using them to organize the samples into a hierarchy based on phenotypic similarity. The hierarchical organization is helpful for effective and robust cytometry data mining, including the creation of collections of cell populations’ characteristic of different classes of samples, robust classification, and anomaly detection. We summarize a set of samples belonging to a biological class or category with a statistically derived template for the class. Whereas individual samples are represented in terms of their cell populations (clusters), a template consists of generic meta-populations (a group of homogeneous cell populations obtained from the samples in a class) that describe key phenotypes shared among all those samples. We organize an FC data collection in a hierarchical data structure that supports the identification of immunophenotypes relevant to clinical diagnosis. A robust template-based classification scheme is also developed, but our primary focus is in the discovery of phenotypic signatures and inter-sample relationships in an FC data collection. This collective analysis approach is more efficient and robust since templates describe phenotypic signatures common to cell populations in several samples while ignoring noise and small sample-specific variations. We have applied the template-based scheme to analyze several datasets, including one representing a healthy immune system and one of acute myeloid leukemia (AML) samples. The last task is challenging due to the phenotypic heterogeneity of the several subtypes of AML. However, we identified thirteen immunophenotypes corresponding to subtypes of AML and were able to distinguish acute promyelocytic leukemia (APL) samples with the markers provided. Clinically, this is helpful since APL has a different treatment regimen from other subtypes of AML. Core algorithms used in our data analysis are available in the flowMatch package at www.bioconductor.org. It has been downloaded nearly 6,000 times since 2014.

59 BASIC BIOLOGICAL SCIENCES↗