Search NASA⌕ Search

SEARCH · Search NASA

Results for “Query Tool”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 235 records · Page 13
All sourcesNASA NTRSDOE OSTI
DOE OSTI2019

Safety Analytics Forecasting and Evaluation Reporting (SAFER)

The SAFER Sandbox is a web application for testing various types of data visualization tools that analyze safety data across national laboratories. By enabling the user to visualize this data through visual and textual analyses, the SAFER Sandbox helps to facilitate the research and support for the National Nuclear Security Administration (NNSA) Office of Safety, Infrastructure and Operations (NA-50). Safety data across the national laboratory complex is not currently analyzed in one location. This prohibits administrators from viewing safety data trends and making large scale decisions based on a holistic view of the data. By implementing data analytics web services such as representative text summary, noun phrase extraction and graph exploration, users can view and explore safety data based on search queries. The data visualizations in the SAFER Sandbox reveal trends and point to items of concern based on safety data. The SAFER Sandbox is a testing ground for these visualizations.

98 NUCLEAR DISARMAMENT, SAFEGUARDS, AND PHYSICAL P↗
DOE OSTI2015

Best practices mobile application for D and D KM-IT - 15712

The Deactivation and Decommissioning Knowledge Management Information Tool (D and D KM-IT) is the repository of useful information built specifically for D and D community. The knowledge provided by the D and D KM-IT is the experience and expertise gained by people within the field and fellow employees of the Department of Energy (DOE). This information has been the result of gathering, analyzing, storing, and sharing knowledge to help reduce time spent on discovering previously found data. In order to help in organizing all the data from all the professional contributors, the D and D KM-IT has been divided into 12 different modules. Each individual module pertains to a different area of information for users, such as vendor and technology information, collaboration tools, and even a hotline that contains problems posted by the users. Purpose: - Design and develop the module for the Best Practices on D and D KM-IT mobile platform deployed at https://m.dndkm.org; - The Best Practices mobile module should integrate with existing modules seamlessly; - Use the D and D KM-IT existing database to consume information for the mobile system. jQuery Mobile framework was used to develop Best Practices module for mobile devices. jQuery Mobile framework provides support for responsive web design (RWD). RWD allows adaptation of the layout and user interface of web sites and web apps to different device resolutions, screen densities, and views. UI design was implemented with three fundamental elements: CSS media queries, An adaptable grid, Flexible images and media. Advantages/benefits: - A responsive web design adapts to any-sized screen / resolution and orientation view; - ASP.NET compatible which allows the D and D KM-IT database to support all platforms accessing the system; - Navigating pages is more bandwidth efficient by refreshing data in small section with Ajax instead of reloading the entire page; - Multiple parts of a page can perform different tasks resulting in improved user's experience. Results: - Mobile system is accessible on compatible devices running Android, Apple, Blackberry, Palm, and Windows devices; - Best Practices searches through Characterization, Decontamination, Dismantlement, Material Disposition, and Worker Safety; - Keyword search is enabled for the best practices that are available on both the desktop and mobile systems - Documents are formatted into a PDF file format to ensure compatibility on any device. Conclusion: - Fully integrated with the desktop Best Practices module and displays documents that are uploaded on web platform; - Fast-loading with D and D KM-IT mobile system and downloadable documents; - Responsive design which gives the same user experience when either viewed on smart phone or tablet.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗
DOE OSTI2014

Flexible and Scalable Data Fusion using Proactive Schemaless Information Services

Exascale data environments are fast approaching, driven by diverse structured and unstructured data such as system and application telemetry streams, open-source information capture, and on-demand simulation output. Storage costs having plummeted, the question is now one of converting vast stores of data to actionable information. Complicating this problem are the low degrees of awareness across domain boundaries about what potentially useful data may exist, and write-once- read-never issues (data generation/collection rates outpacing data analysis and integration rates). Increasingly, technologists and researchers need to correlate previously unrelated data sources and artifacts to produce fused data views for domain-specific purposes. New tools and approaches for creating such views from vast amounts of data are vitally important to maintaining research and operational momentum. We propose to research and develop tools and services to assist in the creation, refinement, discovery and reuse of fused data views over large, diverse collections of heterogeneously structured data. We innovate in the following ways. First, we enable and encourage end-users to introduce customized index methods selected for local benefit rather than for global interaction (flexible multi-indexing). We envision rich combinations of such views on application data: views that span backing stores with different semantics, that introduce analytic methods of indexing, and that define multiple views on individual data items. We specifically decline to build a big fused database of everything providing a centralized index over all data, or to export a rigid schema to all comers as in federated query approaches. Second, we proactively advertise these application-specific views so that they may be programmatically reused and extended (data proactivity). Through this mechanism, both changes in state (new data in existing view collected) and changes in structure (new or derived view exists) are made known. Lastly, we embrace found data heterogeneity by coupling multi-indexing to backing stores with appropriate semantics (as opposed to a single store or schema).

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗
DOE OSTI2014

Flexible and Scalable Data Fusion using Proactive, Schemaless Information Services

Exascale data environments are fast approaching, driven by diverse structured and unstructured data such as system and application telemetry streams, open-source information capture, and on-demand simulation output. Storage costs having plummeted, the question is now one of converting vast stores of data to actionable information. Complicating this problem are the low degrees of awareness across domain boundaries about what potentially useful data may exist, and write-once-read- never issues (data generation/collection rates outpacing data analysis and integration rates). Increasingly, technologists and researchers need to correlate previously unrelated data sources and artifacts to produce fused data views for domain-specific purposes. New tools and approaches for creating such views from vast amounts of data are vitally important to maintaining research and operational momentum. We propose to research and develop tools and services to assist in the creation, refinement, discovery and reuse of fused data views over large, diverse collections of heterogeneously structured data. We innovate in the following ways. First, we enable and encourage end-users to introduce customized index methods selected for local benefit rather than for global interaction (flexible multi-indexing). We envision rich combinations of such views on application data: views that span backing stores with different semantics, that introduce analytic methods of indexing, and that define multiple views on individual data items. We specifically decline to build a big fused database of everything providing a centralized index over all data, or to export a rigid schema to all comers as in federated query approaches. Second, we proactively advertise these application-specific views so that they may be programmatically reused and extended (data proactivity). Through this mechanism, both changes in state (new data in existing view collected) and changes in structure (new or derived view exists) are made known. Lastly, we embrace found data heterogeneity by coupling multi-indexing to backing stores with appropriate semantics (as opposed to a single store or schema).

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗
DOE OSTI2014

Dynamic Analysis Methods for Detecting Anomalies in Asynchronously Interacting Systems

Detecting modifications to digital system designs, whether malicious or benign, is problematic due to the complexity of the systems being analyzed. Moreover, static analysis techniques and tools can only be used during the initial design and implementation phases to verify safety and liveness properties. It is computationally intractable to guarantee that any previously verified properties still hold after a system, or even a single component, has been produced by a third-party manufacturer. In this paper we explore new approaches for creating a robust system design by investigating highly-structured computational models that simplify verification and analysis. Our approach avoids the need to fully reconstruct the implemented system by incorporating a small verification component that dynamically detects for deviations from the design specification at run-time. The first approach encodes information extracted from the original system design algebraically into a verification component. During run-time this component randomly queries the implementation for trace information and verifies that no design-level properties have been violated. If any deviation is detected then a pre-specified fail-safe or notification behavior is triggered. Our second approach utilizes a partitioning methodology to view liveness and safety properties as a distributed decision task and the implementation as a proposed protocol that solves this task. Thus the problem of verifying safety and liveness properties is translated to that of verifying that the implementation solves the associated decision task. We develop upon results from distributed systems and algebraic topology to construct a learning mechanism for verifying safety and liveness properties from samples of run-time executions.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2016

DART 7.0 User Guide

The Desktop Analysis Reporting Tool (DART) is a software package that allows users to easily view and analyze daily files that span long periods. DART gives users the capability to quickly determine the state of health of a radiation portal monitor (RPM), troubleshoot and diagnose problems, and view data in various time frames to perform trend analysis. In short, it converts the data strings written in the daily files into meaningful tables and plots. The standalone version of DART (“soloDART”) utilizes a database engine that is included with the application; no additional installations are necessary. There is also a networked version of DART (“polyDART”) that is designed to maximize the benefit of a centralized data repository while distributing the workload to individual desktop machines. This networked approach requires a more complex database manager Structured Query Language (SQL) Server; however, SQL Server is not currently provided with DART. Regardless of which version is used, DART will import daily files from RPMs, store the relevant data in its database, and it can produce reports for status, trend analysis, and reporting purposes.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2025

The AI-Base NDA Catalog

The project objective was to create a catalog of NDA capabilities so that as future challenges emerge, potential solutions can be quickly identified. The initial direction was to create a custom database where we would manually add various NDA technologies. During the year, LANL leadership heavily embraced AI tools and established LANL’s Enterprise ChatGPT license. To align with this vision, instead of a database we created a custom GPT to achieve the same capability. We provided a series of references on NDA technologies, the most comprehensive of which is the 2024 “PANDA manual” which is a 700-page textbook. A custom GPT is a semi-isolated version of OpenAI’s GPT model that can be tailored to specific tasks with context documents and instructions. Anyone with a LANL ChatGPT Enterprise account can access the NDA Catalog. Since the GPT exists on the green network, only non-sensitive questions may be asked. The Catalog can be used by simply asking questions in conversational English. We have found the Catalog to be quite accurate, even for heavily technical, complex queries. The Catalog allows the user to describe a measurement problem and it will respond with potential technology solutions.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗
DOE OSTI2015

Knowledge management information tool analytics with distributed database engine - 15737

The D and D KM-IT is a web-based knowledge management information tool custom built for the deactivation and decommissioning (D and D) user community. The system allows interested users to post information which goes through a custom work flow. To better improve the performance and user experience, the analysis of server logs is being conducted using MongoDB. MongoDB is a document database that provides high performance, high availability, and easy scalability. Objectives: - Feasibility study of Distributed Database Engine, MongoDB for web analytics; - Analyze server logs to evaluate D and D KM-IT module usage; - Generate useful information from the analysis. Method: - The server logs were loaded and stored in MongoDB for analysis; - Analysis and visualization was performed using NodeJS and Java Script library. Results: - It was possible to store log files in MongoDB; - Log files with different time stamps and groups from different servers can be aggregated; - Queries can be written to extract meaningful information from aggregated log files (e.g. users from different countries/organizations). Conclusions: - MongoDB is useful for server log analytics as the analyst can immediately access, analyze, and visualize MongoDB data based on the requirements; - The information can be extracted to develop custom reports; - Optimized queries can be run for fast data access. Path forward: - Custom queries need to be developed to extract information leading to improved user information.

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗
DOE OSTI2014

SU-E-T-544: A Radiation Oncology-Specific Multi-Institutional Federated Database: Initial Implementation

Purpose: To implement a common database structure and user-friendly web-browser based data collection tools across several medical institutions to better support evidence-based clinical decision making and comparative effectiveness research through shared outcomes data. Methods: A consortium of four academic medical centers agreed to implement a federated database, known as Oncospace. Initial implementation has addressed issues of differences between institutions in workflow and types and breadth of structured information captured. This requires coordination of data collection from departmental oncology information systems (OIS), treatment planning systems, and hospital electronic medical records in order to include as much as possible the multi-disciplinary clinical data associated with a patients care. Results: The original database schema was well-designed and required only minor changes to meet institution-specific data requirements. Mobile browser interfaces for data entry and review for both the OIS and the Oncospace database were tailored for the workflow of individual institutions. Federation of database queries--the ultimate goal of the project--was tested using artificial patient data. The tests serve as proof-of-principle that the system as a whole--from data collection and entry to providing responses to research queries of the federated database--was viable. The resolution of inter-institutional use of patient data for research is still not completed. Conclusions: The migration from unstructured data mainly in the form of notes and documents to searchable, structured data is difficult. Making the transition requires cooperation of many groups within the department and can be greatly facilitated by using the structured data to improve clinical processes and workflow. The original database schema design is critical to providing enough flexibility for multi-institutional use to improve each institution s ability to study outcomes, determine best practices, and support research. The project has demonstrated the feasibility of deploying a federated database environment for research purposes to multiple institutions.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2014

Evaluation of Application Space Expansion for the Sensor Fish

The Pacific Northwest National Laboratory has developed an instrument known as the sensor fish that can be released into downstream passage routes at hydropower facilities to collect data on the physical conditions that a fish might be exposed to during passage through a turbine. The US Department of Energy Wind and Water Power Program sees value in expanding the sensor fish application space beyond large Kaplan turbines in the northwest United States to evaluate conditions to which a greater variety of fish species are exposed. Development of fish-friendly turbines requires an understanding of both physical passage conditions and biological responses to those conditions. Expanding the use of sensor fish into other application spaces will add to the knowledge base of physical passage conditions and could also enhance the use of sensor fish as a site-specific tool in mitigating potential impacts to fish populations from hydropower. The Oak Ridge National Laboratory (ORNL) National Hydropower Assessment Program (NHAAP) database contains hydropower facility characteristics that, along with national fish distribution data, were used to evaluate potential interactions between fish species and project characteristics related to downstream passage issues. ORNL developed rankings for the turbine types in the NHAAP database in terms of their potential to impact fish through injury or mortality during downstream turbine passage. National-scale fish distributions for 31 key migratory species were spatially intersected with hydropower plant locations to identify facilities where turbines with a high threat to fish injury or mortality overlap with the potential range of a sensitive fish species. A dataset was produced that identifies hydropower facilities where deployment of the sensor fish technology might be beneficial in addressing issues related to downstream fish passage. The dataset can be queried to target specific geographic regions, fish species, license expiration dates, generation capacity levels, ownership characteristics, turbine characteristics, or any combination of these metrics.

13 HYDRO ENERGY↗
DOE OSTI2016

GenAp: a distributed SQL interface for genomic data

Abstract Background The impressively low cost and improved quality of genome sequencing provides to researchers of genetic diseases, such as cancer, a powerful tool to better understand the underlying genetic mechanisms of those diseases and treat them with effective targeted therapies. Thus, a number of projects today sequence the DNA of large patient populations each of which produces at least hundreds of terra-bytes of data. Now the challenge is to provide the produced data on demand to interested parties. Results In this paper, we show that the response to this challenge is a modified version of Spark SQL, a distributed SQL execution engine, that handles efficiently joins that use genomic intervals as keys. With this modification, Spark SQL serves such joins more than 50× faster than its existing brute force approach and 8× faster than similar distributed implementations. Thus, Spark SQL can replace existing practices to retrieve genomic data and, as we show, allow users to reduce the number of lines of software code that needs to be developed to query such data by an order of magnitude.

Kozanitis, Christos↗
DOE OSTI2015

MerAligner: A Fully Parallel Sequence Aligner

Aligning a set of query sequences to a set of target sequences is an important task in bioinformatics. In this work we present merAligner, a highly parallel sequence aligner that implements a seed-and-extend algorithm and employs parallelism in all of its components. MerAligner relies on a high performance distributed hash table (seed index) and uses onesided communication capabilities of the Unified Parallel C to facilitate a fine-grained parallelism. We leverage communication optimizations at the construction of the distributed hash table and software caching schemes to reduce communication during the aligning phase. Additionally, merAligner preprocesses the target sequences to extract properties enabling exact sequence matching with minimal communication. Finally, we efficiently parallelize the I/O intensive phases and implement an effective load balancing scheme. Results show that merAligner exhibits efficient scaling up to thousands of cores on a Cray XC30 supercomputer using real human and wheat genome data while significantly outperforming existing parallel alignment tools.

Georganas, E↗
DOE OSTI2014

Machine Learning in the Big Data Era: Are We There Yet?

In this paper, we discuss the machine learning challenges of the Big Data era. We observe that recent innovations in being able to collect, access, organize, integrate, and query massive amounts of data from a wide variety of data sources have brought statistical machine learning under more scrutiny and evaluation for gleaning insights from the data than ever before. In that context, we pose and debate the question - Are machine learning algorithms scaling with the ability to store and compute? If yes, how? If not, why not? We survey recent developments in the state-of-the-art to discuss emerging and outstanding challenges in the design and implementation of machine learning algorithms at scale. We leverage experience from real-world Big Data knowledge discovery projects across domains of national security and healthcare to suggest our efforts be focused along the following axes: (i) the data science challenge - designing scalable and flexible computational architectures for machine learning (beyond just data-retrieval); (ii) the science of data challenge the ability to understand characteristics of data before applying machine learning algorithms and tools; and (iii) the scalable predictive functions challenge the ability to construct, learn and infer with increasing sample size, dimensionality, and categories of labels. We conclude with a discussion of opportunities and directions for future research.

Big Data↗
DOE OSTI2015

SU-E-T-235: Data Mining for Evaluating Treatment Performances Over a Large Quantity of Data to Monitor and Improve SBRT Workflow

Purpose: To quality assure a large quantity of retrospective treatment cases for treatment performances by randomly sampling is inefficient. Here we provide a method to efficiently monitor and investigate the QA of SBRT workflow over Mosaiq. Methods: The code developed with Microsoft SQL Server Management Studio 2008R2 and VBA was used for retrieving and sorting data from Mosaiq (version 2.3–2.6 during 2012–2015). SBRT patients were filtered by fractional dose over 350cGy and total fraction number less than 6, which SBRT prescriptions were defined. The quality assurance on the SBRT workflow was focused on the treatment deliveries such as patient positioning setup, CBCT indicated offsets and couch shifted corrections. The treatment delivery were done by Varian Truebeam systems and the record/verify by Mosaiq. Results: Total 82 SBRT patients corresponding to 103 courses and 854 CBCT images were found by the retrieval query. Most centers record daily pre-treatment (Pre-Tx: before treatment shift) image-guided shifts along treatment course for inter-fraction motion record, and it is useful to also verify it with post-treatment imaging (Post-Tx: after treatment CBCT verification) to verify intra-fraction motion. Analyzing the details of daily recorded shifts can reveals the information of patient-setup and staff’s record/verify behaviors. 3 examples were provided as solid evidences and on-going rectification for preventing future mistakes. Conclusions: The report gave feasible examples for inspector to verify a large amount of data during site investigation. This program can also be extended to a scheduled data mining with software to periodical analyze the timely records in Mosaiq, for example, a various control charts for different QA purposes. As the current trend of automation in radiation therapy field, the data mining would be a necessary tool in the future, just as the automatic plan quality evaluation has been under development in Eclipse.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2015

Open Research Challenges with Big Data - A Data-Scientist s Perspective

In this paper, we discuss data-driven discovery challenges of the Big Data era. We observe that recent innovations in being able to collect, access, organize, integrate, and query massive amounts of data from a wide variety of data sources have brought statistical data mining and machine learning under more scrutiny and evaluation for gleaning insights from the data than ever before. In that context, we pose and debate the question - Are data mining algorithms scaling with the ability to store and compute? If yes, how? If not, why not? We survey recent developments in the state-of-the-art to discuss emerging and outstanding challenges in the design and implementation of machine learning algorithms at scale. We leverage experience from real-world Big Data knowledge discovery projects across domains of national security, healthcare and manufacturing to suggest our efforts be focused along the following axes: (i) the data science challenge - designing scalable and flexible computational architectures for machine learning (beyond just data-retrieval); (ii) the science of data challenge the ability to understand characteristics of data before applying machine learning algorithms and tools; and (iii) the scalable predictive functions challenge the ability to construct, learn and infer with increasing sample size, dimensionality, and categories of labels. We conclude with a discussion of opportunities and directions for future research.

Big data↗
DOE OSTI2025

RAG for FLAG: AI Assistance for a Physics Code

Artificial intelligence (AI) has quickly become an important tool in scientific research, where significant efforts are underway to develop tools that will expedite the research process. One area of particular impact is scientific software, which can be particularly complex, and therefore time consuming to learn and use effectively. AI assistants are increasingly helping to streamline the process by performing tasks such as interactively answering user questions or suggesting solutions. Los Alamos National Laboratory (LANL) develops several advanced scientific codes, such as FLAG, which can be used to run multiphysics simulations. With this study, our goal was to develop an AI assistant for FLAG that could help make the process of understanding the software and running physics simulations more efficient. To develop an AI assistant for FLAG, we used a method called retrieval-augmented generation (RAG), which is a technique that uses information from relevant data sources to enhance the accuracy of large language models (LLMs). We used the FLAG user manual and other FLAG documentation as the knowledge base for the RAG system. When a user provides a query, RAG retrieves relevant sections from the knowledge base in response, then uses those excerpts to generate grounded and contextually rich answers. We found that our AI assistant was able to provide context aware answers and source references to user queries. To evaluate performance, we developed a set of 40 benchmark questions and compared the accuracy of the responses to those of two standard LLMs without retrieval. Our AI assistant significantly outperformed the standard LLMs at answering FLAG-related questions, with an 82.5% accuracy rate, compared to 47.5% for both of the standard LLMs. This has the potential to make the process of learning and using FLAG much easier, especially for new users. Ultimately, it supports LANL’s broader mission by empowering scientists and engineers to focus more on discovery and analysis rather than on navigating complex software systems.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2018

Streamlining Stakeholder Data Accessibility Utilizing Web-Based GIS Capabilities - 18672

Environmental data is often complex, and it can be difficult to provide proper context to fully understand and comprehensively analyze the data. One tool identified to better present complicated data is the use of web-based Geographic Information Systems (GIS). Web-based GIS provides much-needed visual context to the complex data collected at Los Alamos National Laboratory (LANL). By overlaying environmental data atop relevant geographical features, such as topography, surface or subsurface geology, hydrogeological features, buildings, roads, and technical areas, LANL's diverse group of stakeholders are better able to understand environmental data collected at the LANL site within the framework of the surrounding geography. Since 2012, Los Alamos has utilized a web-based GIS module in the Intellus New Mexico data-sharing web site. This web site provides unfettered public access to LANL and New Mexico Environment Department (NMED) - Department of Energy (DOE) Oversight Bureau environmental data. Through the past five years of real world experience, LANL identified areas where questions and confusion arose, particularly for those not directly associated with the site's environmental cleanup mission. In order to query the millions of records in Intellus in a meaningful way, users had to have substantial knowledge of the data organization and nomenclature ahead of time. While this was not a challenge for LANL and NMED environmental scientists, it proved to be a substantial barrier for members of the general public to utilize Intellus. In 2017, LANL initiated an Intellus redesign with the goal of deconstructing the type of questions people asked and identifying the types of user personas accessing the site to engineer a simpler path to desired data sets. Greatly simplified click paths ('Quick Paths') were created that walk the user through the basics with a minimum of jargon and a strong reliance on mapping results for easy visualization. The improved mapping was enabled by an upgraded online GIS functionality that allowed LANL to easily integrate their own base-maps to provide users the most up to date views of their data. The integrated Intellus GIS was enhanced to assimilate with LANL's dynamic base maps and site information in relation to data collection activities, to avoid outdated presentations. A simplified user interface (UI) was added and easier data querying was designed. A more robust visual presentation was created that allowed better viewing of spatial data. All data queries in the new simplified 'Quick Path' give the users the option to map the results and save their results to their optional accounts. This brings the power of GIS visualization to all users without the need for GIS knowledge or extensive training. By integrating with Esri{sup R} technologies, base-map layers in ArcGIS Online{sup R} can be updated by LANL or other data providers at any time and reflected immediately in the Intellus mapping application called GIS+ 1. Users can even load additional map layers to GIS+ that contain their own spatial data or information shared from over a million users on ArcGIS Online{sup R}, such as ecological information from the Fish and Wildlife Service, or geology layers from the USGS. The results from the improved quick path search in Intellus can be viewed in GIS+ and analyzed using Esri's{sup R} Smart Mapping technology, which makes it easy to classify results with its visual representation of the results in an interactive histogram, where sliders can be dragged to look at various intervals for each color range for that query. With enhanced GIS+, Los Alamos National Laboratory is able to observe improve Stakeholder engagement and positive Stakeholder interactions within its public environmental database, Intellus New Mexico. (authors)

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗
DOE OSTI2014

Predicting and Detecting Emerging Cyberattack Patterns Using StreamWorks

The number and sophistication of cyberattacks on industries and governments have dramatically grown in recent years. To counter this movement, new advanced tools and techniques are needed to detect cyberattacks in their early stages such that defensive actions may be taken to avert or mitigate potential damage. From a cybersecurity analysis perspective, detecting cyberattacks may be cast as a problem of identifying patterns in computer network traffic. Logically and intuitively, these patterns may take on the form of a directed graph that conveys how an attack or intrusion propagates through the computers of a network. Such cyberattack graphs could provide cybersecurity analysts with powerful conceptual representations that are natural to express and analyze. We have been researching and developing graph-centric approaches and algorithms for dynamic cyberattack detection. The advanced dynamic graph algorithms we are developing will be packaged into a streaming network analysis framework known as StreamWorks. With StreamWorks, a scientist or analyst may detect and identify precursor events and patterns as they emerge in complex networks. This analysis framework is intended to be used in a dynamic environment where network data is streamed in and is appended to a large-scale dynamic graph. Specific graphical query patterns are decomposed and collected into a graph query library. The individual decomposed subpatterns in the library are continuously and efficiently matched against the dynamic graph as it evolves to identify and detect early, partial subgraph patterns. The scalable emerging subgraph pattern algorithms will match on both structural and semantic network properties.

Chin, George↗