Search NASA⌕ Search

SEARCH · Search NASA

Results for “Analyst”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 127 records · Page 7
All sourcesNASA NTRSDOE OSTI
NASA NTRS1994

The Pacor 2 expert system: A case-based reasoning approach to troubleshooting

The Packet Processor 2 (Pacor 2) Data Capture Facility (DCF) acquires, captures, and performs level-zero processing of packet telemetry for spaceflight missions that adhere to communication services recommendations established by the Consultative Committee for Space Data Systems (CCSDS). A major goal of this project is to reduce life-cycle costs. One way to achieve this goal is to increase automation. Through automation, using expert systems, and other technologies, staffing requirements will remain static, which will enable the same number of analysts to support more missions. Analysts provide packet telemetry data evaluation and analysis services for all data received. Data that passes this evaluation is forwarded to the Data Distribution Facility (DDF) and released to scientists. Through troubleshooting, data that fails this evaluation is dumped and analyzed to determine if its quality can be improved before it is released. This paper describes a proof-of-concept prototype that troubleshoots data quality problems. The Pacor 2 expert system prototype uses the case-based reasoning (CBR) approach to development, an alternative to a rule-based approach. Because Pacor 2 is not operational, the prototype has been developed using cases that describe existing troubleshooting experience from currently operating missions. Through CBR, this experience will be available to analysts when Pacor 2 becomes operational. As Pacor 2 unique experience is gained, analysts will update the case base. In essence, analysts are training the system as they learn. Once the system has learned the cases most likely to recur, it can serve as an aide to inexperienced analysts, a refresher to experienced analysts for infrequently occurring problems, or a training tool for new analysts. The Expert System Development Methodology (ESDM) is being used to guide development.

Sary, Charisse↗
DOE OSTI2019

The Iterative Processing Framework: A New Paradigm for Automatic Event Building

Abstract In a traditional data-processing pipeline, waveforms are acquired, a detector makes the signal detections (i.e., arrival times, slownesses, and azimuths) and passes them to an associator. The associator then links the detections to the fitting-event hypotheses to generate an event bulletin. Most of the time, this traditional pipeline requires substantial human-analyst involvement to improve the quality of the resulting event bulletin. For the year 2017, for instance, International Data Center (IDC) analysts rejected about 40% of the events in the automatic bulletin and manually built 30% of the legitimate events. Here, we introduce an iterative processing framework (IPF) that includes a new data-processing module that incorporates automatic analyst behaviors (auto analyst [AA]) into the event-building pipeline. In the proposed framework, through an iterative process, the AA takes over many of the tasks traditionally performed by human analysts. These tasks can be grouped into two major processes: (1) evaluating small events with a low number of location-defining arrival phases to improve their formation; and (2) scanning for and exploiting unassociated arrivals to form potential events missed by previous association runs. To test the proposed framework, we processed a two-week period (15–28 May 2010) of the signal-detections dataset from the IDC. Comparison with an expert analyst-reviewed bulletin for the same time period suggests that IPF performs better than the traditional pipelines (IDC and baseline pipelines). The majority of the additional events built by the AA are low-magnitude events that were missed by these traditional pipelines. The AA also adds additional signal detections to existing events, which saves analyst time, even if the event locations are not significantly affected.

58 GEOSCIENCES↗
DOE OSTI2020

Temporal Analysis and Scene Change Detection in Multispectral Overhead Imagery

Scene change detection can be a tedious and time consuming process especially when concerning large geographical areas, and the process can be even more cumbersome when analyzing changes in an area over large spans of time. Developing a useful way to help analysts recognize at what points in time significant changes to a scene have occurred can allow them to better focus their efforts in characterizing events. Applications include: Facility monitoring, Construction chronology, Monitoring of vehicle/aircraft activity, Characterization of larger sequences of events. In large areas exceeding hundreds to thousands of square kilometers in size, it can be difficult localizing when scene changes have occurred. Analysts can spend hours going through imagery to try to identify new construction, monitor facility activities, monitor vehicle movement, etc. where the object of interest may only be a few square meters. Our goal is to help cut down this time by giving analysts change maps with hot spots of change, allowing them to focus on regions that have experienced actual change in time frames they're interested in. Additionally, by combining these change maps into layers within a data cube, analysts can examine the change maps from a temporal perspective, allowing events to be characterized over spans of time. By opening the data cube in an imaging software capable of separating the layers, we can analyze the change maps sequentially, allowing us to examine scene changes occurring over time. As an example, we examined overhead imagery from Planet Labs of what appears to be a parking lot on Fort Irwin over the course of a year using ENVI, a geospatial satellite imagery analysis software. Using ENVI, we generate a graph of changes over time, and notice a particular segment near the end of our analysis window where no changes are detected. Examination of the actual satellite imagery reveals that during this time span, the parking lot was empty. This could be due to facility shutdown for maintenance or upgrades, or possibly even total workforce/vehicle fleet movement. Information like this could help analysts better characterize events, as well as to help create clearer timelines in larger sequences of events. Workflow steps: - Collect multiple maps of the same AOI (Area of Interest) during a time span of interest; - Generate change maps from AOI maps; - Generate data cube from change maps. An analyst can use the data cube to help inspect an AOI for activities within a time span of interest. If an event of interest is discovered, the analyst can then refer to the maps corresponding to the appropriate dates and times in the data cube to see precisely what is transpiring. The biggest objective being worked on is improving the change detection methodology employed. We currently use PCA-EM (Principal Component Analysis with Expectation Maximization), but we are currently focusing on implementing IR-MAD (Iteratively Reweighted Multivariate Alteration Detection) to be used in conjunction with PCA-EM in an effort to decrease false positivity and noise in the change maps we generate.

42 ENGINEERING↗
DOE OSTI2017

Toward a Visualization-Supported Workflow for Cyber Alert Management using Threat Models and Human-Centered Design

Cyber network analysts follow complex processes in their investigations of potential threats to their network. Much research is dedicated to providing automated tool support in the effort to make their tasks more efficient, accurate, and timely. This tool support comes in a variety of implementations from machine learning algorithms that monitor streams of data to visual analytic environments for exploring rich and noisy data sets. Cyber analysts, however, often speak of a need for tools which help them merge the data they already have and help them establish appropriate baselines against which to compare potential anomalies. Furthermore, existing threat models that cyber analysts regularly use to structure their investigation are not often leveraged in support tools. We report on our work with cyber analysts to understand they analytic process and how one such model, the MITRE ATT&CK Matrix [32], is used to structure their analytic thinking. We present our efforts to map specific data needed by analysts into the threat model to inform our eventual visualization designs. We examine data mapping for gaps where the threat model is under-supported by either data or tools. We discuss these gaps as potential design spaces for future research efforts. We also discuss the design of a prototype tool that combines machine-learning and visualization components to support cyber analysts working with this threat model.

Cyber Security↗
DOE OSTI2022

LANL Accident Analysis and Atmospheric Dispersion Modeling [Slides]

After completion of this course, the analyst will understand the differences between an unmitigated analysis and a mitigated analysis. After completion of this course, the analyst will know the key receptors that a radiological and hazardous chemical accident analysis must consider. After completion of this course, the analyst will understand how to calculate a radiological release source term and a toxic chemical release source term for various phenomenology. After completion of this course, the analyst will understand how to calculate a radiological and toxic chemical health insult to key receptors and compare to consequence thresholds. After completion of this course, the analyst will understand the role of atmospheric dispersion in radiological and toxic consequence calculations. After completion of this course, the analyst will understand atmospheric dispersion modeling and the inputs to and outputs from the MACCS/POSTMAX codes.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗
DOE OSTI2025

Yes, No, Maybe So: Human Factors Considerations for Fostering Calibrated Trust in Foundation Models Under Uncertainty

High-stakes analytical environments require analysts to evaluate evidence and generate conclusions to inform critical decisions often under conditions of uncertainty. Probabilistic decision-making based on incomplete or inaccurate information can reduce productivity, compromise national interests, and endanger public safety. Researchers are developing expert systems built on foundation models (FMs) to support analysts’ decision-making processes by enabling human-artificial intelligence (AI) teaming, in part through the quantification and expression of uncertainty information. As FMs continue to mature, it is imperative to correspondingly consider analysts’ needs for appropriately interpreting and using uncertainty information. However, prior research indicates that it remains unclear how analysts engage with FM-generated uncertainty information and the extent to which these interactions influence trust in, and reliance on, expert systems. We plan to review the state of the science and conduct an exploratory, qualitative study to (a) understand how properly communicated uncertainty can foster calibrated trust and appropriate reliance and (b) identify approaches for effectively conveying FM-generated uncertainty information during analytical workflows. We will administer semi-structured interviews with analysts from a specific high-stakes analytical environment to collect their current experiences with job-related uncertainty and their impressions when viewing FM-generated uncertainty information. During the interview protocol, participants will be presented with several different FM outputs and invited to discuss their thoughts and beliefs about the uncertainty information displayed. Participants may provide insights into how trust and reliance may be influenced by uncertainty. The results of this study will help us to better understand how analysts currently interpret and use uncertainty information. Our findings may inform human factors recommendations for effectively conveying uncertainty information to foster calibrated trust in, and appropriate reliance on, expert systems. Interaction designers and FM developers can use this knowledge to enhance human-AI teaming and ensure the responsible deployment of FM-based expert systems in analytical workflows.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS2000

Fault Tree in the Trenches, A Success Story

Getting caught up in the explanation of Fault Tree Analysis (FTA) minutiae is easy. In fact, most FTA literature tends to address FTA concepts and methodology. Yet there seems to be few articles addressing actual design changes resulting from the successful application of fault tree analysis. This paper demonstrates how fault tree analysis was used to identify and solve a potentially catastrophic mechanical problem at a rocket motor manufacturer. While developing the fault tree given in this example, the analyst was told by several organizations that the piece of equipment in question had been evaluated by several committees and organizations, and that the analyst was wasting his time. The fault tree/cutset analysis resulted in a joint-redesign of the control system by the tool engineering group and the fault tree analyst, as well as bragging rights for the analyst. (That the fault tree found problems where other engineering reviews had failed was not lost on the other engineering groups.) Even more interesting was that this was the analyst's first fault tree which further demonstrates how effective fault tree analysis can be in guiding (i.e., forcing) the analyst to take a methodical approach in evaluating complex systems.

Long, R. Allen↗
NASA NTRS2004

Advanced Technology Lifecycle Analysis System (ATLAS)

Developing credible mass and cost estimates for space exploration and development architectures require multidisciplinary analysis based on physics calculations, and parametric estimates derived from historical systems. Within the National Aeronautics and Space Administration (NASA), concurrent engineering environment (CEE) activities integrate discipline oriented analysis tools through a computer network and accumulate the results of a multidisciplinary analysis team via a centralized database or spreadsheet Each minute of a design and analysis study within a concurrent engineering environment is expensive due the size of the team and supporting equipment The Advanced Technology Lifecycle Analysis System (ATLAS) reduces the cost of architecture analysis by capturing the knowledge of discipline experts into system oriented spreadsheet models. A framework with a user interface presents a library of system models to an architecture analyst. The analyst selects models of launchers, in-space transportation systems, and excursion vehicles, as well as space and surface infrastructure such as propellant depots, habitats, and solar power satellites. After assembling the architecture from the selected models, the analyst can create a campaign comprised of missions spanning several years. The ATLAS controller passes analyst specified parameters to the models and data among the models. An integrator workbook calls a history based parametric analysis cost model to determine the costs. Also, the integrator estimates the flight rates, launched masses, and architecture benefits over the years of the campaign. An accumulator workbook presents the analytical results in a series of bar graphs. In no way does ATLAS compete with a CEE; instead, ATLAS complements a CEE by ensuring that the time of the experts is well spent Using ATLAS, an architecture analyst can perform technology sensitivity analysis, study many scenarios, and see the impact of design decisions. When the analyst is satisfied with the system configurations, technology portfolios, and deployment strategies, he or she can present the concepts to a team, which will conduct a detailed, discipline-oriented analysis within a CEE. An analog to this approach is the music industry where a songwriter creates the lyrics and music before entering a recording studio.

O'Neil, Daniel A.↗
DOE OSTI2023

Using Eye-Tracking to Quantify Reverse Engineering Expertise

Software reverse engineering (RE) requires analysts to closely read and make decisions about code. Little is known about what makes an analyst successful, making it difficult to train new analysts or design tools to augment existing ones. The goal of this project was to quantify the eye movement behaviors supporting RE and code comprehension more generally. We applied eye-tracking methods from the language comprehension literature to understand where analysts direct their attention over time when completing tasks (e.g., function identification, bug detection). Across three studies, we manipulated aspects of code hypothesized to impact comprehension (e.g., variable name meaningfulness, code complexity) and presentation methods (e.g., line-by-line, free viewing, gaze-contingent moving window) to understand effects on accuracy and gaze patterns. Results showed clear benefits of meaningful variable names, and effects of expertise on global and line-specific viewing patterns. Findings could inspire empirically-supported tool or analytic adaptations that help to reduce analyst workload.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS1991

Utilizing expert systems for satellite monitoring and control

Spacecraft analysts in the spacecraft control center for the Cosmic Background Explorer (COBE) satellite are currently utilizing a fault-isolation expert system developed to assist in the isolation and correction of faults in the communications link. This system, the communication link expert assistance resource (CLEAR), monitors real time spacecraft and ground systems performance parameters in search of configuration discrepancies and communications link problems. If such a discrepancy or problem is isolated, CLEAR alerts the analyst and provides advice on how to resolve the problem swiftly and effectively. The CLEAR system is the first real time expert system to be used in the operational environment of a satellite control center at the NASA Goddard Space Flight Center. Clear has not only demonstrated the utility and potential of an expert system in the demanding environment of a satellite control center, but also has revealed many of the pitfalls and deficiencies of development of expert systems. One of the lessons learned from this and other initial expert system projects is that prototypes can often be developed quite rapidly, but operational expert systems require considerable effort. Development is generally a slow, tedious process that typically requires the special skills of trained programmers. Due to the success of CLEAR and several other systems in the control center domain, a large number of expert systems will certainly be developed to support control center operations during the early 1990's. To facilitate the development of these systems, a project was initiated to develop an integrated, domain-specific tool, the generic spacecraft analyst assistent (GenSAA), that alows the spacecraft analysts to rapidly create simple expert systems themselves. By providing a highly graphical point-and-select method of system development, GenSAA allows the analyst to utilize and/or modify previously developed rule bases and system components; thus, facilitating software reuse and reducing development time and effort.

Hughes, Peter M.↗
NASA NTRS2017

Automation of POST Cases via External Optimizer and "Artificial p2" Calculation

During early conceptual design of complex systems, speed and accuracy are often at odds with one another. While many characteristics of the design are fluctuating rapidly during this phase there is nonetheless a need to acquire accurate data from which to down-select designs as these decisions will have a large impact upon program life-cycle cost. Therefore enabling the conceptual designer to produce accurate data in a timely manner is tantamount to program viability. For conceptual design of launch vehicles, trajectory analysis and optimization is a large hurdle. Tools such as the industry standard Program to Optimize Simulated Trajectories (POST) have traditionally required an expert in the loop for setting up inputs, running the program, and analyzing the output. The solution space for trajectory analysis is in general non-linear and multi-modal requiring an experienced analyst to weed out sub-optimal designs in pursuit of the global optimum. While an experienced analyst presented with a vehicle similar to one which they have already worked on can likely produce optimal performance figures in a timely manner, as soon as the "experienced" or "similar" adjectives are invalid the process can become lengthy. In addition, an experienced analyst working on a similar vehicle may go into the analysis with preconceived ideas about what the vehicle's trajectory should look like which can result in sub-optimal performance being recorded. Thus, in any case but the ideal either time or accuracy can be sacrificed. In the authors' previous work a tool called multiPOST was created which captures the heuristics of a human analyst over the process of executing trajectory analysis with POST. However without the instincts of a human in the loop, this method relied upon Monte Carlo simulation to find successful trajectories. Overall the method has mixed results, and in the context of optimizing multiple vehicles it is inefficient in comparison to the method presented POST's internal optimizer functions like any other gradient-based optimizer. It has a specified variable to optimize whose value is represented as optval, a set of dependent constraints to meet with associated forms and tolerances whose value is represented as p2, and a set of independent variables known as the u-vector to modify in pursuit of optimality. Each of these quantities are calculated or manipulated at a certain phase within the trajectory. The optimizer is further constrained by the requirement that the input u-vector must result in a trajectory which proceeds through each of the prescribed events in the input file. For example, if the input u-vector causes the vehicle to crash before it can achieve the orbital parameters required for a parking orbit, then the run will fail without engaging the optimizer, and a p2 value of exactly zero is returned. This poses a problem, as this "non-connecting" region of the u-vector space is far larger than the "connecting" region which returns a non-zero value of p2 and can be worked on by the internal optimizer. Finding this connecting region and more specifically the global optimum within this region has traditionally required the use of an expert analyst.

Dees, Patrick D.↗
DOE OSTI2013

Redeye: A Digital Library for Forensic Document Triage

Forensic document analysis has become an important aspect of investigation of many different kinds of crimes from money laundering to fraud and from cybercrime to smuggling. The current workflow for analysts includes powerful tools, such as Palantir and Analyst s Notebook, for moving from evidence to actionable intelligence and tools for finding documents among the millions of files on a hard disk, such as FTK. However, the analysts often leave the process of sorting through collections of seized documents to filter out the noise from the actual evidence to a highly labor-intensive manual effort. This paper presents the Redeye Analysis Workbench, a tool to help analysts move from manual sorting of a collection of documents to performing intelligent document triage over a digital library. We will discuss the tools and techniques we build upon in addition to an in-depth discussion of our tool and how it addresses two major use cases we observed analysts performing. Finally, we also include a new layout algorithm for radial graphs that is used to visualize clusters of documents in our system.

Bogen, Paul Logasa↗
DOE OSTI2019

SAFETY ANALYSIS REPORT FOR PACKAGING SHIELDING & NUCLEAR CRITICALITY SAFETY COURSES DEVELOPED AND CONDUCTED BY OAK RIDGE NATIONAL LABORATORY

The US Department of Energy (DOE) Packaging Certification Program, Office of Packaging and Transportation, is offering Safety Analysis Report for Packaging (SARP) Shielding and Nuclear Criticality Safety (NCS) courses for SARP generalists and analysts. The SARP generalist course is designed for project managers, supervisors, NCS/shielding subject matter experts (SME), and SMEs in non-NCS/shielding technical areas (e.g., structural, thermal, package design) who need to improve their understanding of how NCS/shielding analyses fit into the broader spectrum of SARP documentation. The generalist course provides an overview of the regulations and guidelines for the criticality and shielding analyses included in a SARP, and it illustrates how the NCS/shielding chapters integrate with the other parts of the SARP. After course material is presented, students in the generalist course will review an actual SARP document to examine key elements of the shielding and criticality analyses. The analyst course provides detailed instructions on the radioactive material package shielding analyses and NCS evaluation fundamentals needed by analysts/practitioners, including safety analysts and/or technical reviewers, to prepare and/or review technical analyses for SARP documentation. The analyst course also provides an overview of regulations and guidelines, in addition to detailed in-class exercises associated with package shielding and NCS analyses. Analysis teams will be faced with staged SARP examples in which several important decision processes in the generation of a SARP will be demonstrated and discussed. Both courses are offered at Oak Ridge National Laboratory, and for interested students, the analyst course is offered for one hour of graduate credit (course ME-690) from the University of Nevada at Reno. An overview of the course will be provided to ensure that the NCS community is aware of the course and its intent to support the shielding and NCS aspects of packaging and transportation analyses for SARP activities.

Bowen, Douglas G.↗
DOE OSTI2019

The Next Generation System Analysis Model: Capabilities for Simulating a Waste Management System - 19131

The U.S. Department of Energy (DOE) Office of Nuclear Energy Office of Spent Fuel and Waste Disposition's Integrated Waste Management System (IWMS) program is applying integrated waste management system architecture analysis, system engineering, and decision analysis principles to inform potential future decisions regarding potential nuclear waste management system architectures. Architecture analyses of the IWMS are being conducted to support the future deployment of a comprehensive system for managing nuclear waste that considers all major aspects of the back end of the nuclear fuel cycle (i.e., transportation, storage, and disposal). The system architecture analyses consider alternative disposition pathways for commercial spent nuclear fuel (SNF) being stored dry in at-reactor Independent Spent Fuel Storage Installations (ISFSIs) or wet in spent-fuel pools. One method for evaluating these alternatives is simulating the movement of SNF throughout the system components. Successfully simulating a system on the scale of a national waste management system requires - accurate baseline information (e.g., historic fuel discharges from reactors and dry storage loadings at reactor sites), - modeling all logistic interactions between components of the proposed system, and - tracking and reporting all relevant details regarding the simulation's execution. The Next Generation System Analysis Model (NGSAM) is an agent-based simulation software tool designed for the express purpose of modeling the IWMS. It is based on the Performance Analysis Tool, a simulation platform that has been used to model logistics for the Department of Defense and the Federal Emergency Management Agency. Using existing, government-owned software allows NGSAM developers to modify the software at the level of source code and to develop flexible usability and modeling solutions for developing and running simulations of the IWMS. NGSAM imports data from the Oak Ridge National Laboratory (ORNL) Unified Database (e.g., historic assembly information, thermal profiles for assembly heat, at-reactor dry storage loadings) to ensure that the simulation initializes with a realistic representation of the state of commercial SNF in the United States. NGSAM is designed with the flexibility to model and track the behavior of virtually any IWMS architecture assumptions or design considerations. For example, NGSAM can model multiple interim storage facilities or repositories or none at all. Similarly, all assumptions regarding shipment schedules, operating schedules, and reactor operation assumptions can be modeled and adjusted in NGSAM with relative ease. The development of NGSAM was targeted to waste management system analysts as the primary users. With that in mind, analysts from ORNL, Sandia National Laboratories (SNL), and Argonne National Laboratory (ANL) were intimately involved in the development process. That involvement extended beyond merely defining requirements. System analysts have been involved with design decisions and software benchmarking and tests, and they have provided subject-matter expertise where necessary. The result is a high degree of interaction between professional software developers and system analysts, with a multitude of benefits to both groups. This paper explores some of the underlying methodology of the agile software development process used in the creation of NGSAM. It illustrates the capabilities of NGSAM. Some example use cases are discussed as well, with relevant results providing context for how NGSAM is used by system analysts to gain a greater understanding of alternative waste management system architectures. This is a technical paper that does not take into account the contractual limitations under the Standard Contract for Disposal of Spent Nuclear Fuel and/or High-Level Radioactive Waste (Standard Contract) (10 CFR Part 961). Under the provisions of the Standard Contract, DOE does not consider spent nuclear fuel in multi-assembly canisters to be an acceptable waste form, absent a mutually agreed-to contract amendment. To the extent discussions or recommendations in this paper conflict with the provisions of the Standard Contract, the Standard Contract provisions prevail. (authors)

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗
DOE OSTI2020

An Assessment of the Usability of Machine Learning Based Tools for the Security Operations Center

Gartner, a large research and advisory company, anticipates that by 2024 80% of security operation centers (SOCs) will use machine learning (ML) based solutions to enhance their operations. 1 1 https://www.ciodive.com/news/how-data-science-tools-can-lighten-the-load-for-cybersecurity-teams/572209/ In light of such widespread adoption, it is vital for the research community to identify and address usability concerns. This work presents the results of the first in situ usability assessment of ML-based tools. With the support of the US Navy, we leveraged the national cyber range-a large, air-gapped cyber testbed equipped with state-of-the-art network and user emulation capabilities-to study six US Naval SOC analysts' usage of two tools. Our analysis identified several serious usability issues, including multiple violations of established usability heuristics for user interface design. We also discovered that analysts lacked a clear mental model of how these tools generate scores, resulting in mistrust a and/or misuse of the tools themselves. Surprisingly, we found no correlation between analysts' level of education or years of experience and their performance with either tool, suggesting that other factors such as prior background knowledge or personality play a significant role in ML-based tool usage. Our findings demonstrate that ML-based security tool vendors must put a renewed focus on working with analysts, both experienced and inexperienced, to ensure that their systems are usable and useful in real-world security operations settings.

Oesch, T↗
DOE OSTI2020

VADER: A Tool for Criticality Safety Validation

The purpose of criticality safety is to prevent any inadvertent criticality from occurring during the handling or storage of fissile material. Calculations are frequently used to demonstrate that a sufficient subcritical margin exists. Validation is a key aspect of the evaluation process, establishing the suitability, accuracy, and associated uncertainty of the computational method and data to be used for the intended application. The validation process is performed by comparing the results of critical experiments with the calculated results from models of the experiments using the computational method to be validated. Laboratory critical experiments are controlled systems that achieve a k eff of approximately 1 in order to investigate the parameters at which such a critical condition is achieved. The validation parameters that are traditionally applied to safety analysis calculations are the bias and the bias uncertainty . The bias is the deviation of the average k eff of the validation suite from unity. The bias uncertainty accounts for the statistical uncertainty in the bias based on the standard deviation, sample size, and distribution of k eff values of the validation suite. The values of bias and bias uncertainty ensure that the systems predicted to be subcritical by the computational method will indeed be subcritical. The bias and bias uncertainty are often combined to determine an upper subcritical limit (USL) or computational margin that can then be applied to safety analysis calculations. Many methods have been developed by different organizations to calculate the bias and bias uncertainty for various types of criticality analyses. Each of these methods typically requires that the validity of various underpinning statistical assumptions be confirmed to demonstrate that the method is appropriate for the analysis of a given validation suite. An example of the validation decision making flow is shown in Fig.1. As shown in Fig. 1, the analyst performing the validation fits a trend line to the data and performs a test to determine if the trend was a statistically better representation of the data than if it were treated as an uncorrelated sample. If the trend line is a better representation of the data, then the analyst uses any one of a number of trending techniques to determine the bias and bias uncertainty. If a trend is not an appropriate representation of the data, then the analyst proceeds to perform a normality assessment for the data. If the normal assumption can be shown to be acceptable, then the analyst calculates the bias and bias uncertainty with the parametric technique. If the assumption of normality cannot be justified, then the nonparametric technique is used. Once the decision flow has been followed and the appropriate technique has been selected, the bias and bias uncertainty is typically combined with an administrative margin to determine a USL below which calculated values of k eff for safety analysis models can be considered subcritical. The calculations used in each decision are often performed with spreadsheets or with small programs available at various sites performing criticality analyses. Expertise in understanding and interpreting the results must be maintained to perform these calculations. This can often be an error-prone process. Oak Ridge National Laboratory (ORNL) is currently developing the Validation and Data Evaluation Resource (VADER) to simplify and automate the criticality safety validation process and to provide a software quality assurance pedigree to the calculational methods used. This paper discusses the use of the Fulcrum user interface with VADER, the anticipated initial capabilities of VADER to perform validation analyses, and the output from the code.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2014

Hybrid methods for cybersecurity analysis :

Early 2010 saw a signi cant change in adversarial techniques aimed at network intrusion: a shift from malware delivered via email attachments toward the use of hidden, embedded hyperlinks to initiate sequences of downloads and interactions with web sites and network servers containing malicious software. Enterprise security groups were well poised and experienced in defending the former attacks, but the new types of attacks were larger in number, more challenging to detect, dynamic in nature, and required the development of new technologies and analytic capabilities. The Hybrid LDRD project was aimed at delivering new capabilities in large-scale data modeling and analysis to enterprise security operators and analysts and understanding the challenges of detection and prevention of emerging cybersecurity threats. Leveraging previous LDRD research e orts and capabilities in large-scale relational data analysis, large-scale discrete data analysis and visualization, and streaming data analysis, new modeling and analysis capabilities were quickly brought to bear on the problems in email phishing and spear phishing attacks in the Sandia enterprise security operational groups at the onset of the Hybrid project. As part of this project, a software development and deployment framework was created within the security analyst work ow tool sets to facilitate the delivery and testing of new capabilities as they became available, and machine learning algorithms were developed to address the challenge of dynamic threats. Furthermore, researchers from the Hybrid project were embedded in the security analyst groups for almost a full year, engaged in daily operational activities and routines, creating an atmosphere of trust and collaboration between the researchers and security personnel. The Hybrid project has altered the way that research ideas can be incorporated into the production environments of Sandias enterprise security groups, reducing time to deployment from months and years to hours and days for the application of new modeling and analysis capabilities to emerging threats. The development and deployment framework has been generalized into the Hybrid Framework and incor- porated into several LDRD, WFO, and DOE/CSL projects and proposals. And most importantly, the Hybrid project has provided Sandia security analysts with new, scalable, extensible analytic capabilities that have resulted in alerts not detectable using their previous work ow tool sets.

Dunlavy, Daniel M.↗
DOE OSTI2016

Streaming Visual Analytics Workshop Report

How can we best enable users to understand complex emerging events and make appropriate assessments from streaming data? This was the central question addressed at a three-day workshop on streaming visual analytics. This workshop was organized by Pacific Northwest National Laboratory for a government sponsor. It brought together forty researchers and subject matter experts from government, industry, and academia. This report summarizes the outcomes from that workshop. It describes elements of the vision for a streaming visual analytic environment and set of important research directions needed to achieve this vision. Streaming data analysis is in many ways the analysis and understanding of change. However, current visual analytics systems usually focus on static data collections, meaning that dynamically changing conditions are not appropriately addressed. The envisioned mixed-initiative streaming visual analytics environment creates a collaboration between the analyst and the system to support the analysis process. It raises the level of discourse from low-level data records to higher-level concepts. The system supports the analyst’s rapid orientation and reorientation as situations change. It provides an environment to support the analyst’s critical thinking. It infers tasks and interests based on the analyst’s interactions. The system works as both an assistant and a devil’s advocate, finding relevant data and alerts as well as considering alternative hypotheses. Finally, the system supports sharing of findings with others. Making such an environment a reality requires research in several areas. The workshop discussions focused on four broad areas: support for critical thinking, visual representation of change, mixed-initiative analysis, and the use of narratives for analysis and communication.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗