Search NASA⌕ Search

Engineering topics

Nichols, Jeff

Publications and source records attributed to Nichols, Jeff.

AI ATAC 1: An Evaluation of Prominent Commercial Malware Detectors

This work presents an evaluation of six prominent commercial endpoint malware detectors, a network malware detector, and a file-conviction algorithm from a cyber technology vendor. The evaluation was administered as the first of the Artificial I ntelligence Applications t o Autonomous Cybersecurity (AI ATAC) prize challenges, funded by / completed in service of the US Navy. The experiment employed 100K files (50/50% benign/malicious) with a stratified distribution of file types, including ~1K zero-day program executables (increasing experiment size two orders of magnitude over previous work). We present an evaluation process of delivering a file to a fresh virtual machine donning the detection technology, waiting 90s to allow static detection, then executing the file and waiting another period for dynamic detection; this allows greater fidelity in the observational data than previous experiments, in particular, resource and time-to-detection statistics. To execute all 800K trials (100K files × 8 tools), a software framework is designed to choreograph the experiment into an automated, time-synced, and reproducible workflow with substantial parallelization. Software with base classes for this framework are provided. A cost-benefit model was configured to integrate the tools’ detection statistics into a comparable quantity by simulating costs of use. This provides a ranking methodology for cyber competitions and a lens for reasoning about the varied statistical results. The results provide insights on state of commercial malware detection.

Bridges, Robert↗

Assembling a Cyber Range to Evaluate Artificial Intelligence / Machine Learning (AI/ML) Security Tools

In this case study, we will describe the design and assembly of a cyber security test range we have built at Oak Ridge National Laboratory in Oak Ridge, TN, USA. The range is designed to provide a flexible environment to evaluate cyber security tools—particularly those involving AI/ML—in a way that provides realistic environments and where we can control the experiments to determine the strengths and weaknesses of the tools. We have designed in the ability to repeat the evaluations, so additional tools can be evaluated and compared at a later time. The system is one that can be scaled up or down for experiment sizes. At the time of the conference we will have completed two full-scale, national, government challenges on this range. These challenges are evaluating the performance and operating costs for AI/ML-based cyber security tools for application into large, government-sized environments. These evaluations will be described, in order to provide motivation and context for various design decisions and adaptations we have made. The first challenge measured end-point security tools against 100K malware samples chosen across a range of types. The second is network detection of attempted penetration and exploitations with varying levels of covertness in a high-volume, business network. The scale of each of these challenges is requiring us to create automation systems to repeat the experiments identically for each tool. Preventing there being easy signs of malicious activity for the AI/ML tools to focus on has been a particularly interesting and challenging aspect of designing and executing these challenge events. After the events, the range continues to be used for other research such as adversarial machine learning where the repeatability, scale, and automation required for the national challenge events become essential elements for research.

Nichols, Jeff↗

AI for Science: Report on the Department of Energy (DOE) Town Halls on Artificial Intelligence (AI) for Science

The report documents the DOE Town Halls held during 2019 at Argonne National Laboratory, Oak Ridge National Laboratory, Lawrence Berkeley National Laboratory, and in Washington, DC. From July to October 2019, the Argonne, Oak Ridge, and Berkeley National Laboratories hosted a series of four town hall meetings attended by more than 1,000 U.S. scientists and engineers. The goal of the town hall series was to examine scientific opportunities in the areas of artificial intelligence (AI), Big Data, and high-performance computing (HPC) in the next decade, and to capture the big ideas, grand challenges, and next steps to realizing these opportunities. In this report and in the Department of Energy (DOE) laboratory community, we use the term “AI for Science” to broadly represent the next generation of methods and scientific opportunities in computing, including the development and application of AI methods (e.g., machine learning, deep learning, statistical methods, data analytics, automated control, and related areas) to build models from data and to use these models alone or in conjunction with simulation and scalable computing to advance scientific research. The AI for Science town hall discussions focused on capturing the transformational uses of AI that employ HPC and/or data analysis, leveraging data sets from HPC simulations or instruments and user facilities, and addressing scientific challenges unique to DOE user facilities and the agency’s wide-ranging fundamental and applied science enterprise.

36 MATERIALS SCIENCE↗

Towards Malware Detection via CPU Power Consumption: Data Collection Design and Analytics

This paper presents an experimental design and data analytics approach aimed at power-based malware detection on general-purpose computers. Leveraging the fact that malware executions must consume power, we explore the postulate that malware can be accurately detected via power data analytics. Our experimental design and implementation allow for programmatic collection of CPU power profiles for fixed tasks during uninfected and infected states using five different rootkits. To characterize the power consumption profiles, we use both simple statistical and novel, sophisticated features. We test a one-class anomaly detection ensemble (that baselines non-infected power profiles) and several kernel-based SVM classifiers (that train on both uninfected and infected profiles) in detecting previously unseen malware and clean profiles. The anomaly detection system exhibits perfect detection when using all features and tasks, with smaller false detection rate than the supervised classifiers. The primary contribution is the proof of concept that baselining power of fixed tasks can provide accurate detection of rootkits. Moreover, our treatment presents engineering hurdles needed for experimentation and allows analysis of each statistical feature individually. This work appears to be the first step towards a viable power-based detection capability for general-purpose computers, and presents next steps toward this goal.

Bridges, Robert↗

Announcing Supercomputer Summit

Summit is the next leap in leadership-class computing systems for open science. With Summit we will be able to address, with greater complexity and higher fidelity, questions concerning who we are, our place on earth, and in our universe. Summit will deliver more than five times the computational performance of Titan’s 18,688 nodes, using only approximately 3,400 nodes when it arrives in 2017. Like Titan, Summit will have a hybrid architecture, and each node will contain multiple IBM POWER9 CPUs and NVIDIA Volta GPUs all connected together with NVIDIA’s high-speed NVLink. Each node will have over half a terabyte of coherent memory (high bandwidth memory + DDR4) addressable by all CPUs and GPUs plus 800GB of non-volatile RAM that can be used as a burst buffer or as extended memory. To provide a high rate of I/O throughput, the nodes will be connected in a non-blocking fat-tree using a dual-rail Mellanox EDR InfiniBand interconnect. Upon completion, Summit will allow researchers in all fields of science unprecedented access to solving some of the world’s most pressing challenges.

97 MATHEMATICS AND COMPUTING↗

An integrative modeling framework to evaluate the productivity and sustainability of biofuel crop production systems

The potential expansion of biofuel production raises food, energy, and environmental challenges that require careful assessment of the impact of biofuel production on greenhouse gas (GHG) emissions, soil erosion, nutrient loading, and water quality. In this study, we describe a spatially explicit integrative modeling framework (SEIMF) to understand and quantify the environmental impacts of different biomass cropping systems. This SEIMF consists of three major components: (1) a geographic information system (GIS)-based data analysis system to define spatial modeling units with resolution of 56 m to address spatial variability, (2) the biophysical and biogeochemical model Environmental Policy Integrated Climate (EPIC) applied in a spatially-explicit way to predict biomass yield, GHG emissions, and other environmental impacts of different biofuel crops production systems, and (3) an evolutionary multiobjective optimization algorithm for exploring the trade-offs between biofuel energy production and unintended ecosystem-service responses. Simple examples illustrate the major functions of the SEIMF when applied to a nine-county Regional Intensive Modeling Area (RIMA) in SW Michigan to (1) simulate biofuel crop production, (2) compare impacts of management practices and local ecosystem settings, and (3) optimize the spatial configuration of different biofuel production systems by balancing energy production and other ecosystem-service variables. Potential applications of the SEIMF to support life cycle analysis and provide information on biodiversity evaluation and marginal-land identification are also discussed. The SEIMF developed in this study is expected to provide a useful tool for scientists and decision makers to understand sustainability issues associated with the production of biofuels at local, regional, and national scales.

bioenergy↗