Search NASA⌕ Search

SEARCH · Search NASA

Results for “Practitioner”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 757 records · Page 42

Precursor Analysis Report: Industroyer2 and Wiper Malware Targeting Ukrainian Energy Provider 2022

The Industroyer2 and Wiper Malware Targeting Ukrainian Energy Provider 2022 Precursor Analysis Report leverages publicly available information about the Industroyer2 cyber attack and catalogs anomalous observables for each technique employed in the attack. This analysis is based upon the methodology of the Cybersecurity for the Operational Technology Environment (CyOTE) program. An adversary attempted to cause a blackout in Ukraine in April 2022 by using the Industroyer2 malware against a regional Ukrainian energy provider. The adversary targeted eight high-voltage electrical substations and utilized the malware in tandem with disk wipers for Windows, Linux, and Solaris operating systems in an attempt to make response and recovery efforts more difficult. The adversary reused a piece of the original Industroyer malware designed to open circuit breakers and de-energize target substations. The adversary gained initial access to the victim’s enterprise network through unknown means in February 2022 and was able to perform reconnaissance, pivot to the operations network, and reside in the system for at least 51 days. This gave the adversary a detailed understanding of the environment and allowed them to customize the Industroyer2 malware to the victim’s operations network. However, defenders detected and stopped the attack before the adversary could achieve their intended impact. Had the Industroyer2 attack been successful, it could have caused a blackout for more than two million people during the early stages of Russia’s invasion of Ukraine. Researchers and analysts identified 22 unique techniques (used in a sequence of 31 steps) utilized during the attack with a total of 297 observables using MITRE ATT&CK® for Industrial Control Systems. The CyOTE program assesses observables accompanying techniques used prior to the triggering event to identify opportunities to detect malicious activity. If observables accompanying the attack techniques are perceived and investigated prior to the triggering event, earlier comprehension of malicious activity can take place. Twenty-three of the identified techniques used during the Industroyer2 cyber attack were precursors to the triggering event. Analysis identified 224 observables associated with these precursor techniques, 122 of which were assessed to have an increased likelihood of being perceived in the 51 days preceding the triggering event. The response and comprehension time could have been reduced if the observables had been identified earlier. The information gathered in this report contributes to a library of observables tied to a repository of artifacts, data sources, and technique detection references for practitioners and developers to support the comprehension of indicators of attack. Asset owners and operators can use these products if they experience similar observables or to prepare for comparable scenarios.

45 MILITARY TECHNOLOGY, WEAPONRY, AND NATIONAL DEF↗

Precursor Analysis Report: Industroyer Targeting Ukraine Electric Power Transport Utility (Ukrenergo) 2016

The Industroyer Targeting Ukraine Electric Power Transport Utility (Ukrenergo) 2016 Precursor Analysis Report leverages publicly available information about the December 2016 cyber attack against the Ukrainian Ukrenergo electric transmission utility and catalogs anomalous observables for each technique employed in the attack. This analysis is based upon the methodology of the Cybersecurity for the Operational Technology Environment (CyOTE) program. Industroyer is a modular malware framework designed to deploy several Industrial Control System (ICS) protocol-specific attack payloads to disrupt electricity distribution. Adversaries deployed Industroyer within the target network on a Microsoft Windows endpoint capable of directly manipulating or communicating with ICS. Industroyer abuses the functionality of a targeted ICS’s legitimate control system to achieve its intended impact. Adversaries likely first gained access to Ukrenergo enterprise networks in early 2016 after a successful spearphishing campaign against organizations in the electric power sector. Adversaries then began capturing credentials beginning on 1 December 2016. This allowed access to the ICS environment at the Pivnichna electric transmission substation outside Kyiv through a device dual-homed on the Information Technology (IT) and ICS networks. Adversaries conducted discovery, targeting, and access to this device using information and previously captured credentials from compromised enterprise IT machines. Finally, the adversaries deployed and launched the Industroyer malware just before midnight on 17 December. By midnight, Ukrenergo had lost control of a targeted substation, resulting in electric power outages for over an hour in the city of Kyiv and the Kyiv region. Researchers and analysts identified 31 unique techniques (used in a sequence of 33 steps) utilized during the attack with a total of 846 observables using MITRE ATT&CK® for Industrial Control Systems. The CyOTE program assesses observables accompanying techniques used prior to the triggering event to identify opportunities to detect malicious activity. If observables accompanying the attack techniques are perceived and investigated prior to the triggering event, earlier comprehension of malicious activity can take place. Twenty-nine of the identified techniques used during the Industroyer cyber attack were precursors to the triggering event. Analysis identified 548 observables associated with these precursor techniques, 353 of which were assessed to have an increased likelihood of being perceived in the 300 days preceding the triggering event. The response and comprehension time could have been reduced if the observables had been identified earlier. The information gathered in this report contributes to a library of observables tied to a repository of artifacts, data sources, and technique detection references for practitioners and developers to support the comprehension of indicators of attack. Asset owners and operators can use these products if they experience similar observables or to prepare for comparable scenarios.

45 MILITARY TECHNOLOGY, WEAPONRY, AND NATIONAL DEF↗

Precursor Analysis Report: Conti Ransomware Attack on the Health Service Executive of Ireland 2021

The Conti Ransomware Attack on the Health Service Executive (HSE) of Ireland 2021 Precursor Analysis Report leverages publicly available information about the attack and catalogs anomalous observables for each technique employed by the adversary. This analysis is based upon the methodology of the Cybersecurity for the Operational Technology Environment (CyOTE) program. The HSE provides public healthcare corporate services and operational services throughout Ireland, with critical functions including the acute national ambulance service, acute hospital service, and community healthcare service. On 14 May 2021, Conti ransomware encrypted 80 percent of the HSE’s Information Technology (IT) infrastructure across corporate, hospital, community, and electronic health record services. Conti is a ransomware-as-a-service operation that encrypts local files, uses double extortion against victims, and is facilitated by many intrusion tools. The attack forced the HSE to shut down its entire IT infrastructure to contain the ransomware, forcing employees to revert to pen and paper recordkeeping and leading to the cancellation of many appointments and procedures. The adversary also exfiltrated 700 GB of data, compromising the confidentiality of patients’ protected health information. Had the adversary targeted the COVID-19 cloud systems or operational technology assets, such as Internet of Medical Things medical devices or smart building management systems, the impact of the attack would almost certainly have been far more severe. Researchers and analysts identified 21 unique techniques (used in a sequence of 23 steps) likely utilized during the attack with a total of 1,185 observables using MITRE ATT&CK® for Industrial Control Systems. The CyOTE program assesses observables accompanying techniques used prior to the triggering event to identify opportunities to detect malicious activity. If observables accompanying the attack techniques are perceived and investigated prior to the triggering event, earlier comprehension of malicious activity can take place. Twenty-one of the identified techniques used during the attack on the HSE were precursors to the triggering event. Analysis identified 1,086 observables associated with these precursor techniques, 850 of which were assessed to have an increased likelihood of being perceived in the 57 days preceding the triggering event. The response and comprehension time could have been reduced if the observables had been identified earlier. The information gathered in this report contributes to a library of observables tied to a repository of artifacts, data sources, and technique detection references for practitioners and developers to support the comprehension of indicators of attack. Asset owners and operators can use these products if they experience similar observables or to prepare for comparable scenarios.

45 MILITARY TECHNOLOGY, WEAPONRY, AND NATIONAL DEF↗

Precursor Analysis Report: Cyber Attack on Thyssenkrupp Blast Furnace 2014

The Cyber Attack on Thyssenkrupp Blast Furnace 2014 Precursor Analysis Report leverages publicly available information about the Thyssenkrupp Steel Mill cyber attack and catalogs anomalous observables for each technique employed in the attack. This analysis is based upon the methodology of the Cybersecurity for the Operational Technology Environment (CyOTE) program. In December 2014, the German Government’s Federal Office for Information Security (BSI) released a report detailing a cyber attack on a German steel mill that occurred earlier that year, though exact dates and details of the attack were not revealed. While the report did not specify the name of the company, multiple sources identified the victim as one of Europe’s largest steel manufacturers, Thyssenkrupp AG. Further, Thyssenkrupp announced on 16 May of that year that Europe’s largest blast furnace, “Schwelgern 2,” located at its facility in Duisburg, Germany, would be offline for several weeks for repairs and upgrades, suggesting Schwelgern 2 was likely the target of the attack. The attack began in early 2014, when adversaries infiltrated the victim steel mill’s Information Technology (IT) network via a spearphishing campaign, then worked their way into the Operational Technology (OT) environment, where they executed software that caused denial of service, denial of control, and eventually a loss of control. This led to the blast furnace shutting down without proper safety procedures, resulting in catastrophic physical damage. No lives were lost in the incident, but ThyssenKrupp suffered $4 million in damage to the blast furnace and an additional $6 million in lost revenue. The adversaries required specialized knowledge and expertise in steel production, which enabled them to compromise a variety of internal systems and components across both IT and OT networks. The attack also demonstrated detailed knowledge of the industrial control systems (ICS) and production processes being used. This combination resulted in one of the earliest known publicly reported cybersecurity incidents resulting in physical damage to ICS equipment. Researchers and analysts identified 19 unique techniques (used in a sequence of 20 steps) utilized during the attack with a total of 454 observables using MITRE ATT&CK® for Industrial Control Systems. The CyOTE program assesses observables accompanying techniques used prior to the triggering event to identify opportunities to detect malicious activity. If observables accompanying the attack techniques are perceived and investigated prior to the triggering event, earlier comprehension of malicious activity can take place. Fifteen of the identified techniques used during the Thyssenkrupp cyber attack were precursors to the triggering event. Analysis identified 369 observables associated with these precursor techniques, 316 of which were assessed to have an increased likelihood of being perceived in the 120 days preceding the triggering event. The response and comprehension time could have been reduced if the observables had been identified earlier. The information gathered in this report contributes to a library of observables tied to a repository of artifacts, data sources, and technique detection references for practitioners and developers to support the comprehension of indicators of attack. Asset owners and operators can use these products if they experience similar observables or to prepare for comparable scenarios.

45 MILITARY TECHNOLOGY, WEAPONRY, AND NATIONAL DEF↗

Precursor Analysis Report: Doppelpaymer Ransomware Attack on Petroleos Mexicanos (PEMEX) 2019

The DoppelPaymer Ransomware Attack on Petroleos Mexicanos (PEMEX) 2019 Precursor Analysis Report leverages publicly available information about the PEMEX cyber attack and catalogs anomalous observables for each technique employed in the attack. This analysis is based upon the methodology of the Cybersecurity for the Operational Technology Environment (CyOTE) program. The 2019 DoppelPaymer ransomware attack on PEMEX, Mexico’s nationalized petroleum corporation, highlights a unique threat that ransomware and cybercriminal extortion poses to Operational Technology (OT) environments in critical infrastructure. The incident began with an employee downloading commodity malware that allowed adversaries to gain initial access to PEMEX’s enterprise environment. After conducting privilege escalation, tool ingress, and data exfiltration, the adversaries deployed DoppelPaymer ransomware throughout the PEMEX enterprise environment, resulting in the company having to take dozens of systems offline for at least several days. Although PEMEX stated that their operations were not affected, the data exfiltrated from PEMEX was made available for download on DoppelPaymer’s leak site, as well as on other illicit criminal forums. This stolen data included not only company information, but also sensitive OT-specific configuration data. This incident showcases how cybercriminal exfiltration and posting of sensitive OT architecture documentation can pose security concerns for the targeted organization for years due to the long lifespan of OT assets and architectures. Researchers and analysts identified 18 unique techniques utilized during the attack with a total of 190 observables using MITRE ATT&CK® for Industrial Control Systems. The CyOTE program assesses observables accompanying techniques used prior to the triggering event to identify opportunities to detect malicious activity. If observables accompanying the attack techniques are perceived and investigated prior to the triggering event, earlier comprehension of malicious activity can take place. Fifteen of the identified techniques used during the DoppelPaymer ransomware attack were precursors to the triggering event. Analysis identified 163 observables associated with these precursor techniques, 34 of which were assessed to have an increased likelihood of being perceived in the 60 days preceding the triggering event. The response and comprehension time could have been reduced if the observables had been identified earlier. The information gathered in this report contributes to a library of observables tied to a repository of artifacts, data sources, and technique detection references for practitioners and developers to support the comprehension of indicators of attack. Asset owners and operators can use these products if they experience similar observables or to prepare for comparable scenarios.

45 MILITARY TECHNOLOGY, WEAPONRY, AND NATIONAL DEF↗

Precursor Analysis Report: Remote Access Attack on Oldsmar Water Treatment Facility 2021

The Remote Access Attack on Oldsmar Water Treatment Facility 2021 Precursor Analysis Report leverages publicly available information about the Oldsmar cyber attack and catalogs anomalous observables for each technique employed in the attack. This analysis is based upon the methodology of the Cybersecurity for the Operational Technology Environment (CyOTE) program. On 5 February 2021, an adversary gained unauthorized remote access to Bruce T. Haddock Water Treatment Plant in Oldsmar, Florida, which provides treated water to 15,000 customers. The adversary accessed the facility’s Supervisory Control and Data Acquisition (SCADA) workstation and human machine interface (HMI) to change the chemical concentration of sodium hydroxide, commonly referred to as lye and used to regulate acidity levels, from 100 parts per million (PPM) to 11,100 PPM. The chemical was raised to lethal levels that if ingested could lead to serious soft tissue damage, burns, or even death. The facility, however, had redundancies and alarms in place to alert personnel of dangerous chemical levels, and facility officials stated it would have taken 24 to 36 hours for the chemical changes to affect the water supply. Researchers and analysts identified six unique techniques utilized during the attack with a total of 23 observables using MITRE ATT&CK® for Industrial Control Systems. The CyOTE program assesses observables accompanying techniques used prior to the triggering event to identify opportunities to detect malicious activity. If observables accompanying the attack techniques are perceived and investigated prior to the triggering event, earlier comprehension of malicious activity can take place. Four of the identified techniques used during the Oldsmar cyber attack were precursors to the triggering event. Analysis identified 21 observables associated with these precursor techniques, 20 of which were assessed to have an increased likelihood of being perceived in the minutes preceding the triggering event. The response and comprehension time could have been reduced if the observables had been identified earlier. The information gathered in this report contributes to a library of observables tied to a repository of artifacts, data sources, and technique detection references for practitioners and developers to support the comprehension of indicators of attack. Asset owners and operators can use these products if they experience similar observables or to prepare for comparable scenarios.

45 MILITARY TECHNOLOGY, WEAPONRY, AND NATIONAL DEF↗

What Is the Agent Doing? Visualizing Agentic AI Querying Workflows

We explore how visualizations can help users understand what an AI agent is doing as it builds and runs queries over data. As part of the LinkQ system, a natural language interface for querying knowledge graphs with a large language model (LLM), we designed two complementary views: A State Diagram that shows where the agent is within a larger workflow, and a Live Action Display that gives real-time updates about the agent's current task. In a study with 14 practitioners, we found that these visuals helped participants build stronger mental models of the agent's behavior while also increasing their confidence in the system. However, we also observed that users sometimes trusted incorrect outputs simply because the agent appeared to be doing the "right" thing. Our findings point to both the value and risk of visualizing agent behavior in interactive AI systems.

97 MATHEMATICS AND COMPUTING↗

Final Report-Optimization Under Uncertainty and Nonconvexity: Algorithms and Software

The goal of this research was to develop new algorithmic techniques for solving large-scale numerical optimization problems, focusing on problems classes that have proven to be among the most challenging for practitioners: those involving uncertainty and those involving nonconvexity. This research advanced the state-of-the-art in solving mixed integer linear programs containing symmetry, mixed integer nonlinear programs, and stochastic optimization problems.

97 MATHEMATICS AND COMPUTING↗

Systems Analysis Programs for Hands-on Integrated Reliability Evaluations (SAPHIRE) Code Reference Manual

The Systems Analysis Programs for Hands-on Integrated Reliability Evaluations (SAPHIRE) is a software application developed for performing a complete probabilistic risk assessment (PRA) using a personal computer. SAPHIRE is funded by the U.S. Nuclear Regulatory Commission (NRC) and developed by the Idaho National Laboratory (INL). The INL's primary role in this project is that of software developer. However, the INL also plays an important role in technology transfer by interfacing and supporting SAPHIRE users comprised of a wide range of PRA practitioners from the NRC, national laboratories, the private sector, and foreign countries. SAPHIRE can be used to model a complex system’s response to initiating events, quantify associated damage outcome frequencies, and identify important contributors to this damage (Level 1 PRA) and to analyze containment performance during a severe accident and quantify radioactive releases (Level 2 PRA). It can be used for a PRA evaluating a variety of operating conditions, for example, for a nuclear reactor at full power, low power, or at shutdown conditions. Furthermore, SAPHIRE can be used to analyze both internal and external initiating events and has special features for transforming models built for internal event analysis to models for external event analysis. It can also be used in a limited manner to quantify risk in terms of release consequences to both the public and the environment (Level 3 PRA). SAPHIRE includes a separate module called the Graphical Evaluation Module (GEM). GEM provides a highly specialized user interface with SAPHIRE that automates SAPHIRE process steps for evaluating operational events at commercial nuclear power plants. Using GEM, an analyst can estimate the risk associated with operational events in a very efficient and expeditious manner. This reference guide will introduce the SAPHIRE Version 7.0 software. A brief discussion of the purpose and history of the software is included along with general information such as installation instructions, starting and stopping the program, and some pointers on how to get around inside the program. Next, database concepts and structure are discussed. Following that discussion are nine sections, one for each of the menu options on the SAPHIRE main menu, wherein the purpose and general capabilities for each option are furnished. Next, the capabilities and limitations of the software are provided.

99 GENERAL AND MISCELLANEOUS↗

Systems Analysis Programs for Hands-on Intergrated Reliability Evaluations (SAPHIRE) Summary Manual

The Systems Analysis Programs for Hands-on Integrated Reliability Evaluations (SAPHIRE) is a software application developed for performing a complete probabilistic risk assessment (PRA) using a personal computer (PC) running the Microsoft Windows operating system. SAPHIRE is primarily funded by the U.S. Nuclear Regulatory Commission (NRC) and developed by the Idaho National Laboratory (INL). INL's primary role in this project is that of software developer and tester. However, INL also plays an important role in technology transfer by interfacing and supporting SAPHIRE users, who constitute a wide range of PRA practitioners from the NRC, national laboratories, the private sector, and foreign countries. SAPHIRE can be used to model a complex system’s response to initiating events and quantify associated consequential outcome frequencies. Specifically, for nuclear power plant applications, SAPHIRE can identify important contributors to core damage (Level 1 PRA) and containment failure during a severe accident which lead to releases (Level 2 PRA). It can be used for a PRA where the reactor is at full power, low power, or at shutdown conditions. Furthermore, it can be used to analyze both internal and external initiating events and has special features for transforming an internal events model to a model for external events, such as flooding and fire analysis. It can also be used in a limited manner to quantify risk in terms of release consequences to the public and environment (Level 3 PRA). SAPHIRE also includes a separate module called the Graphical Evaluation Module (GEM). GEM is a special user interface linked to SAPHIRE that automates the SAPHIRE process steps for evaluating operational events at commercial nuclear power plants. Using GEM, an analyst can estimate the risk associated with operational events (for example, to calculate a conditional core damage probability) very efficiently and expeditiously. This report provides an overview of the functions available in SAPHIRE and presents general instructions for using the software. Section 1 presents SAPHIRE’s historical evolution and summarizes its capabilities. Section 2 presents instructions for installing and using the code. Section 3 explains the database structure used in SAPHIRE and discusses database concepts. Section 4 explains how PRA data (event frequencies, human error probabilities, etc.) can be generated and manipulated using “change sets.” Section 5 deals with fault tree operations, including constructing, editing, solving, and displaying results. Section 6 presents operations associated with event trees, including rule application for event tree linking, partitioning, and editing sequences. Section 7 presents how accident sequences are generated, solved, quantified, and analyzed. Section 8 discusses the functions available for performing end state analysis. Section 9 explains how to modify data stored in a SAPHIRE database. Section 10 illustrates how to generate and customize reports. Section 11 covers SAPHIRE utility options to perform routine functions such as defining constant values, recovering databases, and loading data from external sources. Section 12 provides an overview of GEM’s features and capabilities. Finally, Section 13 summarizes SAPHIRE’s quality assurance process.

97 MATHEMATICS AND COMPUTING↗

Parallel computing in enterprise modeling.

This report presents the results of our efforts to apply high-performance computing to entity-based simulations with a multi-use plugin for parallel computing. We use the term 'Entity-based simulation' to describe a class of simulation which includes both discrete event simulation and agent based simulation. What simulations of this class share, and what differs from more traditional models, is that the result sought is emergent from a large number of contributing entities. Logistic, economic and social simulations are members of this class where things or people are organized or self-organize to produce a solution. Entity-based problems never have an a priori ergodic principle that will greatly simplify calculations. Because the results of entity-based simulations can only be realized at scale, scalable computing is de rigueur for large problems. Having said that, the absence of a spatial organizing principal makes the decomposition of the problem onto processors problematic. In addition, practitioners in this domain commonly use the Java programming language which presents its own problems in a high-performance setting. The plugin we have developed, called the Parallel Particle Data Model, overcomes both of these obstacles and is now being used by two Sandia frameworks: the Decision Analysis Center, and the Seldon social simulation facility. While the ability to engage U.S.-sized problems is now available to the Decision Analysis Center, this plugin is central to the success of Seldon. Because Seldon relies on computationally intensive cognitive sub-models, this work is necessary to achieve the scale necessary for realistic results. With the recent upheavals in the financial markets, and the inscrutability of terrorist activity, this simulation domain will likely need a capability with ever greater fidelity. High-performance computing will play an important part in enabling that greater fidelity.

97 MATHEMATICS AND COMPUTING↗

Review of Evaluation, Measurement and Verification Approaches Used to Estimate the Load Impacts and Effectiveness of Energy Efficiency Programs

Public and private funding for end-use energy efficiency actions is expected to increase significantly in the United States over the next decade. For example, Barbose et al (2009) estimate that spending on ratepayer-funded energy efficiency programs in the U.S. could increase from $3.1 billion in 2008 to $7.5 and 12.4 billion by 2020 under their medium and high scenarios. This increase in spending could yield annual electric energy savings ranging from 0.58% - 0.93% of total U.S. retail sales in 2020, up from 0.34% of retail sales in 2008. Interest in and support for energy efficiency has broadened among national and state policymakers. Prominent examples include {approx}$18 billion in new funding for energy efficiency programs (e.g., State Energy Program, Weatherization, and Energy Efficiency and Conservation Block Grants) in the 2009 American Recovery and Reinvestment Act (ARRA). Increased funding for energy efficiency should result in more benefits as well as more scrutiny of these results. As energy efficiency becomes a more prominent component of the U.S. national energy strategy and policies, assessing the effectiveness and energy saving impacts of energy efficiency programs is likely to become increasingly important for policymakers and private and public funders of efficiency actions. Thus, it is critical that evaluation, measurement, and verification (EM&V) is carried out effectively and efficiently, which implies that: (1) Effective program evaluation, measurement, and verification (EM&V) methodologies and tools are available to key stakeholders (e.g., regulatory agencies, program administrators, consumers, and evaluation consultants); and (2) Capacity (people and infrastructure resources) is available to conduct EM&V activities and report results in ways that support program improvement and provide data that reliably compares achieved results against goals and similar programs in other jurisdictions (benchmarking). The National Action Plan for Energy Efficiency (2007) presented commonly used definitions for EM&V in the context of energy efficiency programs: (1) Evaluation (E) - The performance of studies and activities aimed at determining the effects and effectiveness of EE programs; (2) Measurement and Verification (M&V) - Data collection, monitoring, and analysis associated with the calculation of gross energy and demand savings from individual measures, sites or projects. M&V can be a subset of program evaluation; and (3) Evaluation, Measurement, and Verification (EM&V) - This term is frequently seen in evaluation literature. EM&V is a catchall acronym for determining both the effectiveness of program designs and estimates of load impacts at the portfolio, program and project level. This report is a scoping study that assesses current practices and methods in the evaluation, measurement and verification (EM&V) of ratepayer-funded energy efficiency programs, with a focus on methods and practices currently used for determining whether projected (ex-ante) energy and demand savings have been achieved (ex-post). M&V practices for privately-funded energy efficiency projects (e.g., ESCO projects) or programs where the primary focus is greenhouse gas reductions were not part of the scope of this study. We identify and discuss key purposes and uses of current evaluations of end-use energy efficiency programs, methods used to evaluate these programs, processes used to determine those methods; and key issues that need to be addressed now and in the future, based on discussions with regulatory agencies, policymakers, program administrators, and evaluation practitioners in 14 states and national experts in the evaluation field. We also explore how EM&V may evolve in a future in which efficiency funding increases significantly, innovative mechanisms for rewarding program performance are adopted, the role of efficiency in greenhouse gas mitigation is more closely linked, and programs are increasingly funded from multiple sources often with multiple program administrators and intended to meet multiple purposes.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗

System Assessment Standards: Defining the Market for Industrial Energy Assessments

Improved efficiency of industrial systems (e.g., compressed air or steam) contributes to a manufacturing facility?s bottom line, improves reliability, and better utilizes assets. Despite these advantages, many industrial facilities continue to have unrealized system optimization potential. A barrier to realizing this potential is the lack of market definition for system energy efficiency assessment services, creating problems for both service providers in establishing market value for their services and for consumers in determining the relative quality of these system assessment services. On August 19, 2008, the American Society of Mechanical Engineers (ASME) issued four new draft Standards for trial use that are designed to raise the bar and define the market for these services. These draft Standards set the requirements for conducting an energy assessment at an industrial facility for four different system types: compressed air, process heating, pumping, and steam. The Standards address topics such as organizing and conducting assessments; analyzing the data collected; and reporting and documentation. This paper addresses both the issues and challenges in developing the Standards and the accompanying Guidance Documents, as well as the result of field testing by industrial facilities, consultants, and utilities during the trial use period that ended in January, 2009. These Standards will be revised and released by ASME for public review, and subsequently submitted for approval as American National Standards for publication in late 2009. Plans for a related activity to establish a professional-level program to certify practitioners in the area of system assessments, opportunities to integrate the ASME Standards with related work on industrial energy efficiency, as well as plans to expand the system assessment Standard portfolio are also discussed.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗

Constructing a resilience index for the Enhanced Critical Infrastructure Protection Program

Following recommendations made in Homeland Security Presidential Directive 7, which established a national policy for the identification and increased protection of critical infrastructure and key resources (CIKR) by Federal departments and agencies, the U.S. Department of Homeland Security (DHS) in 2006 developed the Enhanced Critical Infrastructure Protection (ECIP) program. The ECIP program aimed to provide a closer partnership with state, regional, territorial, local, and tribal authorities in fulfilling the national objective to improve CIKR protection. The program was specifically designed to identify protective measures currently in place in CIKR and to inform facility owners/operators of the benefits of new protective measures. The ECIP program also sought to enhance existing relationships between DHS and owners/operators of CIKR and to build relationships where none existed (DHS 2008; DHS 2009). In 2009, DHS and its protective security advisors (PSAs) began assessing CIKR assets using the ECIP program and ultimately produced individual protective measure and vulnerability values through the protective measure and vulnerability indices (PMI/VI). The PMI/VI assess the protective measures posture of individual facilities at their 'weakest link,' allowing for a detailed analysis of the most vulnerable aspects of the facilities (Schneier 2003), while maintaining the ability to produce an overall protective measures picture. The PMI has six main components (physical security, security management, security force, information sharing, protective measures assessments, and dependencies) and focuses on actions taken by a facility to prevent or deter the occurrence of an incident (Argonne National Laboratory 2009). As CIKR continue to be assessed using the PMI/VI and owners/operators better understand how they can prevent or deter incidents, academic research, practitioner emphasis, and public policy formation have increasingly focused on resilience as a necessary component of the risk management framework and infrastructure protection. This shift in focus toward resilience complements the analysis of protective measures by taking into account the three other phases of risk management: mitigation, response, and recovery (Figure 1). Thus, the addition of a robust resilience index (RI) to the established PMI/VI provides vital information to owners/operators throughout the risk management process. Combining a pre-incident focus with a better understanding of resilience, as well as potential consequences from damaged CIKR, allows owners/operators to better understand different ways to decrease risk by (1) increasing physical security measures to prevent an incident, (2) supplementing redundancy to mitigate the effects of an incident, and (3) enhancing emergency action and business continuity planning to increase the effectiveness of recovery procedures. Information provided by the RI methodology is also used by facility owners/operators to better understand how their facilities compare to similar sector/subsector sites and to help them make risk-based decisions. This report provides an overview of the RI methodology developed to estimate resilience and provide resilience comparisons for sectors and subsectors. The information will be used to (1) assist DHS in analyzing existing response and recovery methods and programs at facilities and (2) identify potential ways to increase resilience. The RI methodology is based on principles of Appreciative Inquiry, which is 'the coevolutionary search for the best in people, their organizations, and the relevant world around them' (Cooperrider et al. 2005). Appreciative Inquiry identifies the best of 'what is' and helps to envision 'what might be.' The ECIP program and the RI represent a new model (using Appreciative Inquiry principles) for information sharing between government and industry (Fisher and Petit 2010). A 'dashboard' display, which provides an interactive tool - rather than a static report, presents the results of the RI in a convenient format. Additional resilience measures can be modeled to illustrate how such actions would impact the asset's RI value.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗

Detection of Spatiotemporal Prescription Opioid Hot Spots With Network Scan Statistics: Multistate Analysis

Background: Overuse and misuse of prescription opioids have become significant public health burdens in the United States. About 11.5 million people are estimated to have misused prescription opioids for nonmedical purposes in 2016. This has led to a significant number of drug overdose deaths in the United States. Previous studies have examined spatiotemporal clusters of opioid misuse, but they have been restricted to circular shaped regions. Objective: The goal of this study was to identify spatiotemporal hot spots of opioid users and opioid prescription claims using Medicare data. Methods: We examined spatiotemporal clusters with significantly higher number of beneficiaries and rate of prescriptions for opioids using Medicare payment data from the Centers for Medicare & Medicaid Services. We used network scan statistics to detect significant clusters with arbitrary shapes, the Kulldorff scan statistic to examine the significant clusters for each year (2013, 2014, and 2015) and an expectation-based version to examine the significant clusters relative to past years. Regression analysis was used to characterize the demographics of the counties that are a part of any significant cluster, and data mining techniques were used to discover the specialties of the anomalous providers. Results: We examined anomalous spatial clusters with respect to opioid prescription claims and beneficiary counts and found some common patterns across states: the counties in the most anomalous clusters were fairly stable in 2014 and 2015, but they have shrunk from 2013. In Virginia, a higher percentage of African Americans in a county lower the odds of the county being anomalous in terms of opioid beneficiary counts to about 0.96 in 2015. For opioid prescription claim counts, the odds were 0.92. This pattern was consistent across the 3 states and across the 3 years. A higher number of people in the county with access to Medicaid increased the odds of the county being in the anomalous cluster to 1.16 in both types of counts in Virginia. A higher number of people with access to direct purchase of insurance plans decreased the odds of a county being in an anomalous cluster to 0.85. The expectation-based scan statistic, which captures change over time, revealed different clusters than the Kulldorff statistic. Providers with an unusually high number of opioid beneficiaries and opioid claims include specialties such as physician’s assistant, nurse practitioner, and family practice. Conclusions: Our analysis of the Medicare claims data provides characteristics of the counties and provider specialties that have higher odds of being anomalous. The empirical analysis identifies highly refined spatial hot spots that are likely to encounter prescription opioid misuse and overdose. The methodology is generic and can be applied to monitor providers and their prescription behaviors in regions that are at a high risk of abuse.

60 APPLIED LIFE SCIENCES↗

Explainable Artificial Intelligence Recommendation System by Leveraging the Semantics of Adverse Childhood Experiences: Proof-of-Concept Prototype Development

The study of adverse childhood experiences and their consequences has emerged over the past 20 years. Although the conclusions from these studies are available, the same is not true of the data. Accordingly, it is a complex problem to build a training set and develop machine-learning models from these studies. Classic machine learning and artificial intelligence techniques cannot provide a full scientific understanding of the inner workings of the underlying models. This raises credibility issues due to the lack of transparency and generalizability. Explainable artificial intelligence is an emerging approach for promoting credibility, accountability, and trust in mission-critical areas such as medicine by combining machine-learning approaches with explanatory techniques that explicitly show what the decision criteria are and why (or how) they have been established. Hence, thinking about how machine learning could benefit from knowledge graphs that combine “common sense” knowledge as well as semantic reasoning and causality models is a potential solution to this problem. In this study, we aimed to leverage explainable artificial intelligence, and propose a proof-of-concept prototype for a knowledge-driven evidence-based recommendation system to improve mental health surveillance. We used concepts from an ontology that we have developed to build and train a question-answering agent using the Google DialogFlow engine. In addition to the question-answering agent, the initial prototype includes knowledge graph generation and recommendation components that leverage third-party graph technology. To showcase the framework functionalities, we here present a prototype design and demonstrate the main features through four use case scenarios motivated by an initiative currently implemented at a children’s hospital in Memphis, Tennessee. Ongoing development of the prototype requires implementing an optimization algorithm of the recommendations, incorporating a privacy layer through a personal health library, and conducting a clinical trial to assess both usability and usefulness of the implementation. This semantic-driven explainable artificial intelligence prototype can enhance health care practitioners’ ability to provide explanations for the decisions they make.

adverse childhood experiences↗

The Double-edged Sword of Data-driven Super-Resolution: Adversarial Super-resolution Models

Data-driven super-resolution (SR) methods are often integrated into imaging pipelines as preprocessing steps to improve downstream tasks such as classification and detection. However, these SR models introduce a previously unexplored attack surface into imaging pipelines. In this paper, we present AdvSR, a framework demonstrating that adversarial behavior can be embedded directly into SR model weights during training, requiring no access to inputs at inference time. Unlike prior attacks that perturb inputs or rely on backdoor triggers, AdvSR operates entirely at the model level. By jointly optimizing for reconstruction quality and targeted adversarial outcomes, AdvSR produces models that appear benign under standard image quality metrics while inducing downstream misclassification. We evaluate AdvSR on three SR architectures (SRCNN, EDSR, SwinIR) paired with a YOLOv11 classifier and demonstrate that AdvSR models can achieve high attack success rates with minimal quality degradation. These findings highlight a new model-level threat for imaging pipelines, with implications for how practitioners source and validate models in safety-critical applications.

Sullivan, Haley [ORNL] (ORCID:0000000274069217)↗

An Inventory of AI-ready Benchmark Data for US Fires, Heatwaves, and Droughts

Extreme weather events, including fires, heatwaves, and droughts, have significant impacts on earth, environmental, and energy systems. Mechanistic and predictive understanding, as well as probabilistic risk assessment of these extreme weather events, are crucial for detecting, planning for, and responding to these extremes. Records of extreme weather events provide an important data source for understanding present and future extremes, but the existing data needs preprocessing before it can be used for analysis. Moreover, there are many nonstandard metrics defining the levels of severity or impacts of extremes. In this study, we compile a comprehensive benchmark data inventory of extreme weather events, including fires, heatwaves, and droughts. The dataset covers the period from 2001 to 2020 with a daily temporal resolution and a spatial resolution of 0.5°×0.5° (~55km×55km) over the continental United States (CONUS), and a spatial resolution of 1km × 1km over the Pacific Northwest (PNW) region, together with the co-located and relevant meteorological variables. By exploring and summarizing the spatial and temporal patterns of these extremes in various forms of marginal, conditional, and joint probability distributions, we gain a better understanding of the characteristics of climate extremes. The resulting AI/ML-ready data products can be readily applied to ML-based research, fostering and encouraging AI/ML research in the field of extreme weather. This study can contribute significantly to the advancement of extreme weather research, aiding researchers, policymakers, and practitioners in developing improved preparedness and response strategies to protect communities and ecosystems from the adverse impacts of extreme weather events. Usage Notes We presented a long term (2001-2020) and comprehensive data inventory of historical extreme events with daily temporal resolution covering the separate spatial extents of CONUS (0.5°×0.5°) and PNW(1km×1km) for various applications and studies. The dataset with 0.5°×0.5° resolution for CONUS can be used to help build more accurate climate models for the entire CONUS, which can help in understanding long-term climate trends, including changes in the frequency and intensity of extreme events, predicting future extreme events as well as understanding the implications of extreme events on society and the environment. The data can also be applied for risk accessment of the extremes. For example, ML/AI models can be developed to predict wildfire risk or forecast HWs by analyzing historical weather data, and past fires or heateave , allowing for early warnings and risk mitigation strategies. Using this dataset, AI-driven risk assessment models can also be built to identify vulnerable energy and utilities infrastructure, imrpove grid resilience and suggest adaptations to withstand extreme weather events. The high-resolution 1km×1km dataset ove PNW are advantageous for real-time, localized and detailed applications. It can enhance the accuracy of early warning systems for extreme weather events, helping authorities and communities prepare for and respond to disasters more effectively. For example, ML models can be developed to provide localized HW predictions for specific neighborhoods or cities, enabling residents and local emergency services to take targeted actions; the assessment of drought severity in specific communities or watersheds within the PNW can help local authorities manage water resources more effectively.

Lin, Xinming↗