Search NASA⌕ Search

SEARCH · Search NASA

Results for “Runtime monitoring”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 127 records · Page 7
All sourcesNASA NTRSDOE OSTI
DOE OSTI2020

Achieving balanced execution through runtime detection of performance variation

Systems, apparatuses, and methods for achieving balanced execution in a multi-node cluster through runtime detection of performance variation are described. During a training phase, performance counters and an amount of time spent waiting for synchronization is monitored for a plurality of tasks for each node of the multi-node cluster. These values are utilized to generate a model which correlates the values of the performance counters to the amount of time spent waiting for synchronization. Once the model is built, the values of the performance counters are monitored for a period of time at the start of each task, and these values are input into the model. The model generates a prediction of whether a given node is on the critical path. If the given node is predicted to be on the critical path, the power allocation of the given node is increased.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

A Brief Survey on High Performance Computing Systems Power Management

This paper provides a survey of software-based power management techniques in High Performance Computing (HPC) systems. Seven existing power management and monitoring tools and frameworks are discussed. These are: Variorum, dynamic energy-performance optimizer (DEPO), Powersched, Bull Dynamic Power Optimizer (BDPO), Energy Aware Runtime (EAR), Global Extensible Open Power Manager (GEOPM), and PoLiMEr. Each of these tools is evaluated based on hardware abstraction, optimization methods, usability, and experimental validation. This survey highlights the diversity of approaches in managing energy efficiency, from vendor-neutral APIs to algorithm-driven power capping, and dynamic frequency adjustments. Given that energy requirements for large computational systems is increasing quickly, the importance of integrating these tools into existing HPC environments and the need for further research in this rapidly evolving field is also discussed.

97 - MATHEMATICS AND COMPUTING↗
DOE OSTI2019

Production Application Performance Data Streaming for System Monitoring

In this article, we present an approach to streaming collection of application performance data. Practical application performance tuning and troubleshooting in production high-performance computing (HPC) environments requires an understanding of how applications interact with the platform, including (but not limited to) parallel programming libraries such as Message Passing Interface (MPI). Several profiling and tracing tools exist that collect heavy runtime data traces either in memory (released only at application exit) or on a file system (imposing an I/O load that may interfere with the performance being measured). Although these approaches are beneficial in development stages and post-run analysis, a systemwide and low-overhead method is required to monitor deployed applications continuously. This method must be able to collect information at both the application and system levels to yield a complete performance picture. In our approach, an application profiler collects application event counters. A sampler uses an efficient inter-process communication method to periodically extract the application counters and stream them into an infrastructure for performance data collection. We implement a tool-set based on our approach and integrate it with the Lightweight Distributed Metric Service (LDMS) system, a monitoring system used on large-scale computational platforms. LDMS provides the infrastructure to create and gather streams of performance data in a low overhead manner. We demonstrate our approach using applications implemented with MPI, as it is one of the most common standards for the development of large-scale scientific applications. We utilize our tool-set to study the impact of our approach on an open source HPC application, Nalu. Our tool-set enables us to efficiently identify patterns in the behavior of the application without source-level knowledge. We leverage LDMS to collect system-level performance data and explore the correlation between the system and application events. Finally, we demonstrate how our tool-set can help detect anomalies with a low latency. We run tests on two different architectures: a system enabled with Intel Xeon Phi and another system equipped with Intel Xeon processor. Our overhead study shows our method imposes at most 0.5% CPU usage overhead on the application in realistic deployment scenarios.

59 BASIC BIOLOGICAL SCIENCES↗
NASA NTRS2009

Analog Input Data Acquisition Software

DAQ Master Software allows users to easily set up a system to monitor up to five analog input channels and save the data after acquisition. This program was written in LabVIEW 8.0, and requires the LabVIEW runtime engine 8.0 to run the executable.

Arens, Ellen↗
DOE OSTI2016

Static and Dynamic Frequency Scaling on Multicore CPUs

Dynamic voltage and frequency scaling (DVFS) adapts CPU power consumption by modifying a processor’s operating frequency (and the associated voltage). Typical approaches employing DVFS involve default strategies such as running at the lowest or the highest frequency, or observing the CPU’s runtime behavior and dynamically adapting the voltage/frequency configuration based on CPU usage. In this paper, we argue that many previous approaches suffer from inherent limitations, such as not account- ing for processor-specific impact of frequency changes on energy for different workload types. We first propose a lightweight runtime-based approach to automatically adapt the frequency based on the CPU workload, that is agnostic of the processor characteristics. We then show that further improvements can be achieved for affine kernels in the application, using a compile-time characterization instead of run-time monitoring to select the frequency and number of CPU cores to use. Our framework relies on a one-time energy characterization of CPU-specific DVFS profiles followed by a compile-time categorization of loop-based code segments in the application. These are combined to determine a priori of the frequency and the number of cores to use to execute the application so as to optimize energy or energy-delay product, outperforming runtime approach. Extensive evaluation on 60 benchmarks and five multi-core CPUs show that our approach systematically outperforms the powersave Linux governor, while improving overall performance.

24 POWER TRANSMISSION AND DISTRIBUTION↗
NASA NTRS2017

Unobtrusive Software and System Health Management with R2U2 on a Parallel MIMD Coprocessor

Dynamic monitoring of software and system health of a complex cyber-physical system requires observers that continuously monitor variables of the embedded software in order to detect anomalies and reason about root causes. There exists a variety of techniques for code instrumentation, but instrumentation might change runtime behavior and could require costly software re-certification. In this paper, we present R2U2E, a novel realization of our real-time, Realizable, Responsive, and Unobtrusive Unit (R2U2). The R2U2E observers are executed in parallel on a dedicated 16-core EPIPHANY co-processor, thereby avoiding additional computational overhead to the system under observation. A DMA-based shared memory access architecture allows R2U2E to operate without any code instrumentation or program interference.

Schumann, Johann↗
DOE OSTI2021

A Range and Performance Optimized Version of the Computer-Aided Speckle Interferometry Algorithm for Real-Time Displacement-Strain Field Monitoring

Abstract This work presents an optimized implementation of the Computer-Aided Speckle Interferometry algorithm which enables full-field determination of displacements and strains on commodity Graphics Processing Units at high resolution and frame rates. By combining careful control of the average speckle size in a laser speckle pattern with a simple sampling rate conversion scheme, a compact representation of the optical speckle is achieved. This allows for optimal use of Graphics Processing Unit architecture with robust range extension. The optimal mapping of the Computer-Aided Speckle Interferometry algorithm to Graphics Processing Unit architecture is shown in detail, and a straightforward method for disambiguating large displacements is illustrated. Lastly, this paper demonstrates a two-step subimage-tapering modification to the original algorithm that enables robust range enhancement while maintaining resolution. Results from numerical simulations on synthetic speckle patterns are shown, and runtime performance metrics are provided, with performance ranging up to 60 frames per second in some cases. The method is suitable for interactive experimental mechanics research, process and testing or any application where real-time high-resolution displacement-strain monitoring is needed. A .NET Framework class library enabling the incorporation of the algorithm into 3rd -party applications is available for download.

42 ENGINEERING↗
DOE OSTI2023

Middleware for a Heterogeneous CAV Fleet

This paper introduces CAN to ROS, a model-based code generation tool used in development, testing, and deployment of a heterogeneous fleet of vehicles with robotic sensing in ROS. Code generation supports two main features: (1) self-configuration for deployment in a heterogeneous vehicle fleet, and (2) quick iteration for testing and development of reading vehicle sensors and robotic control. This tool features the ability to detect the vehicle it is in and regenerate and rebuild itself at runtime to provide the proper two-way bridge between ROS and the sensed on-board vehicle sensor network. Code generation relies on a per-model defined JSON to map a CAN database (DBC) to the desired ROS topic names and message types. The live ROS publishing of CAN messages allows for instant feedback, and the code regeneration allows for adjustments in DBC or vehicle JSON to iteratively hone in on new vehicle signals. Generated ROS nodes are written in C++ for runtime use in lightweight embedded computers. This has been tested in vehicles from three different Original Equipment Manufacturers (OEMs), and can be extended to support a wide array of vehicles. By using a unifying ROS specification, a heterogeneous set of vehicles can be unified into a fleet with abstracted model-specific details; this opens the door for developing cross-model software applications for vehicle control, connected vehicle applications, or fleet monitoring systems.

42 ENGINEERING↗
NASA NTRS2017

MESA: Message-Based System Analysis Using Runtime Verification

In this paper, we present a novel approach and framework for run-time verication of large, safety critical messaging systems. This work was motivated by verifying the System Wide Information Management (SWIM) project of the Federal Aviation Administration (FAA). SWIM provides live air traffic, site and weather data streams for the whole National Airspace System (NAS), which can easily amount to several hundred messages per second. Such safety critical systems cannot be instrumented, therefore, verification and monitoring has to happen using a nonintrusive approach, by connecting to a variety of network interfaces. Due to a large number of potential properties to check, the verification framework needs to support efficient formulation of properties with a suitable Domain Specific Language (DSL). Our approach is to utilize a distributed system that is geared towards connectivity and scalability and interface it at the message queue level to a powerful verification engine. We implemented our approach in the tool called MESA: Message-Based System Analysis, which leverages the open source projects RACE (Runtime for Airspace Concept Evaluation) and TraceContract. RACE is a platform for instantiating and running highly concurrent and distributed systems and enables connectivity to SWIM and scalability. TraceContract is a runtime verication tool that allows for checking traces against properties specified in a powerful DSL. We applied our approach to verify a SWIM service against several requirements.We found errors such as duplicate and out-of-order messages.

Message-based System↗
NASA NTRS2005

BEAM Technology Flight Demonstration

As technologies advance, their growing complexity makes them harder to maintain. Detection methods for isolating and identifying impending problems are needed to balance this complexity. Through comparison of signal pairs from onboard sensors, the Beacon-based Exception Analysis For Multimissions (BEAM) algorithm can identify and help classify deviations in system operation from a data-trained statistical model. The goal of this task is to mature BEAM and validate its performance on a flying test bed. A series of F-18 flight demonstrations with BEAM monitoring engine parameters in real time was used to demonstrate in-the-field readiness. Captured F-18 and simulated F-18 engine data were used in model creation and training. The algorithm was then ported to the embedded system with a data buffering, file writing, and data-time-stamp monitoring shell to reduce the impact of embedded system faults on BEAM'S ability to correctly identify engine faults. Embedded system testing identified hardware related restrictions and contributed to iterative improvements in the code's runtime performance. The system was flown with forced engine flameouts and other pilot induced faults to simulate operation out of the norm. Successful detection of these faults, confirmed through post-flight data analysis, helped BEAM achieve TRL6.

Wang, David↗
NASA NTRS2004

ScyFlow: An Environment for the Visual Specification and Execution of Scientific Workflows

With the advent of grid technologies, scientists and engineers are building more and more complex applications to utilize distributed grid resources. The core grid services provide a path for accessing and utilizing these resources in a secure and seamless fashion. However what the scientists need is an environment that will allow them to specify their application runs at a high organizational level, and then support efficient execution across any given set or sets of resources. We have been designing and implementing ScyFlow, a dual-interface architecture (both GUT and APT) that addresses this problem. The scientist/user specifies the application tasks along with the necessary control and data flow, and monitors and manages the execution of the resulting workflow across the distributed resources. In this paper, we utilize two scenarios to provide the details of the two modules of the project, the visual editor and the runtime workflow engine.

McCann, Karen M.↗
NASA NTRS2024

Components Refurbishment and Chemical Analysis Facility, Hot Spot 1 Solid Waste Management Unit #041 Year 4 Annual Performance Monitoring Report Kennedy Space Center, Florida

This Year 4 Annual Performance Monitoring Report (PMR) presents the operations, maintenance, and monitoring activities for the Hydraulic Containment System (HCS) Interim Measure (IM) at the Components Refurbishment and Chemical Analysis (CRCA) facility located at John F. Kennedy Space Center (KSC), Florida. The primary objective of the HCS is to attain hydraulic control of the dissolved-phase chlorinated volatile organic compound (CVOC) plume, with the secondary objective to reduce concentrations of CVOCs in the high-concentration plume to support transition to monitored natural attenuation (MNA). CRCA has been designated Solid Waste Management Unit 041 under the KSC Resource Conservation and Recovery Act Corrective Action Program. The timeframe for activities documented in this Year 4 PMR extends from November 2022 through September 2023. Baseline sampling activities were completed in June 2019, and full-scale startup of the HCS IM was completed in July-August 2019. The operational runtime of the HCS for the Year 4 reporting period was approximately 94%, with the majority of downtime attributed to associated groundwater sampling events, maintenance, and Hurricane Nicole. Almost five million gallons of groundwater were treated during Year 4 of HCS operations, and concentrations of the site’s contaminants of concern (trans-1,2-dichloroethene and vinyl chloride) have been reduced by over 99%. This PMR describes the activities that were performed during Year 4 to operate and monitor the HCS IM, which includes three extraction wells, seven injection wells, and conveyance piping to a modular structure containing the control panel and an air stripper. Influent and effluent sampling results from the air stripper show that the system is operating as designed and is reducing concentrations of contaminants of concern to below detection limits. In addition to HCS operation, this PMR also discusses performance monitoring that has been implemented to assess progress of the HCS IM and overall plume conditions through scheduled groundwater (quarterly and semi-annual) and sub-slab soil gas (quarterly) sampling and analysis. Two ambient air samples were also collected on a quarterly basis in the vicinity of the modular structure and the paved driveway east of the Solvent Reclamation Area during routine operation and maintenance (O&M) activities to ensure safe breathing zone air quality for on-site personnel. All sub-slab soil gas and ambient air sampling conducted during the Year 4 operational period showed results below applicable regulatory air screening limits. Predictions made during the Year 2 groundwater model updates were in close correlation to post Year 4 plume conditions. A supplemental DPT study conducted in 2022 and 2023. This study indicated that low-concentration plume conditions, where concentrations exceed State of Florida Groundwater Cleanup Target Levels, expanded westward to Kennedy Parkway North and northward to the vicinity of the railroad tracks. Based on these results, recommendations were made to install 14 wells to monitor the downgradient and boundary conditions of the expanded LCP. The contents of this Year 4 PMR were presented during the November 2023 KSC Remediation Team meeting, where Team consensus was reached on several items including continued O&M of the HCS, and continued monitoring of groundwater, ambient air, and sub-slab soil gas. Sampling for per- and polyfluoroalkyl substances at CRCA is ongoing and will be submitted under separate cover.

K. Alex Murphy↗
NASA NTRS2023

Components Refurbishment and Chemical Analysis Facility, Hot Spot 1 SWMU #041 Year 3 Annual Performance Monitoring Report Kennedy Space Center, Florida

This Year 3 Annual Performance Monitoring Report (PMR) presents the operations, maintenance, and monitoring activities for the Hydraulic Containment System (HCS) Interim Measure (IM) at the Components Refurbishment and Chemical Analysis (CRCA) facility located at John F. Kennedy Space Center (KSC), Florida. The primary objective of the HCS is to attain hydraulic control of the dissolved-phase chlorinated volatile organic compound (CVOC) plume, with the secondary objective to reduce concentrations of CVOCs in the high-concentration plume to support transition to monitored natural attenuation (MNA). CRCA has been designated Solid Waste Management Unit 041 under the KSC Resource Conservation and Recovery Act Corrective Action Program. The timeframe for activities documented in this Year 3 PMR extends from September 2021 through October 2022. Baseline sampling activities were completed in June 2019, and full-scale startup of the HCS IM was completed in July-August 2019. The operational runtime of the HCS for the Year 3 reporting period was approximately 91%, with the majority of downtime attributed to system maintenance and repair. This is generally consistent with Year 1 and Year 2 runtimes of 85% and 92%, respectively. To help reduce downtime, an anti-scaling amendment, Redux 390, has been used to reduce scaling and help maintain system design parameters. Over nine million gallons of groundwater were treated during Year 3 of HCS operations, and concentrations of the site’s contaminants of concern (trans-1,2-dichloroethene and vinyl chloride) have been reduced by over 97%. This PMR describes the activities that were performed during Year 3 to operate and monitor the HCS IM, which includes three extraction wells, seven injection wells, and conveyance piping to a modular structure containing the control panel and an air stripper. Influent and effluent sampling results from the air stripper show that the system is operating as designed and is reducing concentrations of contaminants of concern to below detection limits. In addition to HCS operation, this PMR also discusses performance monitoring that has been implemented to assess progress of the HCS IM and overall plume conditions through scheduled groundwater (quarterly and semi-annual) and sub-slab soil gas (quarterly) sampling and analysis. Two ambient air samples were also collected on a quarterly basis in the vicinity of the modular structure and the paved driveway east of the Solvent Reclamation Area during routine operation and maintenance (O&M) activities to ensure safe breathing zone air quality for on-site personnel. All sub-slab soil gas and ambient air sampling conducted during the Year 3 operational period showed results below applicable regulatory air screening limits. Predictions made during the Year 2 groundwater model updates were in close correlation to post Year 3 plume conditions. Therefore, it can be assumed that the projected path remains valid for transition to MNA in one to two years of continuous HCS operation. The contents of this Year 3 PMR were presented during the February 2023 KSC Remediation Team meeting, where Team consensus was reached on several items including continued O&M of the HCS, and continued monitoring of groundwater, ambient air, and sub-slab soil gas. Replacement of MW0019 and VMP04 was also recommended, as well as additional direct-push technology sampling to further delineate the downgradient plume and confirm overall site-wide low-concentration plume boundaries. Sampling for per- and polyfluoroalkyl substances at CRCA is ongoing and will be submitted under separate cover.

K. Alex Murphy↗
NASA NTRS2008

Orchestrator Telemetry Processing Pipeline

Orchestrator is a software application infrastructure for telemetry monitoring, logging, processing, and distribution. The architecture has been applied to support operations of a variety of planetary rovers. Built in Java with the Eclipse Rich Client Platform, Orchestrator can run on most commonly used operating systems. The pipeline supports configurable parallel processing that can significantly reduce the time needed to process a large volume of data products. Processors in the pipeline implement a simple Java interface and declare their required input from upstream processors. Orchestrator is programmatically constructed by specifying a list of Java processor classes that are initiated at runtime to form the pipeline. Input dependencies are checked at runtime. Fault tolerance can be configured to attempt continuation of processing in the event of an error or failed input dependency if possible, or to abort further processing when an error is detected. This innovation also provides support for Java Message Service broadcasts of telemetry objects to clients and provides a file system and relational database logging of telemetry. Orchestrator supports remote monitoring and control of the pipeline using browser-based JMX controls and provides several integration paths for pre-compiled legacy data processors. At the time of this reporting, the Orchestrator architecture has been used by four NASA customers to build telemetry pipelines to support field operations. Example applications include high-volume stereo image capture and processing, simultaneous data monitoring and logging from multiple vehicles. Example telemetry processors used in field test operations support include vehicle position, attitude, articulation, GPS location, power, and stereo images.

Powell, Mark↗
DOE OSTI2023

$\mathrm{RADICAL}$-Pilot and $\mathrm{PMIx}$/$\mathrm{PRRTE}$: Executing Heterogeneous Workloads at Large Scale on Partitioned $\mathrm{HPC}$ Resources

Execution of heterogeneous workflows on high-performance computing (HPC) platforms present unprecedented resource management and execution coordination challenges for runtime systems. Task heterogeneity increases the complexity of resource and execution management, limiting the scalability and efficiency of workflow execution. Re-source partitioning and distribution of tasks execution over portioned re-sources promises to address those problems but we lack an experimental evaluation of its performance at scale. Here this paper provides a performance evaluation of the Process Management Interface for Exascale (PMIx) and its reference implementation PRRTE on the leadership-class HPC plat-form Summit, when integrated into a pilot-based runtime system called RADICAL-Pilot. We partition resources across multiple PRRTE Distributed Virtual Machine (DVM) environments, responsible for launching tasks via the PMIx interface. We experimentally measure the work-load execution performance in terms of task scheduling/launching rate and distribution of DVM task placement times, DVM startup and termination overheads on the Summit leadership-class HPC platform. Integrated solution with PMIx/PRRTE enables using an abstracted, standardized set of interfaces for orchestrating the launch process, dynamic process management and monitoring capabilities. It extends scaling capabilities allowing to overcome a limitation of other launching mechanisms (e.g., JSM/LSF). Explored different DVM setup configurations provide insights on DVM performance and a layout to leverage it. Our experimental results show that heterogeneous workload of 65,500 tasks on 2048 nodes, and partitioned across 32 DVMs, runs steady with resource utilization not lower than 52%. While having less concurrently executed tasks resource utilization is able to reach up to 85%, based on results of heterogeneous workload of 8200 tasks on 256 nodes and 2 DVMs.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS2024

Former Central Heat Plant SWMU 045 Year 2 Air Sparge System Performance Monitoring Report

This Air Sparge (AS) Performance Monitoring (PM) Report (PMR) presents Year 2 operation, maintenance, and monitoring (OM&M) activities, PM results, and monitoring well installations supporting the AS Interim Measure (IM) at the Former Central Heat Plant (CHP) at Kennedy Space Center (KSC), Florida. CHP has been designated Solid Waste Management Unit 045 under the KSC Resource Conservation and Recovery Act Corrective Action Program. An AS IM was installed at CHP between 2019 and 2021, which included the installation of an AS system to treat a chlorinated solvent groundwater plume. Contaminants of concern (COCs) identified at CHP for the AS IM include tetrachloroethene (PCE), trichloroethene (TCE), cis-1,2-dichloroethene (cDCE), and vinyl chloride (VC). The completed AS system includes a network of 267 AS wells, which treat approximately 1.3 acres of contaminated groundwater. “Hot” compressor technology is used to treat the source zone, while a “cold” compressor is used to treat two hot spot (HS) areas (HS1 and HS2) and the high concentration plume (HCP). The AS system began operation in June-July 2021 and this document includes Year 2 of operation. The overall runtimes for the AS system for the Year 2 reporting period (October 2022 to September 2023) were approximately 69 percent for the cold trailer and 71 percent for the hot trailer. Air samples and vapor screening results collected during the reporting period showed concentrations less than applicable human health and air emissions permit criteria. Groundwater performance monitoring results show that AS treatment continues to be effective in reducing COC concentrations at CHP. At the shallow interval, COC concentrations were all non-detect, less than, or met their respective State of Florida Groundwater Cleanup Target Levels (GCTLs) at the end of Year 2 in September 2023. In the deep interval, 10 of the 15 PM wells detected COCs greater than their respective GCTLs, with two of these wells also exceeding the Natural Attenuation Default Concentration for VC. Based on Year 2 OM&M and PM results, continued operation of the AS system is required to meet the IM objective. It is therefore recommended to continue with AS IM operations at CHP with the following plan for Year 3.

Kevin Alex Murphy↗
NASA NTRS2023

Launch Complex 34, SWMU CCO542022 DNAPL Source Zone Operations, Maintenance, and Monitoring, Site-Wide Long-Term Monitoring, and Hot Spot 6 Air Sparge System Annual Performance Monitoring Report Cape Canaveral Space Force Station, Florida

This Annual Performance Monitoring Report (PMR) for the Dense Non-Aqueous Phase Liquid (DNAPL) Source Zone (DSZ), Site-Wide Long-Term Monitoring (LTM), and Hot Spot (HS) 6 Air Sparge (AS) System presents the results of Year 13 operation of the hydraulic containment (HC) Interim Measure (IM), the results of performance monitoring direct-push technology (DPT) sampling and monitoring well sampling conducted in the DSZ, results of the biennial site-wide LTM event, and the results of operations and performance sampling of the HS 6 AS IM at Launch Complex 34 (LC34), located at Cape Canaveral Space Force Station (CCSFS), Florida. This site has been designated Solid Waste Management Unit (SWMU) CC054 under the Kennedy Space Center (KSC) Resource Conservation and Recovery Act (RCRA) Corrective Action Program. For the site-wide biennial LTM event, a total of 55 monitoring wells were sampled for volatile organic compounds (VOCs) in February 2023 and one well was sampled for polychlorinated biphenyls (PCBs) in December 2022. One well planned for VOC sampling was found to be destroyed and could not be sampled (CW0002). The LTM wells are screened in two lithologic zones: Layer 1 (0 to 25 feet below land surface [bls]) and Layer 2 (25 to 30 ft bls), and are located in the outlying areas of LC34 to monitor groundwater conditions within the Low-Concentration Plume (LCP), defined as concentrations exceeding Groundwater Cleanup Target Levels (GCTLs), and the High-Concentration Plume (HCP), defined as concentrations exceeding Natural Attenuation Default Concentrations (NADCs). Results from the biennial sampling event indicated overall plume stability and delineation for the plume, which extends over 300 acres. The operational period for Year 13 of the HCS was from April 1, 2022 to March 31, 2023. Operational runtime for the system was 90 percent during Year 13, with downtime events attributed to planned maintenance, system repairs, and power outages. As of March 31, 2023, a total of 313,673,241 cumulative gallons of groundwater containing 88,337 pounds of VOCs have been removed by the HCS. Influent concentrations of trichloroethene (TCE) have decreased since startup from approximately 280,000 μg/L (January 2010) to 12,000 μg/L (March 2023). During the reporting period, all effluent concentrations from the HCS (aqueous and vapor) were below regulatory reporting limits, indicating the system continues to operate as intended. Performance monitoring was conducted in December 2022 within the DSZ to evaluate TCE contamination. Groundwater samples were collected via DPT at nine locations, consistent with previous events between 2017 and 2021. Full vertical profile sampling was completed at each DPT from 8 to 98 feet bls, at 5-foot intervals. The DPT performance monitoring results are summarized in this PMR. The results revealed TCE remains at concentrations greater than 11,000 μg/L in the DSZ (1-percent solubility, indicative of DNAPL) at eight of the nine DPT locations and at depths ranging from 8 to 98 feet bls. In addition to DPT sampling, groundwater samples were collected from deep monitoring wells in the DSZ area (Layers 7 and 8) in December 2022 to verify vertical delineation. Layer 7/8 monitoring well results were non-detect in December 2022, with exception of three wells (IW0162, IW043D2, and IW044D2), where TCE, cis-1,2-dichloroethene (cDCE), and/or vinyl chloride (VC) were detected above GCTLs. These wells are screened 105 to 115 feet bls, which is below the existing recovery well capture zone. TCE was first detected in IW0162 in December 2021 and has since been sampled at least monthly to monitor TCE concentrations. The maximum TCE concentration during this operational period was 15,000 μg/L at IW0162 in March 2023. Because of the increased TCE concentrations in this well, a new recovery well (RW21D), screened 86 to 106 feet bls, was installed in January 2023 and incorporated into existing HCS operations. The HS 6 AS IM was initiated in 2018 with 160 AS wells, and expanded in 2019 with an additional 140 AS wells. Quarterly performance monitoring was reduced to semi-annual in 2020. The HS 6 AS system remained operational during the reporting period covered under this report. The results of the HS 6 system operation and semi-annual performance monitoring are summarized in this report. Semi-annual monitoring results collected in April and October 2022 show concentrations of contaminants of concern (cDCE, trans-1,2-dichloroethene, and vinyl chloride) are generally decreasing and not impacting the surface water drainage canal, indicating the HS 6 IM is meeting objectives. A Phase Two Expansion of the HS 6 AS IM was recently completed. As of the date of this report, the expansion became operational in August 2023 and the first quarter of monitoring was conducted in November 2023. Details of the construction, start-up, and performance monitoring will be included in a future Annual PMR. Overall, the tasks associated with Year 13 operation of the HC IM, operation of the HS 6 AS IM, and biennial site-wide sampling were performed in accordance with recommendations included in the previous 2021 LC34 (Year 12) annual report. Evaluation of results from the HC IM and HS 6 IM show that these systems are operating as designed and meeting performance objectives. Results from the site-wide biennial LTM program also show that the overall network of monitoring wells is adequate to continue monitoring plume-wide conditions.

Complex 34↗