Database Performance Monitoring for DUNE

Abstract

This project improves Checkmk monitoring for DUNE Rucio PostgreSQL database services by adding clearer dashboard visibility for connection and lock behavior. The work began with a request to monitor database performance metrics such as connection usage, configured limits, lock activity, wait behavior, query performance, storage trends, and saturation alerts. Existing Checkmk PostgreSQL checks were reviewed, and gaps were identified in how connection states and lock modes were displayed. To address these gaps, two DSG-specific local checks were added for dune_rucio_prod: one for connection-state monitoring and one for lock-state monitoring. These checks report active, idle, idle-in-transaction, total, usage-percent, lock-mode, waiting-lock, and wait-age metrics. The added metrics supplement built-in Checkmk monitoring and provide DUNE application developers with clearer service states, history graphs, dashboard widgets, and alerts.