DOE OSTI · 3704543
Machine learning method for enforcing variable independence in background estimation with LHC data: ABCDisCoTEC
Abstract
A novel solution is presented for the problem of estimating the backgrounds of a signal search using observed data while simultaneously maximizing the sensitivity of the search to the signal. The 'ABCD method' provides a reliable framework for background estimation by partitioning events into one signal-enhanced region (A) and three background-enhanced control regions (B, C, and D) via two smoothly varying, statistically independent variables. In practice, even slight correlations between the two variables can significantly undermine the method's performance. Thus, choosing appropriate variables by hand can present a formidable challenge, especially when background and signal differ only subtly. To address this issue, the ABCD with distance correlation (ABCDisCo) method was developed to construct two learned variables via a neural network trained to provide strong signal-background discrimination with small values of the distance correlation (DisCo) measure between the two learned variables. However, relying solely on minimizing the DisCo can result in learned variables that may not have distributions of background events that are smoothly varying and localized at extreme values, as necessary for the validity of the background estimation. The ABCDisCo training enhanced with closure (ABCDisCoTEC) method is introduced to solve this issue by directly minimizing the nonclosure, expressed as a dedicated differentiable loss term. This extended method is applied to a data set of proton-proton collisions at a center-of-mass energy of 13 TeV recorded by the CMS detector at the CERN Large Hadron Collider. Additionally, given the complexity of the minimization problem with constraints on multiple loss terms, the modified differential method of multipliers is applied and shown to greatly improve the stability and robustness of the ABCDisCoTEC method, compared to grid search hyperparameter optimization procedures.
Keep this discovery
Explore connections, maps & timelines
Hayrapetyan, Aram [Yerevan Phys. Inst.], Makarenko, Vladimir [Yerevan Phys. Inst.] (ORCID:0000000284068605), Tumasyan, Armen [Yerevan Phys. Inst.] (ORCID:0009000006846742), Adam, Wolfgang [Vienna, OAW] (ORCID:0000000190994341), Andrejkovic, Janik Walter [Vienna, OAW], Benato, Lisa [Vienna, OAW] (ORCID:0000000151357489), Bergauer, Thomas [Vienna, OAW] (ORCID:0000000257860293), Damanakis, Konstantinos [Vienna, OAW] (ORCID:0000000153892872), Dragicevic, Marko [Vienna, OAW] (ORCID:0000000319676783), Giordano, Cristina [Vienna, OAW], Hussain, Priya Sajid [Vienna, OAW] (ORCID:0000000248255278), Jeitler, Manfred [Vienna, OAW; TU Vienna] (ORCID:0000000251419560), Krammer, Natascha [Vienna, OAW] (ORCID:0000000205480985), Li, Ang [Vienna, OAW] (ORCID:000000024547116X), Liko, Dietrich [Vienna, OAW] (ORCID:000000023380473X), Mikulec, Ivan [Vienna, OAW] (ORCID:0000000303852746), Schieck, Jochen [Vienna, OAW; TU Vienna] (ORCID:0000000210588093), Schöfbeck, Robert [Vienna, OAW; TU Vienna] (ORCID:0000000223328784), Schwarz, Dennis [Vienna, OAW] (ORCID:0000000238217331), Shooshtari, Maryam [Vienna, OAW] (ORCID:0009000488824887), Sonawane, Mangesh [Vienna, OAW] (ORCID:0000000305107010), Waltenberger, Wolfgang [Vienna, OAW] (ORCID:0000000262157228), Wulz, Claudia-Elisabeth [Vienna, OAW; TU Vienna] (ORCID:0000000192265812), Janssen, Tahys [Antwerp U.] (ORCID:0000000239984081), Kwon, Hyejin [Antwerp U.] (ORCID:0009000251655018), Van Laer, Thomas [Antwerp U.], Van Mechelen, Pierre [Antwerp U.] (ORCID:0000000287319051), Bierkens, Jas [Vrije U., Brussels] (ORCID:0000000208753977), Breugelmans, Nordin [Vrije U., Brussels], D'Hondt, Jorgen [Vrije U., Brussels] (ORCID:0000000295986241), Dansana, Soumya [Vrije U., Brussels] (ORCID:0000000277527471), De Moor, Alexandre [Vrije U., Brussels] (ORCID:0000000159641935), Delcourt, Martin [Vrije U., Brussels] (ORCID:0000000182061787), Heyen, Felix [Vrije U., Brussels], Hong, Yanwen [Vrije U., Brussels] (ORCID:0000000347522458), Kashko, Pavlo [Vrije U., Brussels] (ORCID:0000000270507152), Lowette, Steven [Vrije U., Brussels] (ORCID:0000000339849987), Makarenko, Inna [Vrije U., Brussels] (ORCID:0000000285534508), Müller, Denise [Vrije U., Brussels] (ORCID:0000000217524527), Song, Juhee [Vrije U., Brussels] (ORCID:0000000327315881), Tavernier, Stefaan [Vrije U., Brussels] (ORCID:0000000267929522), Tytgat, Michael [Vrije U., Brussels; Gent U.] (ORCID:0000000239902074), Van Onsem, Gerrit Patrick [Vrije U., Brussels] (ORCID:0000000216642337), Van Putte, Senne [Vrije U., Brussels] (ORCID:0000000315593606), Vannerom, David [Vrije U., Brussels] (ORCID:0000000227475095), Bilin, Bugra [Brussels U.] (ORCID:0000000314397128), Clerbaux, Barbara [Brussels U.] (ORCID:0000000185478211), Das, Aloke Kumar [Brussels U.], De Bruyn, Isabelle [Brussels U.] (ORCID:0000000317044360), De Lentdecker, Gilles [Brussels U.] (ORCID:0000000151247693), Evard, Hugues [Brussels U.] (ORCID:0009000550391462), Favart, Laurent [Brussels U.] (ORCID:0000000316457454), Gianneios, Paraskevas [Brussels U.] (ORCID:0009000372330738), Khalilzadeh, Ali [Brussels U.], Khan, Fakhri Alam [Brussels U.] (ORCID:000900022039277X), Malara, Andrea [Brussels U.] (ORCID:0000000186459282), Shahzad, Muhammad Aamir [Brussels U.], Thomas, Laurent [Brussels U.] (ORCID:0000000227563853), Vanden Bemden, Max [Brussels U.] (ORCID:0009000077257945), Vander Velde, Catherine [Brussels U.] (ORCID:0000000333927294), Vanlaer, Pascal [Brussels U.] (ORCID:0000000279314496), Zhang, Fengwangdong [Brussels U.] (ORCID:0000000261582468), De Coen, Maarten [Gent U.] (ORCID:0000000258547442), Dobur, Didar [Gent U.] (ORCID:0000000300124866), Gokbulut, Gul [Gent U.] (ORCID:0000000201756454), Knolle, Joscha [Gent U.] (ORCID:0000000247815704), Lambrecht, Luka [Gent U.] (ORCID:0000000191081560), Marckx, David [Gent U.] (ORCID:0000000167522290), Skovpen, Kirill [Gent U.] (ORCID:0000000211600621), Van Den Bossche, Niels [Gent U.] (ORCID:0000000329734991), van der Linden, Jan [Gent U.] (ORCID:000000027174781X), Vandenbroeck, Jules [Gent U.] (ORCID:0009000461413404), Wezenbeek, Liam [Gent U.] (ORCID:000000016952891X), Bein, Samuel [Louvain U.] (ORCID:0000000193877407), Benecke, Anna [Louvain U.] (ORCID:0000000302523609), Bethani, Agni [Louvain U.] (ORCID:0000000281507043), Bruno, Giacomo [Louvain U.] (ORCID:0000000188578197), Cappati, Alessandra [Louvain U.] (ORCID:0000000343860564), De Favereau De Jeneret, Jerome [Louvain U.] (ORCID:0000000317758574), Delaere, Christophe [Louvain U.] (ORCID:0000000187076021), Giammanco, Andrea [Louvain U.] (ORCID:0000000196408294), Guzel, Ahmet Oguz [Louvain U.] (ORCID:0000000294045933), Lemaitre, Vincent [Louvain U.], Lidrych, Jindrich [Louvain U.] (ORCID:0000000314390196), Mastrapasqua, Paola [Louvain U.] (ORCID:0000000220432367), Turkcapar, Semra [Louvain U.; Cukurova U.] (ORCID:0000000326080494), Alves, Gilvan [Rio de Janeiro, CBPF] (ORCID:0000000283691446), Coelho, Eduardo [Rio de Janeiro, CBPF] (ORCID:0000000161149907), Hensel, Carsten [Rio de Janeiro, CBPF] (ORCID:0000000188747624), Menezes De Oliveira, Thales [Rio de Janeiro, CBPF] (ORCID:0009000947298354), Mora Herrera, Clemencia [Rio de Janeiro, CBPF; Rio de Janeiro State U.] (ORCID:0000000339153170), Rebello Teles, Patricia [Rio de Janeiro, CBPF] (ORCID:0000000190298506), Soeiro, Mariana [Rio de Janeiro, CBPF], Tonelli Manganote, Edmilson José [Rio de Janeiro, CBPF; U. Campinas] (ORCID:0000000324598521), Vilela Pereira, Antonio [Rio de Janeiro, CBPF; Rio de Janeiro State U.] (ORCID:0000000331774626), Aldá Júnior, Walter Luiz [Rio de Janeiro State U.] (ORCID:0000000158559817), Barroso Ferreira Filho, Mapse [Rio de Janeiro State U.] (ORCID:0000000339040571), Brandao Malbouisson, Helena [Rio de Janeiro State U.] (ORCID:000000021326318X), Carvalho, Wagner [Rio de Janeiro State U.] (ORCID:0000000307386615), Chinellato, Jose [Rio de Janeiro State U.; U. Campinas]. 2026-08-26. Machine learning method for enforcing variable independence in background estimation with LHC data: ABCDisCoTEC. https://doi.org/10.1088/2632-2153%2Fae8c11
Cite the original work for its findings. Save a collection to share your selection of sources.