Engineering topics

Mendez, Derek

Publications and source records attributed to Mendez, Derek.

5 records

DOE OSTI2024

ExaFEL: extreme-scale real-time data processing for X-ray free electron laser science

ExaFEL is an HPC-capable X-ray Free Electron Laser (XFEL) data analysis software suite for both Serial Femtosecond Crystallography (SFX) and Single Particle Imaging (SPI) developed in collaboration with the Linac Coherent Lightsource (LCLS), Lawrence Berkeley National Laboratory (LBNL) and Los Alamos National Laboratory. ExaFEL supports real-time data analysis via a cross-facility workflow spanning LCLS and HPC centers such as NERSC and OLCF. Our work therefore constitutes initial path-finding for the US Department of Energy's (DOE) Integrated Research Infrastructure (IRI) program. We present the ExaFEL team's 7 years of experience in developing real-time XFEL data analysis software for the DOE's exascale supercomputers. We present our experiences and lessons learned with the Perlmutter and Frontier supercomputers. Furthermore we outline essential data center services (and the implications for institutional policy) required for real-time data analysis. Finally we summarize our software and performance engineering approaches and our experiences with NERSC's Perlmutter and OLCF's Frontier systems. This work is intended to be a practical blueprint for similar efforts in integrating exascale compute resources into other cross-facility workflows.

59 BASIC BIOLOGICAL SCIENCES↗

DOE OSTI2024

Real‐time XFEL data analysis at SLAC and NERSC: A trial run of nascent exascale experimental data analysis

X‐ray scattering experiments using free electron lasers (XFELs) are a powerful tool to determine the molecular structure and function of unknown samples (such as COVID‐19 viral proteins). XFEL experiments are a challenge to computing in two ways: (i) due to the high cost of running XFELs, a fast turnaround time from data acquisition to data analysis is essential to make informed decisions on experimental protocols; (ii) data‐collection rates are growing exponentially, requiring new scalable algorithms. Here we report our experiences analyzing data from two experiments at the Linac Coherent Light Source (LCLS) during September 2020. Raw data were analyzed on NERSC's Cori XC40 system, using the Superfacility paradigm: our workflow automatically moves raw data between LCLS and NERSC, where it is analyzed using the software package CCTBX. We achieved real time data analysis with a turnaround time from data acquisition to full molecular reconstruction in as little as 10 min—sufficient time for the experiment's operators to make informed decisions. By hosting the data analysis on Cori, and by automating LCLS‐NERSC interoperability, we achieved a data analysis rate which matches the data acquisition rate. Completing data analysis within 10 min is a first for XFEL experiments and an important milestone if we are to keep up with data‐collection trends.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

DOE OSTI2024

Deep residual networks for crystallography trained on synthetic data

The use of artificial intelligence to process diffraction images is challenged by the need to assemble large and precisely designed training data sets. To address this, a codebase called Resonet was developed for synthesizing diffraction data and training residual neural networks on these data. Here, two per-pattern capabilities of Resonet are demonstrated: (i) interpretation of crystal resolution and (ii) identification of overlapping lattices. Resonet was tested across a compilation of diffraction images from synchrotron experiments and X-ray free-electron laser experiments. Crucially, these models readily execute on graphics processing units and can thus significantly outperform conventional algorithms. While Resonet is currently utilized to provide real-time feedback for macromolecular crystallography users at the Stanford Synchrotron Radiation Lightsource, its simple Python-based interface makes it easy to embed in other processing frameworks. This work highlights the utility of physics-based simulation for training deep neural networks and lays the groundwork for the development of additional models to enhance diffraction collection and analysis.

36 MATERIALS SCIENCE↗

DOE OSTI2023

In Situ Structural Observation of a Substrate- and Peroxide-Bound High-Spin Ferric-Hydroperoxo Intermediate in the P450 Enzyme CYP121

The P450 enzyme CYP121 from Mycobacterium tuberculosis catalyzes a carbon–carbon (C–C) bond coupling cyclization of the dityrosine substrate containing a diketopiperazine ring, cyclo(l-tyrosine-l-tyrosine) (cYY). An unusual high-spin (S = 5/2) ferric intermediate maximizes its population in less than 5 ms in the rapid freeze-quenching study of CYP121 during the shunt reaction with peracetic acid or hydrogen peroxide in acetic acid solution. Here, we show that this intermediate can also be observed in the crystalline state by EPR spectroscopy. By developing an on-demand-rapid-mixing method for time-resolved serial femtosecond crystallography with X-ray free-electron laser (tr-SFX-XFEL) technology covering the millisecond time domain and without freezing, we structurally monitored the reaction in situ at room temperature. After a 200 ms peracetic acid reaction with the cocrystallized enzyme–substrate microcrystal slurry, a ferric-hydroperoxo intermediate is observed, and its structure is determined at 1.85 Å resolution. The structure shows a hydroperoxyl ligand between the heme and the native substrate, cYY. The oxygen atoms of the hydroperoxo are 2.5 and 3.2 Å from the iron ion. The end-on binding ligand adopts a near-side-on geometry and is weakly associated with the iron ion, causing the unusual high-spin state. This compound 0 intermediate, spectroscopically and structurally observed during the catalytic shunt pathway, reveals a unique binding mode that deviates from the end-on compound 0 intermediates in other heme enzymes. The hydroperoxyl ligand is only 2.9 Å from the bound cYY, suggesting an active oxidant role of the intermediate for direct substrate oxidation in the nonhydroxylation C–C bond coupling chemistry.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

DOE OSTI2023

Accelerating x-ray tracing for exascale systems using Kokkos

The upcoming exascale computing systems Frontier and Aurora will draw much of their computing power from GPU accelerators. The hardware for these systems will be provided by AMD and Intel, respectively, each supporting their own GPU programming model. The challenge for applications that harness one of these exascale systems will be to avoid lock-in and to preserve performance portability. We report here on our results of using Kokkos to accelerate a real-world application on NERSC's Perlmutter Phase 1 (using NVIDIA A100 accelerators) and Crusher, the testbed system for OLCF's Frontier (using AMD MI250X). By porting to Kokkos, we successfully ran the same X-ray tracing code on both systems and achieved speed-ups between 13 % and 66 % compared to the original CUDA code. Finally, these results are a highly encouraging demonstration of using Kokkos to accelerate production science code.

97 MATHEMATICS AND COMPUTING↗