Search NASA⌕ Search

SEARCH · Search NASA

Results for “ATLAS COMPUTERS”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 145 records · Page 8

Data Placement Optimization for ATLAS in a Multi-Tiered Storage System within a Data Center

Scientific experiments and computations, especially in High Energy Physics, are generating and accumulating data at an unprecedented rate. Effectively managing this vast volume of data while ensuring efficient data analysis poses a significant challenge for data centers, which must integrate various storage technologies. This paper proposes addressing this challenge by designing and developing a precise data popularity prediction model utilizing state-of-theart AI/ML techniques. This model is crafted from the analysis of ATLAS data and access patterns. It enables us to migrate infrequently accessed data to more economical storage media, such as tape drives, while storing frequently accessed data on faster yet costlier storage media like HDD or SSD. This strategic approach ensures data is placed optimally into the appropriate storage classes, thereby maximizing storage capacity while minimizing data access latency for end-users. Furthermore, the paper includes a performance evaluation of the prediction model using various key metrics such as F1 score, accuracy, precision and recall. Finally, we present a prototype use case, leveraging real-world file access data to assess the model’s impact on performance.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

US ATLAS and US CMS HPC and Cloud Blueprint

The Large Hadron Collider (LHC) at CERN houses two general purpose detectors - ATLAS and CMS - which conduct physics programs over multi-year runs to generate increasingly precise and extensive datasets. The efforts of the CMS and ATLAS collaborations lead to the discovery of the Higgs boson, a fundamental particle that gives mass to other particles, representing a monumental achievement in the field of particle physics that was recognized with the awarding of the Nobel Prize in Physics in 2013 to François Englert and Peter Higgs. These collaborations continue to analyze data from the LHC and are preparing for the high luminosity data taking phase at the end of the decade. The computing models of these detectors rely on a distributed processing grid hosted by more than 150 associated universities and laboratories worldwide. However, such new data will require a significant expansion of the existing computing infrastructure. To address this, both collaborations have been working for years on integrating High Performance Computers (HPC) and commercial cloud resources into their infrastructure and continue to assess the potential role of such resources in order to cope with the demands of the new high luminosity era. US ATLAS and US CMS computing management have charged the authors to provide a blueprint document looking at current and possibly future use of HPC and Cloud resources, outlining integration models, possibilities, challenges and costs. The document will address key questions such as the optimal use of resources for the experiments and funding agencies, the main obstacles that need to be overcome for resource adoption, and areas that require more attention.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

Monitoring Geothermal Features in Yellowstone National Park with ATLAS Multispectral Imagery

The National Park Service (NPS) must produce an Environmental Impact Statement for each proposed development in the vicinity of known geothermal resource areas (KGRAs) in Yellowstone National Park. In addition, the NPS monitors indicator KGRAs for environmental quality and is still in the process of mapping many geothermal areas. The NPS currently maps geothermal features with field survey techniques. High resolution aerial multispectral remote sensing in the visible, NIR, SWIR, and thermal spectral regions could enable YNP geothermal features to be mapped more quickly and in greater detail In response, Yellowstone Ecosystems Studies, in partnership with NASA's Commercial Remote Sensing Program, is conducting a study on the use of Airborne Terrestrial Applications Sensor (ATLAS) multispectral data for monitoring geothermal features in the Upper Geyser Basin. ATLAS data were acquired at 2.5 meter resolution on August 17, 2000. These data were processed into land cover classifications and relative temperature maps. For sufficiently large features, the ATLAS data can map geothermal areas in terms of geyser pools and hot springs, plus multiple categories of geothermal runoff that are apparently indicative of temperature gradients and microbial matting communities. In addition, the ATLAS maps clearly identify geyserite areas. The thermal bands contributed to classification success and to the computation of relative temperature. With masking techniques, one can assess the influence of geothermal features on the Firehole River. Preliminary results appear to confirm ATLAS data utility for mapping and monitoring geothermal features. Future work will include classification refinement and additional validation.

Spruce, Joseph↗

Langston University - High Energy Physics (LU-HEP)

This final report is presented by Langston University (LU) for the project entitled "Langston University High Energy Physics" (LUHEP) under the direction of principal investigator (PI) and project director Professor Joel Snow. The project encompassed high energy physics research performed at hadron colliders. The PI is a collaborator on the DZero experiment at Fermi National Accelerator Laboratory in Batavia, IL, USA and the ATLAS experiment at CERN in Geneva, Switzerland and was during the entire project period from April 1, 1999 until May 14, 2012. Both experiments seek to understand the fundamental constituents of the physical universe and the forces that govern their interactions. In 1999 as member of the Online Systems group for Run 2 the PI developed a cross-platform Python-based, Graphical User Interface (GUI) application for monitoring and control of EPICS based devices for control room use. This served as a model for other developers to enhance and build on for further monitoring and control tasks written in Python. Subsequently the PI created and developed a cross-platform C++ GUI utilizing a networked client-server paradigm and based on ROOT, the object oriented analysis framework from CERN. The GUI served as a user interface to the Examine tasks running in the D\O\ control room which monitored the status and integrity of data taking for Run 2. The PI developed the histogram server/control interface to the GUI client for the EXAMINE processes. The histogram server was built from the ROOT framework and was integrated into the D\O\ framework used for online monitoring programs and offline analysis. The PI developed the first implementation of displaying histograms dynamically generated by ROOT in a Web Browser. The PI's work resulted in several talks and papers at international conferences and workshops. The PI established computing software infrastructure at LU and U. Oklahoma (OU) to do analysis of DZero production data and produce simulation data for the experiment. Eventually this included the FNAL SAM data grid system, the SAMGrid (SG) infrastructure, and the Open Science Grid software stacks for computing and storage elements. At the end of 2003 Snow took on the role of global Monte Carlo production coordinator for the DØ experiment. A role which continues til this day. In January of 2004 Snow started working with the SAMGrid development team to help debug, deploy, and integrate SAMGrid with DØ Monte Carlo production. Snow installed and configured SG execution and client sites at LUHEP and OUHEP, and a SG scheduler site at LUHEP. The PI developed a python based GUI (DAJ) that acts as a front end for job submission to SAMGrid. The GUI interfaces to the DZero Mone Carlo (MC) request system that uses SAM to manage MC requests by the physics analysis groups. DAJ significantly simplified SG job submission and was deployed in DZero in an effort to increase the user base of SG. The following year was the advent of SAMGrid job submission to the Open Science Grid (OSG) and LHC Computing Grid (LCG) through a forwarding mechanism. The PI oversaw the integration of these grids into the existing production infrastructure. The PI developed an automatic MC (Automc) request processing system capable of operating without user intervention (other than getting grid credentials), and able to submit to any number of sites on various grids. The system manages production at all but 2 sites. The system was deployed at Fermilab and remains operating there today. The PI's work in distributed computing resulted in several talks at international conferences. UTA, OU, and LU were chosen as the collaborating institutions that form the Southwest Tier 2 Center (SWT2) for ATLAS. During the project period the PI contributed to the online and offline software infrastructure through his work with the Run 2 online group, and played a major role in Monte Carlo production for DZero. During the part of the project period in which the PI served as MC production coordinator MC production increased very significantly. In the first year of the PI's tenure as production coordinator production was 159M events and 6.7~TB of data. During the last year of the project period production was 2,342~M events and 262~TB of data. That is a factor of 15 increase in events and 39 in data volume. The increase occurred with improvements in computer hardware and networks, through the use of grid technology on diverse resources, and through increased automation and efficiency of the production process. LU HEP developed and deployed the automatic MC request processing system in use at FNAL. The complementary strategies of automation and grid production served DZero well. Fermilab has recognized LU HEP's contribution to DZero by allowing the PI to devote full time to research activities by appointing him a guest scientist for the last six years of the project period.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

An intelligent Data Delivery Service for and beyond the ATLAS experiment

The intelligent Data Delivery Service (iDDS) has been developed to cope with the huge increase of computing and storage resource usage in the coming LHC data taking. iDDS has been designed to intelligently orchestrate workflow and data management systems, decoupling data pre-processing, delivery, and main processing in various workflows. It is an experiment-agnostic service around a workflow-oriented structure to work with existing and emerging use cases in ATLAS and other experiments. Here we will present the motivation for iDDS, its design schema and architecture, use cases and current status, and plans for the future.

97 MATHEMATICS AND COMPUTING↗

Low-complexity atlas-based prostate segmentation by combining global, regional, and local metrics

Purpose: To improve the efficiency of atlas-based segmentation without compromising accuracy, and to demonstrate the validity of the proposed method on MRI-based prostate segmentation application. Methods: Accurate and efficient automatic structure segmentation is an important task in medical image processing. Atlas-based methods, as the state-of-the-art, provide good segmentation at the cost of a large number of computationally intensive nonrigid registrations, for anatomical sites/structures that are subject to deformation. In this study, the authors propose to utilize a combination of global, regional, and local metrics to improve the accuracy yet significantly reduce the number of required nonrigid registrations. The authors first perform an affine registration to minimize the global mean squared error (gMSE) to coarsely align each atlas image to the target. Subsequently, atarget-specific regional MSE (rMSE), demonstrated to be a good surrogate for dice similarity coefficient (DSC), is used to select a relevant subset from the training atlas. Only within this subset are nonrigid registrations performed between the training images and the target image, to minimize a weighted combination of gMSE and rMSE. Finally, structure labels are propagated from the selected training samples to the target via the estimated deformation fields, and label fusion is performed based on a weighted combination of rMSE and local MSE (lMSE) discrepancy, with proper total-variation-based spatial regularization. Results: The proposed method was applied to a public database of 30 prostate MR images with expert-segmented structures. The authors’ method, utilizing only eight nonrigid registrations, achieved a performance with a median/mean DSC of over 0.87/0.86, outperforming the state-of-the-art full-fledged atlas-based segmentation approach of which the median/mean DSC was 0.84/0.82 when applying to their data set. Conclusions: The proposed method requires a fixed number of nonrigid registrations, independent of atlas size, providing desirable scalability especially important for a large or growing atlas. When applied to prostate segmentation, the method achieved better performance to the state-of-the-art atlas-based approaches, with significant improvement in computation efficiency. The proposed rationale of utilizing jointly global, regional, and local metrics, based on the information characteristic and surrogate behavior for registration and fusion subtasks, can be extended naturally to similarity metrics beyond MSE, such as correlation or mutual information types.

62 RADIOLOGY AND NUCLEAR MEDICINE↗

The ATLAS integrated structural analysis and design software system

The ATLAS system provides an extensive set of integrated technical computer-program modules for the analysis and design of general structural configurations, as well as capabilities that are particularly suited for the aeroelastic design of flight vehicles. The system is based on the stiffness formulation of the finite element structural analysis method and can be executed in batch and interactive computing environments on CDC 6600/CYBER computers. Problem-definition input data are written in an engineering-oriented language using a free field format. Input-data default values, generation options, and data quality checks provided by the preprocessors minimize the amount of data and flowtime for problem definition/verfication. Postprocessors allow selected input and calculated data to be extracted, manipulated, and displayed via on-line and off-line prints or plots for monitoring and verifying problem solutions. The sequence and mode of execution of selected program modules are controlled by a common user-oriented language.

Dreisbach, R. L.↗

A Conditions Data Management System for HEP Experiments

Conditions data infrastructure for both ATLAS and CMS have to deal with the management of several Terabytes of data. Distributed computing access to this data requires particular care and attention to manage request-rates of up to several tens of kHz. Thanks to the large overlap in use cases and requirements, ATLAS and CMS have worked towards a common solution for conditions data management with the aim of using this design for data-taking in Run 3. In the meantime other experiments, including NA62, have expressed an interest in this cross- experiment initiative. For experiments with a smaller payload volume and complexity, there is particular interest in simplifying the payload storage. The conditions data management model is implemented in a small set of relational database tables. A prototype access toolkit consisting of an intermediate web server has been implemented, using standard technologies available in the Java community. Access is provided through a set of REST services for which the API has been described in a generic way using standard Open API specications, implemented in Swagger. Such a solution allows the automatic generation of client code and server stubs and further allows changes in the backend technology transparently. An important advantage of using a REST API for conditions access is the possibility of caching identical URLs, addressing one of the biggest challenges that large distributed computing solutions impose on conditions data access, avoiding direct DB access by means of standard web proxy solutions.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗

Rate-capability of the VMM3a front-end in the RD51 Scalable Readout System

The VMM3a is an Application Specific Integrated Circuit (ASIC), specifically developed for the readout of gaseous detectors. Originally developed within the ATLAS New Small Wheel (NSW) upgrade, it has been successfully integrated into the Scalable Readout System (SRS) of the RD51 collaboration. This allows, to use the VMM3a also in small laboratory set-ups and mid-scale experiments, which make use of Micro-Pattern Gaseous Detectors (MPGDs). As part of the integration of the VMM3a into the SRS, the readout and data transfer scheme was optimised to reach a high rate-capability of the entire readout system and profit from the VMM3a's high single-channel rate-capability of 3.6Mhits/s. The optimisation focused mainly on the handling of the data output stream of the VMM3a, but also on the development of a trigger-logic between the front-end cards and the DAQ computer. In this article, two firmware implementations of the non-ATLAS continuous readout mode are presented, as well as the implementation of the trigger-logic. Afterwards, a short overview on X-ray imaging results is presented, to illustrate the high rate-capability from an application point-of-view.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

High-Throughput Computing on High-Performance Platforms: A Case Study

The computing systems used by LHC experiments has historically consisted of the federation of hundreds to thousands of distributed resources, ranging from small to mid-size resource. In spite of the impressive scale of the existing distributed computing solutions, the federation of small to mid-size resources will be insufficient to meet projected future demands. This paper is a case study of how the ATLAS experiment has embraced Titan -- a DOE leadership facility in conjunction with traditional distributed high- throughput computing to reach sustained production scales of approximately 52M core-hours a years. The three main contributions of this paper are: (i) a critical evaluation of design and operational considerations to support the sustained, scalable and production usage of Titan; (ii) a preliminary characterization of a next generation executor for PanDA to support new workloads and advanced execution modes; and (iii) early lessons for how current and future experimental and observational systems can be integrated with production supercomputers and other platforms in a general and extensible manner.

Oleynik, D↗

P78-2 SCATHA environmental data atlas

The 100 eV to approximately 1 MeV plasma environment encountered by the P78-2 Spacecraft Charging at High Altitudes (SCATHA) satellite during its initial operation period was studied. Forty-four days of 10 minute averages of the four moments of the electron and ion distribution functions calculated from the SC5 and SC9 energetic particle measurements were analyzed to determine occurrence frequency, local time variation, geomagnetic activity variation, and L shell variation. The single and double Maxwellian parameters derived from the four moments were similarly analyzed. The interrelationships between the moments and derived parameters were computed and the results compared with the ATS-5 and ATS-6 atlas. Results of this analysis establish a baseline range for the SCATHA plasma environment.

Mulen, E. G.↗

Enhancing atlas based segmentation with multiclass linear classifiers

Purpose: To present a method to enrich atlases for atlas based segmentation. Such enriched atlases can then be used as a single atlas or within a multiatlas framework. Methods: In this paper, machine learning techniques have been used to enhance the atlas based segmentation approach. The enhanced atlas defined in this work is a pair composed of a gray level image alongside an image of multiclass classifiers with one classifier per voxel. Each classifier embeds local information from the whole training dataset that allows for the correction of some systematic errors in the segmentation and accounts for the possible local registration errors. The authors also propose to use these images of classifiers within a multiatlas framework: results produced by a set of such local classifier atlases can be combined using a label fusion method. Results: Experiments have been made on the in vivo images of the IBSR dataset and a comparison has been made with several state-of-the-art methods such as FreeSurfer and the multiatlas nonlocal patch based method of Coupé or Rousseau. These experiments show that their method is competitive with state-of-the-art methods while having a low computational cost. Further enhancement has also been obtained with a multiatlas version of their method. It is also shown that, in this case, nonlocal fusion is unnecessary. The multiatlas fusion can therefore be done efficiently. Conclusions: The single atlas version has similar quality as state-of-the-arts multiatlas methods but with the computational cost of a naive single atlas segmentation. The multiatlas version offers a improvement in quality and can be done efficiently without a nonlocal strategy.

60 APPLIED LIFE SCIENCES↗

Research on Spectroscopy, Opacity, and Atmospheres

I propose to continue providing observers with basic data for interpreting spectra from stars, novas, supernovas, clusters, and galaxies. These data will include allowed forbidden line lists both laboratory and computed, for the first five to ten ions of all atoms and for all relevant diatomic molecules. I will eventually expend to all ions of the first thirty elements to treat far UV end X-ray spectra, and for envelope opacities. I also include triatomic molecules providing by other researchers. I have made CDs with Partridge and Schwanke's water data for work on M stars.The luna data also serve as input to my model atmosphere and synthesis programs that generated energy distributions, photometry, limb darkening, and spectra that can be used for planning observations and for fitting observed spectra. The spectrum synthesis programs produce detailed plots with the line identified. Grids of stellar spectra can be used for radial velocity-, rotation-, or abundance templates and for population synthesis. I am fitting spectra of bright stars to test the data and to produce atlases to guide observer. For each star the whole spectrum is computed from the UV to the far IR. The line data, opacities, models, spectra, and programs are freely distributed on CDs and on my web site and represent a unique resource for many NASA programs.

Oliversen, Ronald↗

Research on Spectroscopy, Opacity, and Atmospheres

I propose to continue providing observers with basic data for interpreting spectra from stars, novas, supernovas, clusters, and galaxies. These data will include allowed and forbidden line lists, both laboratory and computed, for the first five to ten ions of all atoms and for all relevant diatomic molecules. I will eventually expand to all ions of the first thirty elements to treat far UV and X-ray spectra, and for envelope opacities. I also include triatomic molecules provided by other researchers. I have also made CDs with Partridge and Schwenke's water data for work on UV stars. The line data also serve as input to my model atmosphere and synthesis programs that generate energy distributions, photometry, limb darkening, and spectra that can be used for planning observations and for fitting observed spectra. The spectrum synthesis programs produce detailed plots with the lines identified. Grids of stellar spectra can be used for radial velocity-, rotation-, or abundance templates and for population synthesis. I am fitting spectra of bright stars to test the data and to produce atlases to guide observers. For each star the whole spectrum is computed from the UV to the far IR. The line data, opacities, models, spectra, and programs are freely distributed on CDs and on my Web site and represent a unique resource for many NASA programs. I am now in full production of new line lists for atoms. I am computing all ions of all elements from H to Zn and the first 5 ions of all the heavier elements, about 800 ions. For each ion I treat as many as 61 even and 61 odd configurations, computing all energy levels and eigenvectors. The Hamiltonian is determined from a scaled-Hartree-Fock starting guess by least squares fitting the observed energy levels. The average energy of each configuration is used in computing scaled-Thomas-Fermi-Dirac wavefunctions for each configuration which in turn are used to compute allowed and forbidden transition integrals. These are multiplied into the LS allowed and forbidden transition arrays. The transition arrays are transformed to the observed coupling to yield the allowed and forbidden line lists. Results are put on the web as they are finished. Provided I get funding,there will be more than 500 million lines. I will then compare ion by ion, to all the laboratory and computed data in the literature and make up a working line list for spectrum synthesis and opacity calculations with the best available data. As the laboratory spectrum analyses are improved, I will redo the calculations with the new energy levels. My original plan when I started the new calculations was to run through all the atoms using my old Cray programs from the 1980's that were limited to 1100 x 1100 arrays in the Hamiltonian for each J. Then I would go back and rerun the more complicated cases with 3000 x 3000 arrays so that I could include many more configurations and more configuration interactions. At present I am limited to 61 even and 61 odd configurations and I try to include everything up through n = 9. The current program runs on Alpha workstations. I decided to test the big program on Fe I and Fe II to see whether there was any great difference in the low configurations compared to those from the Cray program. Besides increasing the number of E1 lines by a factor of 6 to 7.7 million, there was an unexpected result: the electric quadrupole transitions were 10 times stronger than before because the transition integrals are weighted by r(exp 2) ---they become very large for high n, and because there are numerous configuration interactions that mix the low and high configurations. As a check I was able to reproduce Carstang's (1962) lower results by running his three configurations with my program. Since my model atom is still only a subset of a real Fe II ion, the true quadrupole A values are probably larger than mine. The magnetic dipole lines are affected by the mixing but the overall scale does not change. Because of this scovery I decided that there was no point in computing the small array cases. I have been running with as many configurations as I can and with thousands of parameters in the Hamiltonian. The computer runs take much longer to set up and produce than I had expected. I have concentrated on redoing the low iron group spectra, especially to get data for supernova modelers. I have done only Ca I -- Zn I, Ca II -- Zn II, CU I -- Cu XXIX, Zn I - Zn XXX, for practice at high stages of ionization, C I, C II, S I, and CL I and Ag I for people who were working on the laboratory spectra. Check my web site kurucz.harvard.edu for current additions. My latest calculations have been for carbon I and sulphur I, and silicon I is under way using the same elaborate approach as for C I, which took many months to do. These line lists greatly increase the number of lines in the ultraviolet, in the visible, and especially in the infrared. They will increase the opacity in A, F, and G stars. They will account for many unidentified lines in the sun.

Kurucz, Robert L.↗

Direct integration transmittance model

A transmittance model was developed for the 200-2000/cm region for interpretation of high spectral resolution measurements of laboratory absorption and of planetary thermal emission. The high spectral resolution requires transmittances to be computed monochromatically by summing the contribution of individual molecular absorption lines. A magnetic tape atlas of H2O,O3, and CO2 molecular line parameters serves as input to the transmittance model with simple empirical representations used for continuum regions wherever suitable laboratory data exist. The theoretical formulation of the transmittance model and the computational procedures used for the evaluation of the transmittances are discussed. Application is demonstrated of the model to several homogenous path laboratory absorption examples.

Kunde, V. G.↗

Direct integration transmittance model

A transmittance model has been developed for interpretation of high spectral resolution measurements of laboratory absorption and of planetary thermal emission. The high spectral resolution requires transmittances to be computed monochromatically by summing the contribution of individual molecular absorption lines. A magnetic tape atlas of H2O, O3, and CO2 molecular line parameters serves as input to the transmittance model with simple empirical representations used for continuum regions wherever suitable laboratory data exist. The theoretical formulation of the transmittance model and the computational procedures used for the evaluation of the transmittances are discussed, and application of the model to several homogeneous-path laboratory absorption examples is demonstrated.

Kunde, V. G.↗

The OSG Open Facility: an on-ramp for opportunistic scientific computing

The Open Science Grid (OSG) is a large, robust computing grid that started primarily as a collection of sites associated with large HEP experiments such as ATLAS, CDF, CMS, and DZero, but has evolved in recent years to a much larger user and resource platform. In addition to meeting the US LHC community’s computational needs, the OSG continues to be one of the largest providers of distributed high-throughput computing (DHTC) to researchers from a wide variety of disciplines via the OSG Open Facility. The Open Facility consists of OSG resources that are available opportunistically to users other than resource owners and their collaborators. In the past two years, the Open Facility has doubled its annual throughput to over 200 million wall hours. More than half of these resources are used by over 100 individual researchers from over 60 institutions in fields such as biology, medicine, math, economics, and many others. Over 10% of these individual users utilized in excess of 1 million computational hours each in the past year. The largest source of these cycles is temporary unused capacity at institutions affiliated with US LHC computational sites. An increasing fraction, however, comes from university HPC clusters and large national infrastructure supercomputers offering unused capacity. Such expansions have allowed the OSG to provide ample computational resources to both individual researchers and small groups as well as sizable international science collaborations such as LIGO, AMS, IceCube, and sPHENIX. Opening up access to the Fermilab FabrIc for Frontier Experiments (FIFE) project has also allowed experiments such as mu2e and NOvA to make substantial use of Open Facility resources, the former with over 40 million wall hours in a year. We present how this expansion was accomplished as well as future plans for keeping the OSG Open Facility at the forefront of enabling scientific research by way of DHTC.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

An intelligent Data Delivery Service for and beyond the ATLAS experiment

The intelligent Data Delivery Service (iDDS) has been developed to cope with the huge increase of computing and storage resource usage in the coming LHC data taking. It has been designed to intelligently orchestrate workflows and data management systems, decoupling data pre-processing, delivery, and primary processing in large scale workflows. It is an experiment-agnostic service that has been deployed to serve data carousel (orchestrating efficient processing of tape-resident data), machine learning hyperparameter optimization, active learning, and other complex multi-stage workflows defined via DAG (Directed Acyclic Graph), CWL (Common Workflow Language) and other descriptions, including a growing number of analysis workflows. We will at first introduce some deployed use cases in a summary. Then we will focus on new improvements and use cases under developments in ATLAS, Rubin Observatory and sPHENIX, together with future efforts.

97 MATHEMATICS AND COMPUTING↗