Search NASA⌕ Search

SEARCH · Search NASA

Results for “Distribution Grid Atlas”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

47 records · Page 3

BigPanDA: PanDA Workload Management System and its Applications beyond ATLAS

Modern experiments collect peta-scale volumes of data and utilize vast, geographically distributed computing infrastructure that serves thousands of scientists around the world. Requirements for rapid, near real-time data processing, fast analysis cycles and need to run massive detector simulations to support data analysis pose special premium on efficient use of available computational resources. A sophisticated Workload Management System (WMS) is needed to coordinate the distribution and processing of data and jobs in such environment. The ATLAS experiment at CERN uses PanDA (Production and Data Analysis) Workload Management System for managing the workflow for all data processing on over 150 data centers. While PanDAcurrently uses more than 250,000 cores with a peak performance of 0.3 petaFLOPS, it runs around 2 million jobs per day on hundreds of Grid sites and serving thousands of ATLAS users. In 2017 about 1.5 exabytes of data were processed with PanDA.In 2012 BigPanDA project project was started with aim to introduce new types of computing resources into ATLAS computing infrastructure, but also to offering PanDA features to different data-intensive applications for projects and experiments outside of ATLAS and High-Energy and Nuclear Physics. In this article we will present accomplishments and discuss possible directions for future work.

Svirin, Pavlo↗

ATLAS FTK Challenge: Simulation of a Billion-fold Hardware Parallelism

During the current LHC shutdown period the ATLAS experiment will upgrade the Trigger and Data Acquisition system to include a hardware tracker coprocessor: the Fast TracKer (FTK). The FTK receives data from the 80 million of channels of the ATLAS silicon detector, identifying charged tracks and reconstructing their parameters at a rate of up to 100 KHz and within 100 microseconds. To achieve this performance, the FTK system identifies candidate tracks utilizing the computing power of a custom ASIC chip with associative memory (AM) designed to perform “pattern matching” at very high speed; track parameters are then calculated using modern FPGAs. A detailed simulation of this massive system has been developed with the goal of supporting the hardware design and studying its impact in the ATLAS online event selection at high LHC luminosities. We present the issues related to emulating FTK on a general-purpose CPU platform, using ATLAS computing Grid resources, and the solutions developed in order to mitigate these problems and allow the studies required to support the system design, construction and installation.

Distributed Computing↗

Integration Of PanDA Workload Management System With Supercomputers for ATLAS and Data Intensive Science

The.LHC, operating at CERN, is leading Big Data driven scientific explorations. Experiments at the LHC explore the fundamental nature of matter and the basic forces that shape our universe. ATLAS, one of the largest collaborations ever assembled in the sciences, is at the forefront of research at the LHC. To address an unprecedented multi-petabyte data processing challenge, the ATLAS experiment is relying on a heterogeneous distributed computational infrastructure. The ATLAS experiment uses PanDA (Production and Data Analysis) Workload Management System for managing the workflow for all data processing on over 150 data centers. Through PanDA, ATLAS physicists see a single computing facility that enables rapid scientific breakthroughs for the experiment, even though the data centers are physically scattered all over the world. While PanDA currently uses more than 250,000 cores with a peak performance of 0.3 petaFLOPS, LHC data taking runs require more resources than grid can possibly provide. To alleviate these challenges, LHC experiments are engaged in an ambitious program to expand the current computing model to include additional resources such as the opportunistic use of supercomputers. We will describe a project aimed at integration of PanDA WMS with supercomputers in United States, in particular with Titan supercomputer at Oak Ridge Leadership Computing Facility. Current approach utilizes modified PanDA pilot framework for job submission to the supercomputers batch queues and local data management, with light-weight MPI wrappers to run single threaded workloads in parallel on LCFs multi-core worker nodes. This implementation was tested with a variety of Monte-Carlo workloads on several supercomputing platforms for ALICE and ATLAS experiments and it is in full pro duction for the ATLAS since September 2015. We will present our current accomplishments with running PanDA at supercomputers and demonstrate our ability to use PanDA as a portal independent of the computing facilities infrastructure for High Energy and Nuclear Physics as well as other data-intensive science applications, such as bioinformatics and astro-particle physics.

97 MATHEMATICS AND COMPUTING↗

Integration Of PanDA Workload Management System With Supercomputers for ATLAS and Data Intensive Science

The Large Hadron Collider (LHC), operating at the international CERN Laboratory in Geneva, Switzerland, is leading Big Data driven scientific explorations. Experiments at the LHC explore the fundamental nature of matter and the basic forces that shape our universe, and were recently credited for the discovery of a Higgs boson. ATLAS, one of the largest collaborations ever assembled in the sciences, is at the forefront of research at the LHC. To address an unprecedented multi-petabyte data processing challenge, the ATLAS experiment is relying on a heterogeneous distributed computational infrastructure. The ATLAS experiment uses PanDA (Production and Data Analysis) Workload Management System for managing the workflow for all data processing on over 150 data centers. Through PanDA, ATLAS physicists see a single computing facility that enables rapid scientific breakthroughs for the experiment, even though the data centers are physically scattered all over the world. While PanDA currently uses more than 250,000 cores with a peak performance of 0.3 petaFLOPS, LHC data taking runs require more resources than Grid computing can possibly provide. To alleviate these challenges, LHC experiments are engaged in an ambitious program to expand the current computing model to include additional resources such as the opportunistic use of supercomputers. We will describe a project aimed at integration of PanDA WMS with supercomputers in United States, Europe and Russia (in particular with Titan supercomputer at Oak Ridge Leadership Computing Facility (OLCF), MIRA supercomputer at Argonne Leadership Computing Facilities (ALCF), Supercomputer at the National Research Center Kurchatov Institute , IT4 in Ostrava and others). Current approach utilizes modified PanDA pilot framework for job submission to the supercomputers batch queues and local data management, with light-weight MPI wrappers to run single threaded workloads in parallel on LCFs multi-core worker nodes. This implementation was tested with a variety of Monte-Carlo workloads on several supercomputing platforms for ALICE and ATLAS experiments and it is in full production for the ATLAS experiment since September 2015. We will present our current accomplishments with running PanDA WMS at supercomputers and demonstrate our ability to use PanDA as a portal independent of the computing facilities infrastructure for High Energy and Nuclear Physics as well as other data-intensive science applications, such as bioinformatics and astro-particle physics.

Data-intensive science↗

INTEGRATION OF PANDA WORKLOAD MANAGEMENT SYSTEM WITH SUPERCOMPUTERS

Abstract The Large Hadron Collider (LHC), operating at the international CERN Laboratory in Geneva, Switzerland, is leading Big Data driven scientific explorations. Experiments at the LHC explore the funda- mental nature of matter and the basic forces that shape our universe, and were recently credited for the dis- covery of a Higgs boson. ATLAS, one of the largest collaborations ever assembled in the sciences, is at the forefront of research at the LHC. To address an unprecedented multi-petabyte data processing challenge, the ATLAS experiment is relying on a heterogeneous distributed computational infrastructure. The ATLAS experiment uses PanDA (Production and Data Analysis) Workload Management System for managing the workflow for all data processing on over 140 data centers. Through PanDA, ATLAS physicists see a single computing facility that enables rapid scientific breakthroughs for the experiment, even though the data cen- ters are physically scattered all over the world. While PanDA currently uses more than 250000 cores with a peak performance of 0.3+ petaFLOPS, next LHC data taking runs will require more resources than Grid computing can possibly provide. To alleviate these challenges, LHC experiments are engaged in an ambitious program to expand the current computing model to include additional resources such as the opportunistic use of supercomputers. We will describe a project aimed at integration of PanDA WMS with supercomputers in United States, Europe and Russia (in particular with Titan supercomputer at Oak Ridge Leadership Com- puting Facility (OLCF), Supercomputer at the National Research Center Kurchatov Institute , IT4 in Ostrava, and others). The current approach utilizes a modified PanDA pilot framework for job submission to the supercomputers batch queues and local data management, with light-weight MPI wrappers to run single- threaded workloads in parallel on Titan s multi-core worker nodes. This implementation was tested with a variety of Monte-Carlo workloads on several supercomputing platforms. We will present our current accom- plishments in running PanDA WMS at supercomputers and demonstrate our ability to use PanDA as a portal independent of the computing facility s infrastructure for High Energy and Nuclear Physics, as well as other data-intensive science applications, such as bioinformatics and astro-particle physics.

De, K↗

ATLAS Data Carousel

The ATLAS experiment at CERN's LHC stores detector and simulation data in raw and derived data formats across more than 150 Grid sites world-wide, currently in total about 200PB on disk and 250PB on tape. Data have different access characteristics due to various computational workflows, and can be accessed from different media, such as remote I/O, disk cache on hard disk drives or SSDs. Also, larger data centers provide the majority of offline storage capability via tape systems. For the High-Luminosity LHC (HL-LHC), the estimated data storage requirements are several factors bigger than the present forecast of available resources, based on a flat budget assumption. On the computing side, ATLAS Distributed Computing was very successful in the last years with high performance and high throughput computing integration and in using opportunistic computing resources for the Monte Carlo simulation. On the other hand, equivalent opportunistic storage does not exist. ATLAS started the "Data Carousel" project to increase the usage of less expensive storage, i.e. tapes or even commercial storage, so it is not limited to tape technologies exclusively. Data Carousel orchestrates data processing between workload management, data management, and storage services with the bulk data resident on offline storage. The processing is executed by staging and promptly processing a sliding window of inputs onto faster buffer storage, such that only a small percentage of input data are available at any one time. With this project, we aim to demonstrate that this is the natural way to dramatically reduce our storage cost. The first phase of the project was started in the fall of 2018 and was related to I/O tests of the sites archiving systems. Phase II now requires a tight integration of the workload and data management systems. Additionally, the Data Carousel studies the feasibility to run multiple computing workflows from tape. The project is progressing very well and the results presented in this document will be used before the LHC Run 3.

97 MATHEMATICS AND COMPUTING↗

Arctic ice-ocean interactions in an 8-to-2 kilometer resolution global model

In the last decades, the Arctic climate has changed dramatically, with the loss of multiyear sea ice one of the clearest consequences. These changes have occurred on relatively rapid timescales, and both accurate short-term Arctic prediction (e.g., 10 days to three months) and climate projection of future Arctic scenarios present ongoing challenges. Here we describe a representation of the Arctic ocean and sea ice in a ultrahigh resolution simulation in which the horizontal grid mesh reduces from 8 km at the equator to 2 km at the poles (UH8to2) for the years 2017-2020. We find the simulation reproduces observed distributions of seasonal sea-ice thickness and concentration realistically, although concentration is biased low in the spring and summer and low biases in thickness are found in the central and eastern basins in the fall. Volume, fresh water, and heat transports through key passages are realistic, lying within observationally determined ranges. Climatological comparisons reveal that the UH8to2 Atlantic Water is shallower, warmer, and saltier than the World Ocean Atlas 2018 climatology for 2005-2017 in the eastern basin. Our analysis suggests that these biases, combined with a lack of stratification in the upper 100 m of the simulated ocean, contribute to the winter biases in modeled sea ice thickness. This relationship between biases in the sea ice and ocean points to a potential positive feedback within the model, illuminating challenges for long term model predictive power in a changing Arctic climate.

58 GEOSCIENCES↗

A High-resolution Regional Wave Resource Characterization For The U.S. West Coast

Objectives/Scope: Wave resource characterization is a critical step for wave energy converter deployment in the coastal ocean and relies on long-term, high-resolution wave datasets. This study presents a detailed modeling study of the wave resource along the U.S. West Coast (Washington, Oregon, and California), a coastal region that was identified with high wave energy potential in earlier studies. Methods, Procedures, Process: The wave hindcast covers a 32-year period from 1979 to 2010 and is based on a multi-resolution, unstructured-grid SWAN model framework. Model configuration closely follows and meets the requirements recommended by the International Electrotechnical Commission Technical Specification (IEC TS) for wave energy resource assessment and characterization (Class 2 - feasibility study). The model domain covers the entire U.S. Exclusive Economic Zone (EEZ) in the West Coast and has a spatial resolution varying from ~300 m in the nearshore region (20 km from the shoreline) to ~2500 m within the EEZ and ~5000 m at the open boundary, which extends beyond the EEZ. The model was forced by hourly 2-D wave spectra produced by a two-way nested WaveWatch III model, which covers the global ocean domain and the broader U.S. West Coast region domain with spatial resolutions of 0.5 degree and 10 arc-minutes, respectively. Both wave models are forced by hourly, 0.5-degree wind forcing obtained from NCEP’s Climate Forecast System Reanalysis (CFSR) product. Results, Observations, Conclusions: The standard model output for the SWAN model includes 3-hourly output for the six IEC wave resource parameters (e.g., omnidirectional wave power) at each grid point and hourly 2-D spectra at more than 50 NDBC buoys. Extensive model validation was achieved by comparing the six model-predicted IEC parameters with those derived from field observations at representative NDBC buoys. The error statistics indicated the model’s satisfactory performance. Further analyses were conducted to systematically evaluate the temporal and spatial distributions of wave energy potential and wave climate along the U.S. West Coast. Results suggest that Washington and Oregon coasts have similar nearshore wave resource, which is significantly higher than resources in Southern California. Strong seasonal variations are also observed, e.g., high wave energy tends to occur in the winter months. In summary, this study produced the first high-resolution, comprehensive dataset on wave energy distribution along the U.S. West Coast. Novel/Additive Information: The results are being used by the National Renewable Energy Laboratory to update the MHK Atlas, which was originally derived from NOAA’s 4-arc-minute WaveWatch III model output. In addition, the monthly averaged wave energy climatology dataset can be readily shared to support a variety of research and application efforts within the EEZ of the U.S. West Coast.

Wang, Taiping↗

Winter Arctic Sea Ice Thickness from ICESat-2 Freeboards

NASA’s ICESat-2 mission was launched in September 2018 with the primary goal of monitoring our rapidly changing polar regions. The sole instrument onboard, the Advanced Topographic Laser Altimeter System (ATLAS), is now providing routine, very high-resolution, surface elevation data across the globe, including the Arctic and Southern Oceans. In this study we demonstrate our new processing chain for converting the along-track ICESat-2 sea ice freeboard product (ATL10) into sea ice thickness, focusing our initial efforts on the Arctic Ocean. For this conversion we primarily make use of snow depth and density data from the NASA Eulerian Snow on Sea Ice Model (NESOSIM). The coarse resolution (~100 km) snow data are redistributed onto the high resolution (~30 - 100 m) ATL10 freeboards using relationships obtained from snow depth and freeboard data collected by NASA’s Operation IceBridge mission. We present regional sea ice thickness distributions and highlight their seasonal evolution through our first winter-season of data collection. We include ice thickness uncertainty estimates, while also acknowledging the limitations of these estimates. We generate a gridded monthly thickness product and compare this with various monthly sea ice thickness estimates obtained from ESA’s CryoSat-2 satellite mission, with ICE 33 Sat-2 showing consistently lower thicknesses. Finally, we compare our February/March 2019 thickness estimates to ICESat February/March (19th February – 21st March) 2008 ice thickness estimates using the same input assumptions, which show a ~0.40 m or ~21% thinning across an inner Arctic Ocean domain in this 11-year time period.

Alek A Petty↗

AI-assisted detector design for the EIC (AID(2)E)

Artificial Intelligence is poised to transform the design of complex, large-scale detectors like ePIC at the future Electron Ion Collider. Featuring a central detector with additional detecting systems in the far forward and far backward regions, the ePIC experiment incorporates numerous design parameters and objectives, including performance, physics reach, and cost, constrained by mechanical and geometric limits. This project aims to develop a scalable, distributed AI-assisted detector design for the EIC (AID(2)E), employing state-of-the-art multiobjective optimization to tackle complex designs. Supported by the ePIC software stack and using G EANT 4 simulations, our approach benefits from transparent parameterization and advanced AI features. The workflow leverages the PanDA and iDDS systems, used in major experiments such as ATLAS at CERN LHC, the Rubin Observatory, and sPHENIX at RHIC, to manage the compute intensive demands of ePIC detector simulations. Tailored enhancements to the PanDA system focus on usability, scalability, automation, and monitoring. Ultimately, this project aims to establish a robust design capability, apply a distributed AI-assisted workflow to the ePIC detector, and extend its applications to the design of the second detector (Detector-2) in the EIC, as well as to calibration and alignment tasks. Additionally, we are developing advanced data science tools to efficiently navigate the complex, multidimensional trade-offs identified through this optimization process.

97 MATHEMATICS AND COMPUTING↗

ICESat-2 Mission: Status, Outlook, and Contributions to Polar Science

The Ice, Cloud and land Elevation Satellite-2 (ICESat-2) mission carries the Advanced Topographic Laser Altimeter System (ATLAS) lidar to measure the changing height of Earth surface. After more than 3 years of science data collection, ATLAS has emitted well over a trillion laser pulses and continues to operate nominally. The ICESat-2 data products were initially released in May 2019, and have been used in over 150 peer-reviewed publications to date. Recent community white papers have noted the importance of ice elevation measurements for the coming decades, and as CryoSat-2 has entered it’s second decade and future missions are uncertain, ICESat-2 will be a critical part of the Earth System Observatory for the 2020s. In the polar regions, ICESat-2 has enabled year-round sea ice freeboard measurements as well as ice sheet elevation changes on seasonal, annual, and decadal time scales when combined with other missions. These data enable measurement of change at fine spatial and temporal scales to help understand or characterize the processes driving these observations. As of this presentation, Release 005 is the current version of the along-track data products, including surface-specific products for sea ice, land ice, ground and canopy height, ocean, and inland water heights. In addition, first versions of gridded data products for land ice, sea ice, and ocean are all available. Data spans the start of the mission (14 October 2018) through early 2022. The ICESat-2 project has also begun to produce and distribute QuickLook products with a nominal 3-day latency (as opposed to the ~45-day latency of the final data products). Initial versions of these QuickLook products have elevation accuracy of ~3 meters, and geolocation accuracy of ~50 meters. All products are available through the National Snow and Ice Data Center (nsidc.org). This presentation will summarize the current state and health of the observatory, current state of the ICESat-2 data products, data product quality assessment, and outlook for the future.

ICESat-2↗