Search NASA⌕ Search

SEARCH · Search NASA

Results for “Large data”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 649 records · Page 36
All sourcesNASA NTRSDOE OSTI
DOE OSTI2025

Convergent evolution of NFP -facilitated root nodule symbiosis

The origin and phylogenetic distribution of symbiotic associations between nodulating angiosperms and nitrogen-fixing bacteria have long intrigued biologists. Recent comparative evolutionary analyses have yielded alternative hypotheses: a multistep pathway of independent gains and losses of root nodule symbiosis vs. a single gain followed by numerous losses. A detailed reconstruction of the history of genes involved in signaling between nitrogen-fixing bacteria and potential hosts, particularly lipo-chitooligosaccharide (LCO) signaling, is needed to distinguish between these hypotheses. LCO recognition by plants involves the Nod Factor Perception ( NFP ) gene family; in the legume model Medicago truncatula (Fabales), MtNFP is essential for establishing rhizobial symbiosis. Here, we document convergent evolution of NFP , indicating multiple origins of LCO-driven symbiosis. In contrast to previous models that explain the recruitment of NFP via a single duplication in the ancestor of the nitrogen-fixing clade, our phylogenomic and synteny results suggest this duplication does not span the entire clade. Tandem duplication in a common ancestor of Cucurbitales and Rosales resulted in the NFP1 and NFP2 groups. In contrast, the phylogenetically closest paralog of MtNFP is MtLYR1 , located on a different chromosome within a large syntenic block. All available data indicate that a large-scale duplication resulted in MtNFP and MtLYR1 , likely corresponding to a whole-genome duplication in an ancestor of subfamily Papilionoideae of Fabaceae. We show that MtNFP and the NFP2 -like group are not orthologous, indicating multiple independent gains of NFP -based LCO signaling. This molecular convergence provides a possible mechanism for multiple gains of root nodule symbiosis across the nitrogen-fixing clade.

LCO signaling↗
DOE OSTI2020

Characterization and identification of HPC applications at leadership computing facility

High Performance Computing (HPC) is an important method for scientific discovery via large-scale simulation, data analysis, or artificial intelligence. Leadership-class supercomputers are expensive, but essential to run large HPC applications. The Petascale era of supercomputers began in 2008, with the first machines achieving performance in excess of one petaflops, and with the advent of new supercomputers in 2021 (e.g., Aurora, Frontier), the Exascale era will soon begin. However, the high theoretical computing capability (i.e., peak FLOPS) of a machine is not the only meaningful target when designing a supercomputer, as the resources demand of applications varies. A deep understanding of the characterization of applications that run on a leadership supercomputer is one of the most important ways for planning its design, development and operation. In order to improve our understanding of HPC applications, user demands and resource usage characteristics, we perform correlative analysis of various logs for different subsystems of a leadership supercomputer. This analysis reveals surprising, sometimes counter-intuitive patterns, which, in some cases, conflicts with existing assumptions, and have important implications for future system designs as well as supercomputer operations. For example, our analysis shows that while the applications spend significant time on MPI, most applications spend very little time on file I/O. Combined analysis of hardware event logs and task failure logs show that the probability of a hardware FATAL event causing task failure is low. Combined analysis of control system logs and file I/O logs reveals that pure POSIX I/O is used more widely than higher level parallel I/O. Based on holistic insights of the application gained through combined and co-analysis of multiple logs from different perspectives and general intuition, we engineer features to "fingerprint" HPC applications. We use t-SNE (a machine learning technique for dimensionality reduction) to validate the explainability of our features and finally train machine learning models to identify HPC applications or group those with similar characteristic. To the best of our knowledge, this is the first work that combines logs on file I/O, computing, and inter-node communication for insightful analysis of HPC applications in production.

Liu, Zhengchun↗
DOE OSTI2021

FasTensor (FT) v0.0.1

The FasTensor implements the native array data programming mode namely SLOPE for modern data analysis tasks. It is designed to work in parallel on supercomputer and can handle terabyte data analysis tasks. It is thousand times faster than other data analysis systems such as Apache Spark and 38% faster than TensorFlow in dealing with large scale scientific data analysis. It has wide applications in image data analysis and mesh data analysis tasks in physical simulations and other fields.

Wu, Kesheng↗
DOE OSTI2021

Example on how to (intelligently) augment the nuclear-data pipeline with machine learning [Slides]

The presentation discusses how machine learning has helped the Los Alamos National Laboratory (LANL) nuclear-data pipeline. It also discusses the strengths of machine learning as it finds trends in large amounts of data where human brains are overwhelmed and that this information may be crucial to improve our nuclear data. It does stress, however, that machine learning is no "silver bullet" and that it is critical to feed it expert knowledge and use physics intuition to interpret the results. The presentation discusses the need to develop infrastructure and tools to provide data in an easily readable and unambiguously interpretable format (e.g., EXFOR format), to develop experimental data and theory to solve physics questions, and that statisticians and nuclear-data experts must be brought together to correctly interpret the results. The presentation concludes by stating that machine learning is a great tool and that LANL needs to use the algorithms along with developing physics data, tools and infrastructure.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2022

Analytics-at-scale of Sensor Data for Digital Monitoring in Nuclear Plants (3 rd Annual Report)

Nuclear power plants collect and store large volumes of heterogeneous data from various components and systems. With recent advances in machine learning (ML) techniques, these data can be leveraged to develop diagnostic and short-term forecasting models to better predict future equipment condition. Maintenance operations can then be planned in advance whenever degraded performance is predicted, thus resulting in fewer unplanned outages and the optimization of maintenance activities. This enables lower maintenance costs and improves the overall economics of nuclear power. This report primarily focuses on developing a short-term forecasting process that leverages a feature selection process to distill large volumes of heterogeneous data and predict specific equipment parameters. A variety of feature selection methods, including Shapley Additive Explanations (SHAP) and variance inflation factor (VIF), were used to select the optimal features as inputs for three ML methods: long short-term memory (LSTM) networks, support vector regression (SVR), and random forest (RF). Each combination of model and input features was used to predict a pump bearing temperature both 1 and 24 hours in advance, based on actual plant system data. The optimal inputs for the LSTM and SVR were selected using the SHAP values, while the optimal input for the RF consisted solely of the response variable itself. Each model produced similar 1-hour-ahead predictions, with root mean square errors (RMSEs) of roughly 0.006. For the 24-hour-ahead predictions, differences could be seen between LSTM, SVR, and RF, as reflected by model performances of 0.036 ± 0.014, 0.0026 ± 0, and 0.063 ± 0.004 RMSE, respectively. As big data and continuous online monitoring become more widely available, the proposed feature selection process can be used for many applications beyond the prediction of process parameters within nuclear infrastructure. This report summarizes the Fiscal Year 2021 research progress encompassing the (1) data cleaning and feature selection necessary for ML applications; (2) development of short-term forecasting models to predict future plant process parameters for both single and multiple time steps ahead; and (3) validation of the feature selection methods and short-term forecasting models given new data from different systems.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2020

GPS-Based Gamma Survey for Characterizing and Decommissioning NORM Sites - 20389

Gamma survey techniques are an especially powerful decommissioning tool at naturally occurring radioactive material (NORM) sites due to both the low cost to obtain data over a large spatial scale and the abundance of gamma emitters in the uranium and thorium decay series. Gamma surveys are executed by coupling a detector - most often a sodium iodide crystal - to a global positioning system (GPS), then reporting a location and gross gamma reading coincidentally to a data logger. Systems may be carried by workers or mounted to a car, all-terrain vehicle, or unmanned aerial system (UAS). The resulting data set provides a high-resolution but low precision map of the gamma radiation field over the area surveyed. Frequently this map is also correlated to soil concentrations of NORM radionuclides (most often, Ra-226) and/or exposure rate. Gamma survey parameters such as movement speed, transect spacing, and data logging frequency define the spatial resolution of the resulting surface, and can be optimized depending on the desired survey sensitivity. This paper examines gamma survey as a tool for decommissioning NORM sites and provides an overview of current gamma survey technology designed to improve the efficiency and effectiveness of the decommissioning process. Topics to be discussed in the paper include: - An overview of gamma survey systems, and the utility of different delivery vehicles depending on desired cost, desired spatial resolution, and site topography. - The influence of physical detector characteristics on detection sensitivity and survey planning. - The tradeoff between high-resolution and large spatial extent, but inherently uncertain data, and low-resolution, low spatial extent, but highly certain data, as well as the specific utility of each of these types of data during NORM facility decommissioning. - Confounding variables that may limit the utility of gamma survey at some sites (e.g., radon gas and spatial heterogeneity / hot spots), and methods to plan for and control these conditions. Results show that the confounding variables, such as radon and data output can greatly influence the overall data quality associated with the decommissioning process. In addition, the use of real-time and aerial survey platforms provides a method for ensuring proper spatial extent of the data. When applied thoughtfully, gamma survey is a powerful tool for detecting NORM radionuclides in the environment and a cost-effective technique for identifying areas requiring remediation. However, entities performing or using gamma survey as a decommissioning tool must be aware of both its advantages and its limitations before basing remediation or regulatory action on gamma survey results. (authors)

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗
DOE OSTI2020

Fast and Accurate Frequency Response Estimation for Large Power System Disturbances Using Second Derivative of Frequency Data

Accurate Frequency Response Estimation (FRE) is critical to reliability risk management of large power system disturbances. This letter proposes a measurement-driven approach for FRE by calculating the second derivative of frequency from synchrophasor data. Compared with conventional methods, the proposed method has major advantages of low computational burden and high accuracy, which makes it appropriate for real-time FRE in bulk power systems. The effectiveness of this method is validated with actual synchrophasor data on 96 confirmed historical power system disturbances in the U.S. eastern interconnection.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2022

A Scalable Space-Time Domain Decomposition Approach for Solving Large Scale Nonlinear Regularized Inverse Ill Posed Problems in 4D Variational Data Assimilation

We address the development of innovative algorithms designed to solve the strong-constraint Four Dimensional Variational Data Assimilation (4DVar DA) problems in large scale applications. We present a space-time decomposition approach which employs the whole domain decomposition, i.e. both along the spacial and temporal direction in the overlapping case, and the partitioning of both the solution and the operator. Starting from the global functional defined on the entire domain, we get to a sort of regularized local functionals on the set of sub domains providing the order reduction of both the predictive and the Data Assimilation models. The algorithm convergence is developed. Performance in terms of reduction of time complexity and algorithmic scalability is discussed on the Shallow Water Equations on the sphere. The number of state variables in the model, the number of observations in an assimilation cycle, as well as numerical parameters as the discretization step in time and in space domain are defined on the basis of discretization grid used by data available at repository Ocean Synthesis/Reanalysis Directory of Hamburg University.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Towards Lightweight Data Integration Using Multi-Workflow Provenance and Data Observability

Modern large-scale scientific discovery requires multidisciplinary collaboration across diverse computing facilities, including High Performance Computing (HPC) machines and the Edge-to-Cloud continuum. Integrated data analysis plays a crucial role in scientific discovery, especially in the current AI era, by enabling Responsible AI development, FAIR, Reproducibility, and User Steering. However, the heterogeneous nature of science poses challenges such as dealing with multiple supporting tools, cross-facility environments, and efficient HPC execution. Building on data observability, adapter system design, and provenance, we propose MIDA: an approach for lightweight runtime Multi-workflow Integrated Data Analysis. MIDA defines data observability strategies and adaptability methods for various parallel systems and machine learning tools. With observability, it intercepts the dataflows in the background without requiring instrumentation while integrating domain, provenance, and telemetry data at runtime into a unified database ready for user steering queries. We conduct experiments showing end-to-end multi-workflow analysis integrating data from Dask and MLFlow in a real distributed deep learning use case for materials science that runs on multiple environments with up to 276 GPUs in parallel. We show near-zero overhead running up to 100,000 tasks on 1,680 CPU cores on the Summit supercomputer.

Santos Souza, Renan↗
DOE OSTI2024

Subsampling of Parametric Models with Bifidelity Boosting

Least squares regression is a ubiquitous tool for building emulators (a.k.a. surrogate models) of problems across science and engineering for purposes such as design space exploration and uncertainty quantification. When the regression data are generated using an experimental design process (e.g., a quadrature grid) involving computationally expensive models, or when the data size is large, sketching techniques have shown promise at reducing the cost of the construction of the regression model while ensuring accuracy comparable to that of the full data. However, random sketching strategies, such as those based on leverage scores, lead to regression errors that are random and may exhibit large variability. To mitigate this issue, we present a novel boosting approach that leverages cheaper, lower-fidelity data of the problem at hand to identify the best sketch among a set of candidate sketches. This in turn specifies the sketch of the intended high-fidelity model and the associated data. We provide theoretical analyses of this bifidelity boosting (BFB) approach and discuss the conditions the low- and high-fidelity data must satisfy for a successful boosting. In doing so, we derive a bound on the residual norm of the BFB sketched solution relating it to its ideal, but computationally expensive, high-fidelity boosted counterpart. Finally, empirical results on both manufactured and PDE data corroborate the theoretical analyses and illustrate the efficacy of the BFB solution in reducing the regression error, as compared to the nonboosted solution.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Tracking seasonal variability in plant traits from spaceborne PRISMA and NEON AOP across forest types and ecoregions

Plant traits serve as critical indicators of how plants adapt to environmental changes and influence ecosystem functions. While airborne hyperspectral remote sensing effectively maps plant traits through detailed reflectance properties, it is limited by cost and scale, making large-scale and temporal studies challenging. The recently launched spaceborne hyperspectral imager, PRecursore IperSpettrale della Missione Applicativa (PRISMA), offers frequent, large scale and high-fidelity observations on a spatial resolution of 30 m and a revisit time of around 29 days, making it suitable for large-scale seasonal trait mapping. However, their potential remains largely unexplored. This study developed a multi-stage framework by leveraging the PRISMA spaceborne hyperspectral data and National Ecological Observatory Network (NEON) Airborne Observation Platform (AOP) hyperspectral data to investigate the seasonal dynamics of four key plant traits — chlorophyll content, carotenoid content, equivalent water thickness, and nitrogen content — across eleven NEON sites representing diverse forest types and ecoregions in the contiguous U.S. Our results demonstrated that PRISMA hyperspectral data can reliably track seasonal variability in plant traits, achieving overall R 2 values ranging from 0.78 to 0.88 and normalized root mean square error (NRMSE) values ranging from 5.4% to 8.4% for the four traits. Seasonal patterns revealed bell-shaped trajectories for chlorophyll and carotenoids, while equivalent water thickness decreased steadily across most sites, driven by structural changes during leaf maturation and senescence. Nitrogen content exhibited less pronounced seasonal variation but followed expected nutrient resorption patterns. Analysis of environmental drivers showed that seasonal variability is primarily controlled by solar radiation and day length in northern sites, vapor pressure in semi-arid regions, and temperature in mid-southeastern sites. Spatial variability, meanwhile, was primarily driven by soil properties, particularly during the peak growing season. However, the influence of soil variables slightly declines toward the end of the season at several sites, as climatic factors become more prominent. This study highlights the capability of PRISMA, and potentially other similar spaceborne hyperspectral data for large-scale, time-series plant trait mapping and provides valuable insights into the interactions between plant traits and environmental factors. In conclusion, these findings contribute to advancing our understanding of plant functional ecology and improving predictions of ecosystem responses to environmental changes.

Environmental drivers↗
DOE OSTI2020

Evaluating Building Energy Code Compliance and Savings Potential through Large-Scale Simulation with Models Inferred by Field Data

Building energy code compliance is the crucial link between the actual energy savings and the efficiency prescribed in energy codes. A research project aiming to identify opportunities to reduce energy consumption in new single-family residential construction by increasing compliance with the building energy code was conducted in several states of the United States. The study was comprised of three phases: (1) a baseline study to document typical practice and identify opportunities for improvement based on empirical data gathered from the field; (2) an education and training phase targeting the opportunities identified; and (3) a post-study to assess whether a reduction in average state-wide energy use could be achieved following the education and training phase. We proposed a novel methodology based on large-scale building energy simulation inferred by limited field data to assess the performance of a large population of homes. This paper presents the methodology, findings, and results of this study. The state-wide average energy consumption decreased at Phase III from Phase I for five of the seven states involved in the analysis. The measure-level savings potential analysis shows an overall reduction. Overall, the training and education phase plays a recognizable role in improving compliance with building energy codes.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2022

User guide for the Geospatial Risk Analysis Tool (V.1.0.1)

Developed by the Pacific Northwest National Laboratory (PNNL), the Geospatial Risk Analysis Tool (GRAT) links to Hybrid Single-Particle Lagrangian Integrated Trajectory (HYSPLIT), a complete system for computing simple air parcel trajectories, as well as complex transport, dispersion, chemical transformation, and deposition simulations. The HYSPLIT model was developed by the National Oceanic and Atmospheric Administration (NOAA) Air Resources Laboratory and the Australian Bureau of Meteorology Research Center in 1998. GRAT is used to batch process meteorological data into ARL format. While HYSPLIT GUI limits users to convert meteorological data up to 6 date time points per operation, GRAT enables users to convert a significantly large amount of data (For example, 10 years of meteorological data measured with an interval of 5 minutes) within one operation. Incorporated with the Potential source distribution function (PSDF), GRAT can also be used to determine the areas influenced by the emission of hazardous chemicals.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2023

Data and scripts associated with "Coupled primary production and respiration in a large river contrasts with smaller rivers and streams."

This data package is associated with the publication "Coupled primary production and respiration in a large river contrasts with smaller rivers and streams." in review at Limnology and Oceanography (Roley et al. 2023). This study focuses on understanding ecosystem metabolism for the Hanford Reach of the Columbia River in Washington state, a free-flowing stretch with a substantial discharge. Large rivers have been overlooked compared to small and medium rivers due to the challenges associated with measurements. Our study presents novel ways to address these challenges and highlights that metabolism patterns in large rivers differ from those observed in small-medium rivers and requires the application of knowledge and tools beyond those implemented for smaller rivers.This data package includes the data and R scripts for the analyses described in Roley et al. 2023. It includes dissolved oxygen and temperature data from a dissolved oxygen HOBO sensor, light data collected from the National Solar Radiation Database (https://nsrdb.nrel.gov/) and hydrologic variables estimated from the MASS-1 model (Niehus et al.; 2014). It also includes metabolism estimates (gross primary production, ecosystem respiration, and net ecosystem production) estimated via streamMetabolizer (Appling et al.; 2018). All analyses in the paper can be replicated with these data and scripts.The data package is comprised of one main data folder. The folder includes (1) file-level metadata (flmd); (2) a data dictionary (dd) for each data file; (3) data files; and (4) R scripts for metabolism estimates and data analysis. All files are .R, .csv, or .pdf.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2024

DECOVALEX-2023: Task C Final Report

The Full-scale Emplacement (FE) heater experiment at the Mont Terri Underground Rock Laboratory (URL) was designed and conducted by Nagra to replicate an emplacement tunnel of Nagra’s reference repository design at 1:1 scale. Alongside testing the technical feasibility of constructing disposal tunnels, emplacing waste containers in the tunnels and then backfilling them, the main goals of the FE experiment are (1) to obtain a better understanding of the coupled effects of induced thermo-hydro-mechanical (THM) processes that may occur and (2) to validate existing coupled THM models (Müller et al., 2017). A key aspect of ensuring safety for repositories located in low-permeability rock involves minimizing any damage to the rock itself, thereby preserving its integrity and promoting a stable environment Amongst a number of processes that could damage the rock is the increase in pore pressure due to thermal loading caused by heat emitted from the waste. To reduce the potential damage of the rock, it is important to analyse the evolution of heat over time due to the heat load of the containers and assess possible consequences by coupled THM models. The aim of Task C of DECOVALEX-2023 was to build 3D numerical models of the FE experiment, focussing in particular on the heating induced pore pressure change in the Opalinus Clay. Data from a large number of sensors were available from the FE experiment for model comparison. These sensors measured temperature and relative humidity in the bentonite around the heaters, and temperature, pressure and displacement/strain in the surrounding Opalinus clay. Data were available from the start of excavation (April 2012) up to August 2020 for most sensors (more than 5 years from the start of heating in December 2014). To fulfil the overall aim of the task, the work was broken down into a number of steps, starting with simpler models to build confidence in each team’s approach and then moving to more complex models that better represent the FE experiment. Step 0 consisted of 2D benchmark models, gradually increasing the number of processes that are represented from thermal (T) only models in Step 0a, to coupled thermal hydraulic (TH) models in Step 0b with a representation of changing porosity, to coupled thermo-hydro-mechanical (THM) models in Step 0c, where porosity changes are calculated by the mechanical model. A detailed specification of processes, parameters, initial and boundary conditions was provided for this step, with the ambition that all teams would work towards close agreement in their model results, thus building confidence in the model implementations. vi It was not straightforward to achieve agreement between the teams, so additional steps (Step 0b2, 0b3, 0c2, 0c3) were added along with derivation of some analytical solutions against which the models could be compared. The reasons for the differences between teams were investigated and found to be caused primarily by different conceptual model assumptions (including temperature dependence of the thermal expansion of water), different model formulations (including porosity evolution) and differences in modelled domain sizes, boundary conditions and grid discretisation. This demonstrates that comparisons between multiple modelling teams and/or comparison with analytical results and experimental data are highly beneficial in providing an indication of uncertainty in model predictions. At the conclusion of Step 0, almost all teams had achieved a close agreement in model results and those that had not achieved an agreement knew the reason for this. Step 1 moved from 2D models to 3D models of the FE experiment without adding technical features like shotcrete or EDZ, and only considering the heating phase. Initially the 3D model was tightly specified to continue to build confidence in the model implementations (Step 1a). The results of Step 1a were compared to the data from the FE-experiment without the teams seeing the data. The teams were then provided with a sub-set of the data from the FE-experiment and invited to consider how best to use the large dataset for model comparison (Step 1b). Teams were then asked to use the data provided to calibrate their models, only changing material property values rather than adding features or processes to their models (Step 1c). In Step 1, teams were asked to only model the heating phase of the experiment, so pressure in the Opalinus Clay was reported as change in pressure since the initial conditions were specified rather than modelled. The change from 2D to 3D models was accompanied by an increase in the dispersion of results between the teams. Some of this was resolved during the task, but some remained and is potentially due to model discretisation. Calibration of parameters was useful in improving the fit of the models to the data but the remaining differences indicated that the models were missing features or processes. In Step 2, the teams were asked to update their models with additional features and processes as well as calibrating parameters to try and improve the fit of the models to the data. Teams were encouraged to represent ventilation of the open FE tunnel prior to backfilling with heaters and bentonite and in Step 2, the absolute pressure in the Opalinus Clay was compared between the teams. Teams took different approaches, but there was consideration of adding shotcrete and an EDZ into the model, representing stress change during excavation and different approaches to modelling ventilation of the FE tunnel. Overall, the documented results showed a very good agreement for temperature. The results for porewater pressure evolution showed a significant improvement for most teams compared to Step 1c with a good agreement to the measurements for several teams whereas some teams overpredicted the pressure increase and others overpredicted the drainage effect especially for the sensors close to the heater. Step 3 was an opportunity for teams to use the models developed in Step 1 and Step 2 to make predictions about the temperature and pressure changes that will be expected at the FE experiment over the next few years in light of the planned changes in thermal output of the heaters.

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗
DOE OSTI2023

A Vision for Coupling Operation of US Fusion Facilities with HPC Systems and the Implications for Workflows and Data Management

The operation of large US Department of Energy (DOE) research facilities, like the DIII-D National Fusion Facility, results in the collection of complex multi-dimensional scientific datasets, both experimental and model-generated. In the future, it is envisioned that integrated data analysis coupled with large-scale high performance computing (HPC) simulations will be used to improve experimental planning and operation. Practically, massive data sets from these simulations provide the physics basis for generation of both reduced semi-analytic and machine-learning-based models. Storage of both HPC simulation datasets (generated from US DOE leadership computing facilities) and experimental datasets presents significant challenges. In this paper, we present a vision for a DOE-wide data management workflow that integrates US DOE fusion facilities with leadership computing facilities. Data persistence and long-term availability beyond the length of allocated projects is essential, particularly for verification and recalibration of artificial intelligence and machine learning (AI/ML) models. Because these data sets are often generated and shared among hundreds of users across multiple leadership computing facility centers, they would benefit from cross-platform accessibility, persistent identifiers (e.g. DOI, or digital object identifier), and provenance tracking. Here, the ability to handle different data access patterns suggests that a combination of low cost, high latency (e.g. for storing ML training sets) and high cost, low latency systems (e.g. for real-time, integrated machine control feedback) may be needed.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2023

Probe microscopy is all you need *

We pose that microscopy offers an ideal real-world experimental environment for the development and deployment of active Bayesian and reinforcement learning methods. Indeed, the tremendous progress achieved by machine learning (ML) and artificial intelligence over the last decade has been largely achieved via the utilization of static data sets, from the paradigmatic MNIST to the bespoke corpora of text and image data used to train large models such as GPT3, DALL·E and others. However, it is now recognized that continuous, minute improvements to state-of-the-art do not necessarily translate to advances in real-world applications. We argue that a promising pathway for the development of ML methods is via the route of domain-specific deployable algorithms in areas such as electron and scanning probe microscopy and chemical imaging. This will benefit both fundamental physical studies and serve as a test bed for more complex autonomous systems such as robotics and manufacturing. Favorable environment characteristics of scanning and electron microscopy include low risk, extensive availability of domain-specific priors and rewards, relatively small effects of exogenous variables, and often the presence of both upstream first principles as well as downstream learnable physical models for both statics and dynamics. Recent developments in programmable interfaces, edge computing, and access to application programming interfaces (APIs) facilitating microscope control, all render the deployment of ML codes on operational microscopes straightforward. We discuss these considerations and hope that these arguments will lead to create novel set of development targets for the ML community by accelerating both real world ML applications and scientific progress.

47 OTHER INSTRUMENTATION↗
DOE OSTI2021

FMX – the Frontier Microfocusing Macromolecular Crystallography Beamline at the National Synchrotron Light Source II

Two new macromolecular crystallography (MX) beamlines at the National Synchrotron Light Source II, FMX and AMX, opened for general user operation in February 2017 [Schneider et al. (2013). J. Phys. Conf. Ser. 425 , 012003; Fuchs et al. (2014). J. Phys. Conf. Ser. 493 , 012021; Fuchs et al. (2016). AIP Conf. Proc. SRI2015 , 1741 , 030006]. FMX, the micro-focusing Frontier MX beamline in sector 17-ID-2 at NSLS-II, covers a 5–30 keV photon energy range and delivers a flux of 4.0 × 10 12 photons s −1 at 1 Å into a 1 µm × 1.5 µm to 10 µm × 10 µm (V × H) variable focus, expected to reach 5 × 10 12 photons s −1 at final storage-ring current. This flux density surpasses most MX beamlines by nearly two orders of magnitude. The high brightness and microbeam capability of FMX are focused on solving difficult crystallographic challenges. The beamline's flexible design supports a wide range of structure determination methods – serial crystallography on micrometre-sized crystals, raster optimization of diffraction from inhomogeneous crystals, high-resolution data collection from large-unit-cell crystals, room-temperature data collection for crystals that are difficult to freeze and for studying conformational dynamics, and fully automated data collection for sample-screening and ligand-binding studies. FMX's high dose rate reduces data collection times for applications like serial crystallography to minutes rather than hours. With associated sample lifetimes as short as a few milliseconds, new rapid sample-delivery methods have been implemented, such as an ultra-high-speed high-precision piezo scanner goniometer [Gao et al. (2018). J. Synchrotron Rad. 25 , 1362–1370], new microcrystal-optimized micromesh well sample holders [Guo et al. (2018). IUCrJ , 5 , 238–246] and highly viscous media injectors [Weierstall et al. (2014). Nat. Commun. 5 , 3309]. The new beamline pushes the frontier of synchrotron crystallography and enables users to determine structures from difficult-to-crystallize targets like membrane proteins, using previously intractable crystals of a few micrometres in size, and to obtain quality structures from irregular larger crystals.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗