SEARCH · Search NASA

Results for “Statistical Testing Approach”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 937 records · Page 52

All sources NASA NTRS DOE OSTI

DOE OSTI2022

Estimation of stagnation performance metrics in magnetized liner inertial fusion experiments using Bayesian data assimilation

Here we present a new analysis methodology that allows for the self-consistent integration of multiple diagnostics including nuclear measurements, x-ray imaging, and x-ray power detectors to determine the primary stagnation parameters, such as temperature, pressure, stagnation volume, and mix fraction in magnetized liner inertial fusion (MagLIF) experiments. The analysis uses a simplified model of the stagnation plasma in conjunction with a Bayesian inference framework to determine the most probable configuration that describes the experimental observations while simultaneously revealing the principal uncertainties in the analysis. We validate the approach by using a range of tests including analytic and three-dimensional MHD models. An ensemble of MagLIF experiments is analyzed, and the generalized Lawson criterion χ is estimated for all experiments.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

DOE OSTI2024

Measuring Neutron Polarisation in Deuteron Photo-disintegration with the CLAS Start Counter [Thesis]

Deuteron photo-disintegration (γd → γp) is a reaction that represents the simplest case in which nuclear and hadron physics models can be tested. Despite this, associated polarization analyses are limited in terms of angular coverage and energy ranges, especially in observables related to the recoil neutron. This is largely due to a lack in dedicated polarimetry equipment, and represents a roadblock in global progress to understand high-energy phenomena such as hexaquarks, and quark-gluon degrees of freedom. To address this problem, this PhD thesis pioneers a new methodology for the parasitic measurement of nucleon polarization using kinematic reconstruction of (spin-dependent) nucleon-nucleus scattering of reaction products, prior to their detection in large acceptance particle detector apparatus. Following this novel approach, which requires no dedicated polarimeter, a determination of the double polarization observable, $C^n_{x'}$, from deuteron photo-disintegration is presented, using Jefferson Lab’s CLAS detector. The analysis utilizes the (n,p) charge exchange reaction in CLAS’s "start counter" (plastic scintillator) to determine the final state neutron polarizations. The results present the first ever data for this observable above 0.7 GeV (photon beam energy) and significantly extend the angular range of the world data set. This new data is largely statistically consistent with the previous measurement of $C^n_{x'}$ by Bashkanov et al . in the overlapping energy range of 0.4-0.7 GeV. It is planned for the statistical accuracy of the presented result to be increased by the inclusion of additional data. The analysis herein serves as a key proof of concept for future applications, including a recommended similar analysis to be implemented with data from the more modern CLAS12 detector. This paves the way for a plethora of additional analyses using existing data sets that would provide crucial new constraints for hadron and nuclear physics.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

NASA NTRS2008

Use of High-Resolution WRF Simulations to Forecast Lightning Threat

Recent observational studies have confirmed the existence of a robust statistical relationship between lightning flash rates and the amount of large precipitating ice hydrometeors aloft in storms. This relationship is exploited, in conjunction with the capabilities of cloud-resolving forecast models such as WRF, to forecast explicitly the threat of lightning from convective storms using selected output fields from the model forecasts. The simulated vertical flux of graupel at -15C and the shape of the simulated reflectivity profile are tested in this study as proxies for charge separation processes and their associated lightning risk. Our lightning forecast method differs from others in that it is entirely based on high-resolution simulation output, without reliance on any climatological data. short [6-8 h) simulations are conducted for a number of case studies for which three-dmmensional lightning validation data from the North Alabama Lightning Mapping Array are available. Experiments indicate that initialization of the WRF model on a 2 km grid using Eta boundary conditions, Doppler radar radial velocity fields, and METAR and ACARS data y&eld satisfactory simulations. __nalyses of the lightning threat fields suggests that both the graupel flux and reflectivity profile approaches, when properly calibrated, can yield reasonable lightning threat forecasts, although an ensemble approach is probably desirable in order to reduce the tendency for misplacement of modeled storms to hurt the accuracy of the forecasts. Our lightning threat forecasts are also compared to other more traditional means of forecasting thunderstorms, such as those based on inspection of the convective available potential energy field.

McCaul, E. W., Jr.↗

DOE OSTI2016

Diffusion of energetic electrons in turbulent plasmas of the solar wind

A method of calculation of the diffusion coefficients D(v) of particles in velocity space, based on the statistical analysis of the motion of a great number of test electrons, is proposed. In the case of Langmuir turbulence developing in plasmas with fluctuating density inhomogeneities such as the solar wind, simulations provide coefficients D(v) which mainly depend on the Langmuir wave spectra and agree well with the analytical predictions D{sub th}(v) of the quasilinear theory of weak turbulence. Nevertheless, some noticeable differences exist with this theory: in the range of phase velocity of the short waves where the main part of the wave energy is concentrated, D(v) is noticeably smaller than D{sub th}(v), due to the scattering, the reflection, and the focusing effects encountered by the Langmuir waves when they interact with the plasma density inhomogeneities. Moreover, the probability of large velocity jumps in the particles’ trajectories essentially exceeds the probability of a Gaussian distribution. These large jumps, which are connected with the waves’ transformation processes, modify the nature of the particle diffusion, which is no more classical. These higher order effects cause the discrepancies observed with the quasilinear theory, which does not take them into account in its perturbative approach. The solar wind plasmas, which present fluctuating density inhomogeneities of noticeable average levels, are a very good laboratory to study such diffusion processes, which can eventually influence significantly on the development of essential physical phenomena, as electromagnetic radio emissions by type III solar radio bursts, for example.

79 ASTRONOMY AND ASTROPHYSICS↗

DOE OSTI2024

ELECTRONIC STRUCTURE METHODS AND PROTOCOLS WITH APPLICATION TO DYNAMICS, KINETICS AND THERMOCHEMISTRY

Hydrocarbon combustion involves the reaction dynamics of a tremendous number of species beginning with many-component fuel mixtures and proceeding via a complex system of intermediates to form primary and secondary products. Combustion conditions corresponding to new advanced engines and/or alternative fuels rely increasingly on autoignition and low-temperature-combustion chemistry. In these regimes various transient radical species such as HO2, ROO·, ·QOOH, HCO, NO2, HOCO, and Criegee intermediates play important roles in determining the detailed as well as more general dynamics. A clear understanding and accurate representation of these processes is needed for effective modeling. Given the difficulties associated with making reliable experimental measurements of these systems, computation can play an important role in developing these energy technologies. Accurate calculations have their own challenges since even within the simplest dynamical approximations such as transition state theory, the rates depend exponentially on critical barrier heights and these may be sensitive to the level of quantum chemistry. Moreover, it is well-known that in many cases it is necessary to go beyond statistical theories and consider the dynamics. Quantum tunneling, resonances, radiative transitions, and non-adiabatic effects governed by spin-orbit or derivative coupling can be determining factors in those dynamics. Building upon progress made during a period of prior support through the DOE Early Career Program, this project combines developments in the areas of potential energy surface (PES) fitting and multistate multireference quantum chemistry to allow spectroscopically and dynamically/kinetically accurate investigations of key molecular systems (such as those mentioned above), many of which are radicals with strong multireference character and have the possibility of multiple electronic states contributing to the observed dynamics. An ongoing area of investigation is to develop general strategies for robustly convergent electronic structure theory for global multichannel reactive surfaces including diabatization of energy and other relevant surfaces such as dipole transition. Combining advances in ab initio methods with automated interpolative PES fitting allows the construction of high-quality PESs (incorporating thousands of high-level data) to be done rapidly through parallel processing on high-performance computing (HPC) clusters. In addition, new methods and approaches to electronic structure theory will be developed and tested through applications. This project will explore limitations in traditional multireference calculations (e.g., MRCI) such as those imposed by internal contraction, lack of high-order correlation treatment and poor scaling. Methods such as DMRG-based extended active-space CASSCF and various Quantum Monte Carlo (QMC) methods will be applied (including VMC/DMC and FCIQMC). Insight into the relative significance of different orbital spaces and the robustness of application of these approaches on leadership class computing architectures will be gained. Synergy with other components of this research program such as automated PES fitting and multireference quantum chemistry will be used to address challenges encountered by the standard approaches to computational thermochemistry (those being single-reference quantum chemistry and perturbative treatments of the anharmonic vibrational energy, which break down for some cases of electronic structure or floppy strongly coupled vibrational modes).

74 ATOMIC AND MOLECULAR PHYSICS↗

DOE OSTI2025

Information theory optimization of signals from small-angle scattering measurements

Small-angle X-ray scattering (SAXS) of particles in solution informs on the conformational states and assemblies of biological macromolecules (bioSAXS) outside of cryo- and solid-state conditions. In bioSAXS, the SAXS measurement under dilute conditions is resolution limited, and through an inverse Fourier transform, the measured SAXS intensities directly relate to the physical space occupied by the particles via the P (r)-distribution. Yet, this inverse transform of SAXS data has been historically cast as an ill-posed, ill-conditioned problem requiring an indirect approach. Here, we show that through the applications of matrix and information theories, the inverse transform of SAXS intensity data is a well-conditioned problem. The so-called ill-conditioning of the inverse problem is directly related to the Shannon number. By exploiting the oversampling enabled by modern detectors, a direct inverse Fourier transform of the SAXS data is possible, provided the recovered information does not exceed the Shannon number. The Shannon limit corresponds to the maximum number of significant singular values that can be recovered in a SAXS experiment, suggesting this relationship is a fundamental property of band-limited inverse integral transform problems. This correspondence reduces the complexity of the inverse problem to the Shannon limit and maximum dimension. We propose a hybrid scoring function using an information theory framework that assesses both the quality of the model-data fit as well as the quality of the recovered P (r)-distribution. The hybrid score utilizes the Akaike information criteria and Durbin-Watson statistic that considers parameter-model complexity, i.e., degrees of freedom, and the randomness of the model-data residuals. The described tests and findings extend the boundaries for bioSAXS by completing the information theory formalism initiated by Peter B. Moore to enable a quantitative measure of resolution in SAXS, robustly determine maximum dimension, and more precisely define the best parameter model appropriately representing the observed scattering data.

Rambo, Robert P. [Science and Technology Facilitie↗

DOE OSTI2026

Strangeness enhancement at its extremes: multiple (multi-)strange hadron production in pp collisions at $\sqrt{s}=5.02$ TeV

The probability to observe a specific number of strange and multi-strange hadrons (nS), denoted as P(nS), is measured by ALICE at midrapidity (|y| < 0.5) in $$\sqrt{s}=5.02$$ TeV proton-proton (pp) collisions, dividing events into several multiplicity-density classes. Exploiting, for the first time, a technique based on counting the number of strange-particle candidates event-by-event, this measurement allows one to extend the study of strangeness production beyond the mean of the distribution. This constitutes a new test bench for production mechanisms, probing events with a large imbalance between strange and non-strange content. The analysis of a large-statistics data sample makes it possible to extract P(nS) up to a maximum nS of 7 for $${\text{K}}_{\text{S}}^{0}$$, 5 for Λ and $$\overline{\Lambda }$$, 4 for Ξ− and $${\overline{\Xi } }^{+}$$, and 2 for Ω− and $${\overline{\Omega } }^{+}$$. From this, the probability of producing strange hadron multiplets per event is calculated, thereby enabling the extension of the study of strangeness enhancement to extreme situations where several strange quarks hadronize in a single event at midrapidity. Moreover, comparing hadron combinations with different u and d quark compositions and equal overall s quark content, the contribution to the enhancement pattern coming from non-strangeness related mechanisms is isolated. The results are compared with state-of-the-art phenomenological models implemented in commonly used Monte Carlo event generators, including PYTHIA 8 Monash 2013, PYTHIA 8 with QCD-based Color Reconnection and Rope Hadronization (QCD-CR + Ropes), and EPOS LHC, which incorporates both partonic interactions and hydrodynamic evolution. These comparisons show that the new approach dramatically enhances the sensitivity to the different underlying physics mechanisms modeled by each generator.

Abualrob, I J↗

DOE OSTI2020

Methodology for the development of empirical models relating 13C NMR spectral features to fuel properties

Effective formulation of new gasoline or diesel fuels for internal combustion engines would benefit from the development of reliable models for predicting key fuel properties based on a set of molecular descriptors obtained from a single measurement. This is particularly relevant in the case of renewable fuels, where the available fuel sample quantity may be limited. In this work, we present a statistically-based methodology for building empirical models to predict multiple properties from one-dimensional 13C nuclear magnetic resonance (NMR) spectra measured on around 200 microliters of a liquid fuel. NMR spectra contain information about the molecular composition of a sample and the carbon types and molecular substructures therein. Our approach uses this information to build sparse, interpretable models, where the predicted properties are linked to specific molecular features. The approach takes into consideration the constrained nature of the features making up the one-dimensional NMR spectrum, which, after standardization, represent a relative fuel composition. We point to the limitations in interpretability that arise when building this type of empirical predictive model and suggest how these limitations may be diminished. Among the many properties important for maximizing engine performance and minimizing emissions, we build models that predict derived cetane number and distillation temperatures as these are of particular interest because of their links to fuel economy, drivability, and engine-out emissions. Results suggest that the properties of interest may be impacted by only a few of the 27 13C NMR regions represented in the data, pointing to new directions for further testing in the development of improved fuels.

predictive models, biofuels., NMR Analysis↗

NASA NTRS2008

Algorithms for Spectral Decomposition with Applications to Optical Plume Anomaly Detection

The analysis of spectral signals for features that represent physical phenomenon is ubiquitous in the science and engineering communities. There are two main approaches that can be taken to extract relevant features from these high-dimensional data streams. The first set of approaches relies on extracting features using a physics-based paradigm where the underlying physical mechanism that generates the spectra is used to infer the most important features in the data stream. We focus on a complementary methodology that uses a data-driven technique that is informed by the underlying physics but also has the ability to adapt to unmodeled system attributes and dynamics. We discuss the following four algorithms: Spectral Decomposition Algorithm (SDA), Non-Negative Matrix Factorization (NMF), Independent Component Analysis (ICA) and Principal Components Analysis (PCA) and compare their performance on a spectral emulator which we use to generate artificial data with known statistical properties. This spectral emulator mimics the real-world phenomena arising from the plume of the space shuttle main engine and can be used to validate the results that arise from various spectral decomposition algorithms and is very useful for situations where real-world systems have very low probabilities of fault or failure. Our results indicate that methods like SDA and NMF provide a straightforward way of incorporating prior physical knowledge while NMF with a tuning mechanism can give superior performance on some tests. We demonstrate these algorithms to detect potential system-health issues on data from a spectral emulator with tunable health parameters.

Srivastava, Askok N.↗

DOE OSTI2025

System Engineers and Decisions: It?s All about Knowledge

In order to guarantee that a system meets adequate levels of reliability and availability, system performances are continuously monitored and analyzed thanks to the technological advancements driving the Industry 4.0 revolution. An Industry 4.0 approach is typically based on advanced statistical, big data mining, machine learning, and internet-of-things methods designed to detect anomalies in the behavior of system, detect the most likely failure modes, and provide indications to system engineers on when maintenance activities should be performed before system performance are deemed unacceptable (which can be generated by diagnostic and prognostic methods). However, these analyses, which are designed to automatize and increase the efficacy of the system maintenance program, require large amount of data which can come in various forms: numeric, textual, images, sounds etc. Such data constitutes the historic knowledge benchmark to track system performances and support system engineer decisions. Here we claim that data is not sufficient to support this kind of analyses when applied to systems characterized by complex architectures and behaviors. Robust system engineer decisions require the ability to understand the system operational context that lies behind the observed data elements. In this respect, system models are in fact necessary to “put data in context” and capture relationships between data elements. Industry 4.0 methods require in fact contextual knowledge as a basis upon which hypotheses can be generated and assumptions tested. In our view, for complex systems, model-based system engineering (MBSE) models can afford this contextual knowledge, as they are typically used to describe systems architecture and dynamic behaviors. System knowledge is here intended as the blending of collected data and system architecture which takes the form of a “knowledge graph”. A knowledge graph is a database which consists of a large set of nodes (in our case an entity can be either a data or an MBSE element) which are linked to each other. The types of nodes and links follow a pre-defined topology, sometimes also refers as an ontology, that is designed to fit the actual decisions that needs to be performed. We show here how a knowledge graph can be defined to support system engineer maintenance decisions and how the same graph can be built based on system MBSE models and pre-processed data from numeric (through anomaly detections and diagnostic methods) and textual elements (through technical language processing TLP).

97 - MATHEMATICS AND COMPUTING↗

DOE OSTI2019

A fast Fourier transform-based mesoscale field dislocation mechanics study of grain size effects and reversible plasticity in polycrystals

A numerical implementation of a non-local polycrystal plasticity theory based on a mesoscale version of the field dislocation mechanics theory (MFDM) of Acharya and Roy (2006) is presented using small-strain elasto-viscoplastic fast Fourier transform-based (EVPFFT) algorithm developed by Lebensohn et al. (2012). In addition to considering plastic flow and hardening only due to SSDs (statistically stored dislocations) as in the classic EVPFFT framework, the proposed method accounts for the evolution of GND (geometrically necessary dislocations) densities solving a hyperbolic-type partial differential equation, and GND effects on both plastic flow and hardening. This allows consideration of an enhanced strain-hardening law that includes the effect of the GND density tensor. The numerical implementation of a reduced version of the MFDM is presented in the framework of the FFT-based augmented Lagrangian procedure of Michel et al. (2001). A Finite Differences scheme combined with discrete Fourier transforms is applied to solve both incompatibility and equilibrium equations. The numerical procedure named MFDM-EVPFFT is used to perform full field simulations of polycrystal plasticity considering different grain sizes and their mechanical responses during monotonic tensile and reversible tension-compression tests. Using Voronoi tessellation and periodic boundary conditions, voxelized representative volume elements (RVEs) with different grain sizes are generated. With MFDM-EVPFFT, a Hall-Petch type scaling law is obtained in contrast with the conventional crystal plasticity EVPFFT. In the case of reversible plasticity, a stronger Bauschinger effect is observed with the MFDM-EVPFFT approach in comparison with conventional EVPFFT. Here, the origin of these differences is analyzed in terms of heterogeneity, GND density and stress evolutions during the compression stage.

36 MATERIALS SCIENCE↗

DOE OSTI2022

Reliability Evaluation of Bifacial and Monofacial Glass/Glass Modules with EVA and non-EVA Encapsulants

The market share for bifacial modules is projected to be doubled from 30% to 60% in the next ten years. For the monofacial crystalline silicon glass/backsheet (G/B) modules, extensive reliability data has been available for over 40 years. However, practically little/no long-term field or accelerated reliability test data is available for the new generation glass/glass (G/G) bifacial modules and glass/transparent (G/T) bifacial modules. Therefore, the primary motivation of this project was to identify the reliability strengths and weaknesses of new generation G/G modules compared to G/B modules. In this 3-year project, the goal was to objectively recommend the best construction materials for the new generation G/G (bifacial) modules through a systematic experimental approach with appropriate tasks including: Evaluation of field retrieved old-generation G/G modules; Inspection of new-generation G/G modules installed in the plants; Evaluation of new-generation G/G, G/B and G/T modules using EAST (Extended Accelerated Stress Testing), CAST (Combined Accelerated Stress Testing) and FAST (Field Accelerated Stress Testing); and, dynamic literature search and review. Major accomplishments and outcomes of this project are: Accomplishments: Evaluated more than 60 field retrieved modules and constructed and characterized more than 135 mini-modules with three substrate types (G, B, and T), two encapsulant types (EVA and POE), and two cell types (monofacial and bifacial) as well as evaluated more than 30 commercial G/G and G/B modules; Subjected all the modules to multitude characterization tests and various indoor and outdoor accelerated stress tests including FAST (Field Accelerated Stress Testing), EAST (Extended Accelerated Stress Testing), and CAST (Combined Accelerated Stress Testing) to identify and correlate the failure modes in both field and lab tests. Outcomes: Based on the accelerated test results and handling/mounting experience obtained in this project, the recommended best construction for the glass/glass modules is: “Framed GG modules with cut-cells and POE encapsulant (UVpass front; UVpass back).” However, from a statistical and manufacturing point of view (along with stakeholders surveys), the following cautionary notes are added to the above-mentioned recommendation: (i) Cut cells could introduce a higher level of manufacturing issues, including a higher number of cell interconnects; (ii) POE encapsulant is more expensive than EVA and could present a delamination risk and lower throughput during manufacturing due to lower adhesion strength. Other encapsulants, such as coextruded EPE (EVA/POE/EVA), are also recommended to be investigated. The potential public benefit of the proposed project is to present the strengths and weaknesses of glass/glass modules, so an informed procurement decision can be made.

14 SOLAR ENERGY↗

DOE OSTI2016

Investigation of heat transfer in liquid-metal flows under fusion-reactor conditions

The effect discovered in studying a downward liquid-metal flow in vertical pipe and in a channel of rectangular cross section in, respectively, a transverse and a coplanar magnetic field is analyzed. In test blanket modules (TBM), which are prototypes of a blanket for a demonstration fusion reactor (DEMO) and which are intended for experimental investigations at the International Thermonuclear Experimental Reactor (ITER), liquid metals are assumed to fulfil simultaneously the functions of (i) a tritium breeder, (ii) a coolant, and (iii) neutron moderator and multiplier. This approach to testing experimentally design solutions is motivated by plans to employ, in the majority of the currently developed DEMO blanket projects, liquid metals pumped through pipes and/or rectangular channels in a transvers magnetic field. At the present time, experiments that would directly simulate liquid-metal flows under conditions of ITER TBM and/or DEMO blanket operation (irradiation with thermonuclear neutrons, a cyclic temperature regime, and a magnetic-field strength of about 4 to 10 T) are not implementable for want of equipment that could reproduce simultaneously the aforementioned effects exerted by thermonuclear plasmas. This is the reason why use is made of an iterative approach to experimentally estimating the performance of design solutions for liquid-metal channels via simulating one or simultaneously two of the aforementioned factors. Therefore, the investigations reported in the present article are of considerable topical interest. The respective experiments were performed on the basis of the mercury magneto hydrodynamic (MHD) loop that is included in the structure of the MPEI—JIHT MHD experimental facility. Temperature fields were measured under conditions of two- and one-sided heating, and data on averaged-temperature fields, distributions of the wall temperature, and statistical fluctuation features were obtained. A substantial effect of counter thermo gravitational convection (TGC) on averaged and fluctuating quantities were found. The development of TGC in the presence of a magnetic field leads to the appearance of low-frequency fluctuations whose anomalously high intensity exceeds severalfold the level of turbulence fluctuations. This effect manifest itself over a broad region of regime parameters. It was confirmed that low-energy fluctuations penetrate readily through the wall; therefore, it is necessary to study this effect further—in particular, from the point of view of the fatigue strength of the walls of liquid-metal channels.

42 ENGINEERING↗

DOE OSTI2025

AI Model Benchmarking for Nonproliferation Applications: Steel Thread Benchmarking Task Force Technical Report (Rev. 2)

Steel Thread is a NA-22 venture that seeks to build trustworthy, reliable AI models that can be used in a wide variety of nonproliferation tasks. A key aspect of building these models is developing appropriate benchmarks and evaluation methods, which will enable the venture to identify and adapt models to provide the most value in the nonproliferation domain. Benchmarks must be relevant to key tasks in this domain, such as question answering, information retrieval, document summarization and classification, consensus analysis, and image and data analysis. This report 1) provides an overview of benchmark design, evaluation, and challenges; 2) reviews a variety of open benchmarks, with a focus on language models and tasks; and 3) identifies benchmarks that are most relevant to Steel Thread. This report is intended to serve as a basis for further efforts to classify and evaluate benchmarks and their correlation with success on nonproliferation-specific tasks. The Steel Thread venture has defined benchmarks to be a particular combination of a dataset (or datasets) and a metric (or metrics) conceptualized as representing one or more specific tasks or sets of abilities for a specific modality. It is adopted by a research community as a shared framework for comparing methods.1 It includes 1) Data: Labeled (a designated subset not used for training, which could be all the data), 2) Metric: A way to quantify performance, 3) Task/Ability: What the benchmark is testing, 4) Protocol: A structured and repeatable evaluation process, 5) Baseline/Reference Model: For comparison; could be statistical, rule-based, SME-derived, or another model, and 6) Maintenance Plan: to update with new information over time; important for long-term utility. For further clarity, the definition includes what a benchmark, in this context, is not. It is not a corpus of training data, specific to a model (it is intended to apply to a range of models), a universal evaluation of performance, a guarantee that the ‘top’ model on the leaderboard will be the best fit for every specific use case, an all-encompassing proof of a model’s universal quality, nor is it a one-size-fits-all measure of success. It does not cover every real-world constraint (like operational, ethical, or cost considerations), a systems integration test, or a unit test. This definition was inspired by and resulted from discussions within the Steel Thread Benchmarking Task Force. This group was formed to define what we would mean as a benchmark within Steel Thread but persisted as the need to develop a thorough understanding of the large and expanding existing benchmarking space. This technical report is a result of the group’s divide and conquer approach to exploring this space. The release of benchmarks might not be progressing as quickly as model development, but it is moving very fast, as many benchmarks quickly become saturated, when state-of-the-art models score so close to the benchmark’s ceiling that their results are virtually indistinguishable. At that point, the test no longer differentiates between new systems, so researchers usually stop reporting scores as the benchmark no longer informs about improvements from the next generation of models. In the OpenAI announcement of GPT-5, they reported results on six flagship public benchmarks (AIME 2025, SWE-bench Verified, Aider Polyglot, MMMU, HealthBench Hard, GPQA) but the full system-card covers roughly thirty-five separate evaluations, comprising hundreds of test task items in total. There have been some efforts to summarize benchmarks in specific fields, like for text-to-image generation, but these surveys have had a narrow methodology scope. Therefore, a comprehensive survey of all benchmarks or even all benchmarks that could be relevant to Steel Thread is outside of the scope of this report. We chose some specific benchmarks to investigate in detail.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2021

An evaluation of prognostic factors, oncologic outcomes, and management for primary and recurrent squamous cell carcinoma of the vulva

Objective: To evaluate prognostic factors, outcomes, and management patterns of patients treated for squamous cell carcinoma of the vulva. Methods: One hundred sixty-four women were retrospectively identified with primary squamous cell carcinoma of the vulva treated at our institution between 1/1996–12/2018. Descriptive statistics were performed on patient, tumor, and treatment characteristics. The χ 2 tests and t-tests were used to compare categorical variables and continuous variables, respectively. Recurrence free survival (RFS), overall survival (OS), and disease-specific survival (DSS) were analyzed with Kaplan-Meier estimates, the log-rank test, and Cox proportional hazards. Results: Median follow-up was 52.5 months. Five-year RFS was 67.9%, 60.0%, 42.1%, and 20.0% for stage I–IV, respectively. Five-year DSS was 86.2%, 81.6%, 65.0%, and 42.9% for stage I–IV, respectively. On multivariate analysis, positive margins predicted overall RFS (hazard ratio [HR]=3.55; 95% confidence interval [CI]=1.18–10.73; p=0.025), while presence of lichen sclerosus on pathology (HR=2.78; 95% CI=1.30–5.91; p=0.008) predicted local RFS. OS was predicted by nodal involvement (HR=2.51; 95% CI=1.02–6.13; p=0.043) and positive margins (HR=5.19; 95% CI=2.03–13.26; p=0.001). Adjuvant radiotherapy significantly improved RFS (p=0.016) and DSS (p=0.012) in node-positive patients. Median survival after treatment of local, groin, and pelvic/distant recurrence was 52, 8, and 5 months, respectively. Conclusion: For primary treatment, more conservative surgical approaches can be considered with escalation of treatment in patients with concurrent precursor lesions, positive margins, and/or nodal involvement. Further studies are warranted to improve risk stratification in order to optimize treatment paradigms for vulvar cancer patients.

60 APPLIED LIFE SCIENCES↗

NASA NTRS2014

Statistical Mechanics of Turbulent Dynamos

Incompressible magnetohydrodynamic (MHD) turbulence and magnetic dynamos, which occur in magnetofluids with large fluid and magnetic Reynolds numbers, will be discussed. When Reynolds numbers are large and energy decays slowly, the distribution of energy with respect to length scale becomes quasi-stationary and MHD turbulence can be described statistically. In the limit of infinite Reynolds numbers, viscosity and resistivity become zero and if these values are used in the MHD equations ab initio, a model system called ideal MHD turbulence results. This model system is typically confined in simple geometries with some form of homogeneous boundary conditions, allowing for velocity and magnetic field to be represented by orthogonal function expansions. One advantage to this is that the coefficients of the expansions form a set of nonlinearly interacting variables whose behavior can be described by equilibrium statistical mechanics, i.e., by a canonical ensemble theory based on the global invariants (energy, cross helicity and magnetic helicity) of ideal MHD turbulence. Another advantage is that truncated expansions provide a finite dynamical system whose time evolution can be numerically simulated to test the predictions of the associated statistical mechanics. If ensemble predictions are the same as time averages, then the system is said to be ergodic; if not, the system is nonergodic. Although it had been implicitly assumed in the early days of ideal MHD statistical theory development that these finite dynamical systems were ergodic, numerical simulations provided sufficient evidence that they were, in fact, nonergodic. Specifically, while canonical ensemble theory predicted that expansion coefficients would be (i) zero-mean random variables with (ii) energy that decreased with length scale, it was found that although (ii) was correct, (i) was not and the expected ergodicity was broken. The exact cause of this broken ergodicity was explained, after much investigation, by greatly extending the statistical theory of ideal MHD turbulence. The mathematical details of broken ergodicity, in fact, give a quantitative explanation of how coherent structure, dynamic alignment and force-free states appear in turbulent magnetofluids. The relevance of these ideal results to real MHD turbulence occurs because broken ergodicity is most manifest in the ideal case at the largest length scales and it is in these largest scales that a real magnetofluid has the least dissipation, i.e., most closely approaches the behavior of an ideal magnetofluid. Furthermore, the effects grow stronger when cross and magnetic helicities grow large with respect to energy, and this is exactly what occurs with time in a real magnetofluid, where it is called selective decay. The relevance of these results found in ideal MHD turbulence theory to the real world is that they provide at least a qualitative explanation of why confined turbulent magnetofluids, such as the liquid iron that fills the Earth's outer core, produce stationary, large-scale magnetic fields, i.e., the geomagnetic field. These results should also apply to other planets as well as to plasma confinement devices on Earth and in space, and the effects should be manifest if Reynolds numbers are high enough and there is enough time for stationarity to occur, at least approximately. In the presentation, details will be given for both theoretical and numerical results, and references will be provided.

Shebalin, John V.↗

NASA NTRS2014

MHD Turbulence and Magnetic Dynamos

Shebalin, John V↗

DOE OSTI2018

Impact of global atmospheric reanalyses on statistical precipitation downscaling

Statistical downscaling based on a perfect prognosis approach often relies on global reanalyses to infer the statistical relationship between synoptic predictors and the local variable of interest, here daily precipitation. Nowadays, many reanalyses are available and their impact on the downscaled variable is not often considered. The present work assessed the impact of ten reanalyses on the performance of seven variants of analogue methods for statistical precipitation downscaling at 301 stations in Switzerland. Even though the study location is in a data-rich region, significant differences were found between reanalyses and their impact on the performance of the method can be even higher than the choice of the predictor variables. There was no single overall winner, but a selection of recommended reanalyses resulting in higher skill scores depending on the considered predictor variables. The impact of the output spatial resolution was assessed for different types of variables. Output resolutions below one degree were found to be often of low to no interest. Reanalyses with longer archives allow the pool of potential analogues to be increased, resulting in better performance. However, when adding variables affected by errors in a more distant past, the skill score decreased again. The use of multiple members from two reanalyses was also tested over a recent and a past period. The benefit of using members to increase the performance by better incorporating the uncertainties was found to be limited, and even problematic with methods using multiple analogy levels.

Meteorology & Atmospheric Sciences↗