Results for “Report” — Page 19

NASA NTRS2020

Monitoring Water-Related Ecosystems with Earth Observation Data in Support of Sustainable Development Goal (SDG) 6 Reporting

Lack of national data on water-related ecosystems is a major challenge to achieving the Sustainable Development Goal (SDG) 6 targets by 2030. Monitoring surface water extent, wetlands, and water quality from space can be an important asset for many countries in support of SDG 6reporting. We demonstrate the potential for Earth observation (EO) data to support country reporting for SDG Indicator 6.6.1, ‘Change in the extent of water-related ecosystems over time’ and identify important considerations for countries using these data for SDG reporting. The spatial extent of water-related ecosystems, and the partial quality of water within these ecosystems is investigated for seven countries. Data from the Moderate Resolution Imaging Spectroradiometer (MODIS) and Landsat 5, 7, and 8 with Shuttle Radar Topography Mission (SRTM) are used to measure surface water extent at 250 m and 30 m spatial resolution, respectively, in Cambodia, Jamaica, Peru, the Philippines, Senegal, Uganda, and Zambia. The extent of mangroves is mapped at 30 m spatial resolution using Landsat 8 Operational Land Imager (OLI), Sentinel-1, and SRTM data for Jamaica, Peru, and Senegal. Using Landsat 8 and Sentinel 2A imagery, total suspended solids and chlorophyll-a are mapped overtime for a select number of large surface water bodies in Peru, Senegal, and Zambia. All of the EO datasets used are of global coverage and publicly available at no cost. The temporal consistency and long time-series of many of the datasets enable replicability over time, making reporting of change from baseline values consistent and systematic. We find that statistical comparisons between different surface water data products can help provide some degree of confidence for countries during their validation process and highlight the need for accuracy assessments when using EO-based land change data for SDG reporting. We also raise concern that EO data in the context of SDG Indicator 6.6.1reporting may be more challenging for some countries, such as small island nations, than others to use in assessing the extent of water-related ecosystems due to scale limitations and climate variability. Country-driven validation of the EO data products remains a priority to ensure successful data integration in support of SDG Indicator 6.6.1 reporting. Multi-country studies such as this one can be valuable tools for helping to guide the evolution of SDG monitoring methodologies and provide a useful resource for countries reporting on water-related ecosystems. The EO data analyses and statistical methods used in this study can be easily replicated for country-driven validation of EO data products in the future.

Raha Hakimdavar↗

NASA NTRSDate not supplied

An Approach to Identifying Aspects of Positive Pilot Behavior within the Aviation Safety Reporting System

The National Airspace System (NAS) is constantly evolving as air traffic continues to ramp up to pre-pandemic numbers and projected to grow to unprecedented levels in the coming years. As well as increasing demand to the current system, emerging operations such as Unmanned Autonomous Systems are also expected to add to complexity in the airspace. To address these issues, the industry and government agencies supporting the NAS will need to rely upon additional automation and new technologies to address future operational requirements, while continuing to be a world-leading safe transportation system. As these new technologies are implemented, the system continues to rely on human pilots and controllers in the loop to monitor the system and intervene in situations the automation cannot handle. The goal of proactively addressing safety is of foremost concern to ensure passenger confidence. The industry has implemented various Safety Monitoring Systems to identify safety risks and proactively address them before they result in a serious incident or accident. One such program is the Aviation Safety Reporting System (ASRS). ASRS is a long-established system where pilots and controllers voluntarily and anonymously report safety incidents they experienced and observed during line operations by providing rich text narratives describing the events, the environment, and conditions leading to the safety event of concern. These narratives provide insight and context around events of interest and can be used to identify emerging problems. They can trigger investigations within Flight Operational Quality Assurance or Flight Data Monitoring programs. However, this process typically focuses on the adverse events and the unsafe aspects of the operations surrounding the reported or detected events. This perspective of investigating factors that went wrong around an adverse event is commonly referred to as Safety I. Alternatively, characterizing successful actions that operators perform every day under varying conditions that keep the system within safe operating bounds is a concept referred to as Safety II. The benefit of the Safety II view is that the scope is much larger than that of Safety I since a vast majority of the operations result in successful flights. Many of the successful techniques used to manage operational threats are not documented in standard operating procedures or taught during training. They are typically acquired over time by working with experienced pilots during line operations or in many cases after experiencing a problem for the first time and reacting to it in situ, drawing from years of experience to manage the threat. In an attempt to quantify these positive actions, we are proposing an approach to extracting key behaviors within ASRS reports that can support the Safety II concept. Our analysis assumes that ASRS reports contain some descriptions of corrective actions that operators performed to prevent a situation from leading to an accident. Leveraging recent advances in Natural Language Process modeling, we have developed an approach to extract positive sentiment from reports, embed these positive statements in a vector space where they can be numerically analyzed, and clustering these statements into similar contextual categories. From these contextualized categories we can attempt to summarized and distilled aspects of the positive behavior. The goal is to identify categories of behavior that describe consistent operator techniques that supports the Safety II concept. With this information, airlines may enable learning from these positive actions, or address procedures that need to be changed to avoid having pilots implement a workaround. These insights can provide a lens into what is “going right” in the operations that may otherwise not be known widely within the community. It is envisioned that this approach can be extended to other narrative programs such as Line Operation Safety Audit or Learning Improvement Team reports where similar observed behavior can be analyzed to extract positive actions and inform the overall operations.

NLP↗

NASA NTRSDate not supplied

An Approach to Identifying Aspects of Positive Pilot Behavior within the Aviation Safety Reporting System

The National Airspace System (NAS) is constantly evolving as air traffic continues to ramp up to pre-pandemic numbers and projected to grow to unprecedented levels in the coming years. As well as increasing demand to the current system, emerging operations such as Unmanned Autonomous Systems are also expected to add to complexity in the airspace. To address these issues, the industry and government agencies supporting the NAS will need to rely upon additional automation and new technologies to address future operational requirements, while continuing to be a world-leading safe transportation system. As these new technologies are implemented, the system continues to rely on human pilots and controllers in the loop to monitor the system and intervene in situations the automation cannot handle. The goal of proactively addressing safety is of foremost concern to ensure passenger confidence. The industry has implemented various Safety Monitoring Systems to identify safety risks and proactively address them before they result in a serious incident or accident. One such program is the Aviation Safety Reporting System (ASRS). ASRS is a long-established system where pilots and controllers voluntarily and anonymously report safety incidents they experienced and observed during line operations by providing rich text narratives describing the events, the environment, and conditions leading to the safety event of concern. These narratives provide insight and context around events of interest and can be used to identify emerging problems. They can trigger investigations within Flight Operational Quality Assurance or Flight Data Monitoring programs. However, this process typically focuses on the adverse events and the unsafe aspects of the operations surrounding the reported or detected events. This perspective of investigating factors that went wrong around an adverse event is commonly referred to as Safety I. Alternatively, characterizing successful actions that operators perform every day under varying conditions that keep the system within safe operating bounds is a concept referred to as Safety II. The benefit of the Safety II view is that the scope is much larger than that of Safety I since a vast majority of the operations result in successful flights. Many of the successful techniques used to manage operational threats are not documented in standard operating procedures or taught during training. They are typically acquired over time by working with experienced pilots during line operations or in many cases after experiencing a problem for the first time and reacting to it in situ, drawing from years of experience to manage the threat. In an attempt to quantify these positive actions, we are proposing an approach to extracting key behaviors within ASRS reports that can support the Safety II concept. Our analysis assumes that ASRS reports contain some descriptions of corrective actions that operators performed to prevent a situation from leading to an accident. Leveraging recent advances in Natural Language Process modeling, we have developed an approach to extract positive sentiment from reports, embed these positive statements in a vector space where they can be numerically analyzed, and clustering these statements into similar contextual categories. From these contextualized categories we can attempt to summarized and distilled aspects of the positive behavior. The goal is to identify categories of behavior that describe consistent operator techniques that supports the Safety II concept. With this information, airlines may enable learning from these positive actions, or address procedures that need to be changed to avoid having pilots implement a workaround. These insights can provide a lens into what is “going right” in the operations that may otherwise not be known widely within the community. It is envisioned that this approach can be extended to other narrative programs such as Line Operation Safety Audit or Learning Improvement Team reports where similar observed behavior can be analyzed to extract positive actions and inform the overall operations.

NLP↗

NASA NTRSDate not supplied

An Approach to Identifying Aspects of Positive Pilot Behavior within the Aviation Safety Reporting System

The National Airspace System (NAS) is constantly evolving as air traffic continues to ramp up to pre-pandemic numbers and projected to grow to unprecedented levels in the coming years. As well as increasing demand to the current system, emerging operations such as Unmanned Autonomous Systems are also expected to add to complexity in the airspace. To address these issues, the industry and government agencies supporting the NAS will need to rely upon additional automation and new technologies to address future operational requirements, while continuing to be a world-leading safe transportation system. As these new technologies are implemented, the system continues to rely on human pilots and controllers in the loop to monitor the system and intervene in situations the automation cannot handle. The goal of proactively addressing safety is of foremost concern to ensure passenger confidence. The industry has implemented various Safety Monitoring Systems to identify safety risks and proactively address them before they result in a serious incident or accident. One such program is the Aviation Safety Reporting System (ASRS). ASRS is a long-established system where pilots and controllers voluntarily and anonymously report safety incidents they experienced and observed during line operations by providing rich text narratives describing the events, the environment, and conditions leading to the safety event of concern. These narratives provide insight and context around events of interest and can be used to identify emerging problems. They can trigger investigations within Flight Operational Quality Assurance or Flight Data Monitoring programs. However, this process typically focuses on the adverse events and the unsafe aspects of the operations surrounding the reported or detected events. This perspective of investigating factors that went wrong around an adverse event is commonly referred to as Safety I. Alternatively, characterizing successful actions that operators perform every day under varying conditions that keep the system within safe operating bounds is a concept referred to as Safety II. The benefit of the Safety II view is that the scope is much larger than that of Safety I since a vast majority of the operations result in successful flights. Many of the successful techniques used to manage operational threats are not documented in standard operating procedures or taught during training. They are typically acquired over time by working with experienced pilots during line operations or in many cases after experiencing a problem for the first time and reacting to it in situ, drawing from years of experience to manage the threat. In an attempt to quantify these positive actions, we are proposing an approach to extracting key behaviors within ASRS reports that can support the Safety II concept. Our analysis assumes that ASRS reports contain some descriptions of corrective actions that operators performed to prevent a situation from leading to an accident. Leveraging recent advances in Natural Language Process modeling, we have developed an approach to extract positive sentiment from reports, embed these positive statements in a vector space where they can be numerically analyzed, and clustering these statements into similar contextual categories. From these contextualized categories we can attempt to summarized and distilled aspects of the positive behavior. The goal is to identify categories of behavior that describe consistent operator techniques that supports the Safety II concept. With this information, airlines may enable learning from these positive actions, or address procedures that need to be changed to avoid having pilots implement a workaround. These insights can provide a lens into what is “going right” in the operations that may otherwise not be known widely within the community. It is envisioned that this approach can be extended to other narrative programs such as Line Operation Safety Audit or Learning Improvement Team reports where similar observed behavior can be analyzed to extract positive actions and inform the overall operations.

NLP↗

NASA NTRS2023

International VLBI Service for Geodesy and Astrometry 2021+2022 Biennial Report

Since its creation in 1999, the International VLBI Service for Geodesy and Astrometry (IVS) has documented its progress and current status in the form of annual or biennial reports. The first sixteen years were recorded in the form of annual reports. Starting in 2015/2016 the cadence was changed to two years; hence, this Biennial Report marks eight years of using the two-yearly reporting scheme. The general structure of the reporting has remained stable over the years: the individual components of the IVS contributed short reports describing their numerous activities, progress, and future plans. Without the continued input from the VLBI groups of the international geodetic and astrometric community, this publication could not be compiled and the IVS itself would not be able to flourish. So, once again many thanks to all IVS components who contributed to this Biennial Report

Space Geodesy↗

DOE OSTI2024

Bridging the time scale in exascale computing of chemical systems (Final Technical Report)

This report summarizes the work carried out with support of the United States Department of Energy under Award DE-SC0019441. The theme of this project was to develop and apply methods that allowed for the acceleration of atomistic calculations, particularly in challenging areas such as multiphase systems, electrified interfaces, uncertainty estimation, and applications requiring chemical accuracy, which tend to be applications where simulation time is severely bottlenecked by the computational time requirements. Much of the focus was on the application of emerging machine-learning methodologies, although a wide range of methodologies were employed. This report has two major sections. The first focuses on the methodological advances themselves. Within this part, we report a number of major advances, a few examples of which are described here. We report the first machine-learning scheme for the acceleration of electronically grand-canonical calculations (that is, those applicable to electrochemistry). We report new methods of performing transfer learning, in which physics-based priors can be used to provide predictions, often with uncertainty estimates, of images well outside of training sets; we also offer ways to fine-tune these transfer-learning models. We provide a new systematic means to generate and apply minimal training data sets to very large (10,000’s of atoms) systems, with only small training sets appropriate for electronic structure. We developed new methodologies to integrate surface vibrations into surface adsorption calculations. We made advances to the applicability of diffusion Monte Carlo methods to allow (learned) force prediction, finite-size error correction, and force-free means of searching for transition states. We integrated machine-learned atomistic predictions into mechanism generation codes. Additionally, we released new software including AmpTorch, a modernized version of our original atomistic machine-learning code Amp. The second part of this report focuses on the scientific applications that accompanied, and were often enabled by, the methodological advances described earlier. A few examples follow, but full details are in the individual chapters of the report. For example, we developed a general theory of phonon-induced friction on molecular adsorbates. We showed fundamentally how solvent influences the adsorption and desorption process and how it differs from the processes typically involved at the solid–gas interface, making aqueous-phase and electrocatalysis different from traditional thermocatalysis. We examined how metal–insulator and magnetic transitions can be probed, and accelerated exciton dynamics via Frenkel Hamiltonian parameters. We showed that the nearsighted force-training approach, developed within this project, can predict both the stability and reactivity of large nanoparticles, and can also lead to insights on catalyst coverage on binding energies and entropies. These applied studies, which generally integrated with our method development, allowed us to push forward the theoretical understanding of several reaction classes.

08 HYDROGEN↗

DOE OSTI2025

Mechanical Solutions Scan Report

Power lines, poles, and towers are the backbone of the United States (U.S.) electric-power grid. These transmission and distribution networks route electricity from generator to loads. The characteristics of these routes are rapidly changing -- trending towards decentralized renewable generation, electric heating, vehicle charging, and large data-center loads. Coupled with aging infrastructure and the increased frequency of extreme weather events, there is concern about the future reliability and transmission capacity of conductors and adjacent components. This scan report seeks to provide an overview of mechanical solutions to challenges caused by extreme weather events associated with components of transmission and distribution infrastructure, including conductor heat sag, ice accumulation, wind, and wildfire. Many options could increase transmission capacity or reliability, and these are at various stages of technological readiness. Some have only been lab tested, while some have been widely deployed in the U.S. or overseas for decades. The solution categories and providers featured in this report are intended to be comprehensive at the time of publication and to serve as a reference for decision-makers concerned about transmission and distribution reliability. There are two other categories of large, complex solutions, which are not covered in this report: replacing existing conductors with advanced conductors and implementing digital grid enhancing technologies. A separate scan report titled “Advanced Conductor Scan Report,” which discusses advanced carbon-core conductors, was published by the Idaho National Laboratory (INL) in 2023. Information on digital technologies, such as dynamic line ratings, power-flow controllers, and other power electronics and communications-based devices, can be found on the Grid- Enhancing Technologies landing page. Mechanical grid-enhancing technologies, or solutions covered in this report, often do not require full equipment replacement and do not rely on digital components. Mechanical technologies are overlooked because they may be older, simpler, or seemingly “more obvious” than digital or carbon-core technologies. However, it is wise to consider mechanical solutions in a thorough evaluation of grid enhancing technology solutions.

24 - POWER TRANSMISSION AND DISTRIBUTION↗

NASA NTRSDate not supplied

A Survey Protocol to Assess Meaningfulness and Usefulness of Automated Topic Finding in the NASA Aviation Safety Reporting System

Context: The NASA Aviation Safety Reporting System (ASRS) is a voluntary confidential aviation safety reporting system. The ASRS receives reports from pilots, air traffic controllers, flight attendants and other involved in aviation operations. The reports are de-identified and coded by ASRS expert safety analysts and a short descriptive synopsis is written to describe the safety issue. The de-identified reports are then disseminated to the aviation community in a number of ways including entry into an online database, Safety Alert Bulletins and For Your Information Notices, and the CALLBACK newsletter. Key to these publications are the timely processing (de-identification, coding and summarization) of new reports, which is currently done by ASRS expert safety analysts. Thus, we believe topic modelling could decrease effort in ASRS, if topics are comprehensible. Aim: We propose a methodology to evaluate whether automated topic finding using topic modelling provides meaningful and useful topics. Method: We extend the total error survey methodology to evaluate user topic comprehension of machine learning outputs. To accomplish this we performed a literature review to identify existing methods and define a construct for topic comprehension, utilizing existing ASRS synopsis writing practices to more precisely define meaningfulness and usefulness. Results: A survey protocol was created that addresses the limitations of other survey protocols found in the literature review, which we found lacking in rationale and clear protocol definition. Conclusion: The surveying of user understanding in machine learning outputs presents challenges due to the explosion of parameters to control for and the lack of systematic approach presented in the literature. More reproducible work and survey protocols are needed in the literature and our work is one step towards that direction.

topic finding↗

DOE OSTI2024

Weak baselines and reporting biases lead to overoptimism in machine learning for fluid-related partial differential equations

One of the most promising applications of machine learning in computational physics is to accelerate the solution of partial differential equations (PDEs). The key objective of machine-learning-based PDE solvers is to output a sufficiently accurate solution faster than standard numerical methods, which are used as a baseline comparison. Here, we first perform a systematic review of the ML-for-PDE-solving literature. Out of all of the articles that report using ML to solve a fluid-related PDE and claim to outperform a standard numerical method, we determine that 79% (60/76) make a comparison with a weak baseline. Second, we find evidence that reporting biases are widespread, especially outcome reporting and publication biases. We conclude that ML-for-PDE-solving research is overoptimistic: weak baselines lead to overly positive results, while reporting biases lead to under-reporting of negative results. To a large extent, these issues seem to be caused by factors similar to those of past reproducibility crises: researcher degrees of freedom and a bias towards positive results. We call for bottom-up cultural changes to minimize biased reporting as well as top-down structural reforms to reduce perverse incentives for doing so.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2024

OES-Environmental 2024 State of the Science Report: Environmental Effects of Marine Renewable Energy Development Around the World

This report summarizes the state of the science of environmental effects of marine renewable energy (MRE) and serves as an update and a complement to the 2020 State of the Science report. The 2024 State of the Science report was produced by the Ocean Energy Systems (OES)-Environmental initiative, under the International Energy Agency’s OES collaboration. Under OES-Environmental, 16 countries have collaborated to evaluate the “state of the science” of potential environmental effects of MRE development and to understand how they may affect consenting/permitting (hereafter consenting) of MRE devices. This report has brought together the most up-to-date information on potential environmental effects of MRE development, using information that is publicly available as well as from expert inputs. The OES-Environmental analysts from the 16 participating countries helped to scope the entirety of the report and provided valuable contributions to all chapters. The input from these contributors and reviewers has resulted in the most complete compendium of research and monitoring findings possible. This report encompasses an introduction and look ahead, as well as nine chapters that provide details of research and monitoring findings around the world on environmental effects of MRE.

16 TIDAL AND WAVE POWER↗

DOE OSTI2024

Reported Energy and Cost Savings from the DOE ESPC IDIQ Program: FY 2023

The objective of this work was to determine the realization rate of energy and cost savings from the U.S. Department of Energy’s (DOE’s) Energy Savings Performance Contract (ESPC) program based on information reported by the energy services companies (ESCOs) that are carrying out ESPC projects at federal sites. Information was extracted from 201 measurement and verification (M&V) reports covering 191 projects to determine reported, estimated, and guaranteed cost savings and the associated reported and estimated energy savings for the previous contract performance year. This report covers projects that had a performance year ending in fiscal year 2023, between October 1, 2022 and September 30, 2023, and had an M&V report issued. Additionally, the annual cost to perform M&V was extracted from the individual project Task Order (TO) Schedules.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗

DOE OSTI2025

East Tennessee Technology Park Biological Monitoring and Abatement Program 2024 Calendar Year Report

The East Tennessee Technology Park (ETTP) Biological Monitoring and Abatement Program (BMAP) consists of three tasks that reflect different but complementary approaches to evaluating the ecological integrity of waters near ETTP. These tasks include (1) bioaccumulation monitoring of fish and clams, (2) benthic macroinvertebrate species richness and density monitoring, and (3) fish community monitoring. The sampling and analysis requirements for the ETTP BMAP in calendar year 2024, covering in part both FY 2024 and FY 2025, are outlined in the respective FY sampling and analysis plans (UCOR 2023, 2024). Sampled water bodies and locations for the ETTP BMAP are shown in Figures 1 and 2. This ETTP BMAP report presents the CY 2024 results and provides context with results from previous years. The report also includes Oak Ridge National Laboratory (ORNL)–generated biological monitoring data collected for other US Department of Energy programs, including the UCOR Water Resources Restoration Program (WRRP) off-site fish bioaccumulation data (UCOR 2023) and select Y-12 National Security Complex (Y-12) BMAP fish bioaccumulation data. Historical data collected for the ETTP BMAP and other programs in the nearby Poplar Creek and Clinch River are provided where appropriate. This progress report provides an update on the biological monitoring activities supporting the ETTP UCOR Environmental Compliance organization, which sponsors the ETTP BMAP. In addition to this internal reporting, ETTP BMAP results are provided in the annual remediation effectiveness reports and the annual site environmental reports, both of which are publicly available. BMAP data are also available to the public via the Oak Ridge Environmental Information System (https://ucor.com/oak-ridge-environmental-information-system-oreis/).

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2024

Fusion Materials Semiannual Progress Report for the Period Ending June 30, 2024

This is the seventy-sixth in a series of semiannual technical progress reports on fusion materials science activity supported by the Fusion Energy Sciences Program of the U.S. Department of Energy. It covers the period ending June 30, 2024. This report focuses on research addressing the effects on materials properties and performance of exposure to the neutronic, thermal and chemical environments anticipated in the chambers of fusion experiments and energy systems. This research is a major element of the national effort to establish the materials knowledge base for an economically and environmentally attractive fusion energy source. Research activities on issues related to the interaction of materials with plasmas are reported separately. The results reported are the products of a national effort involving a number of national laboratories and universities. A large fraction of this work, particularly in relation to fission reactor irradiations, is carried out collaboratively with partners in Japan, Russia, and the European Union. The purpose of this series of reports is to provide a working technical record for the use of program participants, and to provide a means of communicating the efforts of fusion materials scientists to the broader fusion community, both nationally and worldwide. This report has been compiled by Stephanie Melton, Oak Ridge National Laboratory. Her efforts, and the efforts of the many persons who made technical contributions, are gratefully acknowledged.

36 MATERIALS SCIENCE↗

DOE OSTI2025

AI Model Benchmarking for Nonproliferation Applications: Steel Thread Benchmarking Task Force Technical Report (Rev. 2)

Steel Thread is a NA-22 venture that seeks to build trustworthy, reliable AI models that can be used in a wide variety of nonproliferation tasks. A key aspect of building these models is developing appropriate benchmarks and evaluation methods, which will enable the venture to identify and adapt models to provide the most value in the nonproliferation domain. Benchmarks must be relevant to key tasks in this domain, such as question answering, information retrieval, document summarization and classification, consensus analysis, and image and data analysis. This report 1) provides an overview of benchmark design, evaluation, and challenges; 2) reviews a variety of open benchmarks, with a focus on language models and tasks; and 3) identifies benchmarks that are most relevant to Steel Thread. This report is intended to serve as a basis for further efforts to classify and evaluate benchmarks and their correlation with success on nonproliferation-specific tasks. The Steel Thread venture has defined benchmarks to be a particular combination of a dataset (or datasets) and a metric (or metrics) conceptualized as representing one or more specific tasks or sets of abilities for a specific modality. It is adopted by a research community as a shared framework for comparing methods.1 It includes 1) Data: Labeled (a designated subset not used for training, which could be all the data), 2) Metric: A way to quantify performance, 3) Task/Ability: What the benchmark is testing, 4) Protocol: A structured and repeatable evaluation process, 5) Baseline/Reference Model: For comparison; could be statistical, rule-based, SME-derived, or another model, and 6) Maintenance Plan: to update with new information over time; important for long-term utility. For further clarity, the definition includes what a benchmark, in this context, is not. It is not a corpus of training data, specific to a model (it is intended to apply to a range of models), a universal evaluation of performance, a guarantee that the ‘top’ model on the leaderboard will be the best fit for every specific use case, an all-encompassing proof of a model’s universal quality, nor is it a one-size-fits-all measure of success. It does not cover every real-world constraint (like operational, ethical, or cost considerations), a systems integration test, or a unit test. This definition was inspired by and resulted from discussions within the Steel Thread Benchmarking Task Force. This group was formed to define what we would mean as a benchmark within Steel Thread but persisted as the need to develop a thorough understanding of the large and expanding existing benchmarking space. This technical report is a result of the group’s divide and conquer approach to exploring this space. The release of benchmarks might not be progressing as quickly as model development, but it is moving very fast, as many benchmarks quickly become saturated, when state-of-the-art models score so close to the benchmark’s ceiling that their results are virtually indistinguishable. At that point, the test no longer differentiates between new systems, so researchers usually stop reporting scores as the benchmark no longer informs about improvements from the next generation of models. In the OpenAI announcement of GPT-5, they reported results on six flagship public benchmarks (AIME 2025, SWE-bench Verified, Aider Polyglot, MMMU, HealthBench Hard, GPQA) but the full system-card covers roughly thirty-five separate evaluations, comprising hundreds of test task items in total. There have been some efforts to summarize benchmarks in specific fields, like for text-to-image generation, but these surveys have had a narrow methodology scope. Therefore, a comprehensive survey of all benchmarks or even all benchmarks that could be relevant to Steel Thread is outside of the scope of this report. We chose some specific benchmarks to investigate in detail.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2026

Fusion Materials Semiannual Progress Report for the Period Ending June 30, 2025

This is the seventy-eighth in a series of semiannual technical progress reports on fusion materials science activity supported by the Fusion Energy Sciences Program of the U.S. Department of Energy. It covers the period ending June 30 th , 2025. This report focuses on research addressing the effects on materials properties and performance of exposure to the neutronic, thermal and chemical environments anticipated in the chambers of fusion experiments and energy systems. This research is a major element of the national effort to establish the materials knowledge base for an economically and environmentally attractive fusion energy source. Research activities on issues related to the interaction of materials with plasmas are reported separately. The results reported are the products of a national effort involving a number of national laboratories and universities. A large fraction of this work, particularly in relation to fission reactor irradiations, is carried out collaboratively with international partners, e.g., Japan and the UK. The purpose of this series of reports is to provide a working technical record for the use of program participants, and to provide a means of communicating the efforts of fusion materials scientists to the broader fusion community, both nationally and worldwide. This report has been compiled by Stephanie Melton, Oak Ridge National Laboratory. Her efforts, and the efforts of the many persons who made technical contributions, are gratefully acknowledged.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

DOE OSTI2026

U.S. ESCO Industry Report: Industry Size and Recent Market Trends, 2022- 2024

The latest edition of the U.S. Energy Service Company (ESCO) Industry Report by Lawrence Berkeley National Laboratory (LBNL) finds that the U.S. ESCO industry continues to show strong growth. The report draws from ESCO industry reported revenue data for the 2022-2024 period, detailing the current size and characteristics of the U.S. ESCO industry. Following 20 years of ESCO industry reports, the 2024 report explores significant revenue trends across market segments, geographic regions, ESCO size, financing structures, and business activities. New analysis in this report outlines customer priorities and non-energy benefit drivers of Energy Savings Performance Contract projects, adjusted revenue analysis detailing the impacts of inflation on industry growth, and project challenges by market segment.

Chelminski, Kathryn↗

NASA NTRS1968

NASA scientific and technical reports for 1967. A selected listing

NASA Scientific and Technical Reports for 1967, A Selected Listing (NASA SP-7029) combines in a single publication a selective listing of NASA reports that were announced during 1967 in Scientific and Technical Aerospace Reports (STAR), NASA's semimonthly announcement journal for the aerospace sciences The documents listed were issued as part of the following NASA report series Special Publications (NASA SP-), Technical Reports (NASA TR-). Technical Notes (NASA TN D-), Technical Memorandums (NASA TM X-1, Technical Translations (NASA TT F-), and Contractor Reports (NASA CR-) The listing does not include references to journal articles that were published as a result of NASA-sponsored activities.

Source record↗

NASA NTRS1971

Pilot reports of disorientation across 14 years of flight.

The purpose of this study was to compare recent incidents involving disorientation in flight reported by 336 Air Force, Army, and Navy pilots with incidents reported by 137 pilots in 1956. The pilots reported their experiences using a check list and a written description of an experience with disorientation in the aircraft they were flying at the time. The latter included 40 incidents which occurred in support of operations in Vietnam. The reports of disorientation showed a striking similarity across types of aircraft flown over 14 years of flying, as well as with the incidents occurring in Vietnam. However, some variation in reports between aircraft types was noted. These reports of disorientation suggest that disorientation is currently experienced in a wide variety of flight operations and that it will continue to be experienced by aircraft pilots.

Clark, B.↗