Search NASA⌕ Search

SEARCH · Search NASA

Results for “Software Quality Models”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 271 records · Page 15
All sourcesNASA NTRSDOE OSTI
NASA NTRS1986

Development of algorithms for understanding the temporal and spatial variability of the earth's radiation balance

A brief description is given of how temporal and spatial variability in the earth's radiative behavior influences the goals of satellite radiation monitoring systems and how some previous systems have addressed the existing problems. Then, results of some simulations of radiation budget monitoring missions are presented. These studies led to the design of the Earth Radiation Budget Experiment (ERBE). A description is given of the temporal and spatial averaging algorithms developed for the ERBE data analysis. These algorithms are intended primarily to produce monthly averages of the net radiant exitance on regional, zonal, and global scales and to provide insight into the regional diurnal variability of radiative parameters such as albedo and long-wave radiant exitance. The algorithms are applied to scanner and nonscanner data for up to three satellites. Modeling of dialy shortwave albedo and radiant exitance with satellite samling that is insufficient to fully account for changing meteorology is discussed in detail. Studies performed during the ERBE mission and software design are reviewed. These studies provide quantitative estimates of the effects of temporally sparse and biased sampling on inferred diurnal and regional radiative parameters. Other topics covered include long-wave diurnal modeling, extraction of a regional monthly net clear-sky radiation budget, the statistical significance of observed diurnal variability, quality control of the analysis, and proposals for validating the results of ERBE time and space averaging.

Brooks, D. R.↗
NASA NTRS1990

IDEF4 technical report, version 1.0

A language for the representation of object oriented software designs is described. IDEF4, a methodology for object-oriented design, is being developed as a design tool for software designers who use such object-oriented languages. Such languages include the Common LISP Object System, Flavors, C++, Smalltalk, Objective C, and others. Since effective usage of the object-oriented paradigm requires a different thought process than that used with conventional procedural or database languages, standard methodologies such as structure charts, data flow diagrams, and traditional data design models (hierarchical, relational, and network) are not sufficient. IDEF4 seeks to provide the necessary facilities to support the object-oriented design decision making process. Specifically, the two primary design goals of IDEF4 are: (1) to provide support for creating object oriented designs whose implementations will exhibit desirable life cycle qualities and reduce total implementation development time; and (2) to make it easy to evaluate object oriented code to determine whether or not the delivered product both conforms to the design and exhibits the desired life cycle qualities. The application of IDEF4 in the life cycle of a software development project is intended to be focused on those activities after a decision has been made to employ object oriented programming technology, but prior to detailed code specification.

Mayer, Richard J.↗
DOE OSTI2024

Automating the Analysis of Large Language Models Responses through Zero-Shot Question Answering

Recent advancements in Large Language Models (LLMs) have shown significant potential in various applications, yet their evaluation, particularly in zero-shot question answering scenarios, remains a challenging task. In this study, our objective was to explore precision metrics for Large Language Models (LLM) and design and implement a software pipeline to automatically evaluate LLMs' outputs under zero-shot question answering. Zero-shot question answering involves a model providing answers to questions about topics it hasn't seen during training. It leverages the principles of zero-shot learning by relying on semantic understanding and generalization from related knowledge. The data used was metadata from medical databases on congenital heart disease. We explored eleven LLM metrics and selected three for our evaluation: BLEU, BERTScore, and MoverScore. BLEU calculates a score based on the overlap of n-grams (contiguous sequences of n items, typically words) between the machine-generated translation and the reference translations. Higher BLEU scores indicate better correspondence between the machine-generated and human-generated translations. BERTScore is a metric used to evaluate the quality of machine-generated text by measuring the similarity of token embeddings produced by BERT (Bidirectional Encoder Representations from Transformers) between the generated text and reference text. MoverScore is a metric that quantifies the dissimilarity between the distributions of word embeddings from machine-generated text and reference text, emphasizing semantic similarity over exact token overlap. We also introduced HBKI, a composite metric summarizing these approaches. We tested five models —GPT-3, Llama-2, Gemini 1.5 Pro, Solar 10.7B, and Mixtral-8x7b. Our software pipeline, designed and implemented using Object-Oriented Programming principles, allows users to customize the selection and extraction of features for topics of interest in their own research. Our results show that MoverScore delivered the most precise evaluation of the LLM's outputs, while Mixtral-8x7b achieved the best overall performance in extracting metadata from the databases.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS2001

Cost-benefit based assurance planning

We have extended an existing risk management framework with a refined cost-benefit model. Benefits are measured in terms of reduction of risk.

risk requirements tradeoffs design quality assuran↗
DOE OSTI2022

Strym: A Python Package for Real-time CAN Data Logging, Analysis and Visualization to Work with USB-CAN Interface

In this report, we describe a data analysis tool developed for decoding and analyzing vehicle data obtained from a passenger vehicle’s onboard controller area network (CAN) bus. The tool developed in this paper provides a timeseries framework to perform domain-specific analysis at scale when interpreting data from a vehicle or a collection of vehicles in light of how to design intelligent vehicle applications. The tool, called Strym, exploits the CAN bus mechanism of modern vehicles to capture data using commercially available CAN-to-USB hardware Comma.ai Panda devices, managed through open-source software Libpanda. Strym permits the decoding of vendor-specific CAN messages in a vehicle-agnostic manner. Through this, a researcher can characterize data throughput, assess data quality, and perform analyses. Such analyses are useful in a number of research such as studying human driving behavior in mixed-autonomy, new driver models, rare-event detection, traffic flow estimation, and custom control of vehicles.

Performance evaluation, Smart cities, Intelligent ↗
DOE OSTI2021

ARC Software Validation Work for the FFTF Reactor

Extensive efforts have been carried out at ANL for the verification and validation of the Argonne Reactor Codes (ARC) software package currently used for the design of Versatile Test Reactor (VTR). The ARC software package consists of steady state neutronics and thermal hydraulics modeling capabilities which are being used by the VTR program to develop most of the VTR reactor design details which will be part of the licensing application. It is anticipated that this software will continue to be used for the design work and for initial operations although additional software may be introduced at a later time. The validation work was focused primarily on obtaining validation data consistent with VTR and usable for the ARC software. Because no critical facilities or operating fast spectrum reactors are available to do experiments for the VTR, the next best option is to identify historical experimental data that can be used as validation data. Early on in VTR, the ZPPR-15 set of experiments was identified as good validation data because of 1) the availability and quality of the data, 2) existing staff that are already familiar with the experimental machine and measurements, 3) most of the ZPPR-15 loadings of interest have already been processed into ARC models, and 4) a full uncertainty quantification has already been done for several loadings of ZPPR-15. The FFTF startup and operations data was identified as the most consistent reactor type that has validation data usable for VTR. Finally, the EBR-II fuel depletion measurements were identified as the best available validation data for VTR. It is important to note that both the FFTF and EBR-II reactors typically come with higher uncertainties than the ZPPR. In the frame of the discussed verification and validation efforts, the present document discusses the analysis of selected FFTF measurements included in the benchmark specifications of the International Reactor Physics Experiment (IRPhE) handbook. The FFTF reactor core configurations from the benchmark specification are presented in Section 2. The analysis is performed with the use of the ARC code suite available at ANL for fast reactor studies and is discussed in Section 3. The reactor parameters from the benchmark include criticality, neutron spectra, effective delayed neutron spectra, control rod worth, isothermal temperature coefficient and low energy gamma-ray spectra. The calculated values and the comparison with the experimental data is discussed in Sections 4 to 9 for each considered reactor parameter. Finally, conclusions are presented in Section 10.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
NASA NTRSDate not supplied

FUN3D Analyses in Support of the 1st AIAA Stability and Control Prediction Workshop

The 1st AIAA Stability and Control Prediction Workshop was created to establish best practices for the prediction of stability & control derivatives using computational fluid dynamics and assess the limitations of these computational methods when those best practices are applied. The inaugural workshop considers the ONERA version of the NASA/Boeing Common Research Model(CRM), which includes the wing, body, horizontal tail, and a vertical tail designed by ONERA. Wind tunnel tests have been conducted for this configuration with longitudinal tests having been previously published, in addition to unpublished data at small sideslip angles that will serve as ‘blind’ data for workshop data comparisons. Participants were provided a ‘family’ of unstructured grids for the full-span ONERA CRM model with the wind-tunnel sting included. This family of mixed-element grids consists of 5 levels of refinement (tiny, coarse, medium, fine, and extra fine) with surface and volume mesh scaling, resulting in a size range of 14.6 to 53.4 million nodes. In addition, a medium refinement mesh has been provided for the ONERA CRM configuration without a sting to evaluate the sting’s impact on static longitudinal stability characteristics. In addition to these workshop-provided grids, the present work also considers an equivalent ‘family’ of computational grids generated using Heldenmesh™, a rapid grid generation software by Helden Aerospace Corporation for creating high-quality, three-dimensional, mixed-element unstructured meshes. Because of the authors’ familiarity with this software, these additional grids were generated as a comparison to the workshop-provided grids and to better understand the implications of using volume-mirrored grids for stability and control predictions. The present work will contribute to the workshop with test case data generated using the NASA FUN3D code, a parallelized, unstructured, node-based, finite-volume discretization, Reynolds-averaged Navier-Stokes flow solver. Numerical simulations will be conducted using the Quadratic Constitutive Relationship (QCR) version of the Spalart-Allmaras (SA) turbulence model with negative turbulence variable provisions. Both steady and2nd-order, time-accurate simulation results are to be generated and compared for select test cases, as time permits, to investigate their impact on FUN3D predictions. The present work will consider the three primary workshop test cases: (1) grid convergence study, (2) Mach number effect on static stability, and (3) wind tunnel sting increments. Additionally, data will be provided for the two optional test cases, which include:(1) static stability derivative calculations and (2)sideslip angle sweeps. In each of the test cases, the vehicle is stationary, and the body is assumed to be rigid, where vehicle deformation has been accounted for in the model configuration geometry. For all test cases, longitudinal and lateral force and moment aerodynamic coefficients will be provided for the total configuration, in addition to a component-level breakdown that includes the port wing, starboard wing, fuselage, and tail.

CFD↗
DOE OSTI2020

Image quality indicator and dechromator setup for static LANSCE proton radiography experiment

This document describes a static geometry consisting of a sphere of gold and Ti-6Al-4V alloy, along with an energy-loss matching dechromator composed of polycarbonate, to field with the X3 electromagnetic magnifying lens at LANSCE during the next beam cycle. This static configuration will allow us to: (1) characterize the system’s resolution at the material interface between the gold and titanium alloy, and test the image quality for fine features associated with the screws used to fasten the titanium alloy hemispheres; (2) tune our models of proton scattering and imaging for these materials; and (3) test a procedure to align the polycarbonate dechromator and the sphere within the proton beamline. The data will also be suitable for validation of the software options available for proton radiography simulations.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2026

FAIR Data Meets FAIR Software

Modern scientific research is increasingly defined by the interplay between data, software, and the workflows that connect them. Yet while the FAIR (Findable, Accessible, Interoperable, Reusable) principles have become foundational for scientific data stewardship, the same level of structure and expectation has only recently begun to extend to research software. This talk covers why and how FAIR principles are being applied to data and software to support data reuse. It outlines the gaps in current sharing norms, the growing federal emphasis on persistent identifiers and public access, and the opportunities created when datasets, computational workflows, code, and models are linked through rich, standardized metadata. Practical implementation pathways for the EIC and JLab communities are described, including datacards for structured dataset documentation and provenance-aware workflows. By aligning data lifecycle management with FAIR-aligned software practices, the scientific community can advance toward autonomous knowledge graphs, generative workflows, and high-quality, AI-ready scientific datasets.

McSpadden, Diana [Thomas Jefferson National Accele↗
NASA NTRS1982

Conceptual design and analysis of a large antenna utilizing electrostatic membrane management

Conceptual designs and associated technologies for deployment 100 m class radiometer antennas were developed. An electrostatically suspended and controlled membrane mirror and the supporting structure are discussed. The integrated spacecraft including STS cargo bay stowage and development were analyzed. An antenna performance evaluation was performed as a measure of the quality of the membrane/spacecraft when used as a radiometer in the 1 GHz to 5 GHz region. Several related LSS structural dynamic models differing by their stiffness property (and therefore, lowest modal frequencies) are reported. Control system whose complexity varies inversely with increasing modal frequency regimes are also reported. Interactive computer-aided-design software is discussed.

Brooks, A. L.↗
NASA NTRS2019

An Automated Detection Methodology for Dry Well-Mixed Layers

The intense surface heating over arid land surfaces produces dry well-mixed layers (WML) via dry convection. These layers are characterized by nearly constant potential temperature and low, nearly constant water vapor mixing ratio. To further the study of dry WMLs, we created a detection methodology and supporting software to automate the identification and characterization of dry WMLs from multiple data sources including rawinsondes, remote sensing platforms, and model products. The software is a modular code written in Python, an open source language. Radiosondes from a network of synoptic stations in North Africa were used to develop and test the WML detection process. The detection involves an iterative decision tree that ingests a vertical profile from an input data file, performs a quality check for sufficient data density, and then searches upward through the column for successive points where the simultaneous changes in water vapor mixing ratio and potential temperature are less than the specified maxima. If points in the vertical profile meet the dry WML identification criteria, statistics are generated detailing the characteristics of each layer in the profile. At the end of the vertical profile analysis, there is an option to plot analyzed profiles in a variety of file formats. Initial results show that the detection methodology can be successfully applied across a wide variety of input data and North African environments and for all seasons. It is sensitive enough to identify dry WMLs from other types of isentropic phenomena such as subsidence layers and distinguish the current day’s dry WML from previous days.

Stephen D. Nicholls↗
NASA NTRS2020

Characterizing 15 Years of Saharan-like, Dry, Well-Mixed Air Layers in North Africa

The Saharan Air Layer (SAL) is a dry, well-mixed layer (WML) of warm and sometimes dusty air of nearly constant water vapor mixing ratio generated by the intense surface heating and strong, dry convection in the Sahara Desert, which has notable downstream impacts on the surface energy balance, organized convective system development, seasonal precipitation, and air quality. Characterizing both WMLs and SALs from the existing rawinsonde network has proven challenging because of its sparseness and inconsistent data reporting. Spurred on by this challenge, we previously created a detection methodology and supporting software to automate the identification and characterization of WMLs from multiple data sources including rawinsondes, remote sensing platforms, and model products. We applied our algorithm to each dataset at both its native and at a common (most coarse data product) vertical resolution to detect WMLs and their characteristics (temperature, mixing ratio, AOD, etc.) at each of the 53 rawinsonde launch sites in north Africa.

Nicholls, Stephen D.↗
DOE OSTI2021

A Comprehensive Open-Source R Software For Statistical Metrology Calculations: From Uncertainty Evaluation To Risk Analysis

Whether calibrating equipment or inspecting products on the factory floor, metrology requires many complicated statistical calculations to achieve a full understanding and evaluation of measurement uncertainty and quality. In order to assist its workforce in performing these calculations in a consistent and rigorous way, the Primary Standards Lab at Sandia National Laboratories (SNL) has developed a free and open-source software package for computing various metrology calculations from uncertainty propagation to risk analysis. In addition to propagating uncertainty through a measurement model using the well-known Guide to Expression of Uncertainty in Measurement or Monte Carlo approaches, evaluating the individual Type A and Type B uncertainty components that go into the measurement model often requires other statistical methods such as analysis of variance or determining uncertainty in a fitted curve. Once the uncertainty in a measurement has been calculated, it is usually evaluated from a risk perspective to ensure the measurement is suitable for making a particular conformance decision. Finally, SNL’s software can perform all these calculations in a single application via an easy-to-use graphical interface, where the different functions are integrated so the results of one calculation can be used as inputs to another calculation.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Evaluation of New Additions to OLI Software in Predicting Mercuric and Mercurous Species in Liquid Waste Operations

Speciation of mercury during the pretreatment steps of tank waste processing is critical to successful mercury removal prior to vitrification during Liquid Waste Operations (LWO) at SRS. OLI software has been used to predict mercury speciation and activity throughout LWO. The OLI software operates based on a thermodynamic framework called the Mixed Solvent Electrolyte (MSE) framework. The MSE framework allows prediction in theoretically infinitely dilute to concentrated mixtures (e.g., purely solute solutions). Before modification to the MSE framework databanks, certain critical mercury species were missing in the MSE databank, and some thermodynamic data needed to be updated for the OLI software to accurately predict mercury chemical species in SRS waste tanks. To better reflect streams across LWO, new mercury species were integrated into the MSE database. To evaluate the changes to the OLI MSE framework per the Technical Task Request (TTR) and the Task Technical and Quality Assurance Plan (TTQAP), waste stream compositions from Tanks 38, 43, and Tank 50 decontaminated salt solution (DSS) were used as model inputs. Models were developed and executed using both the old and new databases. Compositional analyses from caustic Tank 50 DSS and caustic Tanks 38 and 43 were used as the input streams. These streams represent the most comprehensive chemical data sets where both mercury and tank constituents were measured together. Results for Tank 50 DSS predict HgO as the predominant species in both databases. Both methyl and dimethyl Hg species are present when the new database is ‘on’ and are not predicted with the new database turned ‘off’. The new database predicts a greater amount of HgO and a greater fraction of it in the solid phase. Pourbaix diagrams (potential vs. pH) generated for each Tank 50 DSS were identical regardless of which database was used. Elemental Hg and HgO were predicted in the water stable region under basic conditions. Tanks 38 and 43 follow similar trends as the Tank 50 DSS models. Unlike Tanks 38 and 50 DSS, the Tank 43 Pourbaix plot shows a region of stability for an aqueous HgOHCO3 - species between approximately pH 7-11. In all streams, when MeHg+ is included in the inputs, the new database predicts aqueous MeHgOH as the dominant species. If elemental or dimethyl mercury is in the waste stream, the new database model predicts they are unchanged and remain in those states and quantities. Additionally, the total mercury values are reported for both the measured input data and the OLI output data for all considered tanks. The summary indicates that the percentage error between the measured and calculated values is less than 1% in all cases The reconciliations and generation of the Pourbaix diagrams for Tank 50 DSS took approximately ten times longer with the new database ‘on’. In addition, over the course of that time, models with the new database ‘on’ were more likely to crash or display an error. Some modest performance improvements were noted when modeling with an i7 processor versus an i5. An example error is found in Appendix A. Furthermore, Appendix B provides V&V for two chemical systems analyzed with the OLI software, results were satisfactory. It is recommended to utilize the new databases (i.e., HCO.ddb and SR-Hg.ddb) in future Savannah River Mission Completion applications of OLI to represent pseudo steady-state. Furthermore, the integration and utilization of the new databases (i.e., HCO.ddb and SR-Hg.ddb) in modeling applications (e.g., Aspen) is also recommended.

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗
DOE OSTI2025

LLM Benchmarking with LLaMA2: Evaluating Code Development Performance Across Multiple Programming Languages

The rapid evolution of large language models (LLMs) has opened new possibilities for automating various tasks in software development. This paper evaluates the capabilities of the LLaMA 2-70B model in automating these tasks for scientific applications written in commonly used programming languages. Using representative test problems, we assess the model's capacity to generate code, documentation, and unit tests, as well as its ability to translate existing code between commonly used programming languages. Our comprehensive analysis evaluates the compilation, runtime behavior, and correctness of the generated and translated code. Additionally, we assess the quality of automatically generated code, documentation, and unit tests. Here, our results indicate that while LLaMA 2-70B frequently generates syntactically correct and functional code for simpler numerical tasks, it encounters substantial difficulties with more complex, parallelized, or distributed computations, requiring considerable manual corrections. We identify key limitations and suggest areas for future improvements to better leverage AI-driven automation in scientific computing workflows.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS2019

Assessing the Impact of Observations in a Multi-Year Reanalysis (MERRA-2)

Operational and quasi-operational weather prediction centers have been routinely assessing the contribution from various observing systems to reducing errors in short-range forecasts for a number of years now. The original technique, Forecast Sensitivity-based Observation Impact (FSOI), involves definition of a forecast error measure and evaluation of sensitivities with respect to changes in the observations that require adjoint operators of both the underlying tangent linear model and corresponding analysis technique. The present work applies FSOI to reanalysis and aims at providing an expanded view of the contribution of various observing systems over nearly 40 years of assimilation. Specifically, this study uses MERRA-2 given that its supporting software includes all ingredients necessary to calculate FSOI. Part of this work shows how the quality of forecasts improves over the course of the reanalysis, and examines forecast sensitivities relevant to FSOI. The assessment here finds, for example, that: conventional observations are a major player in reducing forecast error throughout the 40 years of reanalysis, even when their volume reduces from 45\% in the earlier periods to about 5% in the modern era; satellite radiances, especially microwave instruments are major contributors to error reduction from the early single platform TIROS-N days to the current multi-platform scenario, though their fractional contribution reduces slightly from the early 2000's onward after the increased availability of wind observation from aircraft and atmospheric motion vectors, and the introduction of GPSRO; infrared instruments play a secondary role to microwave but are significant still, with the peculiar result of fractional impacts contribution from modern hyperspectral instruments being roughly similar to those from early infrared instruments. The dependence of results on the chosen error measure is emphasized throughout.

Todling, Ricardo↗
NASA NTRS2019

Assessing the Impact of Observations in a Multi-Year Reanalysis

Operational and quasi-operational weather prediction centers have been routinely assessing the contribution from various observing systems to reducing errors in short-range forecasts for a number of years now. The original technique, Forecast Sensitivity Observation Impact (FSOI), involves definition of a forecast error measure and evaluation of sensitivities with respect to changes in the observing system that require adjoint operators of both the underlying tangent linear model and corresponding analysis technique. The present work applies FSOI to Reanalysis and aims at providing an expanded view of the contribution of various observing systems over nearly 40 years of assimilation. Specifically, this study uses MERRA-2 given that its supporting software includes all ingredients necessary to calculate FSOI. Part of this work shows how the quality of forecasts improves over the course of the reanalysis, and examines forecast sensitivities relevant to FSOI. The assessment here finds, for example, that: conventional observations are a major player in reducing forecast error throughout the 40 years of reanalysis, even when their volume reduces from 45% in the earlier periods to about 5% in the modern era; satellite radiances, especially microwave instruments are major contributors to error reduction from the early single platform TIROS-N days to the current multi-platform scenario; infrared instruments play a secondary role to microwave but are significant still, with the peculiar result of fractional impacts contribution from modern hyperspectral instruments being roughly similar to those from early infrared instruments. The dependence of results on the chosen error measure is emphasized throughout.

Fabio L R Diniz↗
DOE OSTI2022

CTF: A modernized, production-level, thermal hydraulic solver for the solution of industry-relevant challenge problems in pressurized water reactors

CTF is a thermal hydraulic (T/H) subchannel tool that has been extensively developed over the past ten years as part of the Consortium for Advanced Simulation of Light Water Reactors (CASL) program. The code was selected early in the CASL program for support of high-impact challenge problems that were found to be relevant to the nuclear industry and its currently operating fleet of pressurized water reactors (PWRs), including issues such as departure from nucleate boiling (DNB), crud-induced power shifts (CIPSs), and reactivity-insertion accidents (RIAs). By incorporating CTF into the multiphysics Virtual Environment for Reactor Application (VERA) core simulator software developed by CASL, CTF has become the primary means of providing fluid and fuel thermal feedback, as well as T/H figure-of-merits (FOMs) in large-scale reactor simulations. With the goal of solving industry challenge problems, CASL placed great emphasis on developing high-quality, high-performance, validated software tools that offer higher fidelity than what is currently possible with current industry methods. In support of this effort, CTF was developed from a research tool into an nuclear quality assurance (NQA-1)–compliant, production-level software tool that is capable of addressing the stated challenge problems and goals of CASL. This work presents a review of the major technological achievements that were realized in developing CTF over the past decade of the CASL program and presents an overview of the code solution approach and closure models.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗