SEARCH · Search NASA

Results for “model skill”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 271 records · Page 15

All sources NASA NTRS DOE OSTI

NASA NTRSDate not supplied

Predictability of the Minimum Sea Ice Extent from Late Winter Fram Strait Ice Export: Model vs Observations

Late winter coastal divergence along the Eurasian coastline (referred to as the ice factory) – or the Fram Strait ice export, a proxy for coastal divergence in the ice factory – is a skillful predictor of the minimum sea ice extent (Williams et al., 2016; Brunette et al., 2018; Sesternikov and, 1979). Coastal divergence leads to the formation of coastal polynya where new ice grows but to a thickness that is not large enough to survive the following summer melt. This signal is then amplified by the ice albedo feedback and leads to more open water at the end of the summer melt season. In this thesis project, we will identify if this source of predictive skill in the seasonal forecast of the minimum sea ice extent is present in General Circulation Models, more precisely in the CESM2-LE, GISS-E2.1-G, GFDL FLOR-LE, CNRM-CM6-1 and the CanESM5. Since even small biases in the large-scale atmospheric circulation simulated by a model can short circuit this coupling between dominant modes of atmospheric variability (NAO and AO), coastal divergence along the Eurasian coastline, Fram Strait ice export and therefore seasonal forecasting skill of the model, failure to reproduce this coupling observed in the real Arctic will be used to identify biases in GCM. Preliminary results show that subtle changes in the large-scale atmospheric circulation leads to opposite statistical relationship between Fram Strait ice area export, coastal divergence along the Eurasian coastline, and seasonal predictability of the minimum sea ice extent both in the CESM2-LE and the GISS-E2.1-G models. Differences are linked with the partitioning between recirculation within the Beaufort Gyre, ice exported through Fram Strait and ridging north of the Canadian Arctic Archipelago.

Sea ice↗

NASA NTRS1991

Development and application of virtual reality for man/systems integration

While the graphical presentation of computer models signified a quantum leap over presentations limited to text and numbers, it still has the problem of presenting an interface barrier between the human user and the computer model. The user must learn a command language in order to orient themselves in the model. For example, to move left from the current viewpoint of the model, they might be required to type 'LEFT' at a keyboard. This command is fairly intuitive, but if the viewpoint moves far enough that there are no visual cues overlapping with the first view, the user does not know if the viewpoint has moved inches, feet, or miles to the left, or perhaps remained in the same position, but rotated to the left. Until the user becomes quite familiar with the interface language of the computer model presentation, they will be proned to lossing their bearings frequently. Even a highly skilled user will occasionally get lost in the model. A new approach to presenting type type of information is to directly interpret the user's body motions as the input language for determining what view to present. When the user's head turns 45 degrees to the left, the viewpoint should be rotated 45 degrees to the left. Since the head moves through several intermediate angles between the original view and the final one, several intermediate views should be presented, providing the user with a sense of continuity between the original view and the final one. Since the primary way a human physically interacts with their environment should monitor the movements of the user's hands and alter objects in the virtual model in a way consistent with the way an actual object would move when manipulated using the same hand movements. Since this approach to the man-computer interface closely models the same type of interface that humans have with the physical world, this type of interface is often called virtual reality, and the model is referred to as a virtual world. The task of this summer fellowship was to set up a virtual reality system at MSFC and begin applying it to some of the questions which concern scientists and engineers involved in space flight. A brief discussion of this work is presented.

Brown, Marcus↗

NASA NTRS2013

The Influence of Observation Errors on Analysis Error and Forecast Skill Investigated with an Observing System Simulation Experiment

The Global Modeling and Assimilation Office (GMAO) observing system simulation experiment (OSSE) framework is used to explore the response of analysis error and forecast skill to observation quality. In an OSSE, synthetic observations may be created that have much smaller error than real observations, and precisely quantified error may be applied to these synthetic observations. Three experiments are performed in which synthetic observations with magnitudes of applied observation error that vary from zero to twice the estimated realistic error are ingested into the Goddard Earth Observing System Model (GEOS-5) with Gridpoint Statistical Interpolation (GSI) data assimilation for a one-month period representing July. The analysis increment and observation innovation are strongly impacted by observation error, with much larger variances for increased observation error. The analysis quality is degraded by increased observation error, but the change in root-mean-square error of the analysis state is small relative to the total analysis error. Surprisingly, in the 120 hour forecast increased observation error only yields a slight decline in forecast skill in the extratropics, and no discernable degradation of forecast skill in the tropics.

Observation Error↗

NASA NTRS1982

Integrated and spectral energetics studies of the GLAS general circulation model

Integrated and spectral energetics of the Goddard Laboratory for Atmospheric Sciences (GLAS) general circulation model are compared with observations and examined when subdivided beyond hemispheric integrals. In the monthly mean zonal averages, qualitative improvements are found over previous versions of the model in eddy kinetic energy and barotropic conversions although vertical shear above the subtropical jet remains weak. This paper traces this problem to the erroneous growth of kinetic energy above strong jets, shows that this difficulty is common to all orographic general circulation models, and suggests that this problem has significant effects on upper-level long-wave predictability. In particular, it is considered whether hemispherically integrated kinetic energy or low-wavenumber spectral coefficients retain any skill as the predictability limit. In the time-dependent comparisons, the model shows significant skill in predicting the hemispherically integrated eddy kinetic energy out to two weeks for one of two cases.

Tenenbaum, J.↗

NASA NTRS2017

Skillful Spring Forecasts of September Arctic Sea Ice Extent Using Passive Microwave Data

In this study, we demonstrate skillful spring forecasts of detrended September Arctic sea ice extent using passive microwave observations of sea ice concentration (SIC) and melt onset (MO). We compare these to forecasts produced using data from a sophisticated melt pond model, and find similar to higher skill values, where the forecast skill is calculated relative to linear trend persistence. The MO forecasts shows the highest skill in March-May, while the SIC forecasts produce the highest skill in June-August, especially when the forecasts are evaluated over recent years (since 2008). The high MO forecast skill in early spring appears to be driven primarily by the presence and timing of open water anomalies, while the high SIC forecast skill appears to be driven by both open water and surface melt processes. Spatial maps of detrended anomalies highlight the drivers of the different forecasts, and enable us to understand regions of predictive importance. Correctly capturing sea ice state anomalies, along with changes in open water coverage appear to be key processes in skillfully forecasting summer Arctic sea ice.

melt onset↗

NASA NTRS2021

The Contributions of Gauge-Based Precipitation and SMAP Brightness Temperature Observations to the Skill of the SMAP Level-4 Soil Moisture Product

Soil Moisture Active Passive (SMAP) mission L-band brightness temperature (Tb) observations are routinely assimilated into the Catchment land surface model to generate Level-4 Soil Moisture (L4_SM) estimates of global surface and root-zone soil moisture at 9-km, 3-hourly resolution with ~2.5-day latency. The Catchment model in the L4_SM algorithm is driven with ¼-degree, hourly surface meteorological forcing data from the Goddard Earth Observing System (GEOS). Outside of Africa and the high latitudes, GEOS precipitation is corrected using Climate Prediction Center Unified (CPCU) gauge-based, ½-degree, daily precipitation. L4_SM soil moisture was previously shown to improve over land model-only estimates that use CPCU precipitation but no Tb assimilation (CPCU_SIM). Here, we additionally examine the skill of model-only (CTRL) and Tb assimilation-only (SMAP_DA) estimates derived without CPCU precipitation. Soil moisture is assessed versus in situ measurements in well-instrumented regions and globally through the Instrumental Variable (IV) method using independent soil moisture retrievals from the Advanced Scatterometer. At the in situ locations, SMAP_DA and CPCU_SIM have comparable soil moisture skill improvements relative to CTRL for the unbiased root-mean-square error (surface and root-zone) and correlation metrics (root-zone only). In the global average, SMAP Tb assimilation increases the surface soil moisture anomaly correlation by 0.10-0.11 compared to an increase of 0.02-0.03 from the CPCU-based precipitation corrections. The contrast is particularly strong in central Australia, where CPCU is known to have errors and observation-minus-forecast Tb residuals are larger when CPCU precipitation is used. Validation versus streamflow measurements in the contiguous U.S. reveals that CPCU precipitation provides most of the skill gained in L4_SM runoff estimates over CTRL.

SMAP↗

NASA NTRS2022

Multi-Variate LSTM Prediction of Alaska Magnetometer Chain Utilizing a Coupled Model Approach

During periods of rapidly changing geomagnetic conditions electric fields form within the Earth’s surface and induce currents known as geomagnetically induced currents(GICs), which interact with unprotected electrical systems our society relies on. In this study, we train multi-variate Long-Short Term Memory neural networks to predict magnitude of north-south component of the geomagnetic field (|BN|) at multiple ground magnetometer stations across Alaska provided by the SuperMAG database with a future goal of predicting geomagnetic field disturbances. Each neural network is driven by solar wind and interplanetary magnetic field inputs from the NASA OMNI database spanning from 2000–2015 and is fine tuned for each station to maximize the effectiveness in predicting |BN|. The neural networks are then compared against multivariate linear regression models driven with the same inputs at each station using Heidke skill scores with thresholds at the 50, 75, 85, and 99 percentiles for |BN|. The neural network models show significant increases over the linear regression models for |BN| thresholds. We also calculate the Heidke skill scores for d|BN|/dt by deriving d|BN|/dt from |BN| predictions. However, neural network models do not show clear outperformance compared to the linear regression models. To retain the sign information and thus predict BN instead of |BN|, a secondary so-called polarity model is utilized. The polarity model is run in tandem with the neural networks predicting geomagnetic field in a coupled model approach and results in a high correlation between predicted and observed values for all stations. We find this model a promising starting point for a machine learned geomagnetic field model to be expanded upon through increased output time history and fast turnaround times.

Matthew Blandin↗

NASA NTRS2013

Predicting Operator Execution Times Using CogTool

Researchers and developers of NextGen systems can use predictive human performance modeling tools as an initial approach to obtain skilled user performance times analytically, before system testing with users. This paper describes the CogTool models for a two pilot crew executing two different types of a datalink clearance acceptance tasks, and on two different simulation platforms. The CogTool time estimates for accepting and executing Required Time of Arrival and Interval Management clearances were compared to empirical data observed in video tapes and registered in simulation files. Results indicate no statistically significant difference between empirical data and the CogTool predictions. A population comparison test found no significant differences between the CogTool estimates and the empirical execution times for any of the four test conditions. We discuss modeling caveats and considerations for applying CogTool to crew performance modeling in advanced cockpit environments.

Santiago-Espada, Yamira↗

NASA NTRS2018

Overview of the Digitization Workflow Post Image Acquisition of Apollo Lunar and Antarctic Meteorite Samples Using Agisoft Photoscan for the NASA 3D Astromaterials Virtual Samples Collection

The 3D Virtual Astromaterials Samples (3DVAS) collection is a multi-year funded project to create a digital database of sixty Apollo Lunar and Antarctic Meteorite samples following non-destructive documentation conservation protocols. After initial image processing, the photos are evaluated and processed using unique structure-from-motion photogrammetric techniques in a high performance modelling software designed to create a 3D model from 2D images: Agisoft Photoscan Pro. Agisoft Photoscan Pro uses image processing algorithms and techniques originating in computer vision to resolve 3D models for accurate and detailed visualization of a subject. The software provides a stepwise process that is tailored per model based on spatial and specular reflectance properties, for example. The process includes: photo alignment, creation of a dense point cloud, mesh, and finally texture. Photo alignment is dependent on model properties. The 3DVAS process requires a special rotation platform with calibrated photogrammetric targets, specific distance rotation protocols, and a contrasting background for alignment and scale accuracy. As a result of the photographic process, alignment will complete with two mirrored hemispheres that, in a sense, represent the 2D images overlapping to create a 3D model. Each dense point cloud is analyzed with provided statistical measures in a gradual selection process to eliminate outliers. The point cloud is reduced to include only data valuable to the final model. When a precise dense point cloud is achieved, a mesh and texture are applied. Each model is scaled with scale bar accuracies within 100 microns. Each sample has its own intimate process for modelling; there is no standard for the parameters required in the final creation of a high resolution model. By processing multiple samples, a skill is gained in practice to allow a close definition of the original sample and will result in the most detailed version of the sample shell. This process completes one-fifth of the 3DVAS protocol for providing accurate digital documentation. Each model shell is merged with X-ray Computed Tomography data to create a full volumetric sample. All 3DVAS data will be served on NASA's Astromaterials Acquisition and Curation website with an early subset of data available in 2019 and the 3D Virtual Astromaterials Samples Collection launch in 2020.

Thomas, Andi B.↗

NASA NTRS2021

Evaluating the Impact of Model Resolutions and Cumulus Parameterization on Precipitation in NU-WRF: A Case Study in the Central Great Plains

Regional climate models are expected to exhibit improved skill at finer spatial resolutions due to improved representation of land surface heterogeneity. However, at spatial scales between 1 to 10 km (grey scales), these improvements are often illusive due to the competing benefits from spatial resolution and cumulus parameterization. This study provides insights into the impact of model resolution and cumulus parameterization on precipitation prediction in the Central Great Plains by using an object-based evaluation method. Our results show limited improvement solely from finer resolution but larger improvement without using the cumulus scheme at a 4-kmresolution. Compared to traditional evaluation methods, the object-based analysis shows that without the cumulus scheme the spatial properties of precipitation are better represented. In contrast, all model configurations show a dry bias in precipitation days and a tendency to produce widespread precipitation but with fewer hours with precipitationwhich indicates other shortcomings in the model.

Yuqi Zhang↗

NASA NTRS1984

Effect of high latitude filtering on NWP skill

The high latitude filtering techniques commonly employed in global grid point models to eliminate the high frequency waves associated with the convergence of meridians, can introduce serious distortions which ultimately affect the solution at all latitudes. Experiments completed so far with the 4 deg x 5 deg, 9-level GLAS Fourth Order Model indicate that the high latitude filter currently in operation affects only minimally its forecasting skill. In one case, however, the use of pressure gradient filter significantly improved the forecast. Three day forecasts with the pressure gradient and operational filters are compared as are 5-day forecasts with no filter.

Kalnay, E.↗

DOE OSTI2026

OpenStudio®-MCP [SWR-26-035]

OpenStudio®-MCP is a Model Context Protocol (MCP) server that lets AI assistants perform building energy modeling through natural language. Rather than requiring users to learn the OpenStudio® SDK, EnergyPlus® scripting, or Ruby/Python automation, the server translates conversational requests into sequences of tool calls that create models, design HVAC systems, run simulations, and extract results — all within a single chat session. The server's 124 tools are organized into a skills architecture where each skill encapsulates a domain of building energy modeling (envelope, HVAC, loads, weather, simulation, results) behind typed, LLM-friendly interfaces. High-leverage operations like applying ASHRAE 90.1 baseline systems or generating standards-compliant typical buildings are exposed as single tool calls that internally wire dozens of OpenStudio® objects. Bundled measures from ComStock™ and Openstudio® -common-measures-gem are wrapped with dedicated tools and typed arguments rather than exposed through a generic measure interface, so AI models get consistent, error-resistant recipes without needing to discover measure arguments at runtime. A key design decision is structured results extraction: six SQL-based tools return surgical ~300–1,000 token responses (end-use breakdowns, envelope summaries, HVAC sizing, timeseries data) instead of requiring the AI to parse ~100K-token raw HTML reports, making iterative design exploration practical within context window limits. The codebase is designed as a reference implementation — explicit, well-commented, and modular — so that other simulation engines (EnergyPlus® standalone, TRNSYS, DOE-2) can use it as a template for building their own MCP servers.

Ball, Brian [National Laboratory of the Rockies (N↗

NASA NTRS1992

Analysis of CCD images of the coma of comet P/Halley

The modeling analysis objective of this project is to make use of the skill acquired in the development of Monte Carlo particle trajectory models for the distributions of gas species in cometary comae as a basis for a new dust coma model. This model will include a self-consistent picture of the time-dependent dusty-gas dynamics of the inner coma and the three-dimensional time-dependent trajectories of the dust particles under the influence of solar gravity and solar radiation pressure in the outer coma. Our purpose is to use this model as a tool to analyze selected images from two sets of data of the comet P/Halley with the hope that we can help to understand the effects of a number of important processes on the spatial morphology of the observed dust coma. The study will proceed much in the same way as our study of the spatially extended hydrogen coma where we were able to understand the spatial morphology of the Lyman-alpha coma in terms of the partial thermalization of the hot H atoms produced by the photodissociation of cometary H2O and OH. The processes of importance to the observed dust coma include: (1) the dust particle size distribution function; (2) the terminal velocities of various sized dust particles in the inner coma; (3) the radiation scattering properties of dust particles, which are important both in terms of the observed scattered radiation and the radiation pressure acceleration on dust particles; (4) the fragmentation and/or vaporization of dust particles; (5) the relative importance of CHON and silicate dust particles as they contribute both to the dusty-gas dynamics in the inner coma (that produce the dust particle terminal velocities) and to the observed spatial morphology of the outer dust coma; and (6) the time and direction dependence of the source of dust.

Combi, Michael R.↗

NASA NTRS2024

Tropical Cyclones in the GEOS-S2S-2 Subseasonal Forecasts

This paper analyzes the climatology, prediction skill, and predictability of tropical cyclones (TCs) in NASA’s Global Earth Observing System Subseasonal to Seasonal (GEOS-S2S) forecast system version 2. GEOS reasonably simulates the number and spatial distribution of TCs compared to observations except in the Atlantic where the model simulates too few TCs due to low genesis rates in the Caribbean Sea and Gulf of Mexico. The environmental conditions, diagnosed through a genesis potential index, do not clearly explain model biases in the genesis rates, especially in the Atlantic. At the storm-scale, GEOS reforecasts replicate several key aspects of the thermodynamic and dynamic structure of observed TCs, such as a warm core and the secondary circulation. The model, however, fails to simulate an off-center eyewall when evaluating vertical velocity, precipitation and moisture. The analysis of prediction skill of TC genesis and occurrence shows that GEOS has comparable skill to other global models in WMO S2S archive and that its skill could be further improved by increasing the ensemble size. After calibration, GEOS forecasts are skillful in the Western North Pacific and Southern Indian Ocean up to 20 days in advance. A model-based predictability analysis demonstrates the importance of the Madden-Julian Oscillation (MJO) as a source of predictability of TC occurrence beyond the 14 day lead-time. Forecasts initialized under strong MJO conditions show evidence of predictability beyond week 3. However, due to model biases in the forecast distribution there are notable gaps between MJO-related prediction skill and predictability which require further study.

Jorge L. Garcia-Franco↗

DOE OSTI2026

OmicsMLMentor: A Web Application for Guided Machine Learning Analysis of Omics Data

Expression-based omics technologies (e.g. proteomics, metabolomics, transcriptomics, etc.) increasingly rely on supervised and unsupervised machine learning (ML) models to find key biomolecules distinguishing conditions, identify natural groupings in biological data, or generate predictions for outcomes of interest. Fitting ML models to omics data presents several challenges, including handling missing data, selecting a normalization method, choosing a valid model, and optimizing hyperparameters, all requiring statistical programming skills to address these challenges. Thus, the open-source web application SLOPE was designed to lower the barrier to ML modeling for omics data. SLOPE supports the fitting of 15 ML models (10 supervised and 5 unsupervised) tailored to omics datasets, such as proteomics, metabolomics, lipidomics, and transcriptomics. SLOPE offers several omics-specific features, including methods for handling missingness (imputation, conversion, removal), normalization tests, ranking of models based on the structure of a user’s data and user input, and optimal hyperparameter selections using cross-validation splits. By streamlining ML workflows for omics analysis, SLOPE address critical gaps in existing online web tools, facilitating a broader adoption of these models for omics research. Here, SLOPE is applied to data from a lignin exposure study to highlight the workflow for fitting both supervised and unsupervised models to data.

lipidomics↗

DOE OSTI2025

Extraction and Analysis of Time Series Data from Building Automation Systems Using Large Language Models

Semantic schemas like Haystack 4, Brick and ASHRAE standard 223 enable the structured, standardized, and machine-readable representation of building data, facilitating interoperability, data integration, and advanced analytics. However, extracting information from these models requires specialized expertise in SPARQL and other programming languages, skills that are not commonly found among building professionals. Recent advancements in Large Language Models (LLMs), such as ChatGPT, enable the construction of queries using natural language, making it easier for individuals to interact with these systems in a manner that resembles everyday speech. However, these methods have not yet been tested on building semantic ontologies. This paper introduces a novel workflow and tool for enabling users to ask questions about a specific building's data, using natural language and receive answers automatically generated by GPT-4o. Our approach integrates semantic ontologies with advanced LLM capabilities to automate three critical steps: (1) generating SPARQL queries to retrieve time series references from ontological models, (2) extracting the corresponding time series data from the Building Automation System, and (3) performing computations and visualizations tailored to the user's query. The proposed method simplifies access to BAS data, allowing both domain experts and non-specialists to conduct sophisticated analyses without needing extensive technical knowledge of semantic web technologies. By demonstrating this pipeline, we facilitate more accessible and scalable data-driven decision-making in building operations and management.

Mulayim, Ozan Baris↗

DOE OSTI2025

Extraction and Analysis of Time Series Data from Building Automation Systems Using Large Language Models

Mulayim, Ozan Baris↗

NASA NTRS2019

Assessment of Satellite Sea Surface Salinity Products Using a Coupled ENSO Prediction Model

Much work has gone into revising and updating algorithms for converting satellite-measured radiances to useful ocean variables like sea surface salinity (e.g. SMOS - Boutin et al., 2017, SMAP - Fore et al., 2016 and Aquarius - Meissner et al., 2018). As part of our Ocean Salinity Science Team work, we utilize an intermediate-complexity air/sea coupled model as a transfer function to test if more mature satellite SSS model algorithms actually improve ENSO forecast skill. For initialization of the coupled forecast, we demonstrate that the positive impact of SSS assimilation is brought about by surface freshening near the eastern edge of the western Pacific warm pool and density changes that lead to shallower mixed layer between 10S-5N. In addition, salting near the ITCZ leads to a deepening of the mixed layer and thermocline near 8N. These patterns together provide the background state to amplify equatorial Kelvin waves and improve ENSO hindcasts (Hackert et al., 2019). Here we extend this work to compare the impact of various pairs of original and improved satellite SSS algorithms. For instance we compare SMAP V4.1 with the latest, SMAP V4.2, to see what impact algorithm improvements may have on ENSO forecasts. SSS observations are tested on seasonal to interannual variability of tropical Indo-Pacific Ocean dynamics as well as on dynamical ENSO forecasts by initializing twelve-month forecasts for each month of available data. All experiments assimilate satellite sea level (SL), sea surface temperature (SST), and in situ subsurface temperature and salinity observations (Tz, Sz). Additionally various satellite, blended, and in-situ SSS products are assimilated. We find that including satellite SSS significantly improves NiÃ±o3.4 sea surface temperature anomaly validation, more mature SSS model algorithms are generally improving ENSO forecasts over time, and more satellite SSS data coverage helps to extend useful forecasts.

Ocean↗