Search NASA⌕ Search

SEARCH · Search NASA

Results for “validation data sets”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 217 records · Page 12

Lessons Learned Study Final Report for the Exploration Systems Mission Directorate

This report is the final product of a 90-day study performed for the Exploration Systems Mission Directorate. The study was to assemble lessons NASA has learned from previous programs that could help the Exploration Systems Mission Directorate pursue the Exploration vision. It focuses on those lessons that should have the greatest significance to the Directorate during the formulation of program and mission plans. The study team reviewed a large number of lessons learned reports and data bases, including the Columbia Accident Investigation Board and Rogers Commission reports on the Shuttle accidents, accident reports from robotic space flight systems, and a number of management reviews by the Defense Sciences Board, Government Accountability Office, and others. The consistency of the lessons, findings, and recommendations validate the adequacy of the data set. In addition to reviewing existing databases, a series of workshops was held at each of the NASA centers and headquarters that included senior managers from the current workforce as well as retirees. The full text of the workshop reports is included in Appendix A. A lessons learned website was opened up to permit current and retired NASA personnel and on-site contractors to input additional lessons as they arise. These new lessons, when of appropriate quality and relevance, will be brought to the attention of managers. The report consists of four parts: Part 1 provides a small set of lessons, called the Executive Lessons Learned, that represent critical lessons that the Exploration Systems Mission Directorate should act on immediately. This set of Executive Lessons and their supporting rationale have been reviewed at length and fully endorsed by a team of distinguished NASA alumni; Part 2 contains a larger set of lessons, called the Selected Lessons Learned, which have been chosen from the lessons database and center workshop reports on the basis of their specific significance and relevance to the near-term work of the Exploration Directorate. These lessons frequently support the Executive lessons but are more general in nature; Part 3 consists of the reports of the center workshops that were conducted as part of this activity. These reports are included in their entirety (approximately 200 pages) in Appendix G and have significance for specific managers; Part 4 consists of the remainder of the lessons that have been selected by this effort and assembled into a database for the use of the Explorations Directorate. The database is archived and hosted in the Lessons Learned Knowledge Network, which provides a flexible search capability using a wide variety of search terms. Finally, a spreadsheet lists databases searched and a bibliography identifies reports that have been reviewed as sources of lessons for this task. NASA has been presented with many learning opportunities. We have conducted numerous programs, some extremely successful and others total failures. Most have been documented with a formal lessons learned activity, but we have not always incorporated these learning opportunities into our normal modes of business. For example, the Robbins Report of 2001 clearly indicates that many project failures of the past two decades were the result of violating well documented best practices, often in direct violation of management instructions and directives. An overarching lesson emerges: that disciplined execution in accordance with proven best practices is the greatest single contributor to a successful program. The Lessons Learned task team offers a sincere hope that the lessons presented herein will be helpful to the Exploration Systems Directorate in charting and executing their course. The success of the Directorate and of NASA in general depends on our collective ability to move forward without having to relearn the lessons of those who have gone before.

Van Laak, Jim↗

Assessing Dual-Polarization Radar Estimates of Extreme Rainfall During Hurricane Harvey

Hurricane Harvey hit the Texas Gulf Coast as a major hurricane on August 25, 2017 before exiting the state as a tropical storm on August 29, 2017. Left in its wake was historic flooding, with some locations measuring more than 60 inches of rain over a five-day period. The WSR-88D radar (KHGX) maintained operations for the entirety of the event. Rain gauge data from the Harris County Flood Warning System (HCFWS) was used for validation with the full radar data set to retrieve daily and event-total precipitation estimates for the period August 25-29, 2017. The KHGX precipitation estimates were then compared to the HCFWS gauges. Three different hybrid polarimetric rainfall retrievals were used, along with attenuation-based retrieval that employs the radar-observed differential propagation. An advantage of using a attenuation-based retrieval is its immunity to partial beam blockage and calibration errors in reflectivity and differential reflectivity. All of the retrievals are susceptible to changes in the observed Drop Size Distribution (DSD). No in situ DSD data were available over the study area, so changes in the DSD were interpreted by examining the observed radar data. We examined the parameter space of two key values in the attenuation retrieval to test the sensitivity of the rain retrieval. Selecting a value of α=0.015 and β=0.600 and β=0.625 provided the best overall results, relative to the gauges, but more work needs to be done to develop an automated technique to account for changes in the ambient DSD.

David B Wolff↗

Measured and Simulated Acoustic Signature of a Full-Scale Aircraft with Airframe Noise Reduction Technology Installed

Microphone phased-array and pole-mounted microphone data gathered during the NASA Acoustics Research Measurements flight tests were used to benchmark results from companion full-scale aeroacoustics simulations. Conducted with the lattice Boltzmann solver PowerFLOW®, the simulations predicted the acoustic behavior of various tested aircraft configurations. Emphasis was placed on those flown during the third flight test - a Fowler flap-equipped Gulfstream G-III with and without noise abatement technology on the main landing gear. Direct comparisons between experimental and synthetic microphone phasedarray data were achieved by applying the same processing and deconvolution technique to both sets of data. To extend the validation of the computations to the metric used for noise certification, the Effective Perceived Noise Level, a high-fidelity digital model of the nose landing gear, which was excluded from earlier computations, was developed and integrated into the G-III aircraft geometry. The acoustic study presented here demonstrates that the simulated beamform maps and corresponding integrated farfield spectra accurately predict the locations and strengths of the prominent airframe noise sources present on the G-III aircraft.

Khorrami, Mehdi R.↗

Investigation of the Drivers and Atmospheric Impacts of Energetic Electron Precipitation

The drivers and atmospheric impacts of energetic electron precipitation are not yet well understood. Further, electron precipitation is often poorly represented in atmospheric modeling. Additional investigations of the drivers and impacts of electron precipitation are needed to improve models and space weather forecasting requirements. To accurately represent the troposphere through the ionosphere in model simulations, it is vital to account for the chemistry accurately. Electron precipitation is a frequent, yet often ignored middle to high latitude forcing that can have dramatic effects on the middle and upper atmosphere. Over the past decade, several electron precipitation data sets have been developed, however, validation has been difficult due to the lack of independent observations of electron fluxes. Additionally, the limited number of satellites making measurements of global magnetospheric wave activity in concert with the resulting electron precipitation restricts our ability to accurately capture the drivers simultaneously with the precipitation. Accurate characterization of the drivers is needed for physics-based magnetosphere modeling. Likewise, accurate precipitating electron fluxes and relative energies are needed to improve our atmospheric modeling studies. Finally, in order to properly validate and improve our current modeling efforts, observations of atmospheric composition are necessary.

Joshua Pettit↗

Snow Depth from AMSR-2 Using Multispectral Satellite Data in an Artificial Neural Network

By using diffusion theory and Monte Carlo lidar radiative transfer simulations, Hu et al. (2022b) has derived snow depth from the first-, second- and third-order moments of the lidar backscattering pathlength distribution. Lu et al. (2022) calculated the snow depth by applying the methods to the satellite ICESat-2 lidar measurements over the Arctic sea ice, as well as land surfaces of Northern Hemisphere. In this paper, an artificial neural network (ANN) algorithm, employing several channels from Advanced Microwave Scanning Radiometer 2 (AMSR-2) and the humidity vertical profiles from Global Modeling and Assimilation Office (GMAO) Goddard Earth Observing System for Instrument Teams (GEOS-IT) product, is trained to determine snow depth identified by time and geolocation matched 2019 ICESat-2 snow-depth data during winter months over the Arctic sea ice. The trained ANN snow-depth was applied to 2018 AMSR-2 clear pixel data, although the algorithms perform reasonably well in thinner clouds. The validation data (different from the training set) of ANN snow depth from AMSR-2 showed a good agreement with time matched and co-located snow-depth values from ICESat-2. The bias was near zero, with mean absolute error (MAE) 0.05 cm and a root-mean-square-error (RMSE) 0.08 cm. Prior applying the trained ANN snow depth to AMSR-2 data, a cloud screening algorithm was developed with a similar approach. A separate ANN cloud mask was trained to determine an AMSR-2 pixel is clear or cloudy with time and geolocation matched 2015 CALIOP Vertical Feature Mask (VFM) over Arctic sea ice. The ANN cloud mask from AMSR-2 under-estimated cloud fraction by 3-6% compared to CALIOP . The additional research is needed to conclusively evaluate the ANN cloud mask accuracy. Finally, this paper will lay the foundation for a sustained long-term snowfall and snow-storm monitoring system. The future Cloud Aerosol LIdar for Global scale Observations of the ocean-Land Atmosphere system (CALIGOLA) mission will provide a means to calculate snow depth from the lidar backscattering pathlength distribution, benefiting from the UV, visible and infrared pulses. With the calculated snow depth as the truth one could develop a machine learning algorithm, as it was done in this paper, using a passive microwave instrument available at that time to generate a wide range of snow depth data, covering extensive spatial areas in the cross-orbit direction.

Neural Network↗

Testing the Accuracy of Data-Driven MHD Simulations of Active Region Evolution

Models for the evolution of the solar coronal magnetic field are vital for understanding solar activity, yet the best measurements of the magnetic field lie at the photosphere, necessitating the development of coronal models which are "data-driven" at the photosphere. We present an investigation to determine the feasibility and accuracy of such methods. Our validation framework uses a simulation of active region (AR) formation, modeling the emergence of magnetic flux from the convection zone to the corona, as a ground-truth data set, to supply both the photospheric information and to perform the validation of the data-driven method. We focus our investigation on how the accuracy of the data-driven model depends on the temporal frequency of the driving data. The Helioseismic and Magnetic Imager on NASA's Solar Dynamics Observatory produces full-disk vector magnetic field measurements at a 12-minute cadence. Using our framework we show that ARs that emerge over 25 hours can be modeled by the data-driving method with only approximately 1 percent error in the free magnetic energy, assuming the photospheric information is specified every 12 minutes. However, for rapidly evolving features, under-sampling of the dynamics at this cadence leads to a strobe effect, generating large electric currents and incorrect coronal morphology and energies. We derive a sampling condition for the driving cadence based on the evolution of these small-scale features, and show that higher-cadence driving can lead to acceptable errors. Future work will investigate the source of errors associated with deriving plasma variables from the photospheric magnetograms as well as other sources of errors, such as reduced resolution, instrument bias, and noise.

Leake, James E.↗

Reducing Multisensor Satellite Monthly Mean Aerosol Optical Depth Uncertainty: 1. Objective Assessment of Current AERONET Locations

Various space-based sensors have been designed and corresponding algorithms developed to retrieve aerosol optical depth (AOD), the very basic aerosol optical property, yet considerable disagreement still exists across these different satellite data sets. Surface-based observations aim to provide ground truth for validating satellite data; hence, their deployment locations should preferably contain as much spatial information as possible, i.e., high spatial representativeness. Using a novel Ensemble Kalman Filter (EnKF)- based approach, we objectively evaluate the spatial representativeness of current Aerosol Robotic Network (AERONET) sites. Multisensor monthly mean AOD data sets from Moderate Resolution Imaging Spectroradiometer, Multiangle Imaging Spectroradiometer, Sea-viewing Wide Field-of-view Sensor, Ozone Monitoring Instrument, and Polarization and Anisotropy of Reflectances for Atmospheric Sciences coupled with Observations from a Lidar are combined into a 605-member ensemble, and AERONET data are considered as the observations to be assimilated into this ensemble using the EnKF. The assessment is made by comparing the analysis error variance (that has been constrained by ground-based measurements), with the background error variance (based on satellite data alone). Results show that the total uncertainty is reduced by approximately 27% on average and could reach above 50% over certain places. The uncertainty reduction pattern also has distinct seasonal patterns, corresponding to the spatial distribution of seasonally varying aerosol types, such as dust in the spring for Northern Hemisphere and biomass burning in the fall for Southern Hemisphere. Dust and biomass burning sites have the highest spatial representativeness, rural and oceanic sites can also represent moderate spatial information, whereas the representativeness of urban sites is relatively localized. A spatial score ranging from 1 to 3 is assigned to each AERONET site based on the uncertainty reduction, indicating its representativeness level.

aerosol optical depth↗

A diagnostic model for studying daytime urban air quality trends

A single cell Eulerian photochemical air quality simulation model was developed and validated for selected days of the 1976 St. Louis Regional Air Pollution Study (RAPS) data sets; parameterizations of variables in the model and validation studies using the model are discussed. Good agreement was obtained between measured and modeled concentrations of NO, CO, and NO2 for all days simulated. The maximum concentration of O3 was also predicted well. Predicted species concentrations were relatively insensitive to small variations in CO and NOx emissions and to the concentrations of species which are entrained as the mixed layer rises.

Brewer, D. A.↗

Technical Report Series on Global Modeling and Data Assimilation: Soil Moisture Active Passive (SMAP) Project Assessment Report for the Beta-Release L4_SM Data Product - Volume 40

During the post-launch SMAP calibration and validation (Cal/Val) phase there are two objectives for each science data product team: 1) calibrate, verify, and improve the performance of the science algorithm, and 2) validate the accuracy of the science data product as specified in the science requirements and according to the Cal/Val schedule. This report provides an assessment of the SMAP Level 4 Surface and Root Zone Soil Moisture Passive (L4_SM) product specifically for the product's public beta release scheduled for 30 October 2015. The primary objective of the beta release is to allow users to familiarize themselves with the data product before the validated product becomes available. The beta release also allows users to conduct their own assessment of the data and to provide feedback to the L4_SM science data product team. The assessment of the L4_SM data product includes comparisons of SMAP L4_SM soil moisture estimates with in situ soil moisture observations from core validation sites and sparse networks. The assessment further includes a global evaluation of the internal diagnostics from the ensemble-based data assimilation system that is used to generate the L4_SM product. This evaluation focuses on the statistics of the observation-minus-forecast (O-F) residuals and the analysis increments. Together, the core validation site comparisons and the statistics of the assimilation diagnostics are considered primary validation methodologies for the L4_SM product. Comparisons against in situ measurements from regional-scale sparse networks are considered a secondary validation methodology because such in situ measurements are subject to upscaling errors from the point-scale to the grid cell scale of the data product. Based on the limited set of core validation sites, the assessment presented here meets the criteria established by the Committee on Earth Observing Satellites for Stage 1 validation and supports the beta release of the data. The validation against sparse network measurements and the evaluation of the assimilation diagnostics address Stage 2 validation criteria by expanding the assessment to regional and global scales.

ubRMSE↗

Ozone reference models for the middle atmosphere (new CIRA)

Models of ozone vertical structure were generated that were based on multiple data sets from satellites. The very good absolute accuracy of the individual data sets allowed the data to be directly combined to generate these models. The data used for generation of these models are from some of the most recent satellite measurements over the period 1978 to 1983. A discussion is provided of validation and error analyses of these data sets. Also, inconsistencies in data sets brought about by temporal variations or other factors are indicated. The models cover the pressure range from from 20 to 0.003 mb (25 to 90 km). The models for pressures less than 0.5 mb represent only the day side and are only provisional since there was limited longitudinal coverage at these levels. The models start near 25 km in accord with previous COSPAR international reference atmosphere (CIRA) models. Models are also provided of ozone mixing ratio as a function of height. The monthly standard deviation and interannual variations relative to zonal means are also provided. In addition to the models of monthly latitudinal variations in vertical structure based on satellite measurements, monthly models of total column ozone and its characteristic variability as a function of latitude based on four years of Nimbus 7 measurements, models of the relationship between vertical structure and total column ozone, and a midlatitude annual mean model are incorporated in this set of ozone reference atmospheres. Various systematic variations are discussed including the annual, semiannual, and quasibiennial oscillations, and diurnal, longitudinal, and response to solar activity variations.

Keating, G. M.↗

An Efficient Ray-Tracing Method for Determining Terrain Intercepts in EDL Simulations

The calculation of a ray's intercept from an arbitrary point in space to a prescribed surface is a common task in computer simulations. The arbitrary point often represents an object that is moving according to the simulation, while the prescribed surface is fixed in a defined frame. For detailed simulations, this surface becomes complex, taking the form of real-world objects such as mountains, craters or valleys which require more advanced methods to accurately calculate a ray's intercept location. Incorporation of these complex surfaces has commonly been implemented in graphics systems that utilize highly optimized graphics processing units to analyze such features. This paper proposes a simplified method that does not require computationally intensive graphics solutions, but rather an optimized ray-tracing method for an assumed terrain dataset. This approach was developed for the Mars Science Laboratory mission which landed on the complex terrain of Gale Crater. First, this paper begins with a discussion of the simulation used to implement the model and the applicability of finding surface intercepts with respect to atmosphere modeling, altitude determination, radar modeling, and contact forces influencing vehicle dynamics. Next, the derivation and assumptions of the intercept finding method are presented. Key assumptions are noted making the routines specific to only certain types of surface data sets that are equidistantly spaced in longitude and latitude. The derivation of the method relies on ray-tracing, requiring discussion on the formulation of the ray with respect to the terrain datasets. Further discussion includes techniques for ray initialization in order to optimize the intercept search. Then, the model implementation for various new applications in the simulation are demonstrated. Finally, a validation of the accuracy is presented along with the corresponding data sets used in the validation. A performance summary of the method will be shown using the analysis from the Mars Science Laboratory's terminal descent sensing model. Alternate uses will also be shown for determining horizon maps and orbiter set times.

Shidner, Jeremy D.↗

Improvements to TOVS retrievals over sea ice and applications to estimating Arctic energy fluxes

Modeling studies suggest that polar regions play a major role in modulating the Earth's climate and that they may be more sensitive than lower latitudes to climate change. Until recently, however, data from meteorological stations poleward of 70 degs have been sparse, and consequently, our understanding of air-sea-ice interaction processes is relatively poor. Satellite-borne sensors now offer a promising opportunity to observe polar regions and ultimately to improve parameterizations of energy transfer processes in climate models. This study focuses on the application of the TIROS-N operational vertical sounder (TOVS) to sea-ice-covered regions in the nonmelt season. TOVS radiances are processed with the improved initialization inversion ('3I') algorithm, providng estimates of layer-average temperature and moisture, cloud conditions, and surface characteristics at a horizontal resolution of approximately 100 km x 100 km. Although TOVS has flown continuously on polar-orbiting satellites since 1978, its potential has not been realized in high latitudes because the quality of retrievals is often significantly lower over sea ice and snow than over the surfaces. The recent availability of three Arctic data sets has provided an opportunity to validate TOVS retrievals: the first from the Coordinated Eastern Arctic Experiment (CEAREX) in winter 1988/1989, the second from the LeadEx field program in spring 1992, and the third from Russian drifting ice stations. Comparisons with these data reveal deficiencies in TOVS retrievals over sea ice during the cold season; e.g., ice surface temperature is often 5 to 15 K too warm, microwave emissivity is approximately 15% too low at large view angles, clear/cloudy scenes are sometimes misidentified, and low-level inversions are often not captured. In this study, methods to reduce these errors are investigated. Improvements to the ice surface temperature retrieval have reduced rms errors from approximately 7 K to 3 K; correction of microwaves (50 GHz) brightness temperatures for view angle dependence now allows the surface type (open water versus sea ice) to be determined; modifications to the clear/cloud tests have improved cloud detection over sea ice, especially where it is inhomogeneous; and improved surface temperature estimates have resulted in more successful diagnoses of low-level stratification. Applications of improved retrievals to studies of the Arctic energy budget are encouraging. Preliminary calculations of air-ice stress vectors, 10-m wind speed, and horizontal advection of heat and moisture are presented.

Francis, Jennifer A.↗

Radiative Transfer Modeling

Our research efforts addressed theoretical work in radiative transfer which focused the following five major items: (1) Development of three dimensional radiative transfer methods; (2) Improvement to and maintenance of radiative transfer codes; (3) Creating new data-sets to define absorption by atmos.gases; (4) Model tests and validations with data from field experiments; and (5) Model application to study climatic impact of, aerosol/clouds.

Source record↗

A Spatio-Temporal Approach for Global Validation and Analysis of MODIS Aerosol Products

With the launch of the MODIS sensor on the Terra spacecraft, new data sets of the global distribution and properties of aerosol are being retrieved, and need to be validated and analyzed. A system has been put in place to generate spatial statistics (mean, standard deviation, direction and rate of spatial variation, and spatial correlation coefficient) of the MODIS aerosol parameters over more than 100 validation sites spread around the globe. Corresponding statistics are also computed from temporal subsets of AERONET-derived aerosol data. The means and standard deviations of identical parameters from MOMS and AERONET are compared. Although, their means compare favorably, their standard deviations reveal some influence of surface effects on the MODIS aerosol retrievals over land, especially at low aerosol loading. The direction and rate of spatial variation from MODIS are used to study the spatial distribution of aerosols at various locations either individually or comparatively. This paper introduces the methodology for generating and analyzing the data sets used by the two MODIS aerosol validation papers in this issue.

Ichoku, Charles↗

Computation and use of the reflectivity at 3.75 micrometers from AVHRR thermal channels

Global study of land surface properties uses AVHRR channels 1 and 2, but channel 3 may be of interest, although its use requires preprocessing. It consists of both a reflective part and an emissive part, the former can be derived from T3, T4 and T5. Since the water vapor affects channel 3, its content is retrieved from the channel 4 and 5 using the split window technique. A formula of reflective part retrieval at 3.75 micrometers is tested in the case of sunglint observations where the emissivities of channels 4 and 5 can be set to the unity. The formula is adapted and validated to land surface using the FIFE-87 data set. Preliminary applications of the reflectance at 3.75 micrometers to the studies of surface properties retrieval, aerosol retrieval over land, and desertic aerosol retrieval, are addressed.

Roger, J. C.↗

Generation of Level 3 SMMR and SSM/I Brightness Temperatures for the Period 1978-1999

The NOAA/NASA Pathfinder Program was initially designed to assure that certain key remote sensing data sets of particular significance to global change research were scientifically validated, consistently processed and made readily available to the research community at minimal cost. Through this Program the National Snow and Ice Data Center (NSIDC), University of Colorado has successfully processed, archived and distributed the Scanning Multichannel Microwave Radiometer (SMMR) and Special Sensor Microwave/Imager (SSM/I) Level 3 (EASE-Grid format) Pathfinder data sets for the period 1978 to 1999. These data are routinely distributed to approximately 150 researchers through various media including CD-ROM, 8 mm tape, ftp and the EOS Information Management System (IMS). At NSIDC these data are currently being applied in the development and validation of algorithms to derive snow water equivalent (NASA NAG5-6636), the mapping of frozen ground and the detection of the onset of melt over ice sheets, sea ice and snow cover. The EASE-Grid format, developed at NSIDC in conjunction with the SMMR-SSM/I Pathfinder project has also been applied to Advanced Very High Resolution Radiometer (AVHRR) and TOVS Pathfinder data, as well as ancillary data such as digital elevation, land cover classification and several in situ data sets. EASE-Grid will also be used for all land products derived from the NASA EOS AMSR-E instrument.

Partington, Kim↗

Collaboration on Development and Validation of the AMSR-E Snow Water Equivalent Algorithm

The National Snow and Ice Data Center (NSIDC) has produced a global SMMR and SSM/I Level 3 Brightness Temperature data set in the Equal Area Scalable Earth (EASE) Grid for the period 1978 to 2000. Processing of current data is-ongoing. The EASE-Grid passive microwave data sets are appropriate for algorithm development and validation prior to the launch of AMSR-E. Having the lower frequency channels of SMMR (6.6 and 10.7 GHz) and the higher frequency channels of SSM/I (85.5 GHz) in the same format will facilitate the preliminary development of applications which could potentially make use of similar frequencies from AMSR-E (6.9, 10.7, 89.0 GHz).

Armstrong, Richard L.↗

Soil Moisture Active Passive Mission L4_SM Data Product Assessment (Version 2 Validated Release)

During the post-launch SMAP calibration and validation (Cal/Val) phase there are two objectives for each science data product team: 1) calibrate, verify, and improve the performance of the science algorithm, and 2) validate the accuracy of the science data product as specified in the science requirements and according to the Cal/Val schedule. This report provides an assessment of the SMAP Level 4 Surface and Root Zone Soil Moisture Passive (L4_SM) product specifically for the product's public Version 2 validated release scheduled for 29 April 2016. The assessment of the Version 2 L4_SM data product includes comparisons of SMAP L4_SM soil moisture estimates with in situ soil moisture observations from core validation sites and sparse networks. The assessment further includes a global evaluation of the internal diagnostics from the ensemble-based data assimilation system that is used to generate the L4_SM product. This evaluation focuses on the statistics of the observation-minus-forecast (O-F) residuals and the analysis increments. Together, the core validation site comparisons and the statistics of the assimilation diagnostics are considered primary validation methodologies for the L4_SM product. Comparisons against in situ measurements from regional-scale sparse networks are considered a secondary validation methodology because such in situ measurements are subject to up-scaling errors from the point-scale to the grid cell scale of the data product. Based on the limited set of core validation sites, the wide geographic range of the sparse network sites, and the global assessment of the assimilation diagnostics, the assessment presented here meets the criteria established by the Committee on Earth Observing Satellites for Stage 2 validation and supports the validated release of the data. An analysis of the time average surface and root zone soil moisture shows that the global pattern of arid and humid regions are captured by the L4_SM estimates. Results from the core validation site comparisons indicate that "Version 2" of the L4_SM data product meets the self-imposed L4_SM accuracy requirement, which is formulated in terms of the ubRMSE: the RMSE (Root Mean Square Error) after removal of the long-term mean difference. The overall ubRMSE of the 3-hourly L4_SM surface soil moisture at the 9 km scale is 0.035 cubic meters per cubic meter requirement. The corresponding ubRMSE for L4_SM root zone soil moisture is 0.024 cubic meters per cubic meter requirement. Both of these metrics are comfortably below the 0.04 cubic meters per cubic meter requirement. The L4_SM estimates are an improvement over estimates from a model-only SMAP Nature Run version 4 (NRv4), which demonstrates the beneficial impact of the SMAP brightness temperature data. L4_SM surface soil moisture estimates are consistently more skillful than NRv4 estimates, although not by a statistically significant margin. The lack of statistical significance is not surprising given the limited data record available to date. Root zone soil moisture estimates from L4_SM and NRv4 have similar skill. Results from comparisons of the L4_SM product to in situ measurements from nearly 400 sparse network sites corroborate the core validation site results. The instantaneous soil moisture and soil temperature analysis increments are within a reasonable range and result in spatially smooth soil moisture analyses. The O-F residuals exhibit only small biases on the order of 1-3 degrees Kelvin between the (re-scaled) SMAP brightness temperature observations and the L4_SM model forecast, which indicates that the assimilation system is largely unbiased. The spatially averaged time series standard deviation of the O-F residuals is 5.9 degrees Kelvin, which reduces to 4.0 degrees Kelvin for the observation-minus-analysis (O-A) residuals, reflecting the impact of the SMAP observations on the L4_SM system. Averaged globally, the time series standard deviation of the normalized O-F residuals is close to unity, which would suggest that the magnitude of the modeled errors approximately reflects that of the actual errors. The assessment report also notes several limitations of the "Version 2" L4_SM data product and science algorithm calibration that will be addressed in future releases. Regionally, the time series standard deviation of the normalized O-F residuals deviates considerably from unity, which indicates that the L4_SM assimilation algorithm either over- or under-estimates the actual errors that are present in the system. Planned improvements include revised land model parameters, revised error parameters for the land model and the assimilated SMAP observations, and revised surface meteorological forcing data for the operational period and underlying climatological data. Moreover, a refined analysis of the impact of SMAP observations will be facilitated by the construction of additional variants of the model-only reference data. Nevertheless, the “Version 2” validated release of the L4_SM product is sufficiently mature and of adequate quality for distribution to and use by the larger science and application communities.

SMAP L4_SM↗