Search NASA⌕ Search

SEARCH · Search NASA

Results for “mean squared error”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 199 records · Page 11

Quantifying the Storm Time Thermospheric Neutral Density Variations Using Model and Observations

Accurate determination of thermospheric neutral density holds crucial importance for satellited rag calculations. The problem is twofold and involves the correct estimation of the quiet time climatology and storm time variations. In this work, neutral density estimations from two empirical and threephysics based models of the ionosphere thermosphere are compared with the neutral densities along the Challenging MicroSatellite Payload satellite track for six geomagnetic storms. Storm time variations are extracted from neutral density by (1) subtracting the mean difference between model and observation (bias),(2) setting climatological variations to zero, and (3) multiplying model data with the quiet time ratio between the model and observation. Several metrics are employed to evaluate the model performances. We find that the removal of bias or climatology reveals actual performance of the model in simulating the storm time variations. When bias is removed, depending on event and model, storm time errors in neutral density can decrease by an amount of 113% or can increase by an amount of 12% with respect to error in models with quiet time bias. It is shown that using only average and maximum values of neutral density to determine the model performances can be misleading since a model can estimate the averages fairly well but may not capture the maximum value or vice versa. Since each of the metrics used for determining model performances provides different aspects of the error, among these, we suggest employing mean absolute error, prediction efficiency, and normalized root mean square error together as a standard set ofmetrics for the neutral density.

Eyiguler, E. Ceren Kalafatoglu↗

Weekly Mapping of Sea Ice Freeboard in the Ross Sea from ICESat-2

NASA’s ICESat-2 has been providing sea ice freeboard measurements across the polar regions since October 2018. In spite of the outstanding spatial resolution and precision of ICESat-2, the spatial sparsity of the data can be a critical issue for sea ice monitoring. This study employs a geostatistical approach (i.e., ordinary kriging) to characterize the spatial autocorrelation of the ICESat-2 freeboard measurements (ATL10) to estimate weekly freeboard variations in 2019 for the entire Ross Sea area, including where ICESat-2 tracks are not directly available. Three variogram models (exponential, Gaussian, and spherical) are compared in this study. According to the cross-validation results, the kriging-estimated freeboards show correlation coefficients of 0.56–0.57, root mean square error (RMSE) of ~0.12 m, and mean absolute error (MAE) of ~0.07 m with the actual ATL10 freeboard measurements. In addition, the estimated errors of the kriging interpolation are low in autumn and high in winter to spring, and low in southern regions and high in northern regions of the Ross Sea. The effective ranges of the variograms are 5–10 km and the results from the three variogram models do not show significant differences with each other. The southwest (SW) sector of the Ross Sea shows low and consistent freeboard over the entire year because of the frequent opening of wide polynya areas generating new ice in this sector. However, the southeast (SE) sector shows large variations in freeboard, which demonstrates the advection of thick multiyear ice from the Amundsen Sea into the Ross Sea. Thus, this kriging-based interpolation of ICESat-2 freeboard can be used in the future to estimate accurate sea ice production over the Ross Sea by incorporating other remote sensing data.

Satellite altimeter↗

Assessment of the global monthly mean surface insolation estimated from satellite measurements using global energy balance archive data

Global sets of surface radiation budget (SRB) have been obtained from satellite programs. These satellite-based estimates need validation with ground-truth observations. This study validates the estimates of monthly mean surface insolation contained in two satellite-based SRB datasets with the surface measurements made at worldwide radiation stations from the Global Energy Balance Archive (GEBA). One dataset was developed from the Earth Radiation Budget Experiment (ERBE) using the algorithm of Li et al. (ERBE/SRB), and the other from the International Satellite Cloud Climatology Project (ISCCP) using the algorithm of Pinker and Laszlo and that of Staylor (GEWEX/SRB). Since the ERBE/SRB data contain the surface net solar radiation only, the values of surface insolation were derived by making use of the surface albedo data contained GEWEX/SRB product. The resulting surface insolation has a bias error near zero and a root-mean-square error (RMSE) between 8 and 28 W/sq m. The RMSE is mainly associated with poor representation of surface observations within a grid cell. When the number of surface observations are sufficient, the random error is estimated to be about 5 W/sq m with present satellite-based estimates. In addition to demonstrating the strength of the retrieving method, the small random error demonstrates how well the ERBE derives from the monthly mean fluxes at the top of the atmosphere (TOA). A larger scatter is found for the comparison of transmissivity than for that of insolation. Month to month comparison of insolation reveals a weak seasonal trend in bias error with an amplitude of about 3 W/sq m. As for the insolation data from the GEWEX/SRB, larger bias errors of 5-10 W/sq m are evident with stronger seasonal trends and almost identical RMSEs.

Li, Zhanqing↗

Robust Timing Synchronization in Aeronautical Mobile Communication Systems

This work details a study of robust synchronization schemes suitable for satellite to mobile aeronautical applications. A new scheme, the Modified Sliding Window Synchronizer (MSWS), is devised and compared with existing schemes, including the traditional Early-Late Gate Synchronizer (ELGS), the Gardner Zero-Crossing Detector (GZCD), and the Sliding Window Synchronizer (SWS). Performance of the synchronization schemes is evaluated by a set of metrics that indicate performance in digital communications systems. The metrics are convergence time, mean square phase error (or root mean-square phase error), lowest SNR for locking, initial frequency offset performance, midstream frequency offset performance, and system complexity. The performance of the synchronizers is evaluated by means of Matlab simulation models. A simulation platform is devised to model the satellite to mobile aeronautical channel, consisting of a Quadrature Phase Shift Keying modulator, an additive white Gaussian noise channel, and a demodulator front end. Simulation results show that the MSWS provides the most robust performance at the cost of system complexity. The GZCD provides a good tradeoff between robustness and system complexity for communication systems that require high symbol rates or low overall system costs. The ELGS has a high system complexity despite its average performance. Overall, the SWS, originally designed for multi-carrier systems, performs very poorly in single-carrier communications systems. Table 5.1 in Section 5 provides a ranking of each of the synchronization schemes in terms of the metrics set forth in Section 4.1. Details of comparison are given in Section 5. Based on the results presented in Table 5, it is safe to say that the most robust synchronization scheme examined in this work is the high-sample-rate Modified Sliding Window Synchronizer. A close second is its low-sample-rate cousin. The tradeoff between complexity and lowest mean-square phase error determines the rankings of the Gardner Zero-Crossing Detector and both versions of the Early-Late Gate Synchronizer. The least robust models are the high and low-sample-rate Sliding Window Synchronizers. Consequently, the recommended replacement synchronizer for NASA's Advanced Air Transportation Technologies mobile aeronautical communications system is the high-sample-rate Modified Sliding Window Synchronizer. By incorporating this synchronizer into their system, NASA can be assured that their system will be operational in extremely adverse conditions. The quick convergence time of the MSWS should allow the use of high-level protocols. However, if NASA feels that reduced system complexity is the most important aspect of their replacement synchronizer, the Gardner Zero-Crossing Detector would be the best choice.

Xiong, Fu-Qin↗

Evaluation of “Autotune” calibration against manual calibration of building energy models

Our paper demonstrates the application of Autotune, a methodology aimed at automatically producing calibrated building energy models using measured data, in two case studies. In the first case, a building model is de-tuned by deliberately injecting faults into more than 60 parameters. This model was then calibrated using Autotune and its accuracy with respect to the original model was evaluated in terms of the industry-standard normalized mean bias error and coefficient of variation of root mean squared error metrics set forth in ASHRAE Guideline 14. In addition to whole-building energy consumption, outputs including lighting, plug load profiles, HVAC energy consumption, zone temperatures, and other variables were analyzed. In the second case, Autotune calibration is compared directly to experts’ manual calibration of an emulated-occupancy, full-size residential building with comparable calibration results in much less time. Lastly, our paper concludes with a discussion of the key strengths and weaknesses of auto-calibration approaches.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗

Web tools concerning performance analysis and planning support for solar energy plants starting from remotely sensed optical images

We present innovative web tools, developed also in the frame of the FP7 ENDORSE (ENergy DOwnstReam SErvices) project, for the performance analysis and the support in planning of solar energy plants (PV, CSP, CPV). These services are based on the combination between the detailed physical model of each part of the plants and the near real-time satellite remote sensing of incident solar irradiance. Starting from the solar Global Horizontal Irradiance (GHI) data provided by the Monitoring Atmospheric Composition and Climate (GMES-MACC) Core Service and based on the elaboration of Meteosat Second Generation (MSG) satellite optical imagery, the Global Tilted Irradiance (GTI) or the Beam Normal Irradiance (BNI) incident on plant's solar PV panels (or solar receivers for CSP or CPV) is calculated. Combining these parameters with the model of the solar power plant, using also air temperature values, we can assess in near-real-time the daily evolution of the alternate current (AC) power produced by the plant. We are therefore able to compare this satellite-based AC power yield with the actually measured one and, consequently, to readily detect any possible malfunctions and to evaluate the performances of the plant (so-called “Controller” service). Besides, the same method can be applied to satellite-based averaged environmental data (solar irradiance and air temperature) in order to provide a Return on Investment analysis in support to the planning of new solar energy plants (so-called “Planner” service). This method has been successfully applied to three test solar plants (in North, Centre and South Italy respectively) and it has been validated by comparing satellite-based and in-situ measured hourly AC power data for several months in 2013 and 2014. The results show a good accuracy: the overall Normalized Bias (NB) is − 0.41%, the overall Normalized Mean Absolute Error (NMAE) is 4.90%, the Normalized Root Mean Square Error (NRMSE) is 7.66% and the overall Correlation Coefficient (CC) is 0.9538. The maximum value of the Normalized Absolute Error (NAE) is about 30% and occurs for time periods with highly variable meteorological conditions. - Highlights: • We developed an online service (Controller) dedicated to solar energy plants real-time monitoring • We developed an online service (Planner) that supports the planning of new solar energy plants • The services are based on the elaboration of satellite optical imagery in near real-time • The validation with respect to in-situ measured hourly AC power data for three test solar plants shows good accuracy • The maximum value of the Normalized Absolute Error is about 30% and occurs for highly variable meteorological conditions.

14 SOLAR ENERGY↗

A machine learning method of modern urban building energy modeling: A case study of Chicago

Urban-scale building energy modeling is vital for urban planning. However, it can be challenging to assimilate reliable non-geometry building data for urban-scale modeling without extensive investment. Here, this study introduces a novel approach to developing modern urban-scale building energy stock data using geographic information systems and machine learning algorithms without necessarily requiring pre-supplied non-geometric metadata. The proposed framework integrates building footprint and height data to estimate gross floor areas, and matches each building to a pool of candidate records from ComStock or ResStock—filtered to the same county and ranked by geometric similarity—demonstrate a proof-of-concept case study in Chicago for predicting energy use intensity (EUI) using scalable datasets. The model achieved a mean bias error (MBE) of 0.08 kWh/m² and root mean square error (RMSE) of 14.84 kWh/m² under full metadata input for EUI prediction. With only location inputs, the model captured 69.2 % of EUI within predicted ranges. These results demonstrate the model’s potential to support early-stage urban planning, identify candidates for energy-efficient retrofits. By removing the dependency on detailed pre-surveys or extensive building metadata, the approach overcomes a key barrier in traditional urban-scale building energy modeling, illustrating a pathway toward broader and more cost-effective application, though further multi-city validation and improved treatment of pre-1925 buildings are needed.

Energy Use Intensity↗

Temporal ensemble learning of univariate methods for short term load forecasting

Short term load forecasting (STLF) is a fundamental component of demand response programs in smart grids. Recent literature has focused on complex neural networks and deep learning models for solving STLF. While these models work well for load forecasting with complex non-linear relationships, they have been shown to be less effective than simpler univariate models for STLF problems with under a 6 hours horizon. Given the lack of multivariate data (such as temperature) in many practical datasets, we need better univariate prediction models for STLF. By partitioning the dataset by temporal features, we develop a novel ensemble learning method that exploits daily seasonality in electricity consumption data to improve accuracy of popular univariate models. We train an ensemble of models from the dataset partitions. We develop a variety of methods, including Ridge Regression, to increase the robustness of the ensemble prediction. To show the effectiveness of our approach, we perform detailed evaluation using an aggregated user electricity consumption dataset collected by the Los Angeles Department of Water and Power (LADWP). We select four popular prediction algorithms in literature for our experiments, including Kernel Regression (KR), Support Vector Regression (SVR) and neural network approaches. We compare the performance of these algorithms applying our ensemble approach to training only one single model. Our approach leads to an 11.2% decrease in mean absolute percentage error (MAPE) and 21.3% decrease in root mean squared error (RMSE) over the single model approach for KR, and a 30% and 32.4% decrease in MAPE and RMSE respectively for SVR. These ensemble models also outperform the neural network approaches.

14 SOLAR ENERGY↗

WE-AB-207A-02: John’s Equation Based Consistency Condition and Incomplete Projection Restoration Upon Circular Orbit CBCT

Purpose: In transmitted X-ray tomography imaging, projections are sometimes incomplete due to a variety of reasons, such as geometry inaccuracy, defective detector cells, etc. To address this issue, we have derived a direct consistency condition based on John’s Equation, and proposed a method to effectively restore incomplete projections based on this consistency condition. Methods: Through parameter substitutions, we have derived a direct consistency condition equation from John’s equation, in which the left side is only projection derivative of view and the right side is projection derivative of other geometrical parameters. Based on this consistency condition, a projection restoration method is proposed, which includes five steps: 1) Forward projecting reconstructed image and using linear interpolation to estimate the incomplete projections as the initial result; 2) Performing Fourier transform on the projections; 3) Restoring the incomplete frequency data using the consistency condition equation; 4) Performing inverse Fourier transform; 5) Repeat step 2)∼4) until our criteria is met to terminate the iteration. Results: A beam-blocking-based scatter correction case and a bad-pixel correction case were used to demonstrate the efficacy and robustness of our restoration method. The mean absolute error (MAE), signal noise ratio (SNR) and mean square error (MSE) were employed as our evaluation metrics of the reconstructed images. For the scatter correction case, the MAE is reduced from 63.3% to 71.7% with 4 iterations. Compared with the existing Patch’s method, the MAE of our method is further reduced by 8.72%. For the bad-pixel case, the SNR of the reconstructed image by our method is increased from 13.49% to 21.48%, with the MSE being decreased by 45.95%, compared with linear interpolation method. Conclusion: Our studies have demonstrated that our restoration method based on the new consistency condition could effectively restore the incomplete projections, especially for their high frequency component.

60 APPLIED LIFE SCIENCES↗

Estimating Field-Level Perennial Bioenergy Grass Biomass Yields Using the Normalized Difference Red-Edge Index and Linear Regression Analysis for Central Virginia, USA

We investigated the indicative power of the normalized difference red-edge index (NDRE) for estimating field-level perennial bioenergy grass biomass yields utilizing Sentinel-2 imagery and a linear regression model as a rapid, cost-effective method for biomass yield estimations for bioenergy. We used 2019 data from three study sites containing mature perennial bioenergy grass stands in central Virginia, USA. Of the simulated daily NDRE values based on the temporally weighted averaging of two temporal neighbors, we found the strongest index–yield correlation on 11 August (R = 0.85). We estimated the perennial bioenergy grass biomass yields for (1) all sites using the data pooled from the three sites (all-site estimation) and (2) each site using the data pooled from the other two sites (cross-site estimation). The estimated field-level perennial bioenergy grass biomass yields strongly correlated with the recorded yields (average R2 = 0.76), with a root mean square error (RMSE) of 1.5 Mg/ha and a mean absolute error (MAE) of 1.2 Mg/ha for the all-site estimation. For the cross-site estimation, the site with diverse perennial grass types had the weakest correlation (R2 = 0.44) of the sites, indicating a difficulty in accounting for heterogeneous index–yield relationships in a single model. In addition to identifying a strong indicative power of the NDRE for estimating the overall perennial bioenergy grass biomass yields at a field level, the findings from this study call for an analysis across multiple perennial grasses and a comparison using multiple sites to understand (1) if the indicative power of the index shifts from the biomass of the specific perennial bioenergy grass type to the overall biomass during the growing season and (2) the level of perennial bioenergy grass heterogeneity that may hinder the remotely sensed biomass yield estimation using a single model.

09 BIOMASS FUELS↗

Quantitative Methods for Comparing Different Polyline Stream Network Models

Two techniques for exploring relative horizontal accuracy of complex linear spatial features are described and sample source code (pseudo code) is presented for this purpose. The first technique, relative sinuosity, is presented as a measure of the complexity or detail of a polyline network in comparison to a reference network. We term the second technique longitudinal root mean squared error (LRMSE) and present it as a means for quantitatively assessing the horizontal variance between two polyline data sets representing digitized (reference) and derived stream and river networks. Both relative sinuosity and LRMSE are shown to be suitable measures of horizontal stream network accuracy for assessing quality and variation in linear features. Both techniques have been used in two recent investigations involving extracting of hydrographic features from LiDAR elevation data. One confirmed that, with the greatly increased resolution of LiDAR data, smaller cell sizes yielded better stream network delineations, based on sinuosity and LRMSE, when using LiDAR-derived DEMs. The other demonstrated a new method of delineating stream channels directly from LiDAR point clouds, without the intermediate step of deriving a DEM, showing that the direct delineation from LiDAR point clouds yielded an excellent and much better match, as indicated by the LRMSE.

58 GEOSCIENCES↗

The Influence of Observation Errors on Analysis Error and Forecast Skill Investigated with an Observing System Simulation Experiment

The Global Modeling and Assimilation Office (GMAO) observing system simulation experiment (OSSE) framework is used to explore the response of analysis error and forecast skill to observation quality. In an OSSE, synthetic observations may be created that have much smaller error than real observations, and precisely quantified error may be applied to these synthetic observations. Three experiments are performed in which synthetic observations with magnitudes of applied observation error that vary from zero to twice the estimated realistic error are ingested into the Goddard Earth Observing System Model (GEOS-5) with Gridpoint Statistical Interpolation (GSI) data assimilation for a one-month period representing July. The analysis increment and observation innovation are strongly impacted by observation error, with much larger variances for increased observation error. The analysis quality is degraded by increased observation error, but the change in root-mean-square error of the analysis state is small relative to the total analysis error. Surprisingly, in the 120 hour forecast increased observation error only yields a slight decline in forecast skill in the extratropics, and no discernable degradation of forecast skill in the tropics.

Observation Error↗

Snow Process Estimation Over the Extratropical Andes Using a Data Assimilation Framework Integrating MERRA Data and Landsat Imagery

A data assimilation framework was implemented with the objective of obtaining high resolution retrospective snow water equivalent (SWE) estimates over several Andean study basins. The framework integrates Landsat fractional snow covered area (fSCA) images, a land surface and snow depletion model, and the Modern Era Retrospective Analysis for Research and Applications (MERRA) reanalysis as a forcing data set. The outputs are SWE and fSCA fields (1985-2015) at a resolution of 90 m that are consistent with the observed depletion record. Verification using in-situ snow surveys showed significant improvements in the accuracy of the SWE estimates relative to forward model estimates, with increases in correlation (0.49-0.87) and reductions in root mean square error (0.316 m to 0.129 m) and mean error (-0.221 m to 0.009 m). A sensitivity analysis showed that the framework is robust to variations in physiography, fSCA data availability and a priori precipitation biases. Results from the application to the headwater basin of the Aconcagua River showed how the forward model versus the fSCA-conditioned estimate resulted in different quantifications of the relationship between runoff and SWE, and different correlation patterns between pixel-wise SWE and ENSO. The illustrative results confirm the influence that ENSO has on snow accumulation for Andean basins draining into the Pacific, with ENSO explaining approximately 25% of the variability in near-peak (1 September) SWE values. Our results show how the assimilation of fSCA data results in a significant improvement upon MERRA-forced modeled SWE estimates, further increasing the utility of the MERRA data for high-resolution snow modeling applications.

SWE↗

Bias Correction in Urban Building Energy Modeling for Chicago Using Machine Learning

Urban-scale building energy modeling (UBEM) holds promise for optimizing energy usage across extensive geographic regions. However, there is a recognized bias between simulated energy consumption and actual measured data. This study, based on building data from Chicago, delved into bias correction techniques for enhancing the accuracy of UBEM energy consumption estimates. Initially, the AutoBEM simulation yielded a normalized mean bias error (NMBE) of 1.1% and 51% of Coefficient of the Variation of the Root Mean Square Error (CVRMSE) after outlier exclusion. To address this, three bias correction methods were deployed: Average Mean Bias Error based bias correction, Quantile mapping bias correction, and Machine learning-based bias correction using Linear Regression and Random Forest models. Post-correction results exhibited marked improvement. The NMBE values were diminished to 0 for Average MBE-based, 0.36 for Quantile Mapping, and 0 for Machine Learning-based corrections. Concurrently, the CVRMSE values registered reductions from an original 51 to 50.8 for Quantile Mapping, and 38.56 for Machine Learning-based corrections, pointing towards the effectiveness of specific bias correction methods in refining the precision of UBEM energy predictions. Such accurate estimations are paramount for informed energy planning and urban policy-making.

Chowdhury, Shovan↗

Entropy-Infused Deep Learning Loss Function for Capturing Extreme Values in Wind Power Forecasting

Extreme scenarios in wind power generation occur with higher frequency and larger magnitude in the recent years due to the ever-increasing extreme meteorological factors. Accurate forecasting of the occurrence of extreme values in wind power generation is of great concern to ensure reliable power system operation. Recently, deep learning models have surged in popularity for wind power forecasting, with the mean squared error (MSE) loss function being commonly used. However, the MSE loss function, being sensitive to extreme values, disproportionately penalizes larger errors, cannot adequately capture the extreme values present in wind energy data, and novel loss functions have seldom been tailored for wind power forecasting. To this end, in this paper, we introduce a novel loss function specifically crafted to capture extreme values in wind power forecasting. The experimental results with four fundamental deep learning methods on open source wind power dataset validate that the new loss function is efficient and superior in all cases compared to MSE in capturing extreme values while maintaining forecasting performance.

17 WIND ENERGY↗

Snow Depth from AMSR-2 Using Multispectral Satellite Data in an Artificial Neural Network

By using diffusion theory and Monte Carlo lidar radiative transfer simulations, Hu et al. (2022b) has derived snow depth from the first-, second- and third-order moments of the lidar backscattering pathlength distribution. Lu et al. (2022) calculated the snow depth by applying the methods to the satellite ICESat-2 lidar measurements over the Arctic sea ice, as well as land surfaces of Northern Hemisphere. In this paper, an artificial neural network (ANN) algorithm, employing several channels from Advanced Microwave Scanning Radiometer 2 (AMSR-2) and the humidity vertical profiles from Global Modeling and Assimilation Office (GMAO) Goddard Earth Observing System for Instrument Teams (GEOS-IT) product, is trained to determine snow depth identified by time and geolocation matched 2019 ICESat-2 snow-depth data during winter months over the Arctic sea ice. The trained ANN snow-depth was applied to 2018 AMSR-2 clear pixel data, although the algorithms perform reasonably well in thinner clouds. The validation data (different from the training set) of ANN snow depth from AMSR-2 showed a good agreement with time matched and co-located snow-depth values from ICESat-2. The bias was near zero, with mean absolute error (MAE) 0.05 cm and a root-mean-square-error (RMSE) 0.08 cm. Prior applying the trained ANN snow depth to AMSR-2 data, a cloud screening algorithm was developed with a similar approach. A separate ANN cloud mask was trained to determine an AMSR-2 pixel is clear or cloudy with time and geolocation matched 2015 CALIOP Vertical Feature Mask (VFM) over Arctic sea ice. The ANN cloud mask from AMSR-2 under-estimated cloud fraction by 3-6% compared to CALIOP . The additional research is needed to conclusively evaluate the ANN cloud mask accuracy. Finally, this paper will lay the foundation for a sustained long-term snowfall and snow-storm monitoring system. The future Cloud Aerosol LIdar for Global scale Observations of the ocean-Land Atmosphere system (CALIGOLA) mission will provide a means to calculate snow depth from the lidar backscattering pathlength distribution, benefiting from the UV, visible and infrared pulses. With the calculated snow depth as the truth one could develop a machine learning algorithm, as it was done in this paper, using a passive microwave instrument available at that time to generate a wide range of snow depth data, covering extensive spatial areas in the cross-orbit direction.

Neural Network↗

Soil Moisture Active Passive Mission L4_SM Data Product Assessment (Version 2 Validated Release)

During the post-launch SMAP calibration and validation (Cal/Val) phase there are two objectives for each science data product team: 1) calibrate, verify, and improve the performance of the science algorithm, and 2) validate the accuracy of the science data product as specified in the science requirements and according to the Cal/Val schedule. This report provides an assessment of the SMAP Level 4 Surface and Root Zone Soil Moisture Passive (L4_SM) product specifically for the product's public Version 2 validated release scheduled for 29 April 2016. The assessment of the Version 2 L4_SM data product includes comparisons of SMAP L4_SM soil moisture estimates with in situ soil moisture observations from core validation sites and sparse networks. The assessment further includes a global evaluation of the internal diagnostics from the ensemble-based data assimilation system that is used to generate the L4_SM product. This evaluation focuses on the statistics of the observation-minus-forecast (O-F) residuals and the analysis increments. Together, the core validation site comparisons and the statistics of the assimilation diagnostics are considered primary validation methodologies for the L4_SM product. Comparisons against in situ measurements from regional-scale sparse networks are considered a secondary validation methodology because such in situ measurements are subject to up-scaling errors from the point-scale to the grid cell scale of the data product. Based on the limited set of core validation sites, the wide geographic range of the sparse network sites, and the global assessment of the assimilation diagnostics, the assessment presented here meets the criteria established by the Committee on Earth Observing Satellites for Stage 2 validation and supports the validated release of the data. An analysis of the time average surface and root zone soil moisture shows that the global pattern of arid and humid regions are captured by the L4_SM estimates. Results from the core validation site comparisons indicate that "Version 2" of the L4_SM data product meets the self-imposed L4_SM accuracy requirement, which is formulated in terms of the ubRMSE: the RMSE (Root Mean Square Error) after removal of the long-term mean difference. The overall ubRMSE of the 3-hourly L4_SM surface soil moisture at the 9 km scale is 0.035 cubic meters per cubic meter requirement. The corresponding ubRMSE for L4_SM root zone soil moisture is 0.024 cubic meters per cubic meter requirement. Both of these metrics are comfortably below the 0.04 cubic meters per cubic meter requirement. The L4_SM estimates are an improvement over estimates from a model-only SMAP Nature Run version 4 (NRv4), which demonstrates the beneficial impact of the SMAP brightness temperature data. L4_SM surface soil moisture estimates are consistently more skillful than NRv4 estimates, although not by a statistically significant margin. The lack of statistical significance is not surprising given the limited data record available to date. Root zone soil moisture estimates from L4_SM and NRv4 have similar skill. Results from comparisons of the L4_SM product to in situ measurements from nearly 400 sparse network sites corroborate the core validation site results. The instantaneous soil moisture and soil temperature analysis increments are within a reasonable range and result in spatially smooth soil moisture analyses. The O-F residuals exhibit only small biases on the order of 1-3 degrees Kelvin between the (re-scaled) SMAP brightness temperature observations and the L4_SM model forecast, which indicates that the assimilation system is largely unbiased. The spatially averaged time series standard deviation of the O-F residuals is 5.9 degrees Kelvin, which reduces to 4.0 degrees Kelvin for the observation-minus-analysis (O-A) residuals, reflecting the impact of the SMAP observations on the L4_SM system. Averaged globally, the time series standard deviation of the normalized O-F residuals is close to unity, which would suggest that the magnitude of the modeled errors approximately reflects that of the actual errors. The assessment report also notes several limitations of the "Version 2" L4_SM data product and science algorithm calibration that will be addressed in future releases. Regionally, the time series standard deviation of the normalized O-F residuals deviates considerably from unity, which indicates that the L4_SM assimilation algorithm either over- or under-estimates the actual errors that are present in the system. Planned improvements include revised land model parameters, revised error parameters for the land model and the assimilated SMAP observations, and revised surface meteorological forcing data for the operational period and underlying climatological data. Moreover, a refined analysis of the impact of SMAP observations will be facilitated by the construction of additional variants of the model-only reference data. Nevertheless, the “Version 2” validated release of the L4_SM product is sufficiently mature and of adequate quality for distribution to and use by the larger science and application communities.

SMAP L4_SM↗