Search NASA⌕ Search

SEARCH · Search NASA

Results for “Error Metrics”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 163 records · Page 9

Preliminary Analysis of Facility Design for Pyro-processing Safeguardability

The deployment of advanced nuclear energy systems can assure sustainable energy use worldwide. For some of these systems, nuclear fuel will be fabricated in remotely-handled facilities in which batch-type processing will occur in heavily-shielded hot cells. Because special nuclear material is in both different physical and chemical form than that of the contemporary fuel cycle, new challenges have arisen in terms of safeguards, safety, and security, integrated with facility design; i.e., safeguardability. There is a need to develop models that can quantitatively assess safeguardability from a design-driven perspective. Previously, the first build of a material throughput model was established using discrete event simulation (DES) for the fuel fabrication process. Results showed that the DES model potentially can detect diversion attempts. This paper continues the study by further developing the the model. We test the use of the probability of the false alarm; i.e., the Type I error, as a metric for safeguardability assessment. Initiating events leading to a false alarm arise from aspects of safeguards, safety, and security. Different conceptual facility designs could exhibit different frequencies for the Type I error, and this could provide a measure of the safeguardability of each design. A quantitative assessment for safeguardable facility designs has not been developed in this manner. Much of the research into safeguardability has been qualitative. Engineering design approaches for a safeguardable facility are now needed. As there are not accepted metrics for safeguardability assessment, our motivation is to develop this design facet of safeguardability further.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗

Increasing the Measured Effective Quantum Volume with Zero Noise Extrapolation

Quantum volume is a full-stack benchmark for near-term quantum computers. It quantifies the largest size of a square circuit which can be executed on the target device with reasonable fidelity. Error mitigation is a set of techniques intended to remove the effects of noise present in the computation of noisy quantum computers when computing an expectation value of interest. Effective quantum volume is a proposed metric that applies error mitigation to the quantum volume protocol to evaluate the effectiveness not only of the target device but also of the error mitigation algorithm. Digital zero-noise extrapolation is an error mitigation technique that estimates the noiseless expectation value using circuit folding to amplify errors by known scale factors and then extrapolating computed expectation values to the zero-noise limit. Here we demonstrate that zero-noise extrapolation, with global and local unitary folding with fractional scale factors, in conjunction with dynamical decoupling, can increase the effective quantum volume over the vendor-measured quantum volume. Specifically, we measure the effective quantum volume of four IBM Quantum superconducting processor units, obtaining values that are larger than the vendor-measured quantum volume on each device. This is the first such increase reported.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Performance-based assessment of reconstructed images

During the early 90s, I engaged in a productive and enjoyable collaboration with Robert Wagner and his colleague, Kyle Myers. We explored the ramifications of the principle that tbe quality of an image should be assessed on the basis of how well it facilitates the performance of appropriate visual tasks. We applied this principle to algorithms used to reconstruct scenes from incomplete and/or noisy projection data. For binary visual tasks, we used both the conventional disk detection and a new challenging task, inspired by the Rayleigh resolution criterion, of deciding whether an object was a blurred version of two dots or a bar. The results of human and machine observer tests were summarized with the detectability index based on the area under the ROC curve. We investigated a variety of reconstruction algorithms, including ART, with and without a nonnegativity constraint, and the MEMSYS3 algorithm. We concluded that the performance of the Raleigh task was optimized when the strength of the prior was near MEMSYS's default 'classic' value for both human and machine observers. A notable result was that the most-often-used metric of rms error in the reconstruction was not necessarily indicative of the value of a reconstructed image for the purpose of performing visual tasks.

99 GENERAL AND MISCELLANEOUS↗

An Analysis of Plan Robustness for Esophageal Tumors: Comparing Volumetric Modulated Arc Therapy Plans and Spot Scanning Proton Planning

Purpose: Planning studies to compare x-ray and proton techniques and to select the most suitable technique for each patient have been hampered by the nonequivalence of several aspects of treatment planning and delivery. A fair comparison should compare similarly advanced delivery techniques from current clinical practice and also assess the robustness of each technique. The present study therefore compared volumetric modulated arc therapy (VMAT) and single-field optimization (SFO) spot scanning proton therapy plans created using a simultaneous integrated boost (SIB) for dose escalation in midesophageal cancer and analyzed the effect of setup and range uncertainties on these plans. Methods and Materials: For 21 patients, SIB plans with a physical dose prescription of 2 Gy or 2.5 Gy/fraction in 25 fractions to planning target volume (PTV){sub 50Gy} or PTV{sub 62.5Gy} (primary tumor with 0.5 cm margins) were created and evaluated for robustness to random setup errors and proton range errors. Dose–volume metrics were compared for the optimal and uncertainty plans, with P<.05 (Wilcoxon) considered significant. Results: SFO reduced the mean lung dose by 51.4% (range 35.1%-76.1%) and the mean heart dose by 40.9% (range 15.0%-57.4%) compared with VMAT. Proton plan robustness to a 3.5% range error was acceptable. For all patients, the clinical target volume D{sub 98} was 95.0% to 100.4% of the prescribed dose and gross tumor volume (GTV) D{sub 98} was 98.8% to 101%. Setup error robustness was patient anatomy dependent, and the potential minimum dose per fraction was always lower with SFO than with VMAT. The clinical target volume D{sub 98} was lower by 0.6% to 7.8% of the prescribed dose, and the GTV D{sub 98} was lower by 0.3% to 2.2% of the prescribed GTV dose. Conclusions: The SFO plans achieved significant sparing of normal tissue compared with the VMAT plans for midesophageal cancer. The target dose coverage in the SIB proton plans was less robust to random setup errors and might be unacceptable for certain patients. Robust optimization to ensure adequate target coverage of SIB proton plans might be beneficial.

62 RADIOLOGY AND NUCLEAR MEDICINE↗

Modeling land use and land cover change: using a hindcast to estimate economic parameters in gcamland v2.0

Abstract. Future changes in land use and cover have important implications for agriculture, energy, water use, and climate. Estimates of future land use and land cover differ significantly across economic models as a result of differences in drivers, model structure, and model parameters; however, these models often rely on heuristics to determine model parameters. In this study, we demonstrate a more systematic and empirically based approach to estimating a few key parameters for an economic model of land use and land cover change, gcamland. Specifically, we generate a large set of model parameter perturbations for the selected parameters and run gcamland simulations with these parameter sets over the historical period in the United States to quantify land use and land cover, determine how well the model reproduces observations, and identify parameter combinations that best replicate observations, assuming other model parameters are fixed. We also test alternate methods for forming expectations about uncertain crop yields and prices, including adaptive, perfect, linear, and hybrid approaches. In particular, we estimate parameters for six parameters used in the formation of expectations and three of seven logit exponents for the USA only. We find that an adaptive expectation approach minimizes the error between simulated outputs and observations, with parameters that suggest that for most crops, landowners put a significant weight on previous information. Interestingly, for corn, where ethanol policies have led to a rapid growth in demand, the resulting parameters show that a larger weight is placed on more recent information. We examine the change in model parameters as the metric of model error changes, finding that the measure of model fitness affects the choice of parameter sets. Finally, we discuss how the methodology and results used in this study could be used for other regions or economic models to improve projections of future land use and land cover change.

54 ENVIRONMENTAL SCIENCES↗

Casting the Coronal Magnetic Field Reconstruction Tools in 3D Using the MHD Bifrost Model

Quantifying the coronal magnetic field remains a central problem in solar physics. Nowadays, the coronal magnetic field is often modeled using nonlinear force-free field (NLFFF) reconstructions, whose accuracy has not yet been comprehensively assessed. Here we perform a detailed casting of the NLFFF reconstruction tools, such as π -disambiguation, photospheric field preprocessing, and volume reconstruction methods, using a 3D snapshot of the publicly available full-fledged radiative MHD model. Specifically, from the MHD model, we know the magnetic field vector in the entire 3D domain, which enables us to perform a “voxel-by-voxel” comparison of the restored and the true magnetic fields in the 3D model volume. Our tests show that the available π -disambiguation methods often fail in the quiet-Sun areas dominated by small-scale magnetic elements, while they work well in the active region (AR) photosphere and (even better) chromosphere. The preprocessing of the photospheric magnetic field, although it does produce a more force-free boundary condition, also results in some effective “elevation” of the magnetic field components. This “elevation” height is different for the longitudinal and transverse components, which results in a systematic error in absolute heights in the reconstructed magnetic data cube. The extrapolations performed starting from the actual AR photospheric magnetogram are free from this systematic error, while other metrics are comparable with those for extrapolations from the preprocessed magnetograms. This finding favors the use of extrapolations from the original photospheric magnetogram without preprocessing. Our tests further suggest that extrapolations from a force-free chromospheric boundary produce measurably better results than those from a photospheric boundary.

79 ASTRONOMY AND ASTROPHYSICS↗

Analyzing the Deployment of Large Amounts of Offshore Wind to Design an Offshore Transmission Grid in the United States: Preprint

This paper revisits the results from the U.S. Department of Energy's '20% Wind Energy By 2030' study, which envisioned that 54 GW of offshore wind would be installed by said year. The analysis is conducted using the Regional Energy Deployment System (ReEDS), a capacity expansion model developed by the National Renewable Energy Laboratory. The model is used to optimize the deployment of the 54 GW of wind capacity along the coasts and lakes of the United States. The graphical representation of the results through maps will be used to provide a qualitative description for planning and designing an offshore grid. ReEDS takes into account many factors in the process of siting offshore wind capacity, such as the quality of the resource, capital and O&M costs, interconnection costs, or variability metrics (wind capacity value, forecast error, expected curtailment). The effect of these metrics in the deployment of offshore wind will be analyzed through examples in the results.

17 WIND ENERGY↗

SU-G-JeP1-15: Sliding Window Prior Data Assisted Compressed Sensing for MRI Lung Tumor Tracking

Purpose: Prior Data Assisted Compressed Sensing (PDACS) is a partial k-space acquisition and reconstruction method for mobile tumour (i.e. lung) tracking using on-line MRI in radiotherapy. PDACS partially relies on prior data acquired at the beginning of dynamic scans, and is therefore susceptible to artifacts in longer duration scan due to slow drifts in MR signal. A novel sliding window strategy is presented to mitigate this effect. Methods: MRI acceleration is simulated by retrospective removal of data from the fully sampled sets. Six lung cancer patients were scanned (clinical 3T MRI) using a balanced steady state free precession (bSSFP) sequence for 3 minutes at approximately 4 frames per second, for a total of 650 dynamics. PDACS acceleration is achieved by undersampling of k-space in a single pseudo-random pattern. Reconstruction iteratively minimizes the total variations while constraining the images to satisfy both the currently acquired data and the prior data in missing k-space. Our novel sliding window technique (SW-PDACS), uses a series of distinct pseudo-random under-sampling patterns of partial k-space – with the prior data drawn from a sliding window of the most recent data available. Under-sampled data, simulating 2 – 5x acceleration are reconstructed using PDACS and SW-PDACS. Three quantitative metrics: artifact power, centroid error and Dice’s coefficient are computed for comparison. Results: Quantitively metric values from all 6 patients are averaged in 3 bins, each containing approximately one minute of dynamic data. For the first minute bin, PDACS and SW-PDACS give comparable results. Progressive decline in image quality metrics in bins 2 and 3 are observed for PDACS. No decline in image quality is observed for SW-PDACS. Conclusion: The novel approach presented (SW-PDACS) is a more robust for accelerating longer duration (>1 minute) dynamic MRI scans for tracking lung tumour motion using on-line MRI in radiotherapy. B.G. Fallone is a co-founder and CEO of MagnetTx Oncology Solutions (under discussions to license Alberta bi-planar linac MR for commercialization).

60 APPLIED LIFE SCIENCES↗

Improving the MODIS LAI Compositing Using Prior Time-Series Information

The Moderate Resolution Imaging Spectroradiometer (MODIS) long-term leaf area index (LAI) products have significantly contributed to global energy fluxes, climate change, and biogeochemistry research. However, the maximum fraction of photosynthetically active radiation absorbed by vegetation (Max-FPAR) compositing strategy of the Collection 6 (C6) products dictates that the main or backup algorithm is always triggered by observations of different quality, which indirectly causes the observed instability in the LAI time-series. Based on MODIS daily LAI retrievals, this study develops a prior knowledge time-series compositing algorithm (PKA) using a linear kernel driven (LKD) model. Our results show that the newly proposed PKA can significantly improve the LAI composites compared to the Max-FPAR strategy using ground-based observations for validation. We found that the PKA performs better than Max-FPAR in various aspects (different sites, seasons, and retrieval index (RI) ranges), with R2 increasing from 0.69 to 0.76 and root means square error (RMSE) decreasing from 1.01 to 0.84 compared to GBOV ground truth. The same improvement was shown for the ground truth LAIs measured at the Honghe and Hailun sites in northeastern China, with R2 increasing from 0.23 to 0.41 and RMSE decreasing from 1.27 to 1.25. In addition, three newly proposed temporal uncertainty metrics (time-series stability, TSS and time-series anomaly, TSA and reconstruction error metric, RE (the proximity to the main RT-based retrievals)) were applied to compare the stability of LAI time-series before and after PKA implementation. We found that the time series stability of PKA LAI was improved, the time series anomalies were reduced, and the retrieval rates of the main algorithm were also greatly enhanced compared to Max-FPAR LAI. A case intercomparison for Max-FPAR-MODIS, Max-FPAR-VIIRS (Visible Infrared Imager Radiometer Suite), and PKA-MODIS LAIs in the Amazon Forest region showed that the PKA is also effective in improving the LAI retrieval over large regions with few qualified observations due to poor atmospheric conditions (RE decreased from 2.37/2.35 (Max-FPAR-MODIS/Max-FPAR-VIIRS) to 2.25 (PKA-MODIS) and RI increased from 61.94%/59.62% to 66.88%). The same improvement was seen in the BELMANIP 2.1 sites for almost all biomes except deciduous broadleaf forest, where the RE decreased from 1.85/2.13 to 1.15 overall. We note that the PKA has the potential to be easily implemented in the operational algorithms of subsequent MODIS and MODIS-like LAI Collections.

MODIS↗

GPU-accelerated denoising of 3D magnetic resonance images

The raw computational power of GPU accelerators enables fast denoising of 3D MR images using bilateral filtering, anisotropic diffusion, and non-local means. In practice, applying these filtering operations requires setting multiple parameters. This study was designed to provide better guidance to practitioners for choosing the most appropriate parameters by answering two questions: what parameters yield the best denoising results in practice? And what tuning is necessary to achieve optimal performance on a modern GPU? To answer the first question, we use two different metrics, mean squared error (MSE) and mean structural similarity (MSSIM), to compare denoising quality against a reference image. Surprisingly, the best improvement in structural similarity with the bilateral filter is achieved with a small stencil size that lies within the range of real-time execution on an NVIDIA Tesla M2050 GPU. Moreover, inappropriate choices for parameters, especially scaling parameters, can yield very poor denoising performance. To answer the second question, we perform an autotuning study to empirically determine optimal memory tiling on the GPU. The variation in these results suggests that such tuning is an essential step in achieving real-time performance. These results have important implications for the real-time application of denoising to MR images in clinical settings that require fast turn-around times.

97 MATHEMATICS AND COMPUTING↗

Review of Onsite Temperature and Solar Forecasting Models to Enable Better Building Design and Operations

Advanced building controls and energy optimization for new constructions and retrofits rely on accurate weather data. Traditionally, most studies utilize airport weather information as the decision inputs. However, most buildings are in environments that are quite different than those at the airport miles away. Tree cover, adjacent buildings, and micro-climate effects caused by the larger surrounding area can all yield deviations in air temperature, humidity, solar irradiance, and wind that are large enough to influence design and operation decisions. In order to overcome this challenge, there are many prior studies on developing weather forecasting algorithms from micro-to meso-scales. Additionally, this paper reviews and complies knowledge on common weather data resources, data processing methodologies and forecasting techniques of weather information. Commonly used statistical, machine learning and physical-based models are discussed and presented as two major categories: deterministic forecasting and probabilistic forecasting. Finally, evaluation metrics for forecasting errors are listed and discussed.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

Dependence of neoclassical toroidal viscosity on the poloidal spectrum of applied nonaxisymmetric fields

This paper presents a single mode model that accurately predicts the coupling of applied nonaxisymmetric fields to the plasma response that induces neoclassical toroidal viscosity (NTV) torque in DIII-D H-mode plasmas. The torque is measured and modeled to have a sinusoidal dependence on the relative phase of multiple nonaxisymmetric field sources, including a minimum in which large amounts of nonaxisymmetric drive is decoupled from the NTV torque. This corresponds to the coupling and decoupling of the applied field to a NTV-driving mode spectrum. Modeling using the perturbed equilibrium nonambipolar transport (PENT) code confirms an effective single mode coupling between the applied field and the resultant torque, despite its inherent nonlinearity. Lastly, the coupling to the NTV mode is shown to have a similar dependence on the relative phasing as that of the IPEC dominant mode, providing a physical basis for the efficacy of this linear metric in predicting error field correction optima in NTV dominated regimes.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

Tuning the Interpolation Basis in a Multigrid Decomposition for Local Error Control

In the compression of scientific data, error-controlled compressors enable to considerably decrease the size of the dataset while maintaining adequate levels of accuracy. In this paper, we note that multi-level refactoring scheme such as MGARD i) rely on an approximation of the data based on the interpolation of coefficients, ii) estimate the resulting error with global metrics on the dataset. To improve on these two aspects, we propose a method that aims to divide the original dataset into blocks based on their smoothness and refactors each block separately with the most relevant interpolation order. We show the relevance of such a method on tailored datasets and the benefits and challenges when applying it to large scientific data.

Vidal, Nicolas [ORNL]↗

Tracking tumor boundary in MV-EPID images without implanted markers: A feasibility study

Purpose: To develop a markerless tracking algorithm to track the tumor boundary in megavoltage (MV)-electronic portal imaging device (EPID) images for image-guided radiation therapy. Methods: A level set method (LSM)-based algorithm is developed to track tumor boundary in EPID image sequences. Given an EPID image sequence, an initial curve is manually specified in the first frame. Driven by a region-scalable energy fitting function, the initial curve automatically evolves toward the tumor boundary and stops on the desired boundary while the energy function reaches its minimum. For the subsequent frames, the tracking algorithm updates the initial curve by using the tracking result in the previous frame and reuses the LSM to detect the tumor boundary in the subsequent frame so that the tracking processing can be continued without user intervention. The tracking algorithm is tested on three image datasets, including a 4-D phantom EPID image sequence, four digitally deformable phantom image sequences with different noise levels, and four clinical EPID image sequences acquired in lung cancer treatment. The tracking accuracy is evaluated based on two metrics: centroid localization error (CLE) and volume overlap index (VOI) between the tracking result and the ground truth. Results: For the 4-D phantom image sequence, the CLE is 0.23 ± 0.20 mm, and VOI is 95.6% ± 0.2%. For the digital phantom image sequences, the total CLE and VOI are 0.11 ± 0.08 mm and 96.7% ± 0.7%, respectively. In addition, for the clinical EPID image sequences, the proposed algorithm achieves 0.32 ± 0.77 mm in the CLE and 72.1% ± 5.5% in the VOI. These results demonstrate the effectiveness of the authors’ proposed method both in tumor localization and boundary tracking in EPID images. In addition, compared with two existing tracking algorithms, the proposed method achieves a higher accuracy in tumor localization. Conclusions: In this paper, the authors presented a feasibility study of tracking tumor boundary in EPID images by using a LSM-based algorithm. Experimental results conducted on phantom and clinical EPID images demonstrated the effectiveness of the tracking algorithm for visible tumor target. Compared with previous tracking methods, the authors’ algorithm has the potential to improve the tracking accuracy in radiation therapy. In addition, real-time tumor boundary information within the irradiation field will be potentially useful for further applications, such as adaptive beam delivery, dose evaluation.

60 APPLIED LIFE SCIENCES↗

Detailed Hydrographic Feature Extraction from High-Resolution LiDAR Data

Detailed hydrographic feature extraction from high-resolution light detection and ranging (LiDAR) data is investigated. Methods for quantitatively evaluating and comparing such extractions are presented, including the use of sinuosity and longitudinal root-mean-square-error (LRMSE). These metrics are then used to quantitatively compare stream networks in two studies. The first study examines the effect of raster cell size on watershed boundaries and stream networks delineated from LiDAR-derived digital elevation models (DEMs). The study confirmed that, with the greatly increased resolution of LiDAR data, smaller cell sizes generally yielded better stream network delineations, based on sinuosity and LRMSE. The second study demonstrates a new method of delineating a stream directly from LiDAR point clouds, without the intermediate step of deriving a DEM. Direct use of LiDAR point clouds could improve efficiency and accuracy of hydrographic feature extractions. The direct delineation method developed herein and termed “mDn”, is an extension of the D8 method that has been used for several decades with gridded raster data. The method divides the region around a starting point into sectors, using the LiDAR data points within each sector to determine an average slope, and selecting the sector with the greatest downward slope to determine the direction of flow. An mDn delineation was compared with a traditional grid-based delineation, using TauDEM, and other readily available, common stream data sets. Although, the TauDEM delineation yielded a sinuosity that more closely matches the reference, the mDn delineation yielded a sinuosity that was higher than either the TauDEM method or the existing published stream delineations. Furthermore, stream delineation using the mDn method yielded the smallest LRMSE.

58 GEOSCIENCES↗

Deployable reflector antenna performance optimization using automated surface correction and array-feed compensation

Methods for increasing the electromagnetic (EM) performance of reflectors with rough surfaces were tested and evaluated. First, one quadrant of the 15-meter hoop-column antenna was retrofitted with computer-driven and controlled motors to allow automated adjustment of the reflector surface. The surface errors, measured with metric photogrammetry, were used in a previously verified computer code to calculate control motor adjustments. With this system, a rough antenna surface (rms of approximately 0.180 inch) was corrected in two iterations to approximately the structural surface smoothness limit of 0.060 inch rms. The antenna pattern and gain improved significantly as a result of these surface adjustments. The EM performance was evaluated with a computer program for distorted reflector antennas which had been previously verified with experimental data. Next, the effects of the surface distortions were compensated for in computer simulations by superimposing excitation from an array feed to maximize antenna performance relative to an undistorted reflector. Results showed that a 61-element array could produce EM performance improvements equal to surface adjustments. When both mechanical surface adjustment and feed compensation techniques were applied, the equivalent operating frequency increased from approximately 6 to 18 GHz.

Schroeder, Lyle C.↗

LUVOIR Primary Mirror Segment Alignment Control With Joint Laser Metrology and Segment Edge Sensing

An approach is developed for the alignment and stability maintenance of the LUVOIR segmented primary mirror using a segment state estimation and wavefront control method based on a hybrid segment motion sensing architecture of laser truss metrology and segment edge sensors. Our current computer model was generated for LUVOIR Architecture Option A with a 15-meter aperture, 120segment primary mirror. The methodology and simulation results will be presented and analyzed. JPL has a long history of technology development in laser metrology and edge sensors, including work in SIM [7], Keck and TMT [8], CCAT [3] and LUVOIR [1]. We will discuss our current efforts of LUVOIR laser metrology and edge-sensor models development, showing sensitivities of sensor measurements to various mirror eigenmodes, removing global modes and strengthening weak modes by performing joint (hybrid) laser-metrology and edge sensing. We will define and derive an important performance metric called wavefront error multiplier (WEM), and show that WEM provides a simple link between sensor errors and the closed-loop (controlled) system wavefront error. We will show WEM values for several hybrid sensor configuration options studied. We will discuss an algorithm for mirror shape control and maintenance through segment state and wavefront estimations using joint edge-metrology sensing. We will compare simulated performance of mirror state estimation, wavefront estimation and wavefront control based on joint edge-metrology sensing among several sensor configurations, and show the impact of sensor error distributions on the segmented mirror alignment performance. Mirror shape control performance will be also be evaluated in terms of imaging contrast between inner working angles (IWA) and outer working angles (OWA) of a LUVOIR coronagraph. The research was carried out at the Jet Propulsion Laboratory, California Institute of Technology, under a contract with the National Aeronautics and Space Administration. LUVOIR is a mission concept under study for the next astronomy Decadal Survey—the information presented in this paper is pre-decisional and is provided for planning and discussion purposes only.

Shelton, Chris↗