Search NASA⌕ Search

SEARCH · Search NASA

Results for “Missing data”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 73 records · Page 4

LANL Meteorological Program: 2023 Data Completeness/Quality Report

Los Alamos National Laboratory (LANL) operates seven mesa-top instrumented meteorology towers: Technical Area (TA) 6, TA-49, TA-53, TA-54, TA-63, TA-54B, and TA-16B. An additional instrumented tower is located in Mortandad Canyon (TA-5 MDCN), and there is a rain gauge at North Community (NCOM), located within the town of Los Alamos. The 10 meter (m) towers at TA-63, TA-54B, and TA-16B have been in testing since they were installed in 2021, and will be included in a future data completeness report. A description of the meteorology monitoring network, prior to the installation of the TA-63, TA-54B, and TA-16B is found in Dewart and Boggs (2014). Four of the mesa-top towers (e.g., TA-6, TA-49, TA-53, and TA-54) are instrumented at the 1.2 m, 11.5 m, 23 m, and 46 m levels. In addition, the TA-6 tower is instrumented at the 92 m level. The TA-5 MDCN tower is 10 m in height and is instrumented at 1.2 m and 10 m. Data are collected and averaged every 15 minutes. Range checking is performed on each measurement every 15 minutes; data that are beyond normal ranges are eliminated from the data set and replaced by a code for missing data. In addition, data are reviewed weekly by qualified meteorologists to identify bad data not identified by the range checking technique. The data steward eliminates these data from the data set and replaces them with a code for missing data. The instrument technicians also review that data and schedule instrument replacement, as required. All instruments are calibrated at a frequency that meets the criteria identified in ANSI/ANS-3.11-2015. Data completeness is determined by the number of total 15-minute records available versus the number of possible measurements for the entire year. As a rule, the meteorologists do not attempt to estimate data that are eliminated as bad data. Original datalogger records, including bad data, can be recalled from program archival storage.

54 ENVIRONMENTAL SCIENCES↗

Recovering missing CFD data for high-order discretizations using deep neural networks and dynamics learning

Data I/O poses a significant bottleneck in large-scale CFD simulations; thus, practitioners would like to significantly reduce the number of times the solution is saved to disk, yet retain the ability to recover any field quantity (at any time instance) a posteriori . The objective of this work is therefore to accurately recover missing CFD data a posteriori at any time instance, given that the solution has been written to disk at only a relatively small number of time instances. We consider in particular high-order discretizations (e.g., discontinuous Galerkin), as such techniques are becoming increasingly popular for the simulation of highly separated flows. To satisfy this objective, this work proposes a methodology consisting of two stages: 1) dimensionality reduction and 2) dynamics learning. For dimensionality reduction, we propose a novel hierarchical approach. First, the method reduces the number of degrees of freedom within each element of the high-order discretization by applying autoencoders from deep learning. Second, the methodology applies principal component analysis to compress the global vector of encodings. This leads to a low-dimensional state, which associates with a nonlinear embedding of the original CFD data. For dynamics learning, we propose to apply regression techniques (e.g., kernel methods) to learn the discrete-time velocity characterizing the time evolution of this low-dimensional state. In conclusion, a numerical example on a large-scale CFD example characterized by nearly 13 million degrees of freedom illustrates the suitability of the proposed method in an industrial setting.

97 MATHEMATICS AND COMPUTING↗

WE-G-18A-03: Cone Artifacts Correction in Iterative Cone Beam CT Reconstruction

Purpose: For iterative reconstruction (IR) in cone-beam CT (CBCT) imaging, data truncation along the superior-inferior (SI) direction causes severe cone artifacts in the reconstructed CBCT volume images. Not only does it reduce the effective SI coverage of the reconstructed volume, it also hinders the IR algorithm convergence. This is particular a problem for regularization based IR, where smoothing type regularization operations tend to propagate the artifacts to a large area. It is our purpose to develop a practical cone artifacts correction solution. Methods: We found it is the missing data residing in the truncated cone area that leads to inconsistency between the calculated forward projections and measured projections. We overcome this problem by using FDK type reconstruction to estimate the missing data and design weighting factors to compensate the inconsistency caused by the missing data. We validate the proposed methods in our multi-GPU low-dose CBCT reconstruction system on multiple patients' datasets. Results: Compared to the FDK reconstruction with full datasets, while IR is able to reconstruct CBCT images using a subset of projection data, the severe cone artifacts degrade overall image quality. For head-neck case under a full-fan mode, 13 out of 80 slices are contaminated. It is even more severe in pelvis case under half-fan mode, where 36 out of 80 slices are affected, leading to inferior soft-tissue delineation. By applying the proposed method, the cone artifacts are effectively corrected, with a mean intensity difference decreased from ∼497 HU to ∼39HU for those contaminated slices. Conclusion: A practical and effective solution for cone artifacts correction is proposed and validated in CBCT IR algorithm. This study is supported in part by NIH (1R01CA154747-01)

60 APPLIED LIFE SCIENCES↗

A review of imputation strategies for isobaric labeling-based shotgun proteomics

The throughput efficiency and increased depth of coverage provided by isobaric-labeled proteomics measurements have led to increased usage of these techniques. However, the structure of missing data is uniquely different than unlabeled studies. In this review, we compare the efficacy of nine imputation methods on a CPTAC proteomics iTRAQ dataset. Imputation methods were evaluated with regard to accuracy, variability, statistical hypothesis test inference and run time over datasets consisting of varying number of iTRAQ plexes and percentages of missing data. In general, expectation maximization and random forest imputation methods yielded the best performances, and constant-based methods performed poorly consistently across all dataset sizes and percentages of missing values. For datasets with small sample sizes and higher percentages of missing data, results indicate that statistical inference with no imputation may be preferable. Based on the findings in this review, there are core imputation methods that perform higher for isobaric-labeled proteomics data, but great care and consideration as to whether imputation should be used should be given for datasets comprised of a small number of samples, as well as to factors such as computational time and reproducibility of imputation values.

Bramer, Lisa M.↗

A Pattern-Recognition-Based Ensemble Data Imputation Framework for Sensors from Building Energy Systems

Building operation data are important for monitoring, analysis, modeling, and control of building energy systems. However, missing data is one of the major data quality issues, making data imputation techniques become increasingly important. There are two key research gaps for missing sensor data imputation in buildings: the lack of customized and automated imputation methodology, and the difficulty of the validation of data imputation methods. In this paper, a framework is developed to address these two gaps. First, a validation data generation module is developed based on pattern recognition to create a validation dataset to quantify the performance of data imputation methods. Second, a pool of data imputation methods is tested under the validation dataset to find an optimal single imputation method for each sensor, which is termed as an ensemble method. The method can reflect the specific mechanism and randomness of missing data from each sensor. The effectiveness of the framework is demonstrated by 18 sensors from a real campus building. The overall accuracy of data imputation for those sensors improves by 18.2% on average compared with the best single data imputation method.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

Integration of ultra-low coverage whole-genome sequences for reconstructing the evolutionary history of Galapagos giant tortoises

Genomic data from contemporary and historical samples often need to be coupled for evolutionary reconstructions of multitaxon complexes. However, the genetic data recovered from historical samples may result only in ultra-low coverage whole-genome sequences (ulcWGS; <0.15× depth), leading to inaccurate evolutionary inferences given a preponderance of missing data. Using the Galapagos giant tortoise radiation as a study system (Chelonoidis spp., composed of 13 extant and four extinct lineages), we assembled a novel methodological pipeline that removes potential noise introduced by the missing data and enhances the evolutionary signal from ulcWGS samples. We leveraged existing tools for phylogenomic placement (EPA-ng), population genomic structure (smartsnp) and admixture (Admixfrog, NGSadmix) to demonstrate that the evolutionary history of samples can be uncovered with sequencing depths as low as 0.008–0.139×. Importantly, these approaches do not use genotype imputation of the ulcWGS samples, which would require extensive reference datasets. Our application to two cases of extinct lineages of Galapagos giant tortoises, with and without references from the same lineage, demonstrates the general value of the approach. We confirm where the extinct lineages from San Cristóbal and Santa Fe islands fit into the Galapagos giant tortoise radiation, and that these lineages were evolutionarily distinct entities.

ancient DNA↗

Electric Field Data from the North Slope of Alaska

File contains raw as well as calibrated electric field values (V/m) from two CS110 electric field meters. The file also contains raw 2-m U and V winds at the same site. The sampling rate is 1Hz. All files contain raw electric field data, as well as calibrated values to a one week ground-based calibration setup. During this week, simultaneous upward-facing ground based electric field measurements were taken alongside the two downward facing instruments on the pole (CS110-3 at 2m and CS110-2 at 5m). Slopes and intercepts were calculated and applied to the values influenced by the pole to convert the raw data to calibrated values. These calibrated values are absolute electric field measurements. There are no missing data codes. The missong data is not included in the files, and thus any files with missing data will be shorter than the normal 86,400 values. In July 2023, the CS110-3 was replaced with an identical instrument named CS110-8, and the CS110-2 was replaced with an identical instrument called CS110-9. All the variable names remained consistent and the same.

54 ENVIRONMENTAL SCIENCES↗

A cross-sectional study of the temporal evolution of electricity consumption of six commercial buildings

Current approaches to building efficiency diagnoses include conventional energy audit techniques that can be expensive and time consuming. In contrast, virtual energy audits of readily available 15-minute-interval building electricity consumption are being explored to provide quick, inexpensive, and useful insights into building operation characteristics. A cross sectional analysis of six buildings in two different climate zones provides methods for data cleaning, population-based building comparisons, and relationships (correlations) of weather and electricity consumption. Data cleaning methods have been developed to categorize and appropriately filter or correct anomalous data including outliers, missing data, and erroneous values (resulting in < 0.5% anomalies). The utility of a cross-sectional analysis of a sample set of building's electricity consumption is found through comparisons of baseload, daily consumption variance, and energy use intensity. Correlations of weather and electricity consumption 15-minute interval datasets show important relationships for the heating and cooling seasons using computed correlations of a Time-Specific-Averaged- Ordered Variable (exterior temperature) and corresponding averaged variables (electricity consumption)(TSAOV method). The TSAOV method is unique as it introduces time of day as a third variable while also minimizing randomness in both correlated variables through averaging. This study found that many of the pair-wise linear correlation analyses lacked strong relationships, prompting the development of the new TSAOV method to uncover the causal relationship between electricity and weather. We conclude that a combination of varied HVAC system operations, building thermal mass, plug load use, and building set point temperatures are likely responsible for the poor correlations in the prior studies, while the correlation of time-specific-averaged-ordered temperature and corresponding averaged variables method developed herein adequately accounts for these issues and enables discovery of strong linear pair-wise correlation R values. TSAOV correlations lay the foundation for a new approach to building studies, that mitigates plug load interferences and identifies more accurate insights into weather-energy relationship for all building types. Over all six buildings analyzed the TSAOV method reported very significant average correlations per building of 0.94 to 0.82 in magnitude. Our rigorous statistics-based methods applied to 15- minute-interval electricity data further enables virtual energy audits of buildings to quickly and inexpensively inform energy savings measures.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

PHOTOMETRIC REDSHIFTS AND QUASAR PROBABILITIES FROM A SINGLE, DATA-DRIVEN GENERATIVE MODEL

We describe a technique for simultaneously classifying and estimating the redshift of quasars. It can separate quasars from stars in arbitrary redshift ranges, estimate full posterior distribution functions for the redshift, and naturally incorporate flux uncertainties, missing data, and multi-wavelength photometry. We build models of quasars in flux-redshift space by applying the extreme deconvolution technique to estimate the underlying density. By integrating this density over redshift, one can obtain quasar flux densities in different redshift ranges. This approach allows for efficient, consistent, and fast classification and photometric redshift estimation. This is achieved by combining the speed obtained by choosing simple analytical forms as the basis of our density model with the flexibility of non-parametric models through the use of many simple components with many parameters. We show that this technique is competitive with the best photometric quasar classification techniques-which are limited to fixed, broad redshift ranges and high signal-to-noise ratio data-and with the best photometric redshift techniques when applied to broadband optical data. We demonstrate that the inclusion of UV and NIR data significantly improves photometric quasar-star separation and essentially resolves all of the redshift degeneracies for quasars inherent to the ugriz filter system, even when included data have a low signal-to-noise ratio. For quasars spectroscopically confirmed by the SDSS 84% and 97% of the objects with Galaxy Evolution Explorer UV and UKIDSS NIR data have photometric redshifts within 0.1 and 0.3, respectively, of the spectroscopic redshift; this amounts to about a factor of three improvement over ugriz-only photometric redshifts. Our code to calculate quasar probabilities and redshift probability distributions is publicly available.

79 ASTRONOMY AND ASTROPHYSICS↗

Genome-Wide Association Study Based on Multiple Imputation with Low-Depth Sequencing Data: Application to Biofuel Traits in Reed Canarygrass

Genotyping by sequencing allows for large-scale genetic analyses in plant species with no reference genome, but sets the challenge of sound inference in presence of uncertain genotypes. We report an imputation-based genome-wide association study (GWAS) in reed canarygrass (Phalaris arundinacea L., Phalaris caesia Nees), a cool-season grass species with potential as a biofuel crop. Our study involved two linkage populations and an association panel of 590 reed canarygrass genotypes. Plants were assayed for up to 5228 single nucleotide polymorphism markers and 35 traits. The genotypic markers were derived from lowdepth sequencing with 78% missing data on average. To soundly infer marker-trait associations, multiple imputation (MI) was used: several imputes of the marker data were generated to reflect imputation uncertainty and association tests were performed on marker effects across imputes. A total of nine significant markers were identified, three of which showed significant homology with the Brachypodium dystachion genome. Because no physical map of the reed canarygrass genome was available, imputation was conducted using classification trees. In general, MI showed good consistency with the complete-case analysis and adequate control over imputation uncertainty. A gain in significance of marker effects was achieved through MI, but only for rare cases when missing data were ,45%. In addition to providing insight into the genetic basis of important traits in reed canarygrass, this study presents one of the first applications of MI to genome-wide analyses and provides useful guidelines for conducting GWAS based on genotyping-by-sequencing data.

59 BASIC BIOLOGICAL SCIENCES↗

Combined Mesonet and Tracker

Title: Combined Mesonet and Trackers (UNL Mobile Mesonets) Authors University of Nebraska PI: Adam Houston, UNL Professor (ahouston2@unl.edu) Mailing Address: 126 Bessey Hall P.O. Box 880340 Lincoln, NE 68588-0340 CoMeT Overview The University of Nebraska-Lincoln operates three Combined Mesonet and Tracker (CoMeTs). CoMeTs are Ford Explorers (model years 2015, 2017, and 2019) with forward-mounted suites of meteorological sensors and dual moonroofs, combining the capability of a mobile mesonet to collect near-surface observations with the capability of an unmanned aircraft systems (UAS) tracker vehicle, which enables an observer in the second row of seats to see the aircraft and maintain compliance with Federal Aviation Administration policies on UAS operation. The CoMeTs collect observations of slow temperature and humidity at ~2 m above ground level (AGL) using a Vaisala HMP155A, fast temperature at ~2 m AGL using a Campbell Scientific 109SS-L thermistor, pressure at ~2.5 m AGL using a Vaisala PTB210, wind speed and direction at ~3.25 m AGL using an R.M. Young 05103 propeller anemometer, and vehicle heading using a KVH Industries C-100 fluxgate compass (Barbieri et al. 2019). The HMP155A and 109SS-L thermistor are shielded and aspirated within a U-tube (Waugh and Frederickson 2010; Houston et al. 2016). This list of sensors is also included in the CoMeT data file metadata. Manufacturer specifications for these instruments are given in Table 1 of Hanft and Houston (2018). The reported measured quantities are summarized below.CoMeT-3 was funded through an equipment allocation included in the NSF TORUS award (AGS-1824649). Instrument Description The specific sensors included on each CoMeT are summarized in the table at the end of this section. In general each CoMeT collects observations of slow temperature and humidity at ~2 m above ground level (AGL) using a Vaisala HMP155, fast temperature at ~2 m AGL using a Campbell Scientific 109SS-L thermistor, pressure at ~2.5 m AGL using a Vaisala PTB210 barometer with a Gill pressure port, wind speed and direction at ~3.25 m AGL using an R.M. Young 05103 propeller anemometer, position using a Garmin 19x HVS receiver, and vehicle heading using a KVH Industries C-100 fluxgate compass. The HMP155 and 109SS are shielded and aspirated within a U-tube (Waugh and Frederickson 2010; Houston et al. 2016). Fast temperature and corrected RH measurements (using sensors housed within the U-tube) have a time constant of 10-12 s based on data collected across a temperature and RH shock during the CLOUD-MAP 2017 calibration/validation tests on June 26, 2017. Vehicle speed was < 10 kts for this test. CoMeT-1 CoMeT-2 CoMeT-3 Slow Temperature Slow RH Vaisala HMP155A-L20-PT Part #: 22280-7 Vaisala HMP155E Part #: E1AA11A0B1A1A0A Vaisala HMP155E Part #: E1AA11A0B1A1A0A Fast temperature Campbell Scientific 109SS-L20-PT Part #: 21448-3 Campbell Scientific 109SS-L12-PW Part #: 21448-109 Campbell Scientific 109SS-L12-PW Part #: 21448-150 Pressure Vaisala PTB-210 Part #: A1A1B Gill Pressure Port Part #: 61002 Vaisala PTB-210 Part #: A1A1B Gill Pressure Port Part #: 61002 Vaisala PTB-210 Part #: A1A1B Gill Pressure Port Part #: 61002 Wind RM Young 05103-L20-PT Part #: 18435-310 RM Young 05103-L20-PW Part #: 18435-244 RM Young 05103-L20-PW Part #: 18435-244 GPS Garmin GPS 19x HVS (NMEA 0183) Part #: 010-01010-00 Garmin GPS 19x HVS (NMEA 0183) Part #: 010-01010-00 Garmin GPS 19x HVS (NMEA 0183) Part #: 010-01010-00 Compass KVH C-100 Part #: 01-0177-15 KVH C-100 Part #: 01-0177-15 KVH C-100 Part #: 01-0177-15 Logger Campbell Scientific CR6-NA-XT-SW Part #: 28385-9 Campbell Scientific CR6-WIFI-XT-SW Part #: 28385-6 Campbell Scientific CR6-WIFI-XT-SW Part #: 28385-6 Data Collection and Real-Time Processing The reported measured quantities are summarized in the table below. Quantity Units Source Epoch time Seconds GPS Latitude and longitude Degrees GPS Altitude m GPS Pressure hPa PTB210 Temperature (fast) deg C 109SS-L Temperature (slow) deg C HMP155 RH (slow) % HMP155 Vehicle speed m/s GPS Vehicle heading deg C-100 and GPS In addition to the measured variables, several derived variables are calculated. Corrected/fast relative humidity (%) Relative humidity is adjusted to the fast temperature following Richardson et al. (1998) and Houston et al. (2016). Water vapor mixing ratio (g/kg) Dew point temperature (&deg;C) Potential temperature (Kelvin) Virtual potential temperature (Kelvin) Equivalent potential temperature (Kelvin) Regular intercomparisons between all three CoMeTs were performed during TORUS 2019. Comparisons were also conducted between CoMeT-1 and CoMeT-2 during LAPSE-RATE (2018) on 14 July. In these intercomparisons, the vehicles were parked adjacent to each other aligned perpendicular to (and facing into) the wind. To minimize engine heating effects, intercomparisons were only conducted when the wind speed was >10 kts. Data Format Original data files for each deployment are saved as text files and then converted to NetCDF. NetCDF versions have units that are CF compliant and may not match the original units in the txt files. The naming convention for the NetCDF files is as follows: UNL.CoMeT3.{deployment date YYYYMMDD}.{start time of observation collection in UTC HHMM}.L2.{post-processing codes}.cdf example: UNL.CoMeT3.20190627.1931.L2.g1.f1.cdf Post-processing codes are included to track modifications to the raw data. These codes are closely connected to error flags associated with each record. Each letter corresponds to a particular instrument: g: GPS p: Barometer tf: Fast temperature ts: Slow temperature rh: Relative humidity f: Compass w: Wind monitor a: All instruments Each number corresponds to a particular post-processing action described more below. Measured and derived variables are included in the following table. Variable Heading Standard Name Units time Time seconds since 00:00:00, 01-01-1970 Alt Altitude meters lat Latitude degrees north lon Longitude degrees east fast_temp Air Temperature kelvin slow_temp Air Temperature kelvin pressure Air Pressure pascals logger_RH Relative Humidity percent calc_corr_RH Relative Humidity percent wind_speed Wind Speed meters per second wind_dir Wind From Direction degrees vehicle_dir Vehicle Direction degrees dewpoint Dew Point Temperature kelvin mixing_ratio Humidity Mixing Ratio g/g theta Air Potential Temperature kelvin theta_v Virtual Potential Temperature kelvin theta_e Equivalent Potential Temperature Kelvin error_flag The error_flag variable is a string that matches the post-processing codes listed above. All instruments will have an associated code, but will have a &ldquo;0&rdquo; if the datum is unchanged from the initial processed value. Error Codes The following table summarizes the error codes for data collected before 2020: Error Code Relevant CoMeT Description g1 1,2,3 Exact correction. GPS position and time reprocessed from raw data g2 1 As far as we can tell this is an exact correction to an error in the GPS time. During the correct time periods the time suddenly went backwards ~250s and stayed at this offset for 750s when it corrected itself. The offset was applied to the &ldquo;time warp&rdquo; period. p1 2 Approximate correction. Hole in the pressure tube connecting the pressure port to the barometer. Resulted in erroneously low air pressure measurements when the vehicle was in motion. Derived variables recalculated (dew point temperature [e depends on qv and p], water vapor mixing ratio, potential temperature, virtual potential temperature, equivalent potential temperature) a1 3 Exact correction. Missing data reprocessed from raw data a2 1 Bug fix to bias correction for ts1, ts2, and rh1: water vapor mixing ratio was off by a factor of 10 and virtual potential temperature was wrong because of this. f1 3 No correction, missing data. Fluxgate compass inoperable. Wind speed and direction calculated using GPS-derived vehicle heading instead. rh1 1 Approximate correction. Constant bias of +1.7% removed from relative humidity. Derived variables recalculated (corrected/fast relative humidity, dew point temperature, water vapor mixing ratio, virtual potential temperature, equivalent potential temperature) ts1 1 Approximate correction. Constant bias of +0.6 K removed from slow temperature. Derived variables recalculated (corrected/fast relative humidity, dew point temperature, water vapor mixing ratio, virtual potential temperature, equivalent potential temperature) ts2 1 Approximate correction. Constant bias of +1.0 K removed. Derived variables recalculated (corrected/fast relative humidity, dew point temperature, water vapor mixing ratio, virtual potential temperature, equivalent potential temperature) References Bolton, D., 1980: The Computation of Equivalent Potential Temperature. Mon. Wea. Rev., 108, 1046&ndash;1053, https://doi.org/10.1175/1520-0493(1980)108<1046:TCOEPT>2.0.CO;2. Hanft, W., and A. L. Houston, 2018: An Observational and Modeling Study of Mesoscale Air Masses with High Theta-E. Mon. Wea. Rev., 146, 2503&ndash;2524, https://doi.org/10.1175/MWR-D-17-0389.1.Wexler Houston, A. L., R. J. Laurence III, T. W. Nichols, S. Waugh, B. Argrow, and C. L. Ziegler, 2016: Intercomparison of unmanned aircraft-borne and mobile mesonet atmospheric sensors. Journal of Atmospheric and Oceanic Technology. 33, 1569-1582, doi: 10.1175/JTECH-D-15-0178.1. Lowe, P. R., 1977: An Approximating Polynomial for the Computation of Saturation Vapor Pressure. J. Applied Meteorology, 16, 100&ndash;103. Richardson, S. J., S. E. Frederickson, F. V. Brock, and J. A. Brotzge, 1998: Combination temperature and relative humidity probes: Avoiding large air temperature errors and associated relative humidity errors. Preprints, 10th Symp. On Meteorological Observations and Instrumentation, Phoenix, AZ, Amer. Meteor. Soc., 278&ndash;283. Waugh, S., and S. E. Frederickson, 2010: An improved aspirated temperature system for mobile meteorological observations, especially in severe weather. 25th Conf. on Severe Local Storms, Denver, CO, Amer. Meteor. Soc., P5.2. [Available online at https://ams.confex.com/ams/25SLS/techprogram/paper_176205.htm.]

54 ENVIRONMENTAL SCIENCES↗

On‐site surrogates for large‐scale calibration

Abstract Motivated by a computer model calibration problem from the oil and gas industry, involving the design of a honeycomb seal, we develop a new Bayesian methodology to cope with limitations in the canonical apparatus stemming from several factors. We propose a new strategy of on‐site design and surrogate modeling for a computer simulator acting on a high‐dimensional input space that, although relatively speedy, is prone to numerical instabilities, missing data, and nonstationary dynamics. Our aim is to strike a balance between data‐faithful modeling and computational tractability in a calibration framework—tailoring the computer model to a limited field experiment. Situating our on‐site surrogates within the canonical calibration apparatus requires updates to that framework. We describe a novel yet intuitive Bayesian setup that carefully decomposes otherwise prohibitively large matrices by exploiting the sparse blockwise structure. Empirical illustrations demonstrate that this approach performs well on toy data and our motivating honeycomb example.

Huang, Jiangeng↗

A novel low-rank matrix completion approach to estimate missing entries in Euclidean distance matrix

A Euclidean distance matrix (EDM) is a table of distance-square between points on a k-dimensional Euclidean space, with applications in many fields (e.g., engineering, geodesy, economics, genetics, biochemistry, and psychology). A problem that often arises is the absence (or uncertainty) of some EDM elements. In many situations, only a subset of all pairwise distances is available and it is desired to have some procedure to estimate the missing distances. In this paper, we address the problem of missing data in EDM through low-rank matrix completion techniques. We exploit the fact that the rank of a EDM is at most k+2 and does not depend on the number of points, which is, in general, much bigger then k. We use a singular value decomposition approach that considers the rank of the matrix to be completed and computes, in each iteration, a parameter that controls the convergence of the method. After performing a number of computational experiments, we could observe that our proposal was able to recover, with high precision, random EDMs with more than 1000 points and up to 98% of missing data in few minutes. In addition, our method required a smaller number of iterations when compared to other competitive state-of-art technique.

97 MATHEMATICS AND COMPUTING↗

QA/QC-ed BCI meteorological drivers

Time series raw datasets of Temperature, Solar Radiation, Relative Humidity, Rainfall, Barometric Pressure, Wind Speed, and Wind Direction for the Barro Colorado (PA-BCI) site for the period from 2003-01-01 to 2016-12-31 were downloaded from the STRI website. The QA/QC protocol of raw datasets included the detection and removal of NAs, outliers and bad data points, imputation of missing data, and creation of the time series with equidistant time stamps. The time series for different meteorological parameters were aligned into a single database to be used for modeling. Statistical QA/QC analysis was performed using a series of R libraries in Rstudio. The changes in the dataset were flagged. The file BCI_met_drivers_2003-2016_QAQC_summary_report.docx provides the details of the QA/QC data analysis.

54 ENVIRONMENTAL SCIENCES↗

User's Guide to Pre-Processing Data in Universal Translator 2 for the Energy Charting and Metrics Tool (ECAM)

This document is a user's guide for the Energy Charting and Metrics Tool to facilitate the examination of energy information from buildings, reducing the time spent analyzing trend and utility meter data. This user guide was generated to help pre-process data with the intention of utilizing the Energy Charting and Metrics (ECAM) tool to improve building operational efficiency. There are numerous occasions when the metered data that is received from the building automation system (BAS) isn't in the right format acceptable for ECAM. This includes, but isn't limited to, cases such as inconsistent time-stamps for the trends (e.g., each trend has its own time-stamp), data with holes (e.g., some time-stamps have data and others are missing data), each point in the BAS is trended and exported into an individual .csv or .txt file, the time-stamp is unrecognizable by ECAM, etc. After reading through this user guide, the user should be able to pre-process all data files and be ready to use this data in ECAM to improve their building operational efficiency.

99 GENERAL AND MISCELLANEOUS↗

Ecosystem-Atmosphere Exchange of Carbon, Water and Energy over a Mixed Deciduous Forest in the Midwest

During the project period we continued to conduct long-term (multi-year) measurements, analysis, and modeling of energy and mass exchange in and over a deciduous forest in the Midwestern United States, to enhance the understanding of soil-vegetation-atmosphere exchange of carbon. At the time when this report was prepared, results from nine years of measurements (1998 - 2006) of above canopy CO 2 and energy fluxes at the AmeriFlux site in the Morgan-Monroe State Forest, Indiana, USA (see Table 1), were available on the Fluxnet database, and the hourly CO 2 fluxes for 2007 are presented here (see Figure 1). The annual sequestration of atmospheric carbon by the forest is determined to be between 240 and 420 g C m -2 a -1 for the first ten years. These estimates are based on eddy covariance measurements above the forest, with a gap-filling scheme based on soil temperature and photosynthetically active radiation. Data gaps result from missing data or measurements that were rejected in qua)lity control (e.g., during calm nights). Complementary measurements of ecological variables (i.e. inventory method), provided an alternative method to quantify net carbon uptake by the forest, partition carbon allocation in each ecosystem components, and reduce uncertainty on annual net ecosystem productivity (NEP). Biometric datasets are available on the Fluxnext database since 1998 (with the exclusion of 2006). Analysis for year 2007 is under completion.

54 ENVIRONMENTAL SCIENCES↗