Search NASA⌕ Search

SEARCH · Search NASA

Results for “validation dataset”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 55 records · Page 3

Optical Characterization of DebriSat Fragments in Support of Orbital Debris Environmental Models

The NASA Orbital Debris Program Office (ODPO) develops, maintains, and updates orbital debris environmental models, such as the NASA Orbital Debris Engineering Model (ORDEM), to support satellite designers and operators by estimating the risk from orbital debris impacts on their vehicles in orbit. Updates to ORDEM utilize the most recent validated datasets from radar, optical, and in situ sources to provide estimates of the debris flux as a function of size, material density, impact speed, and direction along a mission orbit. On-going efforts within the NASA ODPO to update the next version of ORDEM include a new parameter that highly affects the damage risk – shape. Shape can be binned by material density and size to better understand the damage assessments on spacecraft. The in situ and laboratory research activities at the NASA ODPO are focused on cataloging and characterizing fragments from a laboratory hypervelocity-impact test using a high-fidelity, mock-up satellite, DebriSat, in controlled and instrumented laboratory conditions. DebriSat is representative of present-day, low Earth orbit satellites, having been constructed with modern spacecraft materials and techniques. The DebriSat fragment ensemble provides a variety of shapes, bulk densities, and dimensions. Fragments down to 2 mm in size are being characterized by their physical and derived properties. A subset of fragments is being analyzed further in NASA’s Optical Measurement Center (OMC) using broadband, bidirectional reflectance measurements to provide insight into the optical-based NASA Size Estimation Model. Additionally, pre-impact spectral measurements on a subset of DebriSat materials were acquired for baseline material characterization. This paper provides an overview of DebriSat, the status of the project, and ongoing fragment characterization efforts within the OMC.

Heather M Cowardin↗

Optical Characterization of DebriSat Fragments in Support of Orbital Debris Environmental Models

The NASA Orbital Debris Program Office (ODPO) develops, maintains, and updates orbital debris environmental models, such as the NASA Orbital Debris Engineering Model (ORDEM), to support satellite designers and operators by estimating the risk from orbital debris impacts on their vehicles in orbit. Updates to ORDEM utilize the most recent validated datasets from radar, optical, and in situ sources to provide estimates of the debris flux as a function of size, material density, impact speed, and direction along a mission orbit. On-going efforts within the NASA ODPO to update the next version of ORDEM include a new parameter that highly affects the damage risk – shape. Shape can be binned by material density and size to better understand the damage assessments on spacecraft. The in situ and laboratory research activities at the NASA ODPO are focused on cataloging and characterizing fragments from a laboratory hypervelocity-impact test using a high-fidelity, mock-up satellite, DebriSat, in controlled and instrumented laboratory conditions. DebriSat is representative of present-day, low Earth orbit satellites, having been constructed with modern spacecraft materials and techniques. The DebriSat fragment ensemble provides a variety of shapes, bulk densities, and dimensions. Fragments down to 2 mm in size are being characterized by their physical and derived properties. A subset of fragments is being analyzed further in NASA’s Optical Measurement Center (OMC) using broadband, bidirectional reflectance measurements to provide insight into the optical-based NASA Size Estimation Model. Additionally, pre-impact spectral measurements on a subset of DebriSat materials were acquired for baseline material characterization. This paper provides an overview of DebriSat, the status of the project, and ongoing fragment characterization efforts within the OMC.

Heather M. Cowardin↗

Machine Learning Methods for Estimating Propeller Source Noise Spheres

In this work, several neural network function approximations are compared for inter- polating, storing, and sampling acoustic source spheres with applications to propeller noise estimation. These methods are compared using an acoustic model of the three bladed GL-10 propeller at different flight conditions, with training data generated using NASA’s ANOPP-PAS module. The source spheres used to train the networks capture the tonal propeller noise due to both the blade thickness and loading. This tonal noise prediction method allows the vehicle noise to be estimated for auralization and acoustic control. Three radial basis function neural network architectures are compared in this work. The first two networks directly estimate the parameters of the source sphere at different flight conditions but differ in the number of layers used. The third network estimates the parameters of the source sphere using a weighted combination of spherical basis functions. These networks are trained on numerically generated source spheres, with operating points given in terms of the propeller rotation rate, freestream speed, and propeller angle of attack. The performance of the neural network is determined using a validation dataset of withheld data points. This performance is quantified in terms of the approximation error, training time, and sample time. The third network, which estimates the weights of the spherical basis functions, performs the best in both average and maximum approximation errors in all cases. This network’s worst case performance is 5.6 % relative dif- ference of a model parameter associated with acoustic pressure. The direct estimation network with a single layer has the worst approximation error in all cases. Additionally, the spherically defined network has the slowest sample time at 0.05 seconds per thousand points. Both direct estimation methods produce a thousand sample points in approximately 0.001 seconds.

Acoustics↗

Evaluating China's fossil-fuel CO2 emissions from a comprehensive dataset of nine inventories

China's fossil-fuel CO2 (FFCO2) emissions accounted for approximately 28 % of the global total FFCO2 in 2016. An accurate estimate of China's FFCO2 emissions is a prerequisite for global and regional carbon budget analyses and the monitoring of carbon emission reduction efforts. However, significant uncertainties and discrepancies exist in estimations of China's FFCO2 emissions due to a lack of detailed traceable emission factors (EFs) and multiple statistical data sources. Here, we evaluated China's FFCO2 emissions from nine published global and regional emission datasets. These datasets show that the total emissions increased from 3.4 (3.0–3.7) in 2000 to 9.8 (9.2–10.4) Gt CO2 per yr in 2016. The variations in these estimates were largely due to the different EF (0.491–0.746 t C per t of coal) and activity data. The large-scale patterns of gridded emissions showed a reasonable agreement, with high emissions being concentrated in major city clusters, and the standard deviation mostly ranged from 10 % to 40 % at the provincial level. However, patterns beyond the provincial scale varied significantly, with the top 5 % of the grid level accounting for 50 %–90 % of total emissions in these datasets. Our findings highlight the significance of using locally measured EF for Chinese coal. To reduce uncertainty, we recommend using physical CO2 measurements and use these values for dataset validation, key input data sharing (e.g., point sources), and finer-resolution validations at various levels.

Pengfei Han↗

Mass Inferencing Model Creation and Deployment to the RASSOR Lunar Excavation Robot

The Regolith Advanced Surface Systems Operations Robot (RASSOR) Excavator is a teleoperated mobile robotic platform with a unique space regolith excavation capability. The Intelligent Capabilities Enhanced RASSOR research project developed functionality for inferencing regolith mass ingested during RASSOR operation, enhancing RASSOR’s ability to successfully complete ISRU missions. To teleoperate or run autonomously, it is crucial for the quantity of regolith mass ingested by RASSOR to be available as a system state for efficient operation. For example, during autonomous operation, RASSOR should navigate and move to a processing plant to offload the collected regolith when the drums are full; without knowledge of how much mass is in the drums, this type of high-level planning is not possible. Four distinct modeling approaches were employed in developing a mass inferencing approach that could work on RASSOR. All take in system states, such as arm/drum positions, velocities, currents, voltages, and robot pose, and output a mass prediction for each set of the robot’s bucket drums.1) A neural network model that takes a vector of normalized system states; 2) A model that uses the integrated power consumption of an arm-raise (normalized by velocity); 3) A model that uses average drum current over a variable length interval of the drum disengaged from the surface; and 4) A real-time estimation model that aggregates excavation drum current. The developed models run in real time, outputting predictions for the front and rear drums, timestamp of the last prediction, and total mass in RASSOR’s drums. Further testing is required to validate the arm-raise model (2), though initial tests indicate reasonable performance (<10% mean error) on the hardware. The linear fit of average drum-current model (3) had a front value of r^2=0.99 and a rear value of r^2=0.98 on the validation dataset. This model currently has the best performance on unseen data. The real time model (4) is still in development, though initial results on a small subset of the training data show that it has high accuracy in predicting the increase in mass during excavation. Though work remains to be done with deploying a high-fidelity model to the physical system that makes predictions with error below the desired threshold, the modular architecture for model development allows quick adjustment of parameters to increase model fidelity. This architecture can also be adapted to use lunar excavation data to create models that are reflective of RASSOR’s dynamics when operating on the lunar surface. The results are promising as it has been shown that models can be developed that accurately estimate excavated regolith mass.

rassor↗

Updates on the Debrisat Hypervelocity Experiment and Characterization of Fragments in Support of Environmental Models

To develop, maintain, and update orbital debris environmental and break-up models, such as the NASA Orbital Debris Engineering Model (ORDEM) and the NASA Standard Satellite Breakup Model (SSBM), the NASA Orbital Debris Program Office (ODPO) relies on the most recent validated datasets from various measurement sources, both laboratory and environmental. One key project that will provide insight for break-up events using modern-day spacecraft materials and construction techniques is the DebriSat laboratory hypervelocity-impact test. Based on the mass of the target, the projectile, and the impact velocity, the expected number of fragments greater than 2 mm, generated using the NASA SSBM, was estimated to be close to 85,000. To date, the DebriSat fragment database continues to grow, with over 200,000 fragments collected that will help inform updates to the SSBM. Additionally, the growing fragment ensemble will support key parameters for the next release of NASA’s environmental models, employing fragment shapes, densities, and size distributions. To further support these environmental models, specifically the size parameter used in ground-based optical measurements, optical characterization on a subset of DebriSat fragments is being conducted in ODPO’s Optical Measurement Center (OMC). Broadband bidirectional reflectance distribution (BRDF) measurements will provide insight into the optical-based NASA Size Estimation Model (OSEM). The OSEM equates an object’s brightness to size (e.g., diameter of a disk or sphere) given several assumed parameters, including a defined phase function, albedo, and range. To address the first defined parameter, the ODPO has been using ray-tracing software to simulate light conditions in the OMC and to generate phase functions (i.e., specular, Lambertian, and experimentally-derived) of known shapes, materials, and sizes. This simulated data, in addition to the experimentally derived measurements collected in the OMC, will aid in determining whether a new phase function would be suitable for an updated OSEM. The OSEM also assumes a single-value albedo, thus pre-impact spectral measurements on a subset of DebriSat materials were acquired for baseline material characterization and to provide insight into spacecraft material taxonomies. This DebriSat spectral data, along with spectral measurements from other known spacecraft material samples, will allow for better analyses of albedo variations and the effect on size calculations of defined laboratory targets, thus further supporting OSEM updates. This paper provides an overview of DebriSat, the status of the project, updates on the parameter distributions, an overview of the NASA SSBM, and ongoing fragment characterization efforts within the OMC.

Heather Cowardin↗

The Joint Assimilation of Remotely Sensed Leaf Area Index and Surface Soil Moisture into a Land Surface Model

This work tests the hypothesis that jointly assimilating satellite observations of leaf area index and surface soil moisture into a land surface model improves the estimation of land vegetation and water variables. An Ensemble Kalman Filter is used to test such hypothesis across the Contiguous United States during April 2015 – December 2018. The performance of the proposed methodology is assessed for several modeled vegetation and water variables (evapotranspiration, net ecosystem exchange, and soil moisture) in terms of random errors and anomaly correlation coefficients against a set of independent validation datasets (i.e., Global Land Evaporation Amsterdam Model, FLUXCOM, and International Soil Moisture Network). Results show that the assimilation of leaf area index mostly improves the estimation of evapotranspiration and net ecosystem exchange, whereas the assimilation of surface soil moisture alone improves surface soil moisture content, especially in the western US, in terms of both root mean squared error and anomaly correlation coefficient. The joint assimilation of vegetation and soil moisture information combines the results of individual vegetation and soil moisture assimilations and reduces errors (and increases correlations with the reference datasets) in evapotranspiration, net ecosystem exchange, and surface soil moisture simulated by the land surface model. However, because soil moisture satellite observations only provide information on the water content in the top 5 cm of the soil column, the impact of the proposed data assimilation technique on root zone soil moisture is limited. This work moves one step forward in the direction of improving our estimation and understanding of land surface interactions using a multi-variate data assimilation approach, which can be particularly useful in regions of the world where ground observations are sparse or missing altogether.

Data assimilation↗

Seasonal Forecasting Skill for the High Mountain Asia Region in the Goddard Earth Observing System

Seasonal variability of the global hydrologic cycle directly impacts human activities, including hazard assessment and mitigation, agricultural decisions, and water resources management. This is particularly true across the High Mountain Asia (HMA) region, where availability of water resources can change depending on local seasonality of the hydrologic cycle. Forecasting the atmospheric states and surface conditions, including hydrometeorological relevant variables, at subseasonal-to-seasonal (S2S) lead times of weeks-to-months is an area of active research and development. NASA’s 15 Goddard Earth Observing System (GEOS) S2S prediction system has been developed with this research goal in mind. Here, we benchmark the forecast skill of GEOS-S2S (version 2) hydrometeorological forecasts at 1-3 month lead times in the HMA region, including a portion of the Indian Subcontinent, during the retrospective forecast period, 1981-2016. To assess forecast skill, we evaluate 2-m air temperature, total precipitation, fractional snow cover, snow water equivalent, surface soil moisture, and terrestrial water storage forecasts against the Modern-Era Retrospective analysis for Research and 20 Applications, Version 2 (MERRA-2) and independent reanalysis data, satellite observations, and data fusion products. Anomaly correlation is highest when the forecasts are evaluated against MERRA-2 and particularly in variables with long memory in the climate system, likely due to similar initial conditions and model architecture used in GEOS-S2S and MERRA-2. When compared to MERRA-2, results for the 1-month forecast skill range from anomaly correlation of R anom =0.18 for precipitation to R anom =0.62 for soil moisture. Anomaly correlations are consistently lower when forecasts are 25 evaluated against independent observations; results for the 1-month forecast skill range from R anom =0.13 for snow water equivalent to R anom =0.24 for fractional snow cover. We find that, generally, hydrometeorological forecast skill is dependent on the forecast lead time, the memory of the variable within the physical system, and the validation dataset used. Overall, these results benchmark the GEOS-S2S system’s ability to forecast HMA hydrometeorology.

GEOS↗

Updates on the DebriSat Hypervelocity Experiment and Characterization of Fragments in Support of Environmental Models

To develop, maintain, and update orbital debris environmental and break-up models, such as the NASA Orbital Debris Engineering Model (ORDEM) and the NASA Standard Satellite Breakup Model (SSBM), the NASA Orbital Debris Program Office (ODPO) relies on the most recent validated datasets from various measurement sources, both laboratory and environmental. One key project that will provide insight for break-up events using modern-day spacecraft materials and construction techniques is the DebriSat laboratory hypervelocity-impact test. Based on the mass of the target, the projectile, and the impact velocity, the expected number of fragments greater than 2 mm, generated using the NASA SSBM, was estimated to be close to 85,000. To date, the DebriSat fragment database continues to grow, with over 200,000 fragments collected that will help inform updates to the SSBM. Additionally, the growing fragment ensemble will support key parameters for the next release of NASA’s environmental models, employing fragment shapes, densities, and size distributions. To further support these environmental models, specifically the size parameter used in ground-based optical measurements, optical characterization on a subset of DebriSat fragments is being conducted in ODPO’s Optical Measurement Center (OMC). Broadband bidirectional reflectance distribution (BRDF) measurements will provide insight into the optical-based NASA Size Estimation Model (OSEM). The OSEM equates an object’s brightness to size (e.g., diameter of a disk or sphere) given several assumed parameters, including a defined phase function, albedo, and range. To address the first defined parameter, the ODPO has been using ray-tracing software to simulate light conditions in the OMC and to generate phase functions (i.e., specular, Lambertian, and experimentally-derived) of known shapes, materials, and sizes. This simulated data, in addition to the experimentally derived measurements collected in the OMC, will aid in determining whether a new phase function would be suitable for an updated OSEM. The OSEM also assumes a single-value albedo, thus preimpact spectral measurements on a subset of DebriSat materials were acquired for baseline material characterization and to provide insight into spacecraft material taxonomies. This DebriSat spectral data, along with spectral measurements from other known spacecraft material samples, will allow for better analyses of albedo variations and the effect on size calculations of defined laboratory targets, thus further supporting OSEM updates. This paper provides an overview of DebriSat, the status of the project, updates on the parameter distributions, an overview of the NASA SSBM, and ongoing fragment characterization efforts within the OMC.

Heather Cowardin↗

Evaluating Meteorological Dust Events and Machine-Learning Based Dust Identification in Geostationary Satellite Imagery

NASA scientists in the Short-term Prediction Research and Transition Center (SPoRT) developed a physically-based machine learning approach to identify dust in satellite imagery with a focus on night-time dust detection (Berndt et al. 201; DustTracker-AI). NASA/NOAA Geostationary Environmental Operational Satellite-16 (GOES-16) imagery was used for training and model inputs. The training, testing and validation data set consists of 28 events in the Southwest United States, capturing dust and null events in the region from 2018-2020.With 83 distinct images and millions of pixels a random forest model was trained and validated, correctly labeling 85% of dust pixels.For the first time, the model was run in near-real time production during the spring of 2022 and dust probability visualizations were made available to NOAA National Weather Service (NWS) forecasters to assess its utility for dust forecasting. Results indicated the model helped increase the confidence in the presence of dust and enabled dust tracking for a longer period of time into the night-time hours. Forecaster assessment and running the model in near real-time allowed for the team to determine the types of events missed, captured, and false alarms. To gain additional context on model performance,the SPoRT team sought to gather more detailed information on the training database(e.g., meteorological characteristics and drivers). The goal of this project was to identify the meteorological drivers for the dust events and create a database which synthesized information from observations, forecaster discussions, and analyses pertaining to the dust events to understand the types of events currently used to train the model. A more detailed meteorological synopsis was created for each dust event in the training, testing, and validation datasets. Following the completion of the database and documentation, the classification details revealed that 88% of the dust events were synoptically driven while mesoscale events were less prevalent in model datasets. Meteorological conditions found such as mixing layer depth and wind velocity had mean values of 645mb and 21kt respectively.With conditions of deep mixed layers and moderate to strong surface winds a mesoscale thunderstorm outflow event was considered and subsequently added to the model training data set to test the impact of additional mesoscale training data. The model was retrained and then qualitatively tested on a sample thunderstorm outflow case that the original model was unable to identify. Preliminary results showed potential that the addition of more mesoscale events included in the training data could help to better identify indistinct and localized dust events.

Connor Welch↗

Summarizing Multiple Aspects of Triple Collocation Analysis in a Single Diagram

With the ongoing expansion of global observation networks, it is expected that we shall routinely analyze records of geophysical variables such as temperature from multiple collocated instruments. Validating datasets in this situation is not a trivial task because every observing system has its own bias and noise. Triple collocation is a general statistical framework to estimate the error characteristics in three or more observational-based datasets. In a triple colocation analysis, several metrics are routinely reported but traditional multiple-panel plots are not the most effective way to display information. A new formula of error variance is derived for connecting the key terms in the triple collocation theory. A diagram based on this formula is devised to facilitate triple collocation analysis of any data from observations, as illustrated using three aerosol optical depth datasets from the recent Aerosol Cloud meTeorology Interactions oVer the western ATlantic Experiment (ACTIVATE). An observational-based skill score is also derived to evaluate the quality of three datasets by taking into account both error variance and correlation coefficient. Several applications are discussed and sample plotting routines are provided.

triple collocation↗

Establishing an in-Space Joining Ecosystem at NASA Marshall via Laser Beam Welding

NASA Marshall is establishing an ISAM technology development ecosystem leveraging investments in laser beam processing to enable in-space joining via laser beam welding. A number of ground and flight experiments are being performed to develop laser beam welding as a mature process for use in space. These experiments access varied combinations of reduced gravity, reduced atmospheric pressure, and extreme temperatures to simulate relevant space environments. The associated instrumentation needed to exquisitely understand fundamental mechanisms during laser beam welding and to provide adequate validation datasets for computational models is also being developed and/or integrated.

thermal vacuum↗

Establishing an In-Space Joining Ecosystem at NASA Marshall via Laser Beam Welding

NASA Marshall is establishing an ISAM technology development ecosystem leveraging investments in laser beam processing to enable in-space joining via laser beam welding. A number of ground and flight experiments are being performed to develop laser beam welding as a mature process for use in space. These experiments access varied combinations of reduced gravity, reduced atmospheric pressure, and extreme temperatures to simulate relevant space environments. The associated instrumentation needed to exquisitely understand fundamental mechanisms during laser beam welding and to provide adequate validation datasets for computational models is also being developed and/or integrated.

thermal vacuum↗

Laser Beam Welding Benchmark Experiments Performed in Reduced Gravity and Vacuum

Laser beam welding (LBW) is affected by the extreme temperatures, reduced pressure, and reduced gravity present in space environments. Gravity and pressure especially influence its melt pool and solidification dynamics. A compact, modular vacuum chamber adaptable to flight platforms from parabolic to orbital currently hosts an experiment to investigate the combined influence of reduced gravity and pressure on LBW. A swappable cartridge contains a rotating platen on which customizable workpieces can be welded under vacuum, greatly increasing experimental throughput. Instrumentation includes weld and thermal cameras observing the process, thermocouples placed on workpieces, accelerometers, and vacuum sensors. Experimental data gathered during the welding process will be combined with post-flight nondestructive evaluation, metallography, and mechanical testing to provide validation datasets for computational modeling. Phase I of this effort involves a parabolic flight campaign in low gravity while an anticipated Phase II would proceed to in-space demonstration to access extended duration microgravity.

in-space welding↗

High resolution infrared datasets useful for validating stratospheric models

An important objective of the High Speed Research Program (HSRP) is to support research in the atmospheric sciences that will improve the basic understanding of the circulation and chemistry of the stratosphere and lead to an interim assessment of the impact of a projected fleet of High Speed Civil Transports (HSCT's) on the stratosphere. As part of this work, critical comparisons between models and existing high quality measurements are planned. These comparisons will be used to test the reliability of current atmospheric chemistry models. Two suitable sets of high resolution infrared measurements are discussed.

Rinsland, Curtis P.↗

Maximizing efficiency of dataset compression for machine learning potentials with information theory

Machine learning interatomic potentials (MLIPs) balance high accuracy and lower costs compared to density functional theory calculations, but their performance often depends on the size and diversity of training datasets. Large datasets improve model accuracy and generalization but are computationally expensive to produce and train on, while smaller datasets risk discarding rare but important atomic environments and compromising MLIP accuracy/reliability. Here, we develop an information-theoretical framework to quantify the efficiency of dataset compression methods and propose an algorithm that maximizes this efficiency. By framing atomistic dataset compression as an instance of the minimum set cover (MSC) problem over atom-centered environments, our method identifies the smallest subset of structures that contains as much information as possible from the original dataset while pruning redundant information. The approach is extensively demonstrated on the GAP-20 and TM23 datasets and validated on 64 varied datasets from the ColabFit repository. Across all cases, MSC consistently retains outliers, preserves dataset diversity, and reproduces the long-tail distributions of forces even at high compression rates, outperforming other subsampling methods. Furthermore, MLIPs trained on MSC-compressed datasets exhibit reduced error for out-of-distribution data even in low-data regimes. We explain these results using an outlier analysis and show that such quantitative conclusions could not be achieved with conventional dimensionality reduction methods. The algorithm is implemented in the open-source QUESTS package and can be used for several tasks in atomistic modeling, from data subsampling, outlier detection, and training improved MLIPs at a lower cost.

36 MATERIALS SCIENCE↗

Supervised Machine Learning Approach for Classifying Earth Science Publications

The data collections archived and distributed by the GES DISC NASA data center are widely utilized for various Earth Science studies. As these collections are created, many research works are published regarding these collections' algorithms, their validation, and their applications. As NASA data centers collect these publications for public use, it is helpful to categorize them based on how they relate to their associated datasets. Specifically, whether the publication linked to the GES DISC dataset is using it for applicational research, describing the algorithm used for the dataset creation, validating the dataset, or providing a general overview of the data collection. Currently, this process requires simple manual labeling, and as such, it may be possible to solve via automation. To approach this problem, machine learning classifiers were developed to predict a publication's category. Manually labeled publications were used as the training data for the supervised machine learning algorithms, specifically Random Forest and Multinomial Naïve Bayes. After balancing the dataset and implementing the Multinomial Naïve Bayes algorithm, the classification accuracy achieved was substantially higher than the baseline accuracy, thus significantly improving the efficiency of publication labeling.

Rohan Dayal↗