Search NASA⌕ Search

SEARCH · Search NASA

Results for “metric standardization”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 199 records · Page 11
All sourcesNASA NTRSDOE OSTI
NASA NTRSDate not supplied

Objective Structured Clinical Evaluation (OSCE) of an Artificial Intelligence (AI) Clinical Decision Support System (CDSS) Tool

BACKGROUND Objective Structured Clinical Evaluations (OSCEs) have long been established as a robust methodology for summative assessment of clinical skills and decision-making during medical education. The recent integration of Artificial Intelligence (AI) into clinical decision-making processes has prompted the need for novel evaluation frameworks to assess the efficacy and reliability of AI clinical decision support system (CDSS) tools. This abstract outlines the process of quantitatively evaluating a novel CDSS (“Doc in a Box” Google 2024) trained on curated medical spaceflight data in the psychomotor domain as it interfaces with a human volunteer acting as the crew medical officer (CMO). PURPOSE The AI CDSS under review was developed as part of the Lunar Command and Control Interoperability (LuCCI) project, which is intended to address a gap in how Lunar Surface Systems (LSS) would interoperate across multiple programs, commercial partners, and international partners. The project objective is to define, prototype, integrate, and evaluate an interoperable lunar command, control, data, and software reference architecture to enable autonomy and informatics capability through common standards across LSS. A multi-modal AI-based CDSS compatible with Federated LSS will assist clinicians in diagnosing and managing complex medical conditions by providing evidence-based recommendations through predictive analytics. Given the critical role of decision-support as NASA continues to evolve its Earth-independent medical operations (EIMO), it is imperative to ensure that such AI tools perform reliably and align with clinical standards during progressive lunar and Martian exploration class missions. METHODS The OSCE framework, traditionally used for evaluating human clinicians, was adapted to assess the AI tool's decision-making capabilities in simulated clinical scenarios. In this adapted OSCE, the AI CDSS was tested across a series of structured clinical scenarios designed to mimic real-life spaceflight patient cases. These scenarios included a range of conditions and complexities, allowing for comprehensive assessment of the tool's performance. Key evaluation metrics included accuracy of diagnosis, timeliness of decision-making, and appropriate recommendations for therapies. The OSCE was scored by human physician evaluators who assessed the AI's recommendations in comparison with expert clinicians' medical decision making to ensure alignment with best practices and the standard of care. RESULTS Preliminary results indicate that the AI CDSS demonstrated high accuracy in diagnostic recommendations and decision support across various scenarios. However, certain limitations were noted, such as occasional discrepancies in handling complex or nuanced cases that required a more contextual understanding. Additionally, the tool scored higher on the diagnostic portion of the rubric, with lower scores in the therapeutic recommendations. These findings highlight the importance of continuous refinement and validation of AI tools through rigorous evaluation frameworks like the OSCE. The adaptation of OSCEs for AI tools presents several advantages, including a structured and reproducible approach to evaluation, the ability to test AI systems in diverse clinical scenarios, and the opportunity to benchmark AI performance against established clinical standards to permit charting of future progress as aerospace medicine evolves as a discipline. Remaining challenges include ensuring that these evaluations capture the full spectrum of clinical decision-making scenarios that will be confronted by CMOs during missions and adequately reflecting real-world variability of the austere spaceflight environment. CONCLUSION Employing OSCEs to evaluate AI clinical decision support tools offers a promising approach to validating their clinical utility and efficacy. This methodology not only provides insights into the tool's performance but also fosters ongoing improvement and alignment with standard of care practices. Future research should focus on refining these evaluation processes and addressing limitations to enhance the integration of AI tools in clinical spaceflight settings. REFERENCES Scott S, Hearns V, Barker MA. Testing Clinical Skills: A Look at the OSCE and USMLE Clinical Skills Exams. S D Med. 2019 Oct;72(10):451-453. Majumder MAA, Kumar A, Krishnamurthy K, Ojeh N, Adams OP, Sa B. An evaluative study of objective structured clinical examination (OSCE): students and examiners perspectives. Adv Med Educ Pract. 2019 Jun 5;10:387-397. Karam VY, Park YS, Tekian A, Youssef N. Evaluating the validity evidence of an OSCE: results from a new medical school. BMC Med Educ. 2018 Dec 20;18(1):313.

Ariana M Nelson↗
NASA NTRS2019

NASA’s Space Launch System: Enabling a New Generation of Lunar Exploration

Following two decades of operational experience in low-Earth orbit (LEO), NASA has turned its focus once again to deep space exploration. The Agency is building the Space Launch System (SLS) to take astronauts and cargo to the Moon and send robotic spacecraft deep into the solar system. Offering unmatched performance, departure energy and payload capacity, SLS is designed to evolve into progressively more powerful configurations, enabling a new generation of human exploration of the Moon in preparation for future missions to Mars. The first build of the Block 1 vehicle is nearly complete for Exploration Mission-1 (EM-1), the first integrated flight of SLS and the Orion crew vehicle. EM-1 will send an uncrewed Orion to a distant retrograde lunar orbit in order to test and verify new systems, and along the way will deploy 13 6U-class CubeSats in deep space along the upper stage disposal trajectory after separation from Orion. The Agency’s current plans call for the first three missions on the SLS manifest to utilize the Block 1 vehicle in crew and cargo configurations. A more powerful evolved vehicle, Block 1B, will provide additional mass and volume performance using a new Exploration Upper Stage (EUS). Block 1B will lift 34 to 40 metric tons (t) to trans-lunar injection (TLI), depending on crew or cargo configuration. The Block 1B crew configuration will offer as much payload volume as industry-standard 5 m-diameter fairings to co-manifested payloads in a Universal Stage Adapter (USA). The Block 1B cargo variant will accommodate 8.4 meter-diameter fairings in 62.7-foot (19.1 meter) or 90-foot (27.4 meter) lengths. Adding smallsat secondary payloads to ride along with primary and co-manifested payloads on future flights may be possible, depending on mass margins. Leveraging a flight-proven, well-understood propulsion system, SLS’s flexible architecture, unmatched performance and expansive payload accommodations will open exciting new mission possibilities in deep space. Launches of habitat modules for NASA’s new Gateway lunar outpost, the next generation of robotic spacecraft to the far reaches of the solar system, large-aperture deep space telescopes, probes to interstellar space and the return of astronauts to the Moon are all possible with SLS.

Creech, Stephen D.↗
NASA NTRSDate not supplied

Modular Interface for CLPS-Scale Excavators (MICE) – Feasibility Testing of a Modular Disconnect System for Regolith Manipulation Implements Focused on Excavation and Site Preparation Activies

The Artemis Program seeks to establish a sustained lunar presence and robust economy driven by re-sources available on the Moon and enabled by sup-porting infrastructure. This will require robotic manipulation of tens of thousands of metric tons of regolith. Excavation and site preparation tasks in the unforgiving lunar environment will inevitably lead to component wear and necessitate periodic maintenance and repair. System modularity and standard interfaces are key components of enabling robotic repair and maintenance. The Modular Interface for CLPS (Commercial Lunar Payloads Services) -scale Excavators (MICE) project is a funded fiscal year 2024 Internal Research and Development (IRAD) project at NASA KSC. The MICE project seeks to test concepts for modular re-configuration of a CLPS-scale excavation and site preparation rover. These concepts, henceforth referred to as Modular Disconnect Systems (MDS), allow a mobility platform to interface with regolith manipulation implements such as a bucket drum excavator or compactor. The MDS concepts allow the mobility platform to detect one implement and swap to another implement in-situ. Power and communications is fed through a dust-tolerant electrical connection on the MDS to the implement’s actuator(s) and sensors.

MICE↗
NASA NTRS2007

Process Improvement in a Radically Changing Organization

This presentation describes how the NASA Glenn Research Center planned and implemented a process improvement effort in response to a radically changing environment. As a result of a presidential decision to redefine the Agency's mission, many ongoing projects were canceled and future workload would be awarded based on relevance to the Exploration Initiative. NASA imposed a new Procedural Requirements standard on all future software development, and the Center needed to redesign its processes from CMM Level 2 objectives to meet the new standard and position itself for CMMI. The intended audience for this presentation is systems/software developers and managers in a large, research-oriented organization that may need to respond to imposed standards while also pursuing CMMI Maturity Level goals. A set of internally developed tools will be presented, including an overall Process Improvement Action Item database, a formal inspection/peer review tool, metrics collection spreadsheet, and other related technologies. The Center also found a need to charter Technical Working Groups (TWGs) to address particular Process Areas. In addition, a Marketing TWG was needed to communicate the process changes to the development community, including an innovative web site portal.

Varga, Denise M.↗
NASA NTRSDate not supplied

Nearfield Anisotropic Mesh Adaptation for the Third AIAA Sonic Boom Workshop

Sonic boom prediction is critical for the development of low-boom supersonic aircraft and encouraging a replacement of the overland commercial supersonic ban with a certification standard. The Third AIAA Sonic Boom Workshop nearfield test cases provide a unique opportunity to review anisotropic adaptive-mesh strategies for complex geometries with important propulsion interaction effects. Anisotropic metrics formulated to control estimated Mach interpolation and integrated nearfield pressure signature (goal-oriented) errors are compared with the results obtained on workshop-provided meshes. Detailed flow solutions and mesh renderings are combined with computational schlieren images to reveal the complex interactions present in a shock-plume interaction wind tunnel model and a prototype of the Low-Boom Flight Demonstrator. A favorable comparison of independent implementations of the entire simulation process provides evidence that each process is implemented correctly and sets a solid foundation for confidence in the nearfield prediction of each method.

sonic boom↗
NASA NTRSDate not supplied

Short Circuiting the Controller – Missteps in Maintenance and Inspection of Process and Wiring in STS-93

The primary objective of the Space Transportation System mission 93 (STS-93) was to deploy the Chandra X-Ray Observatory. Chandra, the world's most powerful X-Ray telescope, allowed scientists from around the world to study some of the most distant and dynamic objects in the universe. Stripped of nearly 7,000 pounds of its own gear to make room for the payload, the orbiter assigned to this mission was Space Shuttle Columbia. Prior to STS-93, Columbia had flown 25 flights and was NASA's oldest and heaviest orbiter. On July 23, 1999, after two prior launch scrubs, Eileen Collins and her crew of four launched from Kennedy Space Center. About five seconds after launch, Mission Control at Johnson Space Center detected a voltage drop on one of Columbia's electrical buses. As a result of this power fluctuation, a primary and back-up Main Engine controller dropped offline. Given design redundancy, the two remaining controllers supported all three engines. If there had been any other AC bus issues, one engine of the three on the Orbiter would have shut down. The redundant set of digital computer units in each engine controller saved Columbia and her crew from a very risky contingency abort. Post-flight inspection revealed soot on a screw head and a hole in an adjacent Kapton insulated wire. The single strand of polyimide wire was located nearly half-way down the payload bay. The Shuttle Independent Assessment Team (SIAT) reported that the wire had rubbed and chaffed against a burred screw head. The burr was later determined to be the result of overtightening of the screw by a technician during a maintenance refurbishment. Alone, the burr may not have been problematic, but later, during another ground processing event, possibly years after, someone inadvertently stepped on the wiring harness. With the pressure and motion of unintended contact, some of the Kapton insulation rubbed off against the burred screw head. The SIAT suspected the wire damage was pre-existing and was caused 4 or 5 years prior to the flight. Finally, the intense vibrations during the launch sequence allowed contact between the exposed conductor and exposed metal area on the burred screw head, resulting in the arcing and shorting of the wire. Due to the quick turnaround times of Space Shuttle Orbiters, wiring issues caused from multiple maintenance events were often overlooked. Additionally, failing to incorporate thorough and early inclusion of human systems integration (HSI) applications as a crucial part of the decision process can result in these types of misshaps. In order to reduce human error associated with integrated manufacturing, maintenance, refurbishment and flight preparations, wire inspection criteria should be refined and standardized, visual inspection processes should be quantified, and technicians should be certified by specially trained instructors. This case, among many others, unveils why human error management and development of safety metrics is a vital piece in the development of complex systems, and why it should be supported aggressively and implemented program wide.

Human Systems Integration↗
NASA NTRS2025

Time Course of Effects of Carbon Dioxide Exposure on Physical and Cognitive Performance in a Simulated Surface Extravehicular Activity Contingency Scenario

INTRODUCTION Carbon dioxide (CO 2 ) is a metabolic byproduct produced by humans that must be scrubbed from the atmosphere by environmental control and life support systems in space vehicles and spacesuits to prevent buildup. Current nominal CO 2 limits are set by NASA Standard 3001, however, limits during contingency situations are not specified. The purpose of this study was to characterize and quantify the physiological, cognitive, and self-assessed symptom and performance metrics as a function of inspired CO 2 (P I CO 2 ) during a simulated 1-hour contingency lunar EVA scenario in virtual reality (VR). METHODS Fifteen healthy subjects completed 7 simulated EVA scenarios walking on a passive treadmill for one hour breathing dry, ambient air mixed with added CO 2 in a blind, counterbalanced manner (P I CO 2 : 0, 5, 10, 15, 20, 25, 30 mmHg) simulating a 2.0 km contingency EVA walkback to a habitat on the lunar surface in the event of a CO 2 scrubber malfunction. Subjects were immersed in a Lunar VR environment and asked to maintain their walking speed between 2.1 km/h and an upper limit individually calibrated to a fixed metabolic workload of 1.3 L/min of O 2 consumption, and performed periodic cognitive tasks and continuous cardiopulmonary measurements. Self-assessments of symptoms and task performance were measured via verbal Likert-scale survey at the onset of the walkback and at 10-minute intervals thereafter. RESULTS The time course of cardiopulmonary, cognitive, and self-reported responses at each level of P I CO 2 will be presented. All 15 subjects were able to complete the 1-hr walk back task at all CO 2 levels. Physiological responses were rapid, typically within 1 to 5 minutes. No effects of time were detected for cognitive measures. Symptomatic ratings of headache, shortness of breath, and fatigue demonstrated a time-course development, reaching asymptote after 40 minutes of CO 2 exposure. DISCUSSION These findings will inform the posture of risk associated with CO 2 exposure on human health and performance during microgravity and planetary surface operations, This will help inform decisions related to space suit and environmental life support hardware requirements and EVA operational limits for contingency extravehicular activity (EVA) scenarios.

carbon dioxide↗
NASA NTRS2007

Robust Design Optimization via Failure Domain Bounding

This paper extends and applies the strategies recently developed by the authors for handling constraints under uncertainty to robust design optimization. For the scope of this paper, robust optimization is a methodology aimed at problems for which some parameters are uncertain and are only known to belong to some uncertainty set. This set can be described by either a deterministic or a probabilistic model. In the methodology developed herein, optimization-based strategies are used to bound the constraint violation region using hyper-spheres and hyper-rectangles. By comparing the resulting bounding sets with any given uncertainty model, it can be determined whether the constraints are satisfied for all members of the uncertainty model (i.e., constraints are feasible) or not (i.e., constraints are infeasible). If constraints are infeasible and a probabilistic uncertainty model is available, upper bounds to the probability of constraint violation can be efficiently calculated. The tools developed enable approximating not only the set of designs that make the constraints feasible but also, when required, the set of designs for which the probability of constraint violation is below a prescribed admissible value. When constraint feasibility is possible, several design criteria can be used to shape the uncertainty model of performance metrics of interest. Worst-case, least-second-moment, and reliability-based design criteria are considered herein. Since the problem formulation is generic and the tools derived only require standard optimization algorithms for their implementation, these strategies are easily applicable to a broad range of engineering problems.

Crespo, Luis G.↗
NASA NTRS2013

Achieving Climate Change Absolute Accuracy in Orbit

The Climate Absolute Radiance and Refractivity Observatory (CLARREO) mission will provide a calibration laboratory in orbit for the purpose of accurately measuring and attributing climate change. CLARREO measurements establish new climate change benchmarks with high absolute radiometric accuracy and high statistical confidence across a wide range of essential climate variables. CLARREO's inherently high absolute accuracy will be verified and traceable on orbit to Système Internationale (SI) units. The benchmarks established by CLARREO will be critical for assessing changes in the Earth system and climate model predictive capabilities for decades into the future as society works to meet the challenge of optimizing strategies for mitigating and adapting to climate change. The CLARREO benchmarks are derived from measurements of the Earth's thermal infrared spectrum (5-50 micron), the spectrum of solar radiation reflected by the Earth and its atmosphere (320-2300 nm), and radio occultation refractivity from which accurate temperature profiles are derived. The mission has the ability to provide new spectral fingerprints of climate change, as well as to provide the first orbiting radiometer with accuracy sufficient to serve as the reference transfer standard for other space sensors, in essence serving as a "NIST [National Institute of Standards and Technology] in orbit." CLARREO will greatly improve the accuracy and relevance of a wide range of space-borne instruments for decadal climate change. Finally, CLARREO has developed new metrics and methods for determining the accuracy requirements of climate observations for a wide range of climate variables and uncertainty sources. These methods should be useful for improving our understanding of observing requirements for most climate change observations.

Wielicki, Bruce A.↗
NASA NTRS2017

The Relationships Between the Trends of Mean and Extreme Precipitation

This study provides a better understanding of the relationships between the trends of mean and extreme precipitation in two observed precipitation data sets: the Climate Prediction Center Unified daily precipitation data set and the Global Precipitation Climatology Program (GPCP) pentad data set. The study employs three kinds of definitions of extreme precipitation: (1) percentile, (2) standard deviation and (3) generalize extreme value (GEV) distribution analysis for extreme events based on local statistics. Relationship between trends in the mean and extreme precipitation is identified with a novel metric, i.e. area aggregated matching ratio (AAMR) computed on regional and global scales. Generally, more (less) extreme events are likely to occur in regions with a positive (negative) mean trend. The match between the mean and extreme trends deteriorates for increasingly heavy precipitation events. The AAMR is higher in regions with negative mean trends than in regions with positive mean trends, suggesting a higher likelihood of severe dry events, compared with heavy rain events in a warming climate. AAMR is found to be higher in tropics and oceans than in the extratropics and land regions, reflecting a higher degree of randomness and more important dynamical rather than thermodynamical contributions of extreme events in the latter regions.

global warming↗
NASA NTRS2019

Quantifying the Storm Time Thermospheric Neutral Density Variations Using Model and Observations

Accurate determination of thermospheric neutral density holds crucial importance for satellited rag calculations. The problem is twofold and involves the correct estimation of the quiet time climatology and storm time variations. In this work, neutral density estimations from two empirical and threephysics based models of the ionosphere thermosphere are compared with the neutral densities along the Challenging MicroSatellite Payload satellite track for six geomagnetic storms. Storm time variations are extracted from neutral density by (1) subtracting the mean difference between model and observation (bias),(2) setting climatological variations to zero, and (3) multiplying model data with the quiet time ratio between the model and observation. Several metrics are employed to evaluate the model performances. We find that the removal of bias or climatology reveals actual performance of the model in simulating the storm time variations. When bias is removed, depending on event and model, storm time errors in neutral density can decrease by an amount of 113% or can increase by an amount of 12% with respect to error in models with quiet time bias. It is shown that using only average and maximum values of neutral density to determine the model performances can be misleading since a model can estimate the averages fairly well but may not capture the maximum value or vice versa. Since each of the metrics used for determining model performances provides different aspects of the error, among these, we suggest employing mean absolute error, prediction efficiency, and normalized root mean square error together as a standard set ofmetrics for the neutral density.

Eyiguler, E. Ceren Kalafatoglu↗
NASA NTRSDate not supplied

Improving Computational Efficiency of Prognostics Algorithms in Resource-Constrained Settings

In engineering and aerospace applications, it is vital to operational success to have insight into the expected performance and health of physical systems. The field of prognostics and health management provides quantitative methods for monitoring, predicting, and managing system health. Prognostics algorithms can be employed to assess the current state of a system, propagate the system throughout time, and predict potential anomalies or failures that may occur. While they can provide accurate prediction results, effective prognostics algorithms can be challenging to use in resource-constrained settings due to computational limitations and high computational latency, leading to obsolete predictions. Thus, computationally efficient and accurate algorithms are necessary for future remaining useful life predictions. In this work, we implement new algorithmic approaches for prediction, quantitatively compare them via a battery degradation use-case, and provide recommendations of potential improvements to a prognostics framework. One approach to prediction is through sampling, whereby the current state of a physical system is sampled many times and each sample is propagated forward until failure is reached, resulting in a distribution of failure values. To improve the efficiency of this process, we implemented five new algorithmic approaches to prediction, including three distinct sampling methods (standard Monte Carlo, Quasi-Monte Carlo, and Latin Hypercube Sampling), a variable time step algorithm, and a variable sample size algorithm. To compare the algorithms, we employ a variety of metrics designed specifically to analyze both computational efficiency and model accuracy. Our metrics include accuracy to compare the average predicted value to ground truth, mean absolute deviation to illustrate dispersion, specific percentile error to describe accuracy within a user-defined risk tolerance, and code run-time. To quantitatively analyze our results, we employ a use-case of degradation of a Lithium-ion battery. We use an electrochemistry-based model to describe the current health state of the battery, and implement our prediction algorithms to propagate forward in time until end-of-discharge (EOD) is reached. Notably, through this work it was found that none of our sampling approaches had a significant impact on computational efficiency or model accuracy in predicting EOD of the battery. We find that while the sampling methods are unique, the distributions they generate are similar, ultimately producing final predictions that are nearly identical. In exploring the effect of the time step within the prediction algorithm, we found that prediction accuracy was highly dependent on the time step used, and that implementing a variable time step within a particular prediction may provide an increase in computational efficiency while also maintaining prediction accuracy. Finally, implementing a variable sample size also affected prediction, and our results show that tuning both the magnitude and timing of the sample size adjustment can result in improved computation speed and maintained prediction accuracy. Taken together, our findings highlight the challenge of performing prognostics in resource-constrained settings, and illustrate the potential of developing new prediction algorithms to improve computational efficiency.

prognostics↗
NASA NTRS2023

How Have Hydrological Extremes Changed Over the Past 20 Years?

Severe floods and droughts, including their back-to-back occurrences (weather whiplash), have been increasing in frequency and severity around the world. Improved understanding of systematic changes in hydrological extremes is essential for preparation and adaptation. In this study, we identified and quantified extreme wet and dry events globally by applying a clustering algorithm to terrestrial water storage (TWS) data from the Gravity Recovery and Climate Experiment (GRACE) and GRACE Follow-On (FO). The most intense events, ranked using an intensity metric, often reflect impacts of large-scale oceanic oscillations such as El Niño–Southern Oscillation and consequences of climate change. The severity of both wet and dry events, represented by standardized TWS anomalies, increased significantly in most cases, likely associated with intensification of wet and dry weather regimes in a warmer world, and consequently, exhibited strongest correlation with global temperature. In the Dry climate, the number of wet events decreased while the number of dry events increased significantly, suggesting a drying trend that may be attributed to climate variability and possible increases in irrigation and reliance on groundwater. In the Continental climate where temperature has risen faster than global average, dry events increased significantly. Characteristics of extreme events often showed strong correlations with global temperature, especially when averaged over all climates. These results suggest changes in hydrological extremes and underscore the importance of quantifying total water storage changes when studying hydrological extremes. Extending the GRACE/FO record, which spans 2002 to the present, is essential to continuously tracking changes in TWS and hydrological extremes.

Bailing Li↗
NASA NTRSDate not supplied

Visualizing Geospatial Data through ESRI Story Maps for Earth Science Education: Lessons Learned from My NASA Data

For 20 years My NASA Data (MND) has curated NASA Earth science data and provided the data to educators in engaging learner-centered resources. MND has recently featured story maps as an innovative way to engage students in NASA Earth data. A story map is a cloud-based lesson that engages the learner in interactive geospatial maps using NASA data, and other multimedia content, text, and tasks that can be seamlessly incorporated in classroom instruction. This immersive technology eliminates the need for the user to move among tricky interfaces to access and visualize Earth science data, and no special software is required to be downloaded. Each story map integrates data from different NASA satellite missions, retrieved from Distributed Active Archive Centers (DAACs). Story maps also employ data analysis tools, such as time series options and swipe tools that allow learners to view and analyze relationships between scientific variables. MND has produced 25 story map lesson plans on the topics of air quality, the urban heat island effect, Earth’s energy budget, phytoplankton distribution, hurricane formation, solar eclipses, ocean circulation patterns, sea ice extent, and volcanic eruptions. Nine of them are extended story maps and written in the 5E format, which is internationally recognized as best practice based on how children learn science. Each story map resource is developed by the MND team featuring a GIS programming specialist, a lead scientist, and educational specialist/s to ensure the context, content, and methods are scientifically and educationally sound. The MND story maps are written for middle and high school science teachers and students as they connect with the Earth Systems Science phenomena featured in the Next Generation Science Standards. Each story map includes supporting resources for smooth integration in the classroom. During Fiscal Year 2023, The My NASA Data website received over 100,000 story map engagements during. These metrics highlight the interest in story maps as an Earth Science educational resource.

Desiray Wilson↗
NASA NTRS2013

NASA Software Engineering Benchmarking Study

To identify best practices for the improvement of software engineering on projects, NASA's Offices of Chief Engineer (OCE) and Safety and Mission Assurance (OSMA) formed a team led by Heather Rarick and Sally Godfrey to conduct this benchmarking study. The primary goals of the study are to identify best practices that: Improve the management and technical development of software intensive systems; Have a track record of successful deployment by aerospace industries, universities [including research and development (R&D) laboratories], and defense services, as well as NASA's own component Centers; and Identify candidate solutions for NASA's software issues. Beginning in the late fall of 2010, focus topics were chosen and interview questions were developed, based on the NASA top software challenges. Between February 2011 and November 2011, the Benchmark Team interviewed a total of 18 organizations, consisting of five NASA Centers, five industry organizations, four defense services organizations, and four university or university R and D laboratory organizations. A software assurance representative also participated in each of the interviews to focus on assurance and software safety best practices. Interviewees provided a wealth of information on each topic area that included: software policy, software acquisition, software assurance, testing, training, maintaining rigor in small projects, metrics, and use of the Capability Maturity Model Integration (CMMI) framework, as well as a number of special topics that came up in the discussions. NASA's software engineering practices compared favorably with the external organizations in most benchmark areas, but in every topic, there were ways in which NASA could improve its practices. Compared to defense services organizations and some of the industry organizations, one of NASA's notable weaknesses involved communication with contractors regarding its policies and requirements for acquired software. One of NASA's strengths was its software assurance practices, which seemed to rate well in comparison to the other organizational groups and also seemed to include a larger scope of activities. An unexpected benefit of the software benchmarking study was the identification of many opportunities for collaboration in areas including metrics, training, sharing of CMMI experiences and resources such as instructors and CMMI Lead Appraisers, and even sharing of assets such as documented processes. A further unexpected benefit of the study was the feedback on NASA practices that was received from some of the organizations interviewed. From that feedback, other potential areas where NASA could improve were highlighted, such as accuracy of software cost estimation and budgetary practices. The detailed report contains discussion of the practices noted in each of the topic areas, as well as a summary of observations and recommendations from each of the topic areas. The resulting 24 recommendations from the topic areas were then consolidated to eliminate duplication and culled into a set of 14 suggested actionable recommendations. This final set of actionable recommendations, listed below, are items that can be implemented to improve NASA's software engineering practices and to help address many of the items that were listed in the NASA top software engineering issues. 1. Develop and implement standard contract language for software procurements. 2. Advance accurate and trusted software cost estimates for both procured and in-house software and improve the capture of actual cost data to facilitate further improvements. 3. Establish a consistent set of objectives and expectations, specifically types of metrics at the Agency level, so key trends and models can be identified and used to continuously improve software processes and each software development effort. 4. Maintain the CMMI Maturity Level requirement for critical NASA projects and use CMMI to measure organizations developing software for NASA. 5.onsolidate, collect and, if needed, develop common processes principles and other assets across the Agency in order to provide more consistency in software development and acquisition practices and to reduce the overall cost of maintaining or increasing current NASA CMMI maturity levels. 6. Provide additional support for small projects that includes: (a) guidance for appropriate tailoring of requirements for small projects, (b) availability of suitable tools, including support tool set-up and training, and (c) training for small project personnel, assurance personnel and technical authorities on the acceptable options for tailoring requirements and performing assurance on small projects. 7. Develop software training classes for the more experienced software engineers using on-line training, videos, or small separate modules of training that can be accommodated as needed throughout a project. 8. Create guidelines to structure non-classroom training opportunities such as mentoring, peer reviews, lessons learned sessions, and on-the-job training. 9. Develop a set of predictive software defect data and a process for assessing software testing metric data against it. 10. Assess Agency-wide licenses for commonly used software tools. 11. Fill the knowledge gap in common software engineering practices for new hires and co-ops.12. Work through the Science, Technology, Engineering and Mathematics (STEM) program with universities in strengthening education in the use of common software engineering practices and standards. 13. Follow up this benchmark study with a deeper look into what both internal and external organizations perceive as the scope of software assurance, the value they expect to obtain from it, and the shortcomings they experience in the current practice. 14. Continue interactions with external software engineering environment through collaborations, knowledge sharing, and benchmarking.

Rarick, Heather L.↗
NASA NTRS2014

Grid Resolution Study over Operability Space for a Mach 1.7 Low Boom External Compression Inlet

This paper presents a statistical methodology whereby the probability limits associated with CFD grid resolution of inlet flow analysis can be determined which provide quantitative information on the distribution of that error over the specified operability range. The objectives of this investigation is to quantify the effects of both random (accuracy) and systemic (biasing) errors associated with grid resolution in the analysis of the Lockheed Martin Company (LMCO) N+2 Low Boom external compression supersonic inlet. The study covers the entire operability space as defined previously by the High Speed Civil Transport (HSCT) High Speed Research (HSR) program goals. The probability limits in terms of a 95.0% confidence interval on the analysis data were evaluated for four ARP1420 inlet metrics, namely (1) total pressure recovery (PFAIP), (2) radial hub distortion (DPH/P), (3) ) radial tip distortion (DPT/P), and (4) ) circumferential distortion (DPC/P). In general, the resulting +/-0.95 delta Y interval was unacceptably large in comparison to the stated goals of the HSCT program. Therefore, the conclusion was reached that the "standard grid" size was insufficient for this type of analysis. However, in examining the statistical data, it was determined that the CFD analysis results at the outer fringes of the operability space were the determining factor in the measure of statistical uncertainty. Adequate grids are grids that are free of biasing (systemic) errors and exhibit low random (precision) errors in comparison to their operability goals. In order to be 100% certain that the operability goals have indeed been achieved for each of the inlet metrics, the Y+/-0.95 delta Y limit must fall inside the stated operability goals. For example, if the operability goal for DPC/P circumferential distortion is ≤0.06, then the forecast Y for DPC/P plus the 95% confidence interval on DPC/P, i.e. +/-0.95 delta Y, must all be less than or equal to 0.06.

Grid↗
NASA NTRSDate not supplied

Recovery Rate is an Improved Metric for Monitoring Bioburden in Cleanrooms

The Astromaterials Acquisition and Curation Office maintains seven cleanrooms for curating extraterrestrial samples from a variety of sample return missions. These cleanrooms are monitored for inorganic and particulate contamination and are maintained at ISO 7 to ISO 4 equivalent standards as dictated by collection requirements. As part of the advanced curation effort and to prepare for future sample collections from carbonaceous asteroids and eventually Mars we have begun monitoring the microbiology of these cleanrooms. Insights gained from monitoring these labs can be applied to improving planetary protection efforts.

Planetary Protection↗
NASA NTRS2016

The EOSDIS Products Usability for Disaster Response

The focus of the study is to categorize both NRT and standard data products based on applicability to the SDR-defined disaster types. This will identify which datasets from current NASA satellite missions instruments are best suited for disaster response. The distribution metrics of the products that have been used for studying various selected disasters that have occurred over last 5 years will be analyzed that include volume, number of files, number of users, user domains, user country, etc. This data usage analysis will provide information to the data centers staff that can help them develop the functionality and allocate the resources needed for enhanced access and timely availability of the data products that are critical for the time-sensitive analyses.

LANCE↗