Objective Structured Clinical Evaluation (OSCE) of an Artificial Intelligence (AI) Clinical Decision Support System (CDSS) Tool
Explore the source record for details and available documents.
Ariana Nelson↗
SEARCH · Search NASA
Results for “decision tools”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Advanced Technology Lifecycle Analysis System (ATLAS) Technology Tool Box (TTB)
The Advanced Technology Lifecycle Analysis System (ATLAS) is a decision support tool designed to aid program managers and strategic planners in determining how to invest technology research and development dollars. It is an Excel-based modeling package that allows a user to build complex space architectures and evaluate the impact of various technology choices. ATLAS contains system models, cost and operations models, a campaign timeline and a centralized technology database. Technology data for all system models is drawn from a common database, the ATLAS Technology Tool Box (TTB). The TTB provides a comprehensive, architecture-independent technology database that is keyed to current and future timeframes.
Doyle, Monica↗
Evaluation of Usability and Workload with Paper Strips as Compared to Virtual Flight Strips Used for Ramp Operations
This paper describes an experiment designed to compare the use of paper strips with the use of a new user interface, the Ramp Traffic Console (RTC), designed for use by ramp controllers to be used in place of paper strips. A Human-In-the-Loop (HITL) experiment was performed as the fifth study in a series of six HITL simulation experiments designed to evaluate a concept that provided advisories to the users. The RTC was designed to be used as a Decision Support Tool (DST) that provided advisories to ramp controllers regarding metering or pushback such that most of the delay was taken at the gate to save fuel and emissions. In addition to being a DST, an added benefit of the RTC is that it can provide real-time updates of flight data, airport and airspace status to the controller including Traffic Management Initiatives (TMI). The RTC was designed as new user interface that displays virtual strips on a terminal map drawn on a 27-inch touch screen monitor. The RTC was used in some conditions of the experiment by ramp controllers in place of paper strips and paper maps in the HITL environment. In other conditions the controllers were given paper strips and paper maps similar to what they currently use at Charlotte Douglas International Airport (CLT). The study described here, evaluated the use of the virtual strips displayed on the RTC as compared to the use of paper strips and paper map, using current ramp tower controllers at CLT as participants. The research question being asked was - How does management of ramp traffic affect user workload and usability ratings while using RTC to manage traffic in the ramp verses using paper strips? Workload for our purposes is defined by four components of the NASA-TLX (Task Load Index). Usability was assessed with two sets of usability questions - One set of usability questions addressed traffic management performance and the other set addressed issues of resources and efficiency. Both Post Run and Post Study questionnaire responses were gathered and the results were analyzed to assess controller workload and usability ratings under both conditions, virtual strips shown on RTC and Paper Strips. The results indicate that controllers perceived lower workload while using virtual strips displayed on RTC to manage ramp traffic. Usability ratings for Traffic management performance questions are lower in the virtual strip/RTC condition than in the paper strip condition showing a preference for RTC over Paper. Usability ratings for Resources and efficiency questions show mixed results. Additionally, the Post Study Questions show preference for RTC over paper strips. Results of this data analysis will be presented in this paper. This DST evaluation was an important step in researching and improving the tool, which was planned to be deployed in the field.
Aviation Decision Support Tools↗
NASA-Modified Precipitation Products to Improve EPA Nonpoint Source Water Quality Modeling for the Chesapeake Bay
The Environmental Protection Agency (EPA) has estimated that over 20,000 water bodies within the United States do not meet water quality standards. Ninety percent of the impairments are typically caused by nonpoint sources. One of the regulations in the Clean Water Act of 1972 requires States to monitor the Total Maximum Daily Load (TMDL), or the amount of pollution that can be carried by a water body before it is determined to be "polluted", for any watershed in the U.S.. In response to this mandate, the EPA developed Better Assessment Science Integrating Nonpoint Sources (BASINS) as a Decision Support Tool (DST) for assessing pollution and to guide the decision making process for improving water quality. One of the models in BASINS, the Hydrological Simulation Program -- Fortran (HSPF), computes daily stream flow rates and pollutant concentration at each basin outlet. By design, precipitation and other meteorological data from weather stations serve as standard model input. In practice, these stations may be unable to capture the spatial heterogeneity of precipitation events especially if they are few and far between. An attempt was made to resolve this issue by substituting station data with NASA modified/NOAA precipitation data. Using these data within HSPF, stream flow was calculated for seven watersheds in the Chesapeake Bay Basin during low flow periods, convective storm periods, and annual flows. In almost every case, the modeling performance of HSPF increased when using the NASA-modified precipitation data, resulting in better stream flow statistics and, ultimately, in improved water quality assessment.
Nigro, Joseph↗
Development of the Surface Management System Integrated with CTAS Arrival Tools
The Surface Management System (SMS) developed by NASA Ames Research Center in coordination with the Federal Aviation Administration (FAA) is a decision support tool to help tower traffic coordinators and Ground/Local controllers in managing and controlling airport surface traffic in order to increase capacity, efficiency, and flexibility. SMS provides common situation awareness to personnel at various air traffic control facilities such as airport traffic control towers (ATCT s), airline ramp towers, Terminal Radar Approach Control (TRACON), and Air Route Traffic Control Center (ARTCC). SMS also provides a traffic management tool to assist ATCT traffic management coordinators (TMCs) in making decisions such as airport configuration and runway load balancing. The Build 1 of the SMS tool was installed and successfully tested at Memphis International Airport (MEM) and received high acceptance scores from ATCT controllers and coordinators, as well as airline ramp controllers. NASA Ames Research Center continues to develop SMS under NASA s Strategic Airspace Usage (SAU) project in order to improve its prediction accuracy and robustness under various modeling uncertainties. This paper reports the recent development effort performed by the NASA Ames Research Center: 1) integration of Center TRACON Automation System (CTAS) capability with SMS and 2) an alternative approach to obtain airline gate information through a publicly available website. The preliminary analysis results performed on the air/surface traffic data at the DFW airport have shown significant improvement in predicting airport arrival demand and IN time at the gate. This paper concludes with recommendations for future research and development.
Jung, Yoon C.↗
SARDA: An Integrated Concept for Airport Surface Operations Management
The Spot and Runway Departure Advisor (SARDA) is an integrated decision support tool for airlines and air traffic control tower enabling surface collaborative decision making (CDM) and departure metering in order to enhance efficiency of surface operations at congested airports. The presentation describes the concept and architecture of the SARDA as a CDM tool, and the results from a human-in-the-loop simulation of the tool conducted in 2012 at the FutureFlight Central, the tower simulation facility. Also, presented is the current activities and future plan for SARDA development. The presentation was given at the meeting with the FAA senior advisor of the Surface Operations Office.
surface collaborative decision making↗
An Integrated Tool Suite for En Route Radar Controllers in NextGen
This paper describes recent human-in-the-loop research in the Airspace Operations Laboratory at the NASA Ames Research Center focusing on en route air traffic management with advanced trajectory planning tools and increased levels of human-automation cooperation. The decision support tools were exercised in a simulation of seven contiguous high-altitude sectors. Preliminary data suggests the controllers were able to manage higher amounts of traffic as compared to today, while maintaining acceptable levels of workload.
Mercer, Joey↗
Analysis of ETMS Data Quality for Traffic Flow Management Decisions
The data needed for air traffic flow management decision support tools is provided by the Enhanced Traffic Management System (ETMS). This includes both the tools that are in current use and the ones being developed for future deployment. Since the quality of decision support provided by all these tools will be influenced by the quality of the input ETMS data, an assessment of ETMS data quality is needed. Motivated by this desire, ETMS data quality is examined in this paper in terms of the unavailability of flight plans, deviation from the filed flight plans, departure delays, altitude errors and track data drops. Although many of these data quality issues are not new, little is known about their extent. A goal of this paper is to document the magnitude of data quality issues supported by numerical analysis of ETMS data. Guided by this goal, ETMS data for a 24-hour period were processed to determine the number of aircraft with missing flight plan messages at any given instant of time. Results are presented for aircraft above 18,000 feet altitude and also at all altitudes. Since deviation from filed flight plan is also a major cause of trajectory-modeling errors, statistics of deviations are presented. Errors in proposed departure times and ETMS-generated vertical profiles are also shown. A method for conditioning the vertical profiles for improving demand prediction accuracy is described. Graphs of actual sector counts obtained using these vertical profiles are compared with those obtained using the Host data for sectors in the Fort Worth Center to demonstrate the benefit of preprocessing. Finally, results are presented to quantify the extent of data drops. A method for propagating track positions during ETMS data drops is also described.
Chatterji, Gano B.↗
Market Survey 2020: Commercial Clinical Decision Support Systems and Wellness Tools
For long-duration, deep space exploration missions, current methods for managing and supporting crew health and medical conditions will be unsuitable. Communication and data transmission lags will necessitate the use of a sophisticated clinical decision support system (CDSS) that will tailor diagnosis and treatment guidance that is context-sensitive for anticipated astronaut health, wellness, and medical conditions. A variety of clinical decision support (CDS) and wellness tools (WT) are currently available in the commercial market and a broad-brush survey of this market can provide an initial impression of the current state of the art which, in turn, can inform the roadmap of NASA deep space CDSS development and associated requirements. Such a survey was undertaken during the first six months of 2020 using directed convenience sampling to obtain information provided by vendors on their websites; both commercially available CDS and WT (such as those used to track and monitor nutrition, exercise, and sleep) were included. Areas assessed were item type (e.g., software/application, device); primary purpose of the item (e.g., diagnostic support, nutrition tracking); additional purposes (if any); reported features, capabilities, and functionality; setting of use (e.g., inpatient, outpatient); intended user (e.g., clinician, patient); location and sources of data/information used or produced by the item; integration with patient electronic health record (EHR); compliance with interoperability ontologies and standards (e.g., Health Level 7 [HL7], Systematized Nomenclature of Medicine – Clinical Terminology [SNOMED-CT]); and whether the item is knowledge-based (derived from research findings) or non-knowledge-based (derived through artificial intelligence, machine learning, advanced probability and statistics), among others. Ninety-seven (97) vendor websites describing 196 CDS and 73 WT (269 total) were reviewed and coded. The primary purpose of the majority of CDS reviewed is diagnosis or diagnosis/treatment/drug decision support—targeted for clinician use— and the primary purpose of the majority of WT reviewed is the monitoring of different health metrics, most often through the use of a biosensor device (e.g., blood pressure)—targeted for patient use. Very few CDS or WT appear to comply with major international interoperability standards or can be integrated with a patient’s EHR data. None consider contextual factors, such as conditions of the physical environment (e.g., CO2 levels). The majority of CDS and WT reviewed are non-knowledge, cloud- or web-based applications or software. Forty-three (43) major findings were identified and the implications those findings have for NASA will be discussed. Example major findings include: CDS-WT capabilities range from diagnosis to treatment applications, CDS-WT may be wearable or non-wearable and are technologically advanced and only a few CDS-WT tools referenced compliance to ensure interoperability, among other findings. Recommendations will also be offered that will help to address ExMC Gap, Medical-701: Enhance medical capabilities within an exploration medical system.
market survey↗
Real-Time Science Operations to Support a Lunar Polar Volatiles Rover Mission
Future human exploration of the Moon will likely rely on in situ resource utilization (ISRU) to enable long duration lunar missions. Prior to utilizing ISRU on the Moon, the natural resources (in this case lunar volatiles) must be identified and characterized, and ISRU demonstrated on the lunar surface. To enable future uses of ISRU, NASA and the CSA are developing a lunar rover payload that can (1) locate near subsurface volatiles, (2) excavate and analyze samples of the volatile-bearing regolith, and (3) demonstrate the form, extractability and usefulness of the materials. Such investigations are important both for ISRU purposes and for understanding the scientific nature of these intriguing lunar volatile deposits. Temperature models and orbital data suggest near surface volatile concentrations may exist at briefly lit lunar polar locations outside persistently shadowed regions. A lunar rover could be remotely operated at some of these locations for the approx. 2-14 days of expected sunlight at relatively low cost. Due to the limited operational time available, both science and rover operations decisions must be made in real time, requiring immediate situational awareness, data analysis, and decision support tools. Given these constraints, such a mission requires a new concept of operations. In this paper we outline the results and lessons learned from an analog field campaign in July 2012 which tested operations for a lunar polar rover concept. A rover was operated in the analog environment of Hawaii by an off-site Flight Control Center, a rover navigation center in Canada, a Science Backroom at NASA Ames Research Center in California, and support teams at NASA Johnson Space Center in Texas and NASA Kennedy Space Center in Florida. We find that this type of mission requires highly efficient, real time, remotely operated rover operations to enable low cost, scientifically relevant exploration of the distribution and nature of lunar polar volatiles. The field demonstration illustrated the need for science operations personnel in constant communications with the flight mission operators and the Science Backroom to provide immediate and continual science support and validation throughout the mission. Specific data analysis tools are also required to enable immediate data monitoring, visualization, and decision making. The field campaign demonstrated that this novel methodology of real-time science operations is possible and applicable to providing important new insights regarding lunar polar volatiles for both science and exploration.
Moon; Volatiles; Rover; Missions↗
Impact Real World System Validation
Introduction NASA has developed a new evidence-based data-driven probabilistic risk assessment and tradespace analysis tool as a successor to the Integrated Medical Model. This updated decision support tool is known as IMPACT (Informing Mission Planning via Analysis of Complex Tradespaces). IMPACT estimates the frequency and consequences of medical conditions that might arise during exploration missions. A validation analysis of IMPACT was performed with respect to a set of International Space Station (ISS) and Shuttle Transportation System (STS) real world system (RWS) referent data due to the limited referent data available from exploration missions. Methods Observed mission and crew characteristics from STS and ISS missions were used as model inputs within MEDPRAT (Medical Extensible Dynamic Probabilistic Risk Assessment Tool). For each mission, two hundred thousand simulations were generated. For each mission, model outputs included occurrence counts for each condition, total medical events (TME), and the probability of loss of crew life (LOCL). These simulated model outputs were compared to the RWS referent data. Results The predicted number of total medical events exceeded the total RWS medical events for ISS missions and combined ISS and STS missions and fell within the 90% confidence interval for STS missions. For the 32 ISS missions simulated by IMPACT, the number of total medical events was overpredicted for 19 missions and fell within the 90% confidence interval for 13 missions. For the 21 STS missions, the total number of medical events was overpredicted for 3 missions, fell within the 90% confidence interval for 16 missions, and was underpredicted for 2 missions. Combined, 29 missions were in range, 22 were overpredicted, and 2 were underpredicted. The predicted LOCL probability for the 32 ISS missions, the 21 STS missions, and the combined ISS and STS missions was consistent with the zero LOCL events observed in the RWS referent data. The validation analysis included a comparison of the number of medical events predicted by IMPACT and the number of medical events observed in the RWS data on a condition-by-condition basis. For ISS missions, 50 conditions were in range, 52 conditions were statistically underpowered (not enough observed sample to draw any conclusions on precision), 8 conditions were overpredicted, and 9 conditions were underpredicted. Overall, only 14% (17/119) of conditions were out of range for STS missions, 40 conditions were in range, 59 conditions were statistically underpowered, 10 conditions were overpredicted, and 10 conditions were underpredicted. Overall, only 17% (20/119) of conditions were out of range. For combined ISS and STS missions, 11 conditions were overpredicted, and 11 conditions were underpredicted. Overall, only 18% (22/119) of conditions were out of range. For combined ISS and STS missions, 49 conditions were in range, 46 conditions were statistically underpowered, 18 conditions were overpredicted, and 8 conditions were underpredicted. Overall, 21% (26/121) of conditions were out of range. Conclusion The results of this validation analysis should not be interpreted as a pass/fail test of the validity of IMPACT. Instead, this validation analysis should be used to assess some of the IMPACT outcomes in terms of consistencies and inconsistencies with the ISS and STS RWS referent data.
L. Boley↗
Flight Awareness Collaboration Tool Development
This is a PowerPoint presentation covering airline operations center (AOC) research. It reviews a dispatcher decision support tool called the Flight Awareness Collaboration Tool (FACT). FACT gathers information about winter weather onto one screen and includes predictive abilities. FACT should prove to be useful for airline dispatchers and airport personnel when they manage winter storms and their effect on air traffic. This material is very similar to other previously approved presentations.
airlines↗
Baseline Medical System Translation for the Impact Medical Database
NASA has developed a new evidence-based data-driven probabilistic risk assessment and tradespace analysis tool as a successor to the Integrated Medical Model (IMM). This updated decision support tool is known as IMPACT (Informing Mission Planning via Analysis of Complex Tradespaces). Whereas IMM focuses on the resources and risks associated with International Space Station (ISS) and Low Earth Orbit (LEO) missions, IMPACT estimates the frequency and consequences of medical conditions that might arise during exploration missions. One of the services offered by IMM is a series of generic or baseline medical systems associated with typical mission types or DRMs (Design Reference Missions), such that requestors may prioritize questions pertaining to the mission itself over the medical supplies indicated by the model outputs for that DRM. In a mission focused request, the appropriate baseline medical system is used in place of a prepared or optimized medical kit. In preparation, an effort was undertaken to create baseline systems of medical resources within IMPACT suitable for typical DRMs. Using an existing baseline medical system within IMM’s Integrated Medical Evidence Database (iMED) as a starting point, the resources found within the medical kit were compared to and substituted for equivalent resources available within the IMPACT MD (Medical Database). In consultation with clinicians with knowledge of IMPACT’s MD, each resource was matched as closely as possible to a similarly purposed resource in IMPACT, seeking to preserve the treatment capabilities and procedures offered by the IMM medical system while reconciling the differences in modeled resources and medical conditions between the models. To demonstrate the efficacy of this work, a prototype ISS medical system in IMPACT was translated from the baseline used in the IMM. The appropriate medical system was then run through its associated medical model to compare and validate the resultant risks and risk mitigation provided by each baseline ISS medical kit.
S Schwartz↗
User's guide to the Mission Analysis Evaluation and Space Trajectory Operations program (MAESTRO)
The MAESTRO system is a mission analysis tool designed to present to the user information necessary to make the various decisions required in the design and execution of a spaceflight mission. The system was designed so that it can be used in both the pre-launch mission planning phase of a mission and during the flight as an in-flight decision making tool. A description of each of the following modes is presented: (1) trajectory propagation mode; (2) retro-fire determination mode; (3) midcourse analysis determination mode; (4) Monte Carlo mode; (5) verification mode; (6) orbit stability mode; and (7) post injection trim mode. A description of the inputs necessary to run the program mode is given along with a sample case.
Lutzky, D.↗
Overview: Spot and Runway Departure Advisor (SARDA)
Spot and Runway Departure Advisor (SARDA) is a decision support tool to assist airline ramp controllers and ATC tower controllers to manage traffic on the airport surface to significantly improve efficiency and predictability in surface operations. The core function of the tool is the runway scheduler which generates an optimal solution for runway sequence and schedule of departure aircraft, which would minimize system delay and maximize runway throughput. The presentation describes the concept of the SARDA tool and results from human-in-the-loop simulations conducted in 2012 for DallasFt. Worth International Airport. The presentation also explains the latest status of NASA's current surface research through a collaboration with an airline partner. The presentation concludes with a discussion on other on-going as well as future surface research.
decision support tool↗
A Comparison of Satellite Based, Modeled Derived Daily Solar Radiation Data with Observed Data for the Continental US
Many applications of simulation models and related decision support tools for agriculture and natural resource management require daily meteorological data as inputs. Availability and quality of such data, however, often constrain research and decision support activities that require use of these tools. Daily solar radiation (SRAD) data are especially problematic because the instruments require electronic integrators, accurate sensors are expensive, and calibration standards are seldom available. The Prediction Of Worldwide Energy Resources (NASA/POWER; power.larc.nasa.gov) project at the NASA Langley Research Center estimates daily solar radiation based on data that are derived from satellite observations of outgoing visible radiances and atmospheric parameters based upon satellite observations and assimilation models. The solar data are available for a global 1 degree x 1 degree coordinate grid. SRAD can also be estimated based on attenuation of extraterrestrial radiation (Q0) using daily temperature and rainfall data to estimate the optical thickness of the atmosphere. This study compares daily solar radiation data from NASA/POWER (SRADNP) with instrument readings from 295 stations (SRADOB), as well as with values that were estimated with the WGENR solar generator. WGENR was used both with daily temperature and precipitation records from the stations reporting solar data and records from the NOAA Cooperative Observer Program (COOP), thus providing two additional sources of solar data, SRADWG and SRADCO. Values of SRADNP for different grid cells consistently showed higher correlations (typically 0.85 to 0.95) with SRADOB data than did SRADWG or SRADCO for sites within the corresponding cells. Mean values of SRADOB, SRADWG and SRADNP for sites within a grid cell usually were within 1 MJm-2d-1 of each other, but NASA/POWER values averaged 1.1 MJm-2d-1 lower than SRADOB. The magnitude of this bias was greater at lower latitudes and during summer months and may be at least partially explained by assumptions in ambient aerosol properties. Overall, the NASA/POWER solar radiation data are a promising resource for regional modeling studies where realistic accounting of historic variation is required.
White, Jeffrey W.↗
Integrated National and Regional Agricultural Planning with AgMIP Tools and Methods
AgMIP, through its unique integration of multi-modeling approaches with stakeholder engagement, has advanced co-development of decision support tools and adaptation strategies in the face of climate change and other shocks (especially COVID-19). This has helped generate actionable information for adaptation planning processes in developing countries. Local-to-national stakeholders and decision-makers appreciate the research-informed stakeholder dialogue as very helpful in identifying research and policy gaps. These motivate concise recommendations for productivity-enhancing, environmentally sound technologies, and inclusive development approaches. Furthermore, these stakeholder dialogues help bridge the gaps between policy, research, and adaptation strategies, especially for low-input smallholder farming systems.
Agricultural Model Intercomparison and Improvement↗
Noise Mitigation Controller Tools
During the past decade, the National Aeronautics and Space Administration (NASA) has been developing and evaluating a suite of decision support tools (DSTs) to aid the air traffic controller in the management of traffic. These tools are known collectively as the Center/TRACON Automation System (CTAS). The primary focus of CTAS is increased capacity. As part of a new NASA program called Quiet Aircraft Technology (QAT), the following question is being addressed: Can CTAS technology also support the noise mitigation requirements imposed by the community? Controllers currently support a variety of low noise procedures in low traffic densities but, as traffic increases, these must be abandoned due to excessive spacing requirements for vectoring or inter-arrival spacing requirements needed to handle a spectrum of low noise procedures. NASA is currently investigating how to provide controllers with noise-mitigation-based advisories which address these issues without negatively impacting capacity. These issues are of global concern which must be addressed as the demand for air travel continues to increase.
Tobias, Leonard↗