SEARCH · Search NASA

Results for “metric categorization”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

107 records · Page 6

All sources NASA NTRS DOE OSTI

NASA NTRS2000

A Matrix Approach to Software Process Definition

The Software Engineering Laboratory (SEL) is currently engaged in a Methodology and Metrics program for the Information Systems Center (ISC) at Goddard Space Flight Center (GSFC). This paper addresses the Methodology portion of the program. The purpose of the Methodology effort is to assist a software team lead in selecting and tailoring a software development or maintenance process for a specific GSFC project. It is intended that this process will also be compliant with both ISO 9001 and the Software Engineering Institute's Capability Maturity Model (CMM). Under the Methodology program, we have defined four standard ISO-compliant software processes for the ISC, and three tailoring criteria that team leads can use to categorize their projects. The team lead would select a process and appropriate tailoring factors, from which a software process tailored to the specific project could be generated. Our objective in the Methodology program is to present software process information in a structured fashion, to make it easy for a team lead to characterize the type of software engineering to be performed, and to apply tailoring parameters to search for an appropriate software process description. This will enable the team lead to follow a proven, effective software process and also satisfy NASA's requirement for compliance with ISO 9001 and the anticipated requirement for CMM assessment. This work is also intended to support the deployment of sound software processes across the ISC.

Schultz, David↗

DOE OSTI2023

The Current State of Light-Duty Electric Vehicle Supply Equipment Costs: An Assessment of Contemporary Understanding

This study uses a hybrid meta-analysis and literature-review approach to understand the current state of knowledge regarding the costs of electric vehicle supply equipment (EVSE). We present a novel way to consider, categorize, and label measures of cost and show cost measure estimates from a sample of 13 recent studies. We find that in general, there is too much variation and too few commonly represented EVSE cost measures to reasonably provide aggregate figures for these measures. We propose a convention for presenting EVSE cost measures that includes the application (commercial or residential), the power level (Level 1, Level 2, DCFC [with further distinction based on rated power capacity]), and the type of cost measure (hardware, installation, operation, and total cost). We contend that providing researchers with standard cost measures will help to advance our knowledge of EVSE costs by ensuring that future work will use common metrics. Establishing common metrics will enable conventional meta-analyses that will make assessments of EVSE costs even more accessible. Additionally, common metrics will make tracking costs more reliable as the technology continues to evolve and become more ubiquitous.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗

NASA NTRS2008

Aircraft System Analysis of Technology Benefits to Civil Transport Rotorcraft

An aircraft systems analysis was conducted to evaluate the net benefits of advanced technologies on two conceptual civil transport rotorcraft, to quantify the potential of future civil rotorcraft to become operationally viable and economically competitive, with the ultimate goal of alleviating congestion in our airways, runways and terminals. These questions are three of many that must be resolved for the successful introduction of civil transport rotorcraft: 1) Can civil transport rotorcraft actually relieve current airport congestion and improve overall air traffic and passenger throughput at busy hub airports? What is that operational scenario? 2) Can advanced technology make future civil rotorcraft economically competitive in scheduled passenger transport? What are those enabling technologies? 3) What level of investment is necessary to mature the key enabling technologies? This study addresses the first two questions, and several others, by applying a systems analysis approach to a broad spectrum of potential advanced technologies at a conceptual level of design. The method was to identify those advanced technologies that showed the most promise and to quantify their benefits to the design, development, production, and operation of future civil rotorcraft. Adjustments are made to sizing data by subject matter experts to reflect the introduction of new technologies that offer improved performance, reduced weight, reduced maintenance, or reduced cost. This study used projected benefits from new, advanced technologies, generally based on research results, analysis, or small-scale test data. The technologies are identified, categorized and quantified in the report. The net benefit of selected advanced technologies is quantified for two civil transport rotorcraft concepts, a Single Main Rotor Compound (SMRC) helicopter designed for 250 ktas cruise airspeed and a Civil Tilt Rotor (CTR) designed for 350 ktas cruise airspeed. A baseline design of each concept was sized for a representative civil passenger transport mission, using current technology. Individual advanced technologies are quantified and applied to resize the aircraft, thereby quantifying the net benefit of that technology to the rotorcraft. Estimates of development cost, production cost and operating and support costs are made with a commercial cost estimating program, calibrated to Boeing products with adjustments for future civil production processes. A cost metric of cash direct operating cost per available seat-mile (DOC ASM) is used to compare the cost benefit of the technologies. The same metric is used to compare results with turboprop operating costs. Reduced engine SFC was the most advantageous advanced technology for both rotorcraft concepts. Structural weight reduction was the second most beneficial technology, followed by advanced drive systems and then by technology for rotorcraft performance. Most of the technologies evaluated in this report should apply similarly to conventional helicopters. The implicit assumption is that resources will become available to mature the technologies for fullscale production aircraft. That assumption is certainly the weak link in any forecast of future possibilities. The analysis serves the purpose of identifying which technologies offer the most potential benefit, and thus the ones that should receive the highest priority for continued development. This study directly addressed the following NASA Subsonic Rotary Wing (SRW) subtopics: SR W.4.8.I.J Establish capability for rotorcraft system analysis and SRW. 4.8.I.4 Conduct limited technology benefit assessment on baseline rotorcraft configurations.

Wilkerson, Joseph B.↗

DOE OSTI2023

Medium-temperature phase change material integration in domestic heat pump water heaters for improved thermal energy storage

In this review, we examine state-of-the-art developments in integrating phase change materials (PCMs) for thermal energy storage (TES) in domestic heat pump water heaters (HPWHs). The component design optimization and control optimization of HPWHs and TES are reviewed for insight into improving the thermal capacity and efficiency of a PCM-integrated HPWH. The state-of-the-art review is categorized by the stage of development of the PCM for deployment in HPWHs. To select appropriate PCMs for HPWHs, a six-factor down-selection process is used to determine the best material(s) for integration in HPWHs with appropriate heat exchanger design. Ultimately, food-grade PCMs appear to be the best candidate for integration of TES in domestic HPWHs because they are nontoxic, highly cyclable, and have heat transfer properties accommodable to water heating when integrated in a manner to overcome the thermal conductivity limitations of the material. A key parameter of water heating performance is thermal heating power of the PCM component, which is not often reported. Many studies report significant improvement in capacity and efficiency. Furthermore, many performance metrics are identified from the literature to quantify the system performance, but agreement across studies is not found. Unified energy factor and first hour rating performance tests are required for commercially available HPWHs, and these performance metrics could homogenize the literature. Ultimately, we find that select PCMs are ready for deployment with HPWHs for performance improvement, and component design and control optimizations are new avenues of research and development required for a commercially viable PCM-integrated HPWH system.

25 ENERGY STORAGE↗

NASA NTRS1995

Cognitive models of pilot categorization and prioritization of flight-deck information

In the past decade, automated systems on modern commercial flight decks have increased dramatically. Pilots now regularly interact and share tasks with these systems. This interaction has led human factors research to direct more attention to the pilot's cognitive processing and mental model of the information flow occurring on the flight deck. The experiment reported herein investigated how pilots mentally represent and process information typically available during flight. Fifty-two commercial pilots participated in tasks that required them to provide similarity ratings for pairs of flight-deck information and to prioritize this information under two contextual conditions. Pilots processed the information along three cognitive dimensions. These dimensions included the flight function and the flight action that the information supported and how frequently pilots refer to the information. Pilots classified the information as aviation, navigation, communications, or systems administration information. Prioritization results indicated a high degree of consensus among pilots, while scaling results revealed two dimensions along which information is prioritized. Pilot cognitive workload for flight-deck tasks and the potential for using these findings to operationalize cognitive metrics are evaluated. Such measures may be useful additions for flight-deck human performance evaluation.

Jonsson, Jon E.↗

NASA NTRS2012

A Briefing on Metrics and Risks for Autonomous Decision-Making in Aerospace Applications

Significant technology advances will enable future aerospace systems to safely and reliably make decisions autonomously, or without human interaction. The decision-making may result in actions that enable an aircraft or spacecraft in an off-nominal state or with slightly degraded components to achieve mission performance and safety goals while reducing or avoiding damage to the aircraft or spacecraft. Some key technology enablers for autonomous decision-making include: a continuous state awareness through the maturation of the prognostics health management field, novel sensor development, and the considerable gains made in computation power and data processing bandwidth versus system size. Sophisticated algorithms and physics based models coupled with these technological advances allow reliable assessment of a system, subsystem, or components. Decisions that balance mission objectives and constraints with remaining useful life predictions can be made autonomously to maintain safety requirements, optimal performance, and ensure mission objectives. This autonomous approach to decision-making will come with new risks and benefits, some of which will be examined in this paper. To start, an account of previous work to categorize or quantify autonomy in aerospace systems will be presented. In addition, a survey of perceived risks in autonomous decision-making in the context of piloted aircraft and remotely piloted or completely autonomous unmanned autonomous systems (UAS) will be presented based on interviews that were conducted with individuals from industry, academia, and government.

Frost, Susan↗

NASA NTRS1997

A Categorization of Dynamic Analyzers

Program analysis techniques and tools are essential to the development process because of the support they provide in detecting errors and deficiencies at different phases of development. The types of information rendered through analysis includes the following: statistical measurements of code, type checks, dataflow analysis, consistency checks, test data,verification of code, and debugging information. Analyzers can be broken into two major categories: dynamic and static. Static analyzers examine programs with respect to syntax errors and structural properties., This includes gathering statistical information on program content, such as the number of lines of executable code, source lines. and cyclomatic complexity. In addition, static analyzers provide the ability to check for the consistency of programs with respect to variables. Dynamic analyzers in contrast are dependent on input and the execution of a program providing the ability to find errors that cannot be detected through the use of static analysis alone. Dynamic analysis provides information on the behavior of a program rather than on the syntax. Both types of analysis detect errors in a program, but dynamic analyzers accomplish this through run-time behavior. This paper focuses on the following broad classification of dynamic analyzers: 1) Metrics; 2) Models; and 3) Monitors. Metrics are those analyzers that provide measurement. The next category, models, captures those analyzers that present the state of the program to the user at specified points in time. The last category, monitors, checks specified code based on some criteria. The paper discusses each classification and the techniques that are included under them. In addition, the role of each technique in the software life cycle is discussed. Familiarization with the tools that measure, model and monitor programs provides a framework for understanding the program's dynamic behavior from different, perspectives through analysis of the input/output data.

Lujan, Michelle R.↗

NASA NTRSDate not supplied

Preliminary Steps in Developing Rapid Aero Modeling Technology

The Rapid Aero Modeling (RAM) approach is a method to efficiently and automatically obtain aerodynamic models during testing, significantly saving time and resources. Motivation for this technology results from demand for experimental efficiency and model fidelity that has increased with growing aircraft complexity and aerodynamic nonlinearities. These issues are typical in the responses presented by a class of vehicles categorized as Urban Air Mobility aircraft where many features from both airplane and rotorcraft are present. For UAM configurations, with typically many more factors than conventional aircraft, traditional test methods can lead to increased costs and missed interactions. RAM guides the test to obtain high-fidelity, statistically rigorous aircraft models, and the approach is applicable to computational, ground, or flight-test experiments. It combines concepts from Design of Experiment theory and Aircraft System Identification theory that allow the user the freedom to choose, in advance of the test, a specific level of fidelity typically, in terms of prediction error. RAM only collects data required to meet the user-specified fidelity and fidelity is only limited by the facility and test article capabilities. An initial wind tunnel test to support development of RAM was conducted to assess potential metrics, algorithms, and procedures. This paper presents results from initial tests for the development of RAM technology and highlights some of the unique features of RAM applied toa conventional configuration during a ground-based, static, wind-tunnel test.

Patrick C Murphy↗

DOE OSTI2021

Revisiting matrix-based inversion of scanning mobility particle sizer (SMPS) and humidified tandem differential mobility analyzer (HTDMA) data

Abstract. Tikhonov regularization is a tool for reducing noise amplification during data inversion. This work introduces RegularizationTools.jl, a general-purpose software package for applying Tikhonov regularization to data. The package implements well-established numerical algorithms and is suitable for systems of up to ∼ 1000 equations. Included is an abstraction to systematically categorize specific inversion configurations and their associated hyperparameters. A generic interface translates arbitrary linear forward models defined by a computer function into the corresponding design matrix. This obviates the need to explicitly write out and discretize the Fredholm integral equation, thus facilitating fast prototyping of new regularization schemes associated with measurement techniques. Example applications include the inversion involving data from scanning mobility particle sizers (SMPSs) and humidified tandem differential mobility analyzers (HTDMAs). Inversion of SMPS size distributions reported in this work builds upon the freely available software DifferentialMobilityAnalyzers.jl. The speed of inversion is improved by a factor of ∼ 200, now requiring between 2 and 5 ms per SMPS scan when using 120 size bins. Previously reported occasional failure to converge to a valid solution is reduced by switching from the L-curve method to generalized cross-validation as the metric to search for the optimal regularization parameter. Higher-order inversions resulting in smooth, denoised reconstructions of size distributions are now included in DifferentialMobilityAnalyzers.jl. This work also demonstrates that an SMPS-style matrix-based inversion can be applied to find the growth factor frequency distribution from raw HTDMA data while also accounting for multiply charged particles. The outcome of the aerosol-related inversion methods is showcased by inverting multi-week SMPS and HTDMA datasets from ground-based observations, including SMPS data obtained at Bodega Marine Laboratory during the CalWater 2/ACAPEX campaign and co-located SMPS and HTDMA data collected at the US Department of Energy observatory located at the Southern Great Plains site in Oklahoma, USA. Results show that the proposed approaches are suitable for unsupervised, nonparametric inversion of large-scale datasets as well as inversion in real time during data acquisition on low-cost reduced-instruction-set architectures used in single-board computers. The included software implementation of Tikhonov regularization is freely available, general, and domain-independent and thus can be applied to many other inverse problems arising in atmospheric measurement techniques and beyond.

54 ENVIRONMENTAL SCIENCES↗

NASA NTRS2013

US Rocket Propulsion Industrial Base Health Metrics

The number of active liquid rocket engine and solid rocket motor development programs has severely declined since the "space race" of the 1950s and 1960s center dot This downward trend has been exacerbated by the retirement of the Space Shuttle, transition from the Constellation Program to the Space launch System (SLS) and similar activity in DoD programs center dot In addition with consolidation in the industry, the rocket propulsion industrial base is under stress. To Improve the "health" of the RPIB, we need to understand - The current condition of the RPIB - How this compares to past history - The trend of RPIB health center dot This drives the need for a concise set of "metrics" - Analogous to the basic data a physician uses to determine the state of health of his patients - Easy to measure and collect - The trend is often more useful than the actual data point - Can be used to focus on problem areas and develop preventative measures The nation's capability to conceive, design, develop, manufacture, test, and support missions using liquid rocket engines and solid rocket motors that are critical to its national security, economic health and growth, and future scientific needs. center dot The RPIB encompasses US government, academic, and commercial (including industry primes and their supplier base) research, development, test, evaluation, and manufacturing capabilities and facilities. center dot The RPIB includes the skilled workforce, related intellectual property, engineering and support services, and supply chain operations and management. This definition touches the five main segments of the U.S. RPIB as categorized by the USG: defense, intelligence community, civil government, academia, and commercial sector. The nation's capability to conceive, design, develop, manufacture, test, and support missions using liquid rocket engines and solid rocket motors that are critical to its national security, economic health and growth, and future scientific needs. center dot The RPIB encompasses US government, academic, and commercial (including industry primes and their supplier base) research, development, test, evaluation, and manufacturing capabilities and facilities. center dot The RPIB includes the skilled workforce, related intellectual property, engineering and support services, and supply chain operations and management. This definition touches the five main segments of the U.S. RPIB as categorized by the USG: defense, intelligence community, civil government, academia, and commercial sector.

Doreswamy, Rajiv↗

DOE OSTI2024

Predicting Drug Effects from High-dimensional Asymmetric Drug Data Sets using Graph Neural Networks: A Comprehensive Analysis of Multi-target Drug Effect Prediction

Graph neural networks (GNNs) have emerged as one of the most effective Machine learning (ML) techniques for drug effect prediction from drug molecular graphs. Despite having immense potential, GNN models lack performance when using data sets that contain high dimensional asymmetrically co-occurrent drug effects as targets with complex correlations between them. Training individual learning models for each drug effect and incorporating every prediction result for a wide spectrum of drug effects is beyond practicality. Such an implication provides a testbed to address this challenge as multi-target prediction problems, aiming to predict all drug effects at a time. We develop standard and hybrid graph neural networks (GNNs)to perform two separate tasks that are multi-regression for continuous values and multi-label classification for categorical values contained in our data sets. Since this step makes the target data even more sparse and introduces asymmetric label co-occurrence, the learning of multi-label classification models becomes difficult and heavily impacts the GNN's performance. To address these challenges, we propose a new data oversampling technique to improve multi-label classification performances on all the given imbalanced molecular graph data sets. Using the technique, we improve the data imbalance ratio of the drug effects better than before while protecting the data set's integrity. Finally, we evaluate multi-label classification performance using the best-performant hybrid GNN model on all the oversampled data sets obtained from the proposed oversampling technique. These results outperform those of other ML models including GNN models when they are trained on the original data sets or oversampled data sets using MLSMOTE (a well-known oversampling technique) in all evaluation metrics precision, recall, and F1 score by a significant margin.

Bose, Avishek [ORNL]↗

NASA NTRS2016

Characterization of Evidence for Human System Risk Assessment

Understanding the kinds of evidence available and using the best evidence to answer a question is critical to evidenced-based decision-making, and it requires synthesis of evidence from a variety of sources. Categorization of human system risks in spaceflight, in particular, focuses on how well the integration and interpretation of all available evidence informs the risk statement that describes the relationship between spaceflight hazards and an outcome of interest. A mature understanding and categorization of these risks requires: 1) sufficient characterization of risk, 2) sufficient knowledge to determine an acceptable level of risk (i.e., a standard), 3) development of mitigations to meet the acceptable level of risk, and 4) identification of factors affecting generalizability of the evidence to different design reference missions. In the medical research community, evidence is often ranked by increasing confidence in findings gleaned from observational and experimental research (e.g., "levels of evidence"). However, an approach based solely on aspects of experimental design is problematic in assessing human system risks for spaceflight. For spaceflight, the unique challenges and opportunities include: (1) The independent variables in most evidence are the hazards of spaceflight, such as space radiation or low gravity, which cannot be entirely duplicated in terrestrial (Earth-based) analogs, (2) Evidence is drawn from multiple sources including medical and mission operations, Lifetime Surveillance of Astronaut Health (LSAH), spaceflight research (LSDA), and relevant environmental & terrestrial databases, (3) Risk metrics based primarily on LSAH data are typically derived from available prevalence or incidence data, which may limit rigorous interpretation, (4) The timeframe for obtaining adequate spaceflight sample size (n) is very long, given the small population, (5) Randomized controlled trials are unattainable in spaceflight, (6) Collection of personal and environmental data on the astronaut population may create opportunities for advanced analytics and human-environment modeling that goes beyond that achieved in isolated experimental designs; and (7) Translation of relevant research to operations is a complex, transdisciplinary enterprise in which the approach must apply across the physical, biological, behavioral, and social sciences. The approach to synthesizing evidence must address both source and fidelity of data, and reflect the most general attributes of quality of evidence in science and engineering: reliability and validity. The authors are developing a two-factor approach which includes the various kinds of evidence required to understand risks and for the integrated interpretation of all evidence that is essential to develop standards and countermeasures. A unified framework for aggregating and assessing different kinds of evidence provides a consistent, traceable, evidence-based decision-making process to translate research to operations in an environment where engineers, scientists, physicians, and managers all engage in analyzing the trade space of vehicle design, standards, requirements and solutions for spaceflight.

Steinberg, S. L.↗

DOE OSTI2023

Explainable Artificial Intelligence Technology for Predictive Maintenance

The domestic nuclear power plant fleet has relied on labor-intensive and time-consuming preventive maintenance programs, thus driving up operation and maintenance costs to achieve high-capacity factors. Artificial intelligence and machine learning can help simplify complex problems, such as diagnosing equipment degradation, to enable more effective decision-making. Benefits will be felt not only within existing analog and digital instrumentation and control, but also work processes, the integration of people with technology, and most importantly, the business case. Together, these hold promise to make nuclear power more efficient and reduce costs associated with operation and maintenance. While the artificial intelligence and machine learning technologies hold significant promise in the nuclear industry, there are challenges or barriers to their adoption. This report outlines the those different machine learning adoption barriers (categorized as historical, technical, economic, regulatory, and user) that the industry must overcome to realize the full benefits of artificial intelligence and machine learning capabilities for long-term economic sustainability. This report also provides solutions for some of these barriers by focusing on improving the explainability of machine learning to encourage trust from the end-user. Trust and explainability are essential to machine learning adoption. This report focuses on research-developed solutions to some of these barriers while analyzing a non-safety-related system, namely the circulating water system. This system frequently experiences waterbox fouling which our models preemptively diagnoses then explains to the operator how those conclusions were reached. This report presents and discusses the inherent trade-off between machine learning performance (in terms of accuracy) and explainability, where highly accurate machine learning methods (such as deep-learning) are the least explainable, and the most explainable methods (such as decision trees) are the least accurate. In addition, explainability of artificial intelligence techniques in terms of transparency and post-hoc metrics are discussed. This report outlines the importance of data novelty and value of new information in evaluating both the explainability and trustworthiness. Novelty detection helps to establish consistency or inconsistency of the new data with respect to the training data. On the other hand, value of information could be a part of the user-centric visualization recommendation system that request additional information to be collected, thereby strengthening the machine learning outcomes. During this project, a copyrighted user-centric visualization that aligns with a human-in-the-loop approach was developed. The user-centric visualization presents different levels of information and can be tailored as per user credentials to gain user confidence. One of the salient features of the user-centric visualization is it presents machine learning methods with explainability metrics. A simplified version of the user-centric visualization was presented to 32 users with varying levels of machine learning expertise. Feedback was solicited to test the hypothesis that the app contained sufficient explainability and that the users would trust the algorithm. Overall, the app was positively received, and the hypothesis was supported. This report discusses the trust-but-verify framework – a potential approach to build user trust artificial intelligence. The framework discusses trust from the human level to artificial intelligence level. The fundamental premise of the trust but verify framework is derived from an observation of nuclear safety culture (i.e., nuclear power plant personnel do not rely on a singular source of data to make a decision). This also ties back to the user-centric visualization that presents different levels of information to achieve both explainability and trustworthiness of artificial intelligence. Even so, the adoption of artificial intelligence and machine learning in the nuclear industry faces additional barriers, namely regulatory and stakeholder readiness. To overcome these challenges, new solutions must gain regulatory approval and cater to stakeholder needs. The Nuclear Regulatory Committee has a 5-year strategic plan which prepares them for reviewing artificial intelligence technologies in licensee submissions. Early and frequent engagement with the regulator is encouraged. Additionally, artificial intelligence solutions should incorporate human-in-the-loop considerations and offer explainability. Stakeholders must prepare by hiring or training staff to adapt to advancing technology in everyday plant tasks.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

NASA NTRS2019

Predictive Modeling for Differential Diagnosis and Mortality Risk Assessment

The prevalence of electronic health record (EHR) systems has brought prodigious biomedical informatics opportunity. Automated machine learning methods can effectively utilize such data and have become common tools for healthcare predictive modeling. Researches in medical informatics have explored the potential of deep learning and classical models in emergent care scenarios. In particular, predicting differential diagnoses for admissions have proven useful in decreasing unnecessary lab tests and improving inpatient triage decision-making. Moreover, identification of high-risk patients for in-hospital mortality is vitally important to maximize allocation of medical resources.The Medical Information Mart for Intensive Care (MIMIC-III) database, containing de-identified critical care inpatient was used in our study. This data set captures hospital patient laboratory measurements, pharmacologic prescriptions, diagnostic data and procedure event recordings. When considering adult patients and discounting admissions with ICU length of stay less than 24 hours, there were 37,787 unique admissions and 30,414 total patients. We examined the top 25 most prevalent ICD-9 group-level disease specificities in MIMIC-III using a multi-label classification model. In-hospital mortality was modeled as binary classification with 4,155 (13%) adult patients that expired, of which 3,138 (75.5%) were in the ICU setting. The metrics AUC, F1 score, sensitivity and specificity values calculated for each disease label measured prediction performance.The usage of ICD-9 group codes reduced feature dimension from 14,567 to 942 and greatly improved distribution of patient diagnostic categories. Disease temporal patterns were captured by considering the most frequently sampled 6 vital signs and 13 laboratory values. Missing data were imputed at each time-stamp. Time-series raw hourly average values were converted into 5 summary features (mean, standard deviation, number of observations, min & max values). Patient demographic variables such as age, gender, marital status and ethnicity were also factored into the modeling. Choi et al showed that contextual embedding of medical data, diagnostic and procedural codes alone can predict future diagnoses with sensitivity as high as 0.79. We utilized an embedding technique called word2vec which allowed sparse representations of medical history to be transformed into dense word vectors. The mappings captured contextual information by treating each admission as a sentence and learning the most likely neighboring words in a sliding window fashion. Binary and multi-label classification was achieved via collapse models, which do not consider temporal information, as well as recurrent neural networks with regularization, Softmax output layer activation together with categorical cross-entropy as the loss function.

US Army collaboration↗

DOE OSTI2024

Data-Informed Evaluation Framework for Integrated Energy Systems: Insights from Power, Process Heat, and Hydrogen Production Applications

The multi-criteria decision analysis (MCDA) framework provides a systematic evaluation of the diverse preferences and performance metrics associated with alternative solutions. This approach is advantageous over a single-criterion methodology, which are only valid under conditions that assume ceteris paribus or an "apples-to-apples" comparison. However, selecting suitable technologies for integrated energy systems (IES) can be likened to an "apples-to-oranges" comparison, given the heterogeneous factors at stake. These factors include economics and performance parameters, geological compatibility, and environmental impacts. Consequently, past research has often employed a mixture of qualitative and quantitative criteria tailored to the specific interests of each study. While the method proves effective in handling the intricate interplay of criteria, the resulting rankings and scores can vary from study to study. This inconsistency is introduced from the use of subjectively defined thresholds and weights. As a result, decision-makers frequently find it challenging to establish clear connections between specific criteria and the resulting scores, as the transformation of criteria into ordinal scores results in a substantial loss of information. To address this challenge, we introduce a data-informed IES evaluation framework that offers comprehensive, interpretable, and traceable evaluations backed by quantifiable rationale. First, we identified key IES evaluation criteria from a decade of literature, focusing on relevant IES applications in power, process heat, and hydrogen production. We leveraged state-of-the-art cost estimates from the Idaho National Laboratory (INL) and technical data from 78 reactor designs from the International Atomic Energy Agency (IAEA) and the Organization for Economic Co-operation and Development - Nuclear Energy Agency (OECD-NEA). Lastly, we established thresholds by analyzing the mean, variance, root mean square, and slope of values across alternatives, categorizing the preferences of decision-makers into distinct utility functions, such as linear, saturating, exponential, and stepwise. Our approach yielded two main outcomes: (1) it provided consistent assessments across different stakeholder groups and (2) it visualized uncertainties in the decision-making context via comprehensive sensitivity analysis. To demonstrate the impact of our framework, we conducted case studies on 6 reactor designs (AP1000, NuScale, BWRX-300, Xe-100, eVinci, iMSR) for the three applications. Our data-driven framework proved to be highly effective in addressing heterogenous uncertainties faced by varied decision-makers? preferences and IES applications, as well as cost and technical estimates of advanced reactors.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗

DOE OSTI2024

Data-Informed Evaluation Framework for Integrated Energy Systems: Insights from Power, Process Heat, and Hydrogen Production Applications

29 ENERGY PLANNING, POLICY, AND ECONOMY↗

DOE OSTI2020

Accounting for Training Data Error in Machine Learning Applied to Earth Observations

Remote sensing, or Earth Observation (EO), is increasingly used to understand Earth system dynamics and create continuous and categorical maps of biophysical properties and land cover, especially based on recent advances in machine learning (ML). ML models typically require large, spatially explicit training datasets to make accurate predictions. Training data (TD) are typically generated by digitizing polygons on high spatial-resolution imagery, by collecting in situ data, or by using pre-existing datasets. TD are often assumed to accurately represent the truth, but in practice almost always have error, stemming from (1) sample design, and (2) sample collection errors. The latter is particularly relevant for image-interpreted TD, an increasingly commonly used method due to its practicality and the increasing training sample size requirements of modern ML algorithms. TD errors can cause substantial errors in the maps created using ML algorithms, which may impact map use and interpretation. Despite these potential errors and their real-world consequences for map-based decisions, TD error is often not accounted for or reported in EO research. Here we review the current practices for collecting and handling TD. We identify the sources of TD error, and illustrate their impacts using several case studies representing different EO applications (infrastructure mapping, global surface flux estimates, and agricultural monitoring), and provide guidelines for minimizing and accounting for TD errors. To harmonize terminology, we distinguish TD from three other classes of data that should be used to create and assess ML models: training reference data, used to assess the quality of TD during data generation; validation data, used to iteratively improve models; and map reference data, used only for final accuracy assessment. We focus primarily on TD, but our advice is generally applicable to all four classes, and we ground our review in established best practices for map accuracy assessment literature. EO researchers should start by determining the tolerable levels of map error and appropriate error metrics. Next, TD error should be minimized during sample design by choosing a representative spatio-temporal collection strategy, by using spatially and temporally relevant imagery and ancillary data sources during TD creation, and by selecting a set of legend definitions supported by the data. Furthermore, TD error can be minimized during the collection of individual samples by using consensus-based collection strategies, by directly comparing interpreted training observations against expert-generated training reference data to derive TD error metrics, and by providing image interpreters with thorough application-specific training. We strongly advise that TD error is incorporated in model outputs, either directly in bias and variance estimates or, at a minimum, by documenting the sources and implications of error. TD should be fully documented and made available via an open TD repository, allowing others to replicate and assess its use. To guide researchers in this process, we propose three tiers of TD error accounting standards. Finally, we advise researchers to clearly communicate the magnitude and impacts of TD error on map outputs, with specific consideration given to the likely map audience.

58 GEOSCIENCES↗