Search NASA⌕ Search

SEARCH · Search NASA

Results for “scorecard”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

Human-System Integration Scorecard Update to VB.Net

The purpose of this project was to create Human-System Integration (HSI) scorecard software, which could be utilized to validate that human factors have been considered early in hardware/system specifications and design. The HSI scorecard is partially based upon the revised Human Rating Requirements (HRR) intended for NASA's Constellation program. This software scorecard will allow for quick appraisal of HSI factors, by using visual aids to highlight low and rapidly changing scores. This project consisted of creating a user-friendly Visual Basic program that could be easily distributed and updated, to and by fellow colleagues. Updating the Microsoft Word version of the HSI scorecard to a computer application will allow for the addition of useful features, improved easy of use, and decreased completion time for user. One significant addition is the ability to create Microsoft Excel graphs automatically from scorecard data, to allow for clear presentation of problematic areas. The purpose of this paper is to describe the rational and benefits of creating the HSI scorecard software, the problems and goals of project, and future work that could be done.

Sanders, Blaze D.↗

Tool for Human-Systems Integration Assessment: HSI Scorecard

This paper describes the development and rationale for a human-systems integration (HSI) scorecard that can be used in reviews of vehicle specification and design. This tool can be used to assess whether specific HSI related criteria have been met as part of a project milestone or critical event, such as technical reviews, crew station reviews, mockup evaluations, or even review of major plans or processes. Examples of HSI related criteria include Human Performance Capabilities, Health Management, Human System Interfaces, Anthropometry and Biomechanics, and Natural and Induced Environments. The tool is not intended to evaluate requirements compliance and verification, but to review how well the human related systems have been considered for the specific event and to identify gaps and vulnerabilities from an HSI perspective. The scorecard offers common basis, and criteria for discussions among system managers, evaluators, and design engineers. Furthermore, the scorecard items highlight the main areas of system development that need to be followed during system lifecycle. The ratings provide a repeatable quantitative measure to what has been often seen as only subjective commentary. Thus, the scorecard is anticipated to be a useful HSI tool to communicate review results to the institutional and the project office management.

Whitmore, Nihriban↗

Requirements Engineering Scorecard and the Next-Generation Space Suit

The objective for a NASA contractor, the performing organization in this case study, is to provide engineering services to develop and deliver the next generation space suit to NASA, the customer in this case study. A case study with qualitative and quantitative analyses regarding a new process and approach to requirements engineering is described, with the intent that if utilized, these tools may have contributed to improvements across the project in terms of meeting cost, scope, budget and quality while appropriately accounting for risk management. The procedure entails a research method in which the current state of the project, current state of the art, and the identified systems engineering challenges are evaluated. Iterative models are tempered through development by continual improvements by engineering evaluation of engineers on the project. The current results have produced a prototype of a requirements engineering scorecard with implementations of FMEA and quantitative analysis to (i) identify root cause of underdeveloped requirements and (ii) project management impacts with regards to project risk. Forward work includes customer, performing organization, acceptance against applicable INCOSE community accepted practices.

INCOSE↗

Development of Risk Assessment Matrix for NASA Engineering and Safety Center

This paper describes a study, which had as its principal goal the development of a sufficiently detailed 5 x 5 Risk Matrix Scorecard. The purpose of this scorecard is to outline the criteria by which technical issues can be qualitatively and initially prioritized. The tool using this score card has been proposed to be one of the information resources the NASA Engineering and Safety Center (NESC) takes into consideration when making decisions with respect to incoming information on safety concerns across the entire NASA agency. The contents of this paper discuss in detail each element of the risk matrix scorecard, definitions for those elements and the rationale behind the development of those definitions. This scorecard development was performed in parallel with the tailoring of the existing Futron Corporation Integrated Risk Management Application (IRMA) software tool. IRMA was tailored to fit NESC needs for evaluating incoming safety concerns and was renamed NESC Assessment Risk Management Application (NAFMA) which is still in developmental phase.

Malone, Roy W., Jr.↗

Evaluation of remote sensing and automatic data techniques for characterization of wetlands

This investigation has been conducted in the Atchafalaya River Basin of South Central Louisiana. This is a humid area of heavily forested swamps with a large volume of flow mostly from a diversion of the lower Mississippi River. Techniques to obtain enlarged imagery from computer compatible tapes of ERTS data without photographic enlargement is explained and illustrated. Techniques of extraction of environmental information from single bands and multiband pattern recognition procedures are explained and evaluated. A comparison of pattern recognition classifications of the Atchafalaya Basin by aircraft multispectral scanner and ERTS MSS data is made. Data for this comparison were gathered within three weeks of each other in the winter of 1973. Scorecards of the accuracy of the classifications are presented. Recommendations are made concerning the utilization of each sensor platform to perform specific tasks of wetlands characterization.

Cartmill, R. H.↗

HPT Clearance Control: Intelligent Engine Systems-Phase 1

The following work has been completed to satisfy the Phase I Deliverables for the "HPT Clearance Control" project under NASA GRC's "Intelligent Engine Systems" program: (1) Need for the development of an advanced HPT ACC system has been very clearly laid out, (2) Several existing and potential clearance control systems have been reviewed, (3) A scorecard has been developed to document the system, performance (fuel burn, range, payload, etc.), thermal, and mechanical characteristics of the existing clearance control systems, (4) Engine size and flight cycle selection for the advanced HPT ACC system has been reviewed with "large engine"/"long range mission" combination showing the most benefit, (5) A scoring criteria has been developed to tie together performance parameters for an objective, data driven comparison of competing systems, and (6) The existing HPT ACC systems have been scored based on this scoring system.

Source record↗

IV&V Project Assessment Process Validation

The Space Launch System (SLS) will launch NASA's Multi-Purpose Crew Vehicle (MPCV). This launch vehicle will provide American launch capability for human exploration and travelling beyond Earth orbit. SLS is designed to be flexible for crew or cargo missions. The first test flight is scheduled for December 2017. The SLS SRR/SDR provided insight into the project development life cycle. NASA IV&V ran the standard Risk Based Assessment and Portfolio Based Risk Assessment to identify analysis tasking for the SLS program. This presentation examines the SLS System Requirements Review/System Definition Review (SRR/SDR), IV&V findings for IV&V process validation correlation to/from the selected IV&V tasking and capabilities. It also provides a reusable IEEE 1012 scorecard for programmatic completeness across the software development life cycle.

Driskell, Stephen↗

Risky Business

During my internship I worked on two major projects, recommending improvements for the Center's Risk Management Workshop and helping with the strategic planning efforts for Safety and Mission Assurance (S&MA). The risk management improvements is the key project I worked on this semester through my internship, while the strategic planning is the secondary assignment. S&MA Business Office covers both aspects in its delegation, getting both spans some of the work done in the office. A risk is a future event with a negative consequence that has some probability of occurring. Safety and Mission Assurance identifies, analyzes, plans, and tracks risk. The directorate offers the Center a Risk Management Workshop, and part of the ongoing efforts of S&MA is to make continuous improvements to the RM Workshop. By using the Project Management Institute's (PMI) Standard for Risk Management, I performed a gap analysis to make improvements for our materials. I benchmarked the PMI's Risk Management Standard, compared our Risk Management Workshop materials to PMI's standard, and identified any gaps in our material. My major findings were presented to the Business Office of S&MA for a decision on whether or not to incorporate the improvements. These suggestions were made by attending JSC working group meetings, Health, Safety and Environment (HSE) panel reviews and various risk review meetings. The improvements provide better understanding of risk management processes and enhanced risk tracking knowledge and skills. Risk management is an integral part of any engineering discipline, getting exposed to this section of engineering will greatly help shape my career in the future. Johnson Space Center is a world leader in risk management processes; learning risk management here gives me a huge advantage over my peers, as well as understanding decision making in the context of risk management will help me to be a well-rounded engineer. Strategic planning is an area I had not previously studied. Helping with the strategic planning efforts in S&MA has taught me how organizations think and function as a whole. S&MA is adopting a balanced scorecard approach to strategic planning. As part of this planning method strategic themes, objectives, and initiatives are formed. I attended strategic theme team workshops that formed the strategy map for the directorate and gave shape to the plan. Also during these workshops the objectives were discussed and built. Learning the process for strategic planning has helped me better understand how organizations and businesses function, which also helps me to be a more effective employee. Other assignments I had during my internship included completing the Safety and Mission Assurance Technical Excellent Program (STEP) Level 1, as well as doing a two week rotation through the Space Exploration division in S&MA, specifically working with a thermal protection systems (TPS) engineer. While working there, I learned about the Orion capsule and the SpaceX Dragon cargo capsule. I attended meetings to prepare the engineers for the upcoming Critical Design Reviews for both capsules and reviewed test data. Learning risk management, strategic planning, and working in the Space Exploration division has taught me about many aspects of S&MA. My internship at NASA has given me new experiences and taught me numerous subjects that I would have otherwise not learned. This opportunity has expanded my educational horizons and is helping me to become a more useful engineer and employee.

Yarbrough, Katherine↗

Evaluating Process Effectiveness to Reduce Risk

It is well documented that government agencies do not have the same incentive as the private sector to focus on process effectiveness and continual improvement of those processes. It is also well documented whenever government agencies fail to deliver efficient, effective, consistent, and fair services to the citizens. In spite of the various "reinventing government" and "effectiveness initiatives" of the past decades, and in spite of the efforts on the part of many agencies to improve, government in general still lags behind industry in creating a culture of effective processes and systems. While the tragic events that unfolded recently in Flint, Michigan, teach us that running government "like a business" does not always take the needs of the citizenry into account, there are many lessons and techniques from the private sector that government agencies can use to improve. The incentive to improve, while mandated by various administrations1, needs to come from within the workforce, in order to effectively take root. The best, most effective incentive is to reduce, control or eliminate risk. Government agencies face some of the same risks as the private sector, while some are unique. While ISO 310002 has been around since 2009, risk has taken on increased visibility within the private sector with the advent of the emphasis on risk-based thinking in ISO 9001:20153. The relationship between risk-based thinking and effective processes is simple and direct. Those processes that are well thought out and standardized (i.e. Plan-Do-Check-Act), will have taken into account the applicable policy, statutory, regulatory, safety, quality and technical parameters, which may not occur to someone performing the process with minimal experience or training; and thus protect the employees, the public and the agency from statutory and regulatory violations; delay in providing services; non-delivery of services; harm to public or employee safety and health; cost overruns; breaches in security; loss of confidence in government; failure of publicly funded projects; damage to the environment; ethics violations, and the list goes on; with local, national and even international consequences. The Plan-Do-Check-Act process, also known as the "process approach" can be used at any time to establish and standardize a process, and it can also be used to check periodically for "process creep" (i.e., informal, unauthorized changes that have occurred over time), any necessary updates and improvements. While ISO 9001 compliance is not mandated for all government agencies, if interpreted correctly, it can be useful in establishing a framework and implementing effective management systems and processes.4 Another method that can be used to evaluate effectiveness is the scorecard definitions in Mallory's Process Management Standard5 as a basis for evaluating work on the process level on effective, and continuously improved and improving processes. With processes on the lower end of the scale, agencies are vulnerable to a great many risks, with employees and managers making up many of the rules as they go, leading to the above listed negative results. Without clear guidance for nominal operations, off-nominal situations can, and do, increase the likelihood of chaos. In an increasingly technical environment, with inter-agency communication and collaboration becoming the norm, agencies need to come to grips with the fact that processes can become rapidly outdated, and that the technical community should take on an increased role in the maturation of the agency's processes. Industry has long known that effective processes are also efficient, and process improvement methods such as Kaizen, Lean, Six Sigma, 5S, and mistake proofing lead to increased productivity, improved quality, and decreased cost. Again, government agencies have different concerns, but inefficiencies and mistakes can have dire and wide reaching consequences for the public that they serve. While no one goes to work planning to cause harm, it is up to agencies to establish upper level systems, which make establishment and compliance with processes possible. Again, Mallory provides us with a Systems Management Standard6, similar to the Process Management Standard, with a scale of 0-5 for systems effectiveness and maturity. Deming determined that "eighty-five percent of the reasons for failure are deficiencies in the systems and process rather than the employee. The role of management is to change the process rather than badgering individual employees to do better." 7 It is not just the working level employees who need effective processes, but the mid-and upper level managers as well. A disciplined management culture sets the tone for the employees, aids both routine and off-nominal decision-making, and incorporates risk -based thinking into the systems and processes as a matter of normal activity. Figure 1, illustrates the relationship between ineffective and effective processes and risk, through the use of the "stoplight" colors that are commonly used to show serious situations (red), situations which may be improving or deteriorating depending on trends (yellow), and situations that are under control and continuously improved (green).

Shepherd, Christena C.↗

Space Science and Technology Partnership Forum: In-Space Assembly Data Collection and Analysis

The Space Science and Technology Partnership Forum was established in 2015 to identify synergistic efforts and technologies across the government. In-Space Assembly (iSA) is the focus of the topic area that NASA is currently coordinating with other government agencies. This paper focuses on the data collection process, the data analysis of that information, and preliminary insights gleaned from the data. The goal of the analysis is to understand the linkages within the collected data, identifying synergies and gaps, and provide visualization of the current state of iSA needs and capability development across the government. Capability roadmaps, Venn diagrams, bubble charts, and scorecards (an overview of each individual iSA capability) are used to visualize the results of this analysis, which reveals areas of possible inter-agency collaboration, investment gaps in capabilities relative to the need, and capabilities that warrant engagement across multiple agencies to eliminate potential inefficiencies.

Arney, Dale C.↗

Space Science and Technology Partnership Forum: In-Space Assembly Data Collection and Analysis

The Space Science and Technology Partnership Forum was established in 2015 to identify synergistic efforts and technologies across the government. In-Space Assembly (iSA) is the focus of the topic area that NASA is currently coordinating with other government agencies. This paper focuses on the data collection process, the data analysis of that information, and preliminary insights gleaned from the data. The goal of the analysis is to understand the linkages within the collected data, identifying synergies and gaps, and provide visualization of the current state of iSA needs and capability development across the government. Capability roadmaps, Venn diagrams, bubble charts, and scorecards (an overview of each individual iSA capability) are used to visualize the results of this analysis, which reveals areas of possible inter-agency collaboration, investment gaps in capabilities relative to the need, and capabilities that warrant engagement across multiple agencies to eliminate potential inefficiencies.

Dale C Arney↗

PolarMERRA: A Polar-Focused Global Reanalysis Project for Scientific and Stakeholder Needs

The adequate modeling of physical processes in the Arctic and Antarctic is vital for developing prediction capabilities and for obtaining an understanding of rapidly evolving polar conditions, including their potential global impacts. These processes are often poorly represented in global models and reanalyses, owing to a legacy modeling focus on midlatitude processes, as well as a scarcity of observations needed for model development in polar regions. The polarMERRA initiative, a joint effort between NASA’s Cryospheric Sciences and Modeling and Prediction programs, seeks to improve the representation of cryospheric and polar atmospheric processes and to develop an open-source framework for a quantitively evaluation of polar-relevant variables against current and future satellite and in-situ observations, models, and reanalyses. Here, we evaluate the impacts of spatial resolution and modifications to sea ice, ice sheet, and atmospheric parameterizations on the representation of high latitude conditions by using a quantitative scorecard approach that leverages NASA’s extensive satellite record. Investigations are conducted using the NASA Goddard Earth Observing System model (GEOS) and its data assimilation system (GEOS DAS). Through a quantitative identification of process deficiencies, bias reductions in key surface variables, including temperature and precipitation over cryospheric surfaces, may be achieved. The polarMERRA project additionally seeks to identify additional data sources for use in the GEOS DAS, and to incorporate data and parameterization improvements into future model and reanalysis products for scientific and stakeholder use.

Lauren C Andrews↗

Flight Mechanics Modeling and Simulation of the Earth Entry System

Introduction: The Mars Sample Return (MSR) Campaign being planned by NASA and ESA has the ambitious goal to return Mars samples back to Earth. This international collaboration had developed a concept of operations that included a ESA-designed Earth Return Orbiter (ERO) and NASA-designed Capture, Containment, and Return System (CCRS). The Earth Entry System (EES), consisting of a protective aeroshell that houses the samples as well as sample containment vessels, would conduct entry, descent, and landing (EDL) on a direct Earth trajectory. The EES would enter on a spin-stabilized ballistic trajectory with the goal to passively achieve aerodynamic stability throughout all regions of flight. The EDL sequence would end with the EES impacting the soft playa soil of the Utah Test and Training Range (UTTR). As of the submission of this abstract, the MSR campaign is undergoing a re-architecture leading to a pause in EES development. However, the novel approaches developed in flight mechanics modeling and simulation can significantly benefit the greater IPPW community in the development of Earth return vehicles. This paper will present the latest state of EES flight mechanics modeling and simulation. The paper will highlight the simulation architecture developed and key lessons learned from understanding of EDL trajectory sensitivities. Modeling and Simulation: Figure 1 provides a high-level concept of operations for the approach, entry, descent, and landing (AEDL) phase of the CCRS-portion of MSR. The objective of EES flight mechanics is to model and simulate the EES trajectory from ERO separation to ground impact at UTTR. A variety of flight mechanics simulation models were utilized to model both exo-atmopsheric and atmospheric portions of flight. 42, a 6-DOF simulation developed at Goddard Space Flight Center, is utilized for propagating the attitude of EES during exo-atmospheric flight. 42 allows for a variety of spin eject mechanism scenarios to be simulated for analysis. 10 minutes prior to entry, the 42 states are handed off to the EDL sims. The prime EDL sim utilized by EES is the Program to Optimize Simulated Trajectories II (POST2), a 6-DOF sim developed at Langley Research Center, and the independent verification and validation EDL sim utilized is DSENDS, a 6-DOF sim developed at Jet Propulsion Laboratory. Figure 2 provides a visualization of the flight mechanics simulation model flow through various points in the AEDL phase. Due to the existence of a variety of sim models, the EES flight mechanics team developed processes for data hand-off. These processes included the development of a centralized coordinate frame document, utilization of a single, centralized simulation input document for all sims to reference, and hand-off files containing both the technical data to be ingested by other flight mechanics sims as well as annotations of modeling assumptions utilized to generate the data. Figure~\ref{fig:post2simarchitecture} provides an overview of the POST2 sim architecture wherein POST2 ingests numerous subsystem models and input files. The dispersed state file generated by MONTE provides the position/velocity state of the trajectory while the 42 Handoff file provides the attitude. The aerodynamics database, delivered by the EES aeroscience team, is utilized to simulate the aerodynamic forces and moments experienced during EDL. A custom atmosphere model, developed by EES atmosphere team, is utilized to simulate the anticipated atmosphere environment around the region of Earth through which the EES trajectory flys. These inputs and subsystem models can be varied depending on the AEDL flight mechanics scenario being simulated. Monte Carlo simulations are utilized to generate statistical AEDL performance metrics in the form of scorecards and violin plots. Furthermore, outputs from the POST2 simulation are utilized for follow-on analyses including aerothermal and landing performance. \section{Flight Mechanics Lessons Learned} Though the EES flight mechanics team uncovered a variety of lessons learned through the analysis conducted to support CCRS through preliminary design review, this paper will highlight the most important lessons. A key AEDL performance goal is to ensure the landing footprint of EES remains on the UTTR south range. A common modeling strategy used in EDL analysis is One-Variable-At-a-Time (OVAT). OVAT analysis provides insight into the key drivers that affect AEDL performance metrics. Figure 3 shows the landing ellipses for single dispersion sources as compared to the baseline aggregate of all dispersions. The figure shows that atmosphere winds alone dominate the size of the footprint ellipse (note: EES does not use a parachute unlike previous Earth-return missions and is in wind-driven free fall for ~5min). The significance of the wind led the EES flight mechanics team to pursue the development of a Custom Atmosphere Model [4], in lieu of EarthGRAM [1], built on actual radiosonde wind measurements around the UTTR-region. This decision was driven by the realism in the generated footprint ellipses and lessons-learned from Stardust [5]. These findings will be invaluable for future Earth-return missions in providing an early understanding of the key drivers affecting footprint size and modeling considerations for which to account. Another lesson learned is tied to the AEDL performance goal of achieving passive stability throughout all regions of flight. It is well understood that blunt-body aeroshells are less stable as they transition from supersonic to subsonic. Eliminating a backshell does help improvestability; however, other phenomena such as roll-induced instability during terminal descent can still arise. The EES flight mechanics team developed stability metrics as tools to better understand the causes of and better predict the onset of dynamic instability. These tools were built upon analytical models developed by Jaffe [3] and Murphy [2]. The tools were shown to both be very accurate in correlation with actual unstable cases and useful in developing stability margin policies based on the vehicle design and simulation considerations (e.g. sphere-cone angle change, mass change, wind turbulence). These tools allowed for the current EES design to demonstrate the ability to achieve passive stability and can be an invaluable tool for consideration in the design of parachute-less Earth-return vehicles.

Rohan Deshmukh↗