SEARCH · Search NASA
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
This paper sets out our observations, modeling results, and conclusions regarding a novel organizational form applied variously as Extreme Collaboration (XC) or radical collocation.
This is a three-paper dissertation that responds to a call for empirical research on stakeholder influence in the implementation of evaluation recommendations. The literature is consistent but mostly theoretical: stakeholder participation should positively influence and increase the probability of implementation (see e.g., Christie, 2011; Cousins & Earl, 1992; Fetterman, 2004). In response, I use two Goddard Space Flight Center engineering technical training and development programs to build a causal argument supporting my proposition that stakeholder influence in the logic modeling process impacts organizational decision-making and change. I first introduce the logic modeling process as a tool to involve stakeholders in program and evaluation design (paper 1); then use the data collection process to implement the collaboratively constructed logic model outcomes (paper 2); and, finally, I triangulate the data using the Toal (2009) Evaluation Involvement Scale (paper 3). I do this to “increase the validity of the findings” (Noble & Heale, 2019, p. 67) and to garner potential mechanisms. I found that stakeholder participation in the logic modeling process positively influenced the use of data to inform recommendations which in turn positively influenced implementation of evaluation recommendations. I identified the process itself as the mechanism and the key decision points as links or influencers. Potential factors included power and interest (measured by role) and time-on-job.
This is a three-paper dissertation that responds to a call for empirical research on stakeholder influence in the implementation of evaluation recommendations. The literature is consistent but mostly theoretical: stakeholder participation should positively influence and increase the probability of implementation (see e.g., Christie, 2011; Cousins and Earl, 1992); Fetterman, 2004). In response, I use two Goddard Space Flight Center engineering technical training and development programs to build a causal pathway argument supporting my proposition that stakeholder influence in the logic modeling process impacts organizational decision-making and change. I first introduce the logic modeling process as a tool to involve stakeholders in program and evaluation design (paper 1), then use the data collection process to implement the collaboratively constructed logic model outcomes (paper 2), and finally, I triangulate the data using the Toal (2009) Evaluation Involvement Scale (paper 3). I do this to “increase the validity of the findings” (Noble & Heale, 2019, p. 67) and to garner potential mechanisms. I found that stakeholder participation in the logic modeling process positively influenced the use of data to inform recommendations which in turn positively influenced implementation of evaluation recommendations. I identified the process itself as the mechanism and the key decision points as links or influencers. Potential factors included power and interest (measured by role) and time-on-job.
Digital model-based engineering (DMbE) is the use of digital artifacts, digital environments, and digital tools in the performance of engineering functions. DMbE is intended to allow an organization to progress from documentation-based engineering methods to digital methods that may provide greater flexibility, agility, and efficiency. The term 'DMbE' was developed as part of an effort by the Model-Based Systems Engineering (MBSE) Infusion Task team to identify what government organizations might expect in the course of moving to or infusing MBSE into their organizations. The Task team was established by the Interagency Working Group on Engineering Complex Systems, an informal collaboration among government systems engineering organizations. This Technical Memorandum (TM) discusses the work of the MBSE Infusion Task team to date. The Task team identified prerequisites, expectations, initial challenges, and recommendations for areas of study to pursue, as well as examples of efforts already in progress. The team identified the following five expectations associated with DMbE infusion, discussed further in this TM: (1) Informed decision making through increased transparency, and greater insight. (2) Enhanced communication. (3) Increased understanding for greater flexibility/adaptability in design. (4) Increased confidence that the capability will perform as expected. (5) Increased efficiency. The team identified the following seven challenges an organization might encounter when looking to infuse DMbE: (1) Assessing value added to the organization. Not all DMbE practices will be applicable to every situation in every organization, and not all implementations will have positive results. (2) Overcoming organizational and cultural hurdles. (3) Adopting contractual practices and technical data management. (4) Redefining configuration management. The DMbE environment changes the range of configuration information to be managed to include performance and design models, database objects, as well as more traditional book-form objects and formats. (5) Developing information technology (IT) infrastructure. Approaches to implementing critical, enabling IT infrastructure capabilities must be flexible, reconfigurable, and updatable. (6) Ensuring security of the single source of truth (7) Potential overreliance on quantitative data over qualitative data. Executable/ computational models and simulations generally incorporate and generate quantitative vice qualitative data. The Task team also developed several recommendations for government, academia, and industry, as discussed in this TM. The Task team recommends continuing beyond this initial work to further develop the means of implementing DMbE and to look for opportunities to collaborate and share best practices.
To identify best practices for the improvement of software engineering on projects, NASA's Offices of Chief Engineer (OCE) and Safety and Mission Assurance (OSMA) formed a team led by Heather Rarick and Sally Godfrey to conduct this benchmarking study. The primary goals of the study are to identify best practices that: Improve the management and technical development of software intensive systems; Have a track record of successful deployment by aerospace industries, universities [including research and development (R&D) laboratories], and defense services, as well as NASA's own component Centers; and Identify candidate solutions for NASA's software issues. Beginning in the late fall of 2010, focus topics were chosen and interview questions were developed, based on the NASA top software challenges. Between February 2011 and November 2011, the Benchmark Team interviewed a total of 18 organizations, consisting of five NASA Centers, five industry organizations, four defense services organizations, and four university or university R and D laboratory organizations. A software assurance representative also participated in each of the interviews to focus on assurance and software safety best practices. Interviewees provided a wealth of information on each topic area that included: software policy, software acquisition, software assurance, testing, training, maintaining rigor in small projects, metrics, and use of the Capability Maturity Model Integration (CMMI) framework, as well as a number of special topics that came up in the discussions. NASA's software engineering practices compared favorably with the external organizations in most benchmark areas, but in every topic, there were ways in which NASA could improve its practices. Compared to defense services organizations and some of the industry organizations, one of NASA's notable weaknesses involved communication with contractors regarding its policies and requirements for acquired software. One of NASA's strengths was its software assurance practices, which seemed to rate well in comparison to the other organizational groups and also seemed to include a larger scope of activities. An unexpected benefit of the software benchmarking study was the identification of many opportunities for collaboration in areas including metrics, training, sharing of CMMI experiences and resources such as instructors and CMMI Lead Appraisers, and even sharing of assets such as documented processes. A further unexpected benefit of the study was the feedback on NASA practices that was received from some of the organizations interviewed. From that feedback, other potential areas where NASA could improve were highlighted, such as accuracy of software cost estimation and budgetary practices. The detailed report contains discussion of the practices noted in each of the topic areas, as well as a summary of observations and recommendations from each of the topic areas. The resulting 24 recommendations from the topic areas were then consolidated to eliminate duplication and culled into a set of 14 suggested actionable recommendations. This final set of actionable recommendations, listed below, are items that can be implemented to improve NASA's software engineering practices and to help address many of the items that were listed in the NASA top software engineering issues. 1. Develop and implement standard contract language for software procurements. 2. Advance accurate and trusted software cost estimates for both procured and in-house software and improve the capture of actual cost data to facilitate further improvements. 3. Establish a consistent set of objectives and expectations, specifically types of metrics at the Agency level, so key trends and models can be identified and used to continuously improve software processes and each software development effort. 4. Maintain the CMMI Maturity Level requirement for critical NASA projects and use CMMI to measure organizations developing software for NASA. 5.onsolidate, collect and, if needed, develop common processes principles and other assets across the Agency in order to provide more consistency in software development and acquisition practices and to reduce the overall cost of maintaining or increasing current NASA CMMI maturity levels. 6. Provide additional support for small projects that includes: (a) guidance for appropriate tailoring of requirements for small projects, (b) availability of suitable tools, including support tool set-up and training, and (c) training for small project personnel, assurance personnel and technical authorities on the acceptable options for tailoring requirements and performing assurance on small projects. 7. Develop software training classes for the more experienced software engineers using on-line training, videos, or small separate modules of training that can be accommodated as needed throughout a project. 8. Create guidelines to structure non-classroom training opportunities such as mentoring, peer reviews, lessons learned sessions, and on-the-job training. 9. Develop a set of predictive software defect data and a process for assessing software testing metric data against it. 10. Assess Agency-wide licenses for commonly used software tools. 11. Fill the knowledge gap in common software engineering practices for new hires and co-ops.12. Work through the Science, Technology, Engineering and Mathematics (STEM) program with universities in strengthening education in the use of common software engineering practices and standards. 13. Follow up this benchmark study with a deeper look into what both internal and external organizations perceive as the scope of software assurance, the value they expect to obtain from it, and the shortcomings they experience in the current practice. 14. Continue interactions with external software engineering environment through collaborations, knowledge sharing, and benchmarking.
The NASA Human Health and Performance Center (NHHPC) was established in October 2010 to promote collaborative problem solving and project development to advance human health and performance innovations benefiting life in space and on Earth. The NHHPC, which now boasts over 135 corporate, government, academic and non-profit members, has convened four successful workshops and engaged in multiple collaborative projects. The center is currently developing a streamlined partner engagement process to capture technical needs and opportunities of NHHPC members, facilitate partnership development, and establish and manage collaborative projects for NASA. The virtual center facilitates member engagement through a variety of vehicles, including annual inperson workshops, webcasts, quarterly electronic newsletters, web postings, and the new system for partner engagement. The most recent NHHPC workshop was conducted in November 2013 on the topic of "Accelerating Innovation: New Organizational Business Models," and focused on various collaborative approaches successfully used by organizations to achieve their goals. The powerful notion of collaboration across sectors to solve intractable problems was recently highlighted in Williams Eggers' book "The Solution Revolution,"i which provides numerous examples of how business, government and social enterprises partner to solve tough problems. Mr. Eggers was a keynote speaker at the workshop, along with Harvard Business School, Jump Associates, and the Conrad Foundation. The robust program also included an expert panel addressing collaboration across sectors, four interactive breakout sessions, and a concluding keynote on innovative ways to increase science, technology, engineering, and math (STEM) education by NASA Associate Administrator for Education, Leland Melvin. The NHHPC forum also provides a platform for international partners to interact on many topics. Members from around the world include ISS International Partner JAXA; the World Biomimetic Foundation in Spain who is interested in advancing the use of biomimicry to provide technical solutions in many industries; Satellite Application Catapult in London, England who interested in pursuing U.S. collaborations with the Space and Life Sciences Innovation Centre under development in Scotland; and DLR in Cologne, Germany who developed :envihab, a collaborative facility for partners to pursue research and technology projects of mutual interest. The NHHPC has sponsored two global networking forums on innovation by partners Wyle, NASA, and DLR, was featured in the 2013 Humans in Space Symposium Panel on "NHHPC and :envihab - reach out to Future Markets," and is working on an international meeting for Spring 2014 in Cologne with :envihab.
Human collaboration in distributed knowledge sharing groups depends on the functionality of information and communication technologies (ICT) to support performance. Since many of these dynamic environments are constrained by time limits, knowledge must be shared efficiently by adapting the level of information detail to the specific situation. This paper focuses on the process of knowledge and context sharing with and without mediation by ICT, as well as issues to be resolved when determining appropriate ICT channels. Both technology-rich and non-technology examples are discussed.
Long-term data sets with data from many missions are needed to study trends and validate model results that are typical in Earth System Science research. Data and derived products originate from multiple missions (spaceborne, airborne and/or in situ) and from multiple organizations. During the missions as well as well past their termination, it is essential to preserve the data and products to support future studies. Key aspects of preservation are: preserving bits and ensuring data are uncorrupted, preserving understandability with appropriate documentation, and preserving reproducibility of science with appropriate documentation and other artifacts. Computer technology provides adequate standards to ensure that, with proper engineering, bits are preserved as hardware evolves. However, to ensure understandability and reproducibility, it is essential to plan ahead to preserve all the relevant data and information. There are currently no standards to identify the content that needs to be preserved, leading to non-uniformity in content and users not being sure of whether preserved content is comprehensive. Each project, program or agency can specify the items to be preserved as a part of its data management requirements. However, broader community consensus that cuts across organizational or national boundaries would be needed to ensure comprehensiveness, uniformity and long-term utility of archived data. The Federation of Earth Science Information Partners (ESIP), a diverse network of scientists, data stewards and technology developers, has a forum for ESIP members to collaborate on data preservation issues. During early 2011, members discussed the importance of developing a Provenance and Context Content Standard (PCCS) and developed an initial list of content items. This list is based on the outcome of a NASA and NOAA meeting held in 1998 under the auspices of the USGCRP, documentation requirements from NOAA and our experience with some of the NASA Earth science missions. The items are categorized into the following 8 high level categories: Preflight/Pre-Operations, Products (Data), Product Documentation, Mission Calibration, Product Software, Algorithm Input, Validation, Software Tools.
Power system planning software remains fragmented across organizational boundaries, with specialized tools for capacity expansion, production cost modeling, power flow, and dynamic analysis operating on incompatible data models and assumptions. This article argues that the fragmentation is not merely a technical problem but a predictable consequence of Conway's law: software architectures mirror the departmental structures within which they are developed. Regulatory milestones like Federal Energy Regulatory Commission (FERC) Order 888 formalized these divisions, but the roots trace back to the distinct engineering disciplines-mechanical, chemical, and electrical-that staffed generation and transmission planning departments in vertically integrated utilities. As the industry moves toward integrated system planning (ISP) that coordinates generation, transmission, and distribution investment decisions, the software ecosystem must evolve accordingly. We identify five categories of software requirements to enable this transition: coherent data inputs decoupled from individual applications, unified and extensible data schemas, modular component representations that support multiple abstraction levels, lifecycle management of planning datasets, and well-defined application programming interface (API) contracts that separate data exchange from algorithmic control. We examine how these requirements interact with three common workflow patterns-serial gate clearing, sequential multiapplication, and convergence oriented-and discuss the interface design principles each demands. We then outline a vision for platform-based planning architectures where specialized analytical services compose through standardized interfaces and where artificial intelligence (AI)/machine learning (ML) tools augment decision support within a disciplined software infrastructure. The practices proposed here offer a path from today's siloed tool collections toward collaborative planning ecosystems capable of handling the complexity of modern power system transformation.
The biggest users of GRID technologies came from the science and technology communities. These consist of government, industry and academia (national and international). The NASA GRID is moving into a higher technology readiness level (TRL) today; and as a joint effort among these leaders within government, academia, and industry, the NASA GRID plans to extend availability to enable scientists and engineers across these geographical boundaries collaborate to solve important problems facing the world in the 21 st century. In order to enable NASA programs and missions to use IPG resources for program and mission design, the IPG capabilities needs to be accessible from inside the NASA center networks. However, because different NASA centers maintain different security domains, the GRID penetration across different firewalls is a concern for center security people. This is the reason why some IPG resources are been separated from the NASA center network. Also, because of the center network security and ITAR concerns, the NASA IPG resource owner may not have full control over who can access remotely from outside the NASA center. In order to obtain organizational approval for secured remote access, the IPG infrastructure needs to be adapted to work with the NASA business process. Improvements need to be made before the IPG can be used for NASA program and mission development. The Secured Advanced Federated Environment (SAFE) technology is designed to provide federated security across NASA center and NASA partner's security domains. Instead of one giant center firewall which can be difficult to modify for different GRID applications, the SAFE "micro security domain" provide large number of professionally managed "micro firewalls" that can allow NASA centers to accept remote IPG access without the worry of damaging other center resources. The SAFE policy-driven capability-based federated security mechanism can enable joint organizational and resource owner approved remote access from outside of NASA centers. A SAFE enabled IPG can enable IPG capabilities to be available to NASA mission design teams across different NASA center and partner company firewalls. This paper will first discuss some of the potential security issues for IPG to work across NASA center firewalls. We will then present the SAFE federated security model. Finally we will present the concept of the architecture of a SAFE enabled IPG and how it can benefit NASA mission development.
The National Aeronautics and Space Administration (NASA) Air Mobility Pathfinders (AMP) project is developing and evaluating concepts of operations (ConOps) for safe, secure, and scalable Urban Air Mobility (UAM) operations. The AMP project’s Operational Concepts, Architecture, and Requirements Integration (OCARI) Team is using a Model Based System Engineering (MBSE) approach for integration, interoperability, and traceability of Advanced Air Mobility (AAM) ecosystems centered around urban air taxi services. The team’s goal is to define structures and behaviors needed for system feasibility, readiness, and interoperability, establish a UAM knowledge base, and trace and validate assumptions and requirements relevant to AAM. NASA Langley Research Center (LaRC) is spearheading an innovative digital engineering approach to integrate, communicate, and facilitate the research of multi-modal transportation systems. The Knowledge-based Digital Platform (KbDP) is a concept being developed that ties the workflows of Project Managers (PM), Principal Investigators (PI), and System Engineers together across organizational boundaries. It does so through the management of an information database defined by mathematical, data science, and system engineering principles. Machine Learning (ML) algorithms play a key role in this concept by extracting meaningful knowledge from relational and graph databases, document repositories, and system artifacts, which the human user leverages to greatly improve the efficiency and effectiveness of their research. Recent advancements in the field of Large Language Models (LLMs), specifically models trained for tool use, such as Command-R , now allow for the reliable implementation of single-step and multi-step tool-centric systems. These techniques provide the LLM with a set of tools, in our case Python functions, that can be called on to answer a much wider range of questions compared to LLMs implemented using a traditional single-source or Retrieval Augmented Generation (RAG) approach. Through this method, the LLM can pull information from multiple data sources, such as relational or graph databases, document repositories, application programming interfaces (APIs), and SysML artifacts depending on the user’s question. The LLM can also output the information in a variety of different formats, using output generation tools, such as CSV, UML, or SysML artifacts. Additionally, tools can be assigned roles and can work together to provide answers to queries in an “agent” like approach, similar to that implemented by Microsoft’s AutoGen framework where different agents can converse with each other to accomplish tasks. Previously, our team developed a chatbot system with “agent like” functionality in the form of different “modes” the user could select from a user interface (UI), this architecture can be seen on the left in figure 1. Three different modes were implemented, the first mode allowed the LLM to utilize the structures and algorithms within a graph database to trace UAM requirements. The second mode gave the LLM access to a vector search capable of providing relevant information from thousands of document pages related to UAM ConOps and requirements. The third mode served as a general assistant where users could enter open-ended questions and custom prompts to utilize the LLM for different use-cases. This system improved the process surrounding generating and analyzing information related to UAM requirements, however, the implementation provided a clunky user experience. Users were required to know what mode to select within the UI in advance before entering their question to the selected tool. Moreover, the different tools were isolated from each other, they lacked bidirectional links that would allow for tools to collaborate to generate better responses. Our team is working on a new architecture, seen on the right in the below figure, with the goal to address many of the UX shortcomings of our original system while improving the accuracy and depth of responses from the LLM. This new system will automatically select the appropriate tool to use based off the user’s question. Each tool will be capable of calling on any of the other tools available to the LLM, resulting in a collaborative pipeline where tools can pass data between other tools until enough data is received to generate an answer to the user’s question. Using a locally deployed, open-source, LLM, the NASA OCARI team, in collaboration with Collins Aerospace, will implement a prototype application that will bridge knowledge across multiple sources to assist System Engineers (SEs) with requirements discovery and tracing, research question and use case identification, and assumption validation. Such a system will also allow SEs to more easily, and intuitively, explore the AAM ecosystem, ultimately improving the efficiency and effectiveness of the SE's research and decision-making processes surrounding ConOps development and validation. In this session, our team will provide a video demonstration of our new prototype architecture in action. We will also present an overview of our prototype system architecture and talk about its advantages over traditional LLM deployments along with how those advantages can provide additional value to the field of System Engineering.