Search NASA⌕ Search

SEARCH · Search NASA

Results for “peer review”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records

Portable Common Execution Environment (PCEE) project review: Peer review

The purpose of the review was to conduct an independent, in-depth analysis of the PCEE project and to provide the results of said review. The review team was tasked with evaluating the potential contribution of the PCEE project to the improvement of the life cycle support of mission and safety critical (MASC) computing components for large, complex, non-stop, distributed systems similar to those planned for such NASA programs as the space station, lunar outpost, and manned missions to Mars. Some conclusions of the review team are as follow: The PCEE project was given high marks for its breath of vision on the overall problem with MASC software; Correlated with the sweeping vision, the Review Team is very skeptical that any research project can successfully attack such a broad range of problems; and several recommendations are made such as to identify the components of the broad solution envisioned, prioritizing them with respect to their impact and the likely ability of the PCEE or others to attack them successfully, and to rewrite its Concept Document differentiating the problem description, objectives, approach, and results so that the project vision becomes assessible to others.

Locke, C. Douglass↗

Peer Review to Peer Learning in Radiology: Where Have We Been, What Have We Learned and Where Are We Headed?

Purpose of review: To review and contrast varying methods of peer assessment driven practice quality improvement programs in radiology with an emphasis on peer review and peer learning. Recent findings: Review of the literature revealed that the current consensus is that a shift away from the original peer review system toward a peer learning process has the most beneficial effects for organizations seeking to maximally improve performance. This requires altering perceptions towards the peer review process itself, and significant time, effort, and resources. Summary: The transition to a peer learning process is a necessity to advance the field of radiology into an era of delivering near faultless quality health care.

62 RADIOLOGY AND NUCLEAR MEDICINE↗

Redefining the Practice of Peer Review Through Intelligent Automation Part 2: Data-Driven Peer Review Selection and Assignment

In conventional radiology peer review practice, a small number of exams (routinely 5% of the total volume) is randomly selected, which may significantly underestimate the true error rate within a given radiology practice. An alternative and preferable approach would be to create a data-driven model which mathematically quantifies a peer review risk score for each individual exam and uses this data to identify high risk exams and readers, and selectively target these exams for peer review. An analogous model can also be created to assist in the assignment of these peer review cases in keeping with specific priorities of the service provider. An additional option to enhance the peer review process would be to assign the peer review cases in a truly blinded fashion. In addition to eliminating traditional peer review bias, this approach has the potential to better define exam-specific standard of care, particularly when multiple readers participate in the peer review process.

62 RADIOLOGY AND NUCLEAR MEDICINE↗

Does Peer Review of Radiation Plans Affect Clinical Care? A Systematic Review of the Literature

Purpose: Peer review is a recommended component of quality assurance in radiation oncology; however, it is resource-intensive and its effect on patient care is not well understood. We conducted a systematic review of the published data to assess the reported clinical impact of peer review on radiation treatment plans. Methods and Materials: A systematic review of published English studies was performed in accordance with the PRISMA guidelines using the MEDLINE and EMBASE databases and abstracts published from major radiation oncology scientific meeting proceedings. For inclusion, the studies were required to report the effect of peer review on ≥1 element of treatment planning (eg, target volume or organ-at-risk delineation, dose prescription or dosimetry). Results: The initial search strategy identified 882 potentially eligible studies, with 11 meeting the inclusion criteria for full-text review and final analysis. Across a total of 11,491 patient cases, peer review programs led to modifications in a weighted mean of 10.8% of radiation treatment plans. Five studies differentiated between major and minor changes and reported weighted mean rates of change of 1.8% and 7.3%, respectively. The most common changes were related to target volume delineation (45.2% of changed plans), dose prescription or written directives (24.4%), and non-target volume delineation or normal tissue sparing (7.5%). Conclusions: Our findings suggest that peer review leads to changes in clinical care in approximately 1 of every 9 cases overall. This is similar to the reported rates of change in peer review studies from other oncology-related specialties, such as radiology and pathology.

62 RADIOLOGY AND NUCLEAR MEDICINE↗

Thank You to Our Peer Reviewers in 2025

Peer reviewing is the foundation of modern scholarship, with external specialists being asked to fairly check and evaluate submitted work. This difficult and often time‐consuming activity is performed voluntarily, with the understanding that one's own scholarship shall benefit down the line from a careful analysis of its assumption, results, accuracy, and yes, language, as we are now evaluating someone else's work. At Space Weather, we pride ourselves on a fair but quick review process yielding high‐quality articles with a time from submission to first decision of less than 2 months. This would not be possible without the hard work of all our reviewers. Once a year, we take the occasion to name these reviewers to thank them for their service to the journal and the community.

99 GENERAL AND MISCELLANEOUS↗

2023 IEDO Peer Review Report

On behalf of the U.S. Department of Energy (DOE) Industrial Efficiency and Decarbonization Office (IEDO), we are pleased to share the report from our 2023 peer review. Peer review provides us with invaluable insight into our office strategy and program direction from a diverse group of external thought leaders. The findings and actionable recommendations from peer review inform our decision making and help us maintain a constant state of evolution and growth.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗

NASA Product Peer Review Process

This viewgraph presentation describes NASA's product peer review process. The contents include: 1) Inspection/Peer Review at NASA; 2) Reasons for product peer reviews; 3) Different types of peer reviews; and 4) NASA requirements for peer reviews. This presentation also includes a demonstration of an actual product peer review.

Jenks, Ken↗

Summary of Technical Peer Review on the Risk Assessment Framework proposed in Report INL/RPT-22-68656 for Digital Instrumentation and Control Systems

This report summarizes the peer review activities initiated by Idaho National Laboratory (INL) during fiscal year (FY) 2023 for the evaluation and improvement of the methodology developed under the U.S. Department of Energy (DOE) Light Water Reactor Sustainability (LWRS) Program, Risk Informed Systems Analysis (RISA) Pathway, digital instrumentation and control (DI&C) risk assessment project. In FY 2019, the RISA Pathway initiated a project to develop a risk assessment strategy for delivering a technical basis to support effective and secure DI&C technologies for digital upgrades/designs. A framework was proposed for this strategy, which aims to (1) provide a best-estimate, risk-informed capability to quantitatively and accurately estimate the risk impact of plant modernization, considering the introduction of high safety-significant safety-related (HSSSR) DI&C systems, (2) support and supplement existing risk-informed DI&C design guides by providing quantitative risk information and evidence, (3) offer a capability of design architecture evaluation of various DI&C systems, (4) assure the long-term safety and reliability of HSSSR DI&C systems, and (5) reduce uncertainty in costs and support integration of DI&C systems in the plant. The R&D efforts of this project from FY 2019 through FY 2022 were focused on methodology improvement and demonstration of the proposed framework for the risk assessment and design optimization of safety-critical DI&C systems. Collaborations with the nuclear industry have been initiated to support the reliability and risk assessment of their DI&C systems by using the proposed framework. In FY 2023, the framework has reached to a point for a technical peer review and obtain stakeholder feedback. This peer review activity includes coordination of the reviews performed by a group of industry stakeholders, documentation of the peer review suggestions, providing resolutions and responses to the peer review comments. The objective of this technical peer review is to obtain representative feedback on the proposed framework to improve the technical qualities of its methodology and readiness for deployment to the industry. Feedback may identify potential areas for improvement and further development. The Subject Matter experts were invited to review the latest project report documenting the methodology developed in the project and provide evaluations of the technical qualities of the proposed framework and relevant methods. The reviewed project report is “An Integrated Framework for Risk Assessment of High Safety-significant Safety-related Digital Instrumentation and Control Systems in Nuclear Power Plants: Methodology and Demonstration” INL/RPT-22-68656 (short as “INL/RPT-22-68656” in this report). This peer review report documents the technical questions provided for technical peer review and introduces the technical peer reviewers from the stakeholders including nuclear utilities, regulators, and universities. Comments from technical peer reviewers and the resolutions and responses to these comments are outlined. Insights and lessons learned from the technical peer review are summarized in conclusions and future work. The primary audience of this report are DI&C designers, engineers, and probabilistic risk assessment (PRA) practitioners. This includes stakeholders, such as the nuclear utilities and regulators who consider the deployment and upgrade of DI&C systems, DI&C software developers and reviewers, and cybersecurity specialists.

99 GENERAL AND MISCELLANEOUS↗

Peer Review Lessons Learned - DOE and Regulator Perspectives - 17140

The DOE NNSA Nevada Field Office (NFO) Underground Test Area (UGTA) Activity requires an external peer review to complete an extensive stage of site characterization and groundwater flow and contaminant transport model(s) development. The external peer review evaluates whether assumptions, methods, and conclusions derived from model(s) outputs are based on sound scientific principles and examines the scientific appropriateness of the model(s) outputs for informing the regulatory decision. The regulatory decision which is made by the Nevada Division of Environmental Protection (NDEP) is whether the model is acceptable to advance to the next UGTA strategy stage. Depending on the specific corrective action unit, the next stage is a Corrective Action Decision Document/Corrective Action Plan or Closure Stage. Three external peer reviews have taken place to date for the UGTA Activity. The first one resulted in a second phase of characterization and model development. The second external peer review resulted in advancement to the next UGTA strategy stage. While advancement to the next strategy stage was recommended by the third external peer review, a series of supplemental data collection and analyses and model simulation activities to address external peer review comments, including drilling three new wells and sampling multiple wells, occurred before moving into the next UGTA strategy stage. Twelve participants representing different aspects of the peer reviews (NDEP, NNSA/NFO, science advisors, presenters, and external peer reviewers) were interviewed to generate a combined lessons learned. Questions were asked regarding their experience with the process including panel selection, focused questions, presentations, interactions between UGTA Activity participants and reviewers, and results. The interview results were combined with the outcome of a published lessons learned report developed following the first external peer review. In general, those interviewed believed that all three external peer reviews were satisfactory and met the goals of the FFACO process. The following seven lessons learned were identified: (1) Ensure the panel is not biased and has a balance of technical and regulatory background; (2) Clearly state the review objective(s) at the start of the review; (3) Ask focused questions that support the peer review objective(s) but that do not ask the specific regulatory question; (4) Establish a peer review panel earlier in the strategy process; (5) Establish one point of contact for the entire review and conduct periodic meetings with the peer review panel to ensure focus remains on the review objective(s) and to ensure that pertinent information is provided; (6) Ensure that presentations are well coordinated, anomalous data or analysis results are addressed, and are focused on supporting the peer review objective(s); and (7) Carefully consider the panel's recommendations prior to embarking on a particular path forward. (authors)

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗

2017 Water Power Project Peer Review Summary Report - Volume 1

The Water Power Technologies Office Peer Review Report summarizes the findings from a 2017 peer review held in Arlington, Virginia, with nearly 300 representatives from our national laboratories, industry, academic institutions, and other stakeholder organizations. Volume I includes sections 1–5 of the report: the executive summary, synopses of the program- and project-level evaluation results, and WPTO’s response to the 2017 Peer Review findings. This report details the results of the Water Power Peer Review Meeting, which was held in February 2017 in Arlington, VA. The purpose of the review was to evaluate WPTO-funded projects for their contribution to the mission and goals of the WPTO and DOE, assess progress against stated objectives, and appraise WPTO’s overall management and performance.

13 HYDRO ENERGY↗

Redefining the Practice of Peer Review Through Intelligent Automation—Part 3: Automated Report Analysis and Data Reconciliation

One method for addressing existing peer review limitations is the assignment of peer review cases on a completely blinded basis, in which the peer reviewer would create an independent report which can then be cross-referenced with the primary reader report of record. By leveraging existing computerized data mining techniques, one could in theory automate and objectify the process of report data extraction, classification, and analysis, while reducing time and resource requirements intrinsic to manual peer review report analysis. Once inter-report analysis has been performed, resulting inter-report discrepancies can be presented to the radiologist of record for review, along with the option to directly communicate with the peer reviewer through an electronic data reconciliation tool aimed at collaboratively resolving inter-report discrepancies and improving report accuracy. All associated report and reconciled data could in turn be recorded in a referenceable peer review database, which provides opportunity for context and user-specific education and decision support.

42 ENGINEERING↗

Water Power Technologies Office: 2022 Project Peer Review

The Water Power Technologies Office (WPTO), part of the U.S. Department of Energy’s (DOE) Office of Energy Efficiency and Renewable Energy (EERE), held a virtual peer review on July 18–29, 2022. The purpose of peer review is to evaluate DOE-funded projects for their contributions to the mission and goals of the office, progress made against stated objectives, and the office’s overall management and performance. The peer review process enables external stakeholders to provide feedback on the most impactful use of taxpayer funding and develop recommendations for the most efficient and effective ways to accelerate industry development in power technologies. This report includes the results of WPTO’s 2022 Peer Review.

13 HYDRO ENERGY↗

SU-F-T-231: Improving the Efficiency of a Radiotherapy Peer-Review System for Quality Assurance

Purpose: To improve the efficiency of a radiotherapy peer-review system using a commercially available software application for plan quality evaluation and documentation. Methods: A commercial application, FullAccess (Radialogica LLC, Version 1.4.4), was implemented in a Citrix platform for peer-review process and patient documentation. This application can display images, isodose lines, and dose-volume histograms and create plan reports for peer-review process. Dose metrics in the report can also be benchmarked for plan quality evaluation. Site-specific templates were generated based on departmental treatment planning policies and procedures for each disease site, which generally follow RTOG protocols as well as published prospective clinical trial data, including both conventional fractionation and hypo-fractionation schema. Once a plan is ready for review, the planner exports the plan to FullAccess, applies the site-specific template, and presents the report for plan review. The plan is still reviewed in the treatment planning system, as that is the legal record. Upon physician’s approval of a plan, the plan is packaged for peer review with the plan report and dose metrics are saved to the database. Results: The reports show dose metrics of PTVs and critical organs for the plans and also indicate whether or not the metrics are within tolerance. Graphical results with green, yellow, and red lights are displayed of whether planning objectives have been met. In addition, benchmarking statistics are collected to see where the current plan falls compared to all historical plans on each metric. All physicians in peer review can easily verify constraints by these reports. Conclusion: We have demonstrated the improvement in a radiotherapy peer-review system, which allows physicians to easily verify planning constraints for different disease sites and fractionation schema, allows for standardization in the clinic to ensure that departmental policies are maintained, and builds a comprehensive database for potential clinical outcome evaluation.

60 APPLIED LIFE SCIENCES↗

Building Technologies Office: Peer Review Report 2019

To ensure BTO projects are relevant, effective, and productively assisting the Office in meeting its goals, BTO conducts an annual peer review. The peer review is a formal, documented evaluation process that uses objective criteria and qualified independent reviewers to judge BTO-funded projects in terms of their technical, scientific, or business merit; actual or anticipated results; and productivity and effectiveness. Knowledge about the quality and effectiveness of current BTO projects and programs is essential for improving existing activities and informing future efforts. The BTO Peer Review is free and open to the public and provides an opportunity to learn more about BTO’s various projects and initiatives as well as promote collaborations and partnerships.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

The role of peer review and science for advancing public understanding

We live in a time of rapid change. The Covid-19 pandemic has united the world to understand and develop solutions to protect the health and wellbeing of society. At the same time, access to information on the internet and social media has exploded. People are exposed to information with many claims. Some of the information is relevant and accurate. A significant fraction may be inaccurate, biased, and potentially dangerous. With Covid-19, the information might point to cures that are toxic or risks that are not relevant. The scientific community has developed a process to develop and convey knowledge. The validation of results and their interpretation is critical to advancing knowledge. The core of the scientific process is based on developing a hypothesis-driven concept, designing and executing an experimental approach to prove or disprove the hypothesis, conducting a statistical analysis of the results to confirm their validity, and interpreting their meaning and impact. When their work is completed the scientist prepares to publish the work. Publication requires peer review. Hypotheses, experimental approaches, results, and interpretation of results are independently and anonymously reviewed by a scientific peer. The reviewers are selected based on their expertise in the field. The peer review process is essential for the integrity of the scientific process. Without a robust scientific process, society cannot discriminate between validated results and interpretations and biased, inaccurate, and potentially dangerous conjecture. With inaccurate science society and policy makers may make poor decisions that could hurt people and the world around them. While scientists should not censor information available to the public, scientists have a responsibility to inform the public. We must articulate what the scientific process is and why it is essential when considering the validity of information. Scientist are responsible for pointing out when public information has not been vetted through a scientific process and therefore cannot be considered conclusive. Scientists are responsible for highlighting information that has been generated and vetted with the scientific process. Over time, our goal is for the public to value the scientific process based on peer review. The public could become enlightened to the necessity for peer review to support decisions and action. Scientists need to contribute to advancing public knowledge. When invited to serve as a peer reviewer, scientists have a responsibility to contribute to the process. We all are extremely busy making peer review a distraction. While peer review does not directly benefit our own work and career, it benefits our chosen field of science. This is very important for the research area that our journal, Water-Energy Nexus, is focusing on. We cover the convergence of science and technology integrating across large spatial ranges (micro to global) in environmental pollution and the balance of remediation issues with energy use. Water-Energy Nexus wants to provide scientists with a platform for scientific debate and discussion on this rich and complicated topic. With the manuscripts submitted to WEN cover integrated research topics across a broad range of subjects from environment sciences through energy engineering, the role of peer review is essential. Peer review on a new submission to WEN should not be biased towards a specific science field. It should be done considering the complex nature of issues in the water-energy nexus. Scientists need to contribute to the public dialog. To accomplish these goals, we need to be honest, accurate, articulate, and use understandable language. The impact of WEN is very diverse affecting society in many ways. The scientific audience for WEN is diverse and readers may not fully understand the content of the papers outside their research expertise. Therefore, scientists, especially those who want to contribute to WEN, should work with communications specialists to convey message to benefit society. The audience includes the general public and researchers outside the author’s core specialty. While the peer-reviewed journal article is the cornerstone of our careers and critical for advancing scientific knowledge, a general interest story could benefit society even more. The author cannot lose sight that our work has two audiences, the science community, and more importantly, our family, friends, neighbors, and policy makers.

54 ENVIRONMENTAL SCIENCES↗

2014 Wind Program Peer Review Report

The Wind Program Peer Review Meeting was held March 24-28, 2014 in Arlington, VA. Principle investigators from the Energy Department, National Laboratories, academic, and industry representatives presented the progress of their DOE-funded research. This report documents the formal, rigorous evaluation process and findings of nine independent reviewers who examined the technical, scientific, and business results of Wind Program funded projects, as well as the productivity and management effectiveness of the Wind Program itself.

17 WIND ENERGY↗

2014 Water Power Program Peer Review Report

The Water Power Peer Review Meeting was held February 24-28, 2014 in Arlington, VA. Principle investigators from the Energy Department National Laboratories, academic, and industry representatives presented the progress of their DOE-funded research. This report documents the formal, rigorous evaluation process and findings of nine independent reviewers who examined the technical, scientific, and business results of 96 projects of the Water Power Program, as well as the productivity and management effectiveness of the Water Power Program itself.

13 HYDRO ENERGY↗

2016 Building Technologies Office Peer Review Report

The 2016 Building Technologies Office Peer Review Report summarizes the feedback submitted by reviewers of the 67 BTO projects presented at the 2016 BTO Peer Review. The report presents an overview of the goals and activities under each technology program area, a summary of project scores for each program, and a brief analysis of general evaluation trends within each program area or its constituent subprograms.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗