SEARCH · Search NASA

Results for “false negative”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 127 records · Page 7

All sources NASA NTRS DOE OSTI

DOE OSTI2024

EHR-BERT: A BERT-based model for effective anomaly detection in electronic health records

Objective: Physicians and clinicians rely on data contained in electronic health records (EHRs), as recorded by health information technology (HIT), to make informed decisions about their patients. The reliability of HIT systems in this regard is critical to patient safety. Consequently, better tools are needed to monitor the performance of HIT systems for potential hazards that could compromise the collected EHRs, which in turn could affect patient safety. In this paper, we propose a new framework for detecting anomalies in EHRs using sequence of clinical events. This new framework, EHR-Bidirectional Encoder Representations from Transformers (BERT), is motivated by the gaps in the existing deep-learning related methods, including high false negatives, sub-optimal accuracy, higher computational cost, and the risk of information loss. EHR-BERT is an innovative framework rooted in the BERT architecture, meticulously tailored to navigate the hurdles in the contemporary BERT method; thus, enhancing anomaly detection in EHRs for healthcare applications.Methods: The EHR-BERT framework was designed using the Sequential Masked Token Prediction (SMTP) method. This approach treats EHRs as natural language sentences and iteratively masks input tokens during both training and prediction stages. This method facilitates the learning of EHR sequence patterns in both directions for each event and identifies anomalies based on deviations from the normal execution models trained on EHR sequences.Results: Extensive experiments on large EHR datasets across various medical domains demonstrate that EHR-BERT markedly improves upon existing models. It significantly reduces the number of false positives and enhances the detection rate, thus bolstering the reliability of anomaly detection in electronic health records. This improvement is attributed to the model’s ability to minimize information loss and maximize data utilization effectively.Conclusion: EHR-BERT showcases immense potential in decreasing medical errors related to anomalous clinical events, positioning itself as an indispensable asset for enhancing patient safety and the overall standard of healthcare services. The framework effectively overcomes the drawbacks of earlier models, making it a promising solution for healthcare professionals to ensure the reliability and quality of health data.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗

NASA NTRS1995

PCR Based Microbial Monitor for Analysis of Recycled Water Aboard the ISSA: Issues and Prospects

The monitoring of spacecraft life support systems for the presence of health threatening microorganisms is paramount for crew well being and successful completion of missions. Development of technology to monitor spacecraft recycled water based on detection and identification of the genetic material of contaminating microorganisms and viruses would be a substantial improvement over current NASA plans to monitor recycled water samples that call for the use of conventional microbiology techniques which are slow, insensitive, and labor intensive. The union of the molecular biology techniques of DNA probe hybridization and polymerase chain reaction (PCR) offers a powerful method for the detection, identification, and quantification of microorganisms and viruses. This technology is theoretically capable of assaying samples in as little as two hours with specificity and sensitivity unmatched by any other method. A major advance in probe-hybridization/PCR has come about in a technology called TaqMan(TM), which was invented by Perkin Elmer. Instrumentation using TaqMan concepts is evolving towards devices that could meet NASA's needs of size, low power use, and simplicity of operation. The chemistry and molecular biology needed to utilize these probe-hybridization/PCR instruments must evolve in parallel with the hardware. The following issues of chemistry and biology must be addressed in developing a monitor: Early in the development of a PCR-based microbial monitor it will be necessary to decide how many and which organisms does the system need the capacity to detect. We propose a set of 17 different tests that would detect groups of bacteria and fungus, as well as specific eukaryotic parasites and viruses; In order to use the great sensitivity of PCR it will be necessary to concentrate water samples using filtration. If a lower limit of detection of 1 microorganism per 100 ml is required then the microbes in a 100 ml sample must be concentrated into a volume that can be added to a PCR assay; There are not likely to be contaminants in ISSA recycled water that would inhibit PCR resulting in false-negative results; The TaqMan PCR product detection system is the most promising method for developing a rapid, highly automated gene-based microbial monitoring system. The method is inherently quantitative. NASA and other government agencies have invested in other technologies that, although potentially could lead to revolutionary advances, are not likely to mature in the next 5 years into working systems; PCR-based methods cannot distinguish between DNA or RNA of a viable microorganism and that of a non-viable organism. This may or may not be an important issue with reclaimed water on the ISSA. The recycling system probably damages the capacity of the genetic material of any bacteria or viruses killed during processing to serve as a template in a PCR desinged to amplify a large segment of DNA (less than 650 base pairs). If necessary, vital dye staining could be used in addition to PCR, to enumerate the viable cells in a water sample; The quality control methods have been developed to insure that PCR's are working properly, and that reactions are not contaminated with PCR carryover products which could lead to the generation of false-positive results; and The sequences of the small rRNA subunit gene for a large number of microorganisms are known, and they consititue the best database for rational development of the oligonucleotide reagents that give PCR its great specificity. From those gene sequences, sets of oligonucleotide primers for PCR and Taqman detection that could be used in a NASA microbial monitor were constructed using computer based methods. In addition to space utilization, a microbial monitior will have tremendous terrestrial applications. Analysis of patient samples for microbial pathogens, testing industrial effluent for biofouling bacteria, and detection biological warfare agents on the battlefield are but a few of the diverse potential uses for this technology. Once fully developed, gene-based microbial monitors will become the fundamental tool in every lab that tests for microbial contaminants, and serve as a powerful weapon in mankind's war with the germ world.

Cassell, Gail H.↗

DOE OSTI2014

Determination of cellulose crystallinity from powder diffraction diagrams: Powder Diffraction Diagrams

Commonly one-dimensional (1D) (spherically averaged) powder diffraction diagrams are used to determine the degree of cellulose crystallinity in biomass samples. Here, it is shown using molecular modeling how disorder in cellulose fibrils can lead to considerable uncertainty in conclusions drawn concerning crystallinity based on 1D powder diffraction data alone. For example, cellulose microfibrils that contain both crystalline and noncrystalline segments can lead to powder diffraction diagrams lacking identifiable peaks, while microfibrils without any crystalline segments can lead to such peaks. Moreover, this leads to false positives, that is, assigning disordered cellulose as crystalline, and false negatives, that is, categorizing fibrils with crystalline segments as amorphous. Finally, the reliable determination of the fraction of crystallinity in any given biomass sample will require a more sophisticated approach combining detailed experiment and simulation.

36 MATERIALS SCIENCE↗

DOE OSTI2018

Phosphodiesterase (PDE5) inhibition assay for rapid detection of erectile dysfunction drugs and analogs in sexual enhancement products

Abstract Products marketed as dietary supplements for sexual enhancement are frequently adulterated with phosphodiesterase‐5 (PDE5) inhibitors, which are erectile dysfunction drugs or their analogs that can cause adverse health effects. Due to widespread adulteration, a rapid screening assay was developed to detect PDE5 inhibitors in adulterated products. The assay employs fluorescence detection and is based on measuring inhibition of PDE5 activity, the pharmacological mechanism shared among the adulterants. Initially, the assay reaction scheme was established and characterized, followed by analysis of 9 representative PDE5 inhibitors ( IC 50 , 0.4–4.0 ng mL –1 ), demonstrating sensitive detection in matrix‐free solutions. Next, dietary supplements serving as matrix blanks ( n = 25) were analyzed to determine matrix interference and establish a threshold value; there were no false positives. Finally, matrix blanks were spiked with 9 individual PDE5 inhibitors, along with several mixtures. All 9 adulterants were successfully detected (≤ 5 % false negative rate; n = 20) at a concentration of 1.00 mg g –1 , which is over 5 times lower than concentrations commonly encountered in adulterated products. A major distinction of the PDE5 inhibition assay is the ability to detect adulterants without prior knowledge of their chemical structures, demonstrating a broad‐based detection capability that can address a continuously evolving threat of new adulterants. The PDE5 inhibition assay can analyze over 40 samples simultaneously within 15 minutes and involves a single incubation step and simple data analysis, all of which are advantageous for combating the widespread adulteration of sex‐enhancement products.

Santillo, Michael F.↗

DOE OSTI2022

Estimating dermal contact soil exposure for amphibians

Abstract Chemical exposure estimation through the dermal route is an underemphasized area of ecological risk assessment for terrestrial animals. Currently, there are efforts to create exposure models to estimate doses from this pathway for use in ecological risk assessment. One significant limitation has been insufficient published data to characterize exposure and to support the selection and parameterization of appropriate models, particularly for amphibians in terrestrial habitats. Recent publications measuring pesticide doses to terrestrial‐phase amphibians have begun to rectify this situation. We collated and summarized available measurements of terrestrial amphibian dermal exposure to pesticides from 11 studies in which researchers measured tissue concentrations associated with known pesticide experimental application rates. This data set included tissue concentrations in 11 amphibian species and 14 different pesticides. We then compared the results of two screening exposure models that differed based on surface area scaling approaches as a function of body weight (one based on birds as surrogates for amphibians and another amphibian‐specific) to the measured tissue residue concentrations. We define a false‐negative rate for each screening model as the proportion of amphibians for which the predicted concentration is less than the observed concentration (i.e., underestimate), contrary to the intent of screening models, which are intended to have a bias for higher exposure concentrations. The screening model that uses birds as surrogates did not have any instances where estimated expected avian doses were less than measured amphibian body burdens. When using the amphibian‐specific exposure model that corrected for differences between avian and amphibian surface area, measured concentrations were greater than model estimates for 11.3% of the 1158 comparisons. The database of measured pesticide concentrations in terrestrial amphibians is provided for use in calculating bioconcentration factors and for future amphibian dermal exposure model development. Integr Environ Assess Manag 2023;19:9–16. © 2022 SETAC. This article has been contributed to by U.S. Government employees and their work is in the public domain in the USA.

Environmental Sciences & Ecology↗

DOE OSTI2024

Inferring precocial Chinook Salmon production through single‐parentage assignments

Abstract Objective Parentage analysis is a routine methodology in fisheries research, but study systems exist where it is impractical to sample both parents. The ability to reliably assign offspring to a single parent is beneficial in these situations. We applied single‐parentage assignments to a naturally spawning population of Chinook Salmon Oncorhynchus tshawytscha to quantify production of anadromous returns by unsampled precocial males. Methods We used an approach that focused on two important aspects of parentage analyses: (1) addressing the presence of family structure within the set of sampled parents and (2) controlling for false‐positive and false‐negative assignments. Result Results indicated that 30% of reproductively successful males were precocial males, which produced 20% of the returning anadromous offspring. Conclusion This study provides a framework for applying single‐parent assignments in a salmonid study system while explicitly addressing sources of assignment errors.

Steele, Craig A.↗

DOE OSTI2019

Imaging post-stereotactic body radiation therapy responses for hepatocellular carcinoma: typical imaging patterns and pitfalls

Stereotactic body radiation therapy (SBRT) has increased utility in the management of hepatocellular carcinoma (HCC) ranging from local therapy in early-stage HCC not suitable for other focal therapies to end-stage HCC. As the indications for the use of SBRT in HCC expand, diagnostic imaging is being increasingly used to assess response to treatment. The imaging features of tumor response do not parallel those of other focal therapies such as radiofrequency ablation or trans-arterial chemoembolization that immediately devascularize the tumor. The tumor response to SBRT on imaging takes much longer and often shows gradual changes including the reduction of enhancement and size over several months. It is essential to recognize the typical imaging patterns of response, as well as the appearance of focal liver reaction in the non-target liver that can confound image interpretation. The timing of treatment response assessment imaging is fundamental to minimize the potential for false negative response. The purpose of this article is to review the variable post-SBRT imaging features of HCC and adjacent liver parenchyma and discuss the potential pitfalls of imaging evaluation after SBRT for HCC.

62 RADIOLOGY AND NUCLEAR MEDICINE↗

DOE OSTI2016

MRI-Guided Percutaneous Biopsy of Mediastinal Masses Using a Large Bore Magnet: Technical Feasibility

ObjectiveTo evaluate the diagnostic accuracy and safety of magnetic resonance imaging (MRI)-guided percutaneous biopsy of mediastinal masses performed using a wide-bore high-field scanner.Materials and MethodsThis is a retrospective study of 16 consecutive patients (8 male, 8 female; mean age 74 years) who underwent MRI-guided core needle biopsy of a mediastinal mass between February 2010 and January 2014. Size and location of lesion, approach taken, time for needle placement, overall duration of procedure, and post-procedural complications were evaluated. Technical success rates and correlation with surgical pathology (where available) were assessed.ResultsTarget lesions were located in the anterior (n = 13), middle (n = 2), and posterior mediastinum (n = 1), respectively. Mean size was 7.2 cm (range 3.6–11 cm). Average time for needle placement was 9.4 min (range 3–18 min); average duration of entire procedure was 42 min (range 27–62 min). 2–5 core samples were obtained from each lesion (mean 2.6). Technical success rate was 100 %, with specimens successfully obtained in all 16 patients. There were no immediate complications. Histopathology revealed malignancy in 12 cases (4 of which were surgically confirmed), benign lesions in 3 cases (1 of which was false negative following surgical resection), and one inconclusive specimen (treated as inaccurate since repeat CT-guided biopsy demonstrated thymic hyperplasia). Sensitivity, specificity, positive predictive value, negative predictive value, and accuracy in our study were 92.3, 100, 100, 66.7, and 87.5 %, respectively.ConclusionMRI-guided mediastinal biopsy is a safe procedure with high diagnostic accuracy, which may offer a non-ionizing alternative to CT guidance.

62 RADIOLOGY AND NUCLEAR MEDICINE↗

DOE OSTI2013

Cross-Combined Composite Sampling Designs for Identification of Elevated Regions

Analyzing soils for contaminants can be costly. Generally, discrete samples are gathered from within a study area, analyzed by a laboratory and the results are used in a site-specific statistical analysis. Because of the heterogeneities that exist in soil samples within study areas, a large amount of variability and skewness may be present in the sample population. This necessitates collecting a large number of samples to obtain reliable inference on the mean contaminant concentration and to understand the spatial patterns for future remediation. Composite or Incremental sampling is a commonly applied method for gathering multiple discrete samples and physically combining them, such that each combination of discrete samples requires a single laboratory analysis, which reduces cost and can improve the estimates of the mean concentration. While incremental sampling can reduce cost and improve mean estimates, current implementations do not readily facilitate the characterization of spatial patterns or the detection of elevated regions within study areas. The methods we present in this work provide efficient estimation and inference for the mean contaminant concentration over the entire spatial area and enable the identification of high contaminant regions within the area of interest. We develop sample design methodologies that explicitly define the characteristics of these designs (such as sample grid layout) and quantify the number of incremental samples that must be obtained under a design criteria to control false positive and false negative (Type I and II) decision errors. We present the sample design theory and specifications as well as results on simulated and real data.

Hathaway, John E↗

DOE OSTI2019

Crowdsourced reliable labeling of safety-rule violations on images of complex construction scenes for advanced vision-based workplace safety

Construction workplace hazard detection requires engineers to analyze scenes manually against many safety rules, which is time-consuming, labor-intensive, and error-prone. Computer vision algorithms are yet to achieve reliable discrimination of anomalous and benign object relations underpinning safety violation detections. Recently developed deep learning-based computer vision algorithms need tens of thousands of images, including labels of the safety rules violated, in order to train deep-learning networks for acquiring spatiotemporal reasoning capacity in complex workplaces. Such training processes need human experts to label images and indicate whether the relationship between the worker, resource, and equipment in the scenes violate spatiotemporal arrangement rules for safe and productive operations. False alarms in those manual labels (labeling no-violation images as having violations) can significantly mislead the machine learning process and result in computer vision models that produce inaccurate hazard detections. Compared with false alarms, another type of mislabels, false negatives (labeling images having violations as “no violations”), seem to have fewer impacts on the reliability of the trained computer vision models.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2017

Computing the binding affinity of a ligand buried deep inside a protein with the hybrid steered molecular dynamics

Computing the ligand-protein binding affinity (or the Gibbs free energy) with chemical accuracy has long been a challenge for which many methods/approaches have been developed and refined with various successful applications. False positives and, even more harmful, false negatives have been and still are a common occurrence in practical applications. Inevitable in all approaches are the errors in the force field parameters we obtain from quantum mechanical computation and/or empirical fittings for the intra- and inter-molecular interactions. These errors propagate to the final results of the computed binding affinities even if we were able to perfectly implement the statistical mechanics of all the processes relevant to a given problem. And they are actually amplified to various degrees even in the mature, sophisticated computational approaches. In particular, the free energy perturbation (alchemical) approaches amplify the errors in the force field parameters because they rely on extracting the small differences between similarly large numbers. In this paper, we develop a hybrid steered molecular dynamics (hSMD) approach to the difficult binding problems of a ligand buried deep inside a protein. Sampling the transition along a physical (not alchemical) dissociation path of opening up the binding cavity- -pulling out the ligand- -closing back the cavity, we can avoid the problem of error amplifications by not relying on small differences between similar numbers. We tested this new form of hSMD on retinol inside cellular retinol-binding protein 1 and three cases of a ligand (a benzylacetate, a 2-nitrothiophene, and a benzene) inside a T4 lysozyme L99A/M102Q(H) double mutant. In all cases, we obtained binding free energies in close agreement with the experimentally measured values. This indicates that the force field parameters we employed are accurate and that hSMD (a brute force, unsophisticated approach) is free from the problem of error amplification suffered by many sophisticated approaches in the literature.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

DOE OSTI2022

Development and testing of a performance evaluation methodology to assess the reliability of occupancy sensor systems in residential buildings

With the emergence of advanced occupancy sensor technologies to better detect occupancy in buildings, a universal methodology and metrics are required to evaluate and report sensor systems’ reliability and compare the performance across multiple sensor systems. Herein this research presents a methodology to assess the reliability of occupancy sensor systems in residential buildings in a controlled laboratory environment, including both “typical” and “failure” testing scenarios. The developed methodology was then implemented to evaluate a novel occupancy detection sensor system’s reliability. “Typical” testing evaluates the overall accuracy of the sensor system, which suggest how reliable the occupancy sensor system is over time. Results show that on average, the precision and recall are 0.75 and 0.70, indicating similar numbers of false positives and false negatives across the dataset. The overall accuracy of the tested sensor system was 62.4% to 76.4%. Failure testing results indicate whether there are influential variables impacting the sensor performance. For the tested sensor system, the number of occupants, presence of large objects, presence of interior light sources, and number of doors are not influential, while lighting level, location of occupants, additional door in the entry/exit area, and having the TV on are variables determined to impact the sensor system performance.

47 OTHER INSTRUMENTATION↗

DOE OSTI2024

Long-Term field testing of the accuracy and HVAC energy savings potential of occupancy presence sensors in A Single-Family home

The energy-saving potential of occupancy-centric smart thermostats has been extensively explored in simulations but lacked field testing for energy savings quantification and sensor performance assessment in real buildings. This paper presents a long-term field study conducted in a single-family home in Texas, U.S. to evaluate the performance of occupancy-centric controls (OCC) of HVAC (heating, ventilation, and air-conditioning) system in terms of energy savings, sensor accuracy, and impact on electric peak demand. The test site was equipped with a commercial off-the-shelf (COTS) smart thermostat and multiple occupancy presence sensors for OCC implementation. Additionally, a sub-metering system was installed to monitor electricity consumption of various end-use equipment, including the HVAC system. A supplementary device was installed to track the ground-truth occupancy for the accuracy evaluation of the occupancy presence sensor. Scenarios of baseline and OCC controls were alternated weekly over the 20-month testing period. The results indicated an effective OCC execution, as evidenced by indoor temperature profiles. During the 2023 cooling season, OCC achieved total energy savings of 1,958 kWh, corresponding to a 17.6% energy savings ratio. Under certain conditions, daily HVAC energy savings reached as high as 17 kWh, with a savings ratio of 35%. Sensor performance showed an overall accuracy of 83.8%, a False Positive Rate (FPR) of 12.8%, and a False Negative Rate (FNR) of 47.4%. A key limitation was the sensor’s inability to detect stationary occupants during sleep, leading to a midnight FNR of nearly 100% and significantly compromising thermal comfort. Additionally, the implementation of OCC resulted in extended periods of high electricity demand on summer afternoons, affecting occupant’s thermal comfort and posing potential challenges to community-level grid operations if OCC were widely adopted. Furthermore, this study addresses a critical research gap by empirically investigating energy-saving potential and occupancy sensor performance in residential buildings. Through a comprehensive field-testing study, the research examines the interrelationship between sensor accuracy, energy savings, and thermal comfort, an area that has received limited attention in the current literature.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

DOE OSTI2014

Examination of lead concentrations in new decorative enamel paints in four countries with different histories of activity in lead paint regulation

Paints with high lead concentrations (ppm) continue to be sold around the world in many developing countries and those with economies in transition representing a major preventable environmental health hazard that is being increased as the economies expand and paint consumption is increasing. Prior lead paint testing had been performed in Brazil and India and these countries were selected to examine the impact of a new regulatory limit in Brazil and the impact of efforts of non-governmental organizations and others to stop the use of lead compounds in manufacturing paints. Armenia and Kazakhstan, in Central Asia, were selected because no information on lead concentration in those regions was available, no regulatory activities were evident and non-governmental organizations in the IPEN network were available to participate. Another objective of this research was to evaluate the lead loading (µg/cm{sup 2}) limit determined by X-Ray Fluorescence (XRF) for areas on toys that are too small to obtain a sample of sufficient size for laboratory analysis. The lead concentrations in more than three-fourths of the paints from Armenia and Kazakhstan exceeded 90 ppm, the limit in the United States, and 600 ppm, the limit in Brazil. The percentages were about one-half as high in Brazil and India. The average concentration in paints purchased in Armenia, 25,000 ppm, is among the highest that has been previously reported, that in Kazakhstan, 15,700 ppm, and India, 16,600, about median. The average concentration in Brazil, 5600 ppm, is among the lowest observed. Paints in Brazil that contained an average of 36,000 ppm before the regulatory limit became effective were below detection (<9 ppm) in samples collected in the current study. The lack of any apparent public monitoring of paint lead content as part of regulatory enforcement makes it difficult to determine whether the regulation was a major factor contributing to the decline in lead use in these paints. Using data from the current study and those available from other studies 24 of 28 paints from major brands in India decreased from high concentrations to 90 ppm or lower. Since lead concentrations in golden yellow paints from these brands were found to decrease to ≤90 ppm, it is possible that all 28 of these paints now contain ≤90 ppm since yellow paints usually have the highest lead concentrations. Other brands in Brazil and India that have been analyzed only one time had lead concentrations up to 59,000 ppm and 134,000 ppm, respectively. Less than one-third of the paints had notations on their labels with information about lead content and these were sometimes inaccurate. The label from one brand indicating “no added lead” contained paint with 134,000 ppm lead, the highest found in this study. Three percent (3 of 98) of the paints with surface lead loading that did not exceed 2 µg/cm{sup 2}, the limit established by the Consumer Product Safety Improvement Act for small areas on toys, contained greater than 90 ppm lead and thus were false negatives. Of the new paint samples that contained ≤600 ppm, 88% contained ≤90 ppm. Of the samples that contained ≤90 ppm, 97% contained ≤45 ppm and 92% contained ≤15 ppm. Based on these data it appears to be technically feasible to manufacture paints containing ≤90 ppm and in many cases to produce paints that have lead concentrations that do not exceed 15 ppm. - Highlights: • Brazilian paints with high lead (avg. 36,000 ppm) contained <9 ppm after regulation. • Lead in major Indian paint brands decreased to low levels by efforts of NGOs/others. • Most paints with lead ≤600 ppm also are ≤90 ppm and many also ≤15 ppm. • At the CPSIA limit of 2 µg Pb/cm{sup 2} for small areas on toys, 3% exceeded 90 ppm lead. • Most paint brands in Armenia and Kazakhstan based/manufactured outside the country.

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2015

Evaluation and comparison of benchmark QSAR models to predict a relevant REACH endpoint: The bioconcentration factor (BCF)

The bioconcentration factor (BCF) is an important bioaccumulation hazard assessment metric in many regulatory contexts. Its assessment is required by the REACH regulation (Registration, Evaluation, Authorization and Restriction of Chemicals) and by CLP (Classification, Labeling and Packaging). We challenged nine well-known and widely used BCF QSAR models against 851 compounds stored in an ad-hoc created database. The goodness of the regression analysis was assessed by considering the determination coefficient (R{sup 2}) and the Root Mean Square Error (RMSE); Cooper's statistics and Matthew's Correlation Coefficient (MCC) were calculated for all the thresholds relevant for regulatory purposes (i.e. 100 L/kg for Chemical Safety Assessment; 500 L/kg for Classification and Labeling; 2000 and 5000 L/kg for Persistent, Bioaccumulative and Toxic (PBT) and very Persistent, very Bioaccumulative (vPvB) assessment) to assess the classification, with particular attention to the models' ability to control the occurrence of false negatives. As a first step, statistical analysis was performed for the predictions of the entire dataset; R{sup 2}>0.70 was obtained using CORAL, T.E.S.T. and EPISuite Arnot–Gobas models. As classifiers, ACD and log P-based equations were the best in terms of sensitivity, ranging from 0.75 to 0.94. External compound predictions were carried out for the models that had their own training sets. CORAL model returned the best performance (R{sup 2}{sub ext}=0.59), followed by the EPISuite Meylan model (R{sup 2}{sub ext}=0.58). The latter gave also the highest sensitivity on external compounds with values from 0.55 to 0.85, depending on the thresholds. Statistics were also compiled for compounds falling into the models Applicability Domain (AD), giving better performances. In this respect, VEGA CAESAR was the best model in terms of regression (R{sup 2}=0.94) and classification (average sensitivity>0.80). This model also showed the best regression (R{sup 2}=0.85) and sensitivity (average>0.70) for new compounds in the AD but not present in the training set. However, no single optimal model exists and, thus, it would be wise a case-by-case assessment. Yet, integrating the wealth of information from multiple models remains the winner approach. - Highlights: • REACH encourages the use of in silico methods in the assessment of chemicals safety. • The performances of nine BCF models were evaluated on a benchmark database of 851 chemicals. • We compared the models on the basis of both regression and classification performance. • Statistics on chemicals out of the training set and/or within the applicability domain were compiled. • The results show that QSAR models are useful as weight-of-evidence in support to other methods.

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2022

Planning, design and logistics of a decision analysis study: The FBI/Ames study involving forensic firearms examiners

This paper describes design and logistical aspects of a decision analysis study to assess the performance of qualified firearms examiners working in accredited laboratories in the United States in terms of accuracy (error rate), repeatability, and reproducibility of decisions involving comparisons of fired bullets and cartridge cases. The purpose of the study was to validate current practice of the forensic discipline of firearms/toolmarks (F/T) examination. It elicited error rate data by counting the number of false positive and false negative conclusions. Preceded by the experimental design, decisions, and logistics described herein, testing was ultimately administered 173 qualified, practicing F/T examiners in public and private crime laboratories. The first round of testing evaluated accuracy, while two subsequent rounds evaluated repeatability and reproducibility of examiner conclusions. This project expands on previous studies by involving many F/T examiners in challenging comparisons and by executing the study in the recommended double-blind format.

99 GENERAL AND MISCELLANEOUS↗

DOE OSTI2012

Indications for Pelvic Nodal Treatment in Prostate Cancer Should Change. Validation of the Roach Formula in a Large Extended Nodal Dissection Series

Purpose: Previous studies have criticized the predicting ability of the Roach formula in assessing the risk of lymph node invasion (LNI) in contemporary patients with prostate cancer (PCa) due to a significant overestimation of LNI rates. However, all those studies included patients treated with limited pelvic lymph node dissection (PLND), which is associated with high rates of false negative findings. We hypothesized that the Roach formula is still an accurate tool for LNI predictions if an extended PLND (ePLND) is performed. Methods and Materials: We included 3,115 consecutive patients treated with radical prostatectomy and ePLND between 2000 and 2010 at a single tertiary referral center. Extended PLND consisted of removal of obturator, external iliac, and hypogastric lymph nodes. We externally validated the Roach formula by using the area under the receiver operating characteristics curve and calibration plot method. Moreover, we tested the performance characteristics of different formula-generated cutoff values ranging from 1% to 20%. Results: The accuracy of the Roach formula was 80.3%. The calibration showed only a minor underestimation of the LNI risk in high-risk patients (6.7%). According to the Roach formula, the use of 15% cut off would have allowed 74.2% (2,311/3,115) of patients to avoid nodal irradiation, while up to 32.7% (111/336) of all patients with LNI would have been missed. When the cut off was lowered to 6%, nodal treatment would have been spared in 1,541 (49.5%) patients while missing 41 LNI patients. The sensitivity, specificity, and negative predictive values associated with the 6% cut off were 87.9%, 54%, and 97.3%, respectively. Conclusions: The Roach formula is still accurate and does not overestimate the rate of LNI in contemporary prostate cancer patients if they are treated with ePLND. However, the recommended cut off of 15% would miss approximately one-third of patients with LNI. Based on our results, the cut off should be lowered to 6%.

62 RADIOLOGY AND NUCLEAR MEDICINE↗

DOE OSTI2013

Evaluation of 4-dimensional Computed Tomography to 4-dimensional Cone-Beam Computed Tomography Deformable Image Registration for Lung Cancer Adaptive Radiation Therapy

Purpose: To evaluate 2 deformable image registration (DIR) algorithms for the purpose of contour mapping to support image-guided adaptive radiation therapy with 4-dimensional cone-beam CT (4DCBCT). Methods and Materials: One planning 4D fan-beam CT (4DFBCT) and 7 weekly 4DCBCT scans were acquired for 10 locally advanced non-small cell lung cancer patients. The gross tumor volume was delineated by a physician in all 4D images. End-of-inspiration phase planning 4DFBCT was registered to the corresponding phase in weekly 4DCBCT images for day-to-day registrations. For phase-to-phase registration, the end-of-inspiration phase from each 4D image was registered to the end-of-expiration phase. Two DIR algorithms—small deformation inverse consistent linear elastic (SICLE) and Insight Toolkit diffeomorphic demons (DEMONS)—were evaluated. Physician-delineated contours were compared with the warped contours by using the Dice similarity coefficient (DSC), average symmetric distance, and false-positive and false-negative indices. The DIR results are compared with rigid registration of tumor. Results: For day-to-day registrations, the mean DSC was 0.75 ± 0.09 with SICLE, 0.70 ± 0.12 with DEMONS, 0.66 ± 0.12 with rigid-tumor registration, and 0.60 ± 0.14 with rigid-bone registration. Results were comparable to intraobserver variability calculated from phase-to-phase registrations as well as measured interobserver variation for 1 patient. SICLE and DEMONS, when compared with rigid-bone (4.1 mm) and rigid-tumor (3.6 mm) registration, respectively reduced the average symmetric distance to 2.6 and 3.3 mm. On average, SICLE and DEMONS increased the DSC to 0.80 and 0.79, respectively, compared with rigid-tumor (0.78) registrations for 4DCBCT phase-to-phase registrations. Conclusions: Deformable image registration achieved comparable accuracy to reported interobserver delineation variability and higher accuracy than rigid-tumor registration. Deformable image registration performance varied with the algorithm and the patient.

62 RADIOLOGY AND NUCLEAR MEDICINE↗