Search NASA⌕ Search

SEARCH · Search NASA

Results for “text extraction”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 55 records · Page 3
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

Modeling and Simulation of Electrostatics of Ge$_{\text{1-x}}$Sn$_{\text{x}}$ Layers Grown on Ge Substrates

This work introduces a comprehensive simulation tool that provides a robust 1D Schrödinger – Poisson solver for modeling the electrostatics of heterostructures with an arbitrary number of layers, and non-uniform doping profiles along with the treatment of partial ionization of dopants at low temperatures. The effective masses are derived from the first-principles calculations. The solver is used to characterize three Ge 1-x Sn x /Ge heterostructures with non-uniform doping profiles and determine the subband structure at various temperatures. Here, the simulation results of the sheet carrier densities show excellent agreement with the experimentally extracted data, thus demonstrating the capabilities of the solver.

42 ENGINEERING↗
DOE OSTI2025

Explainable Synthesizability Prediction of Inorganic Crystal Polymorphs Using Large Language Models

Abstract We evaluate the ability of machine learning to predict whether a hypothetical crystal structure can be synthesized and explain those predictions to scientists. Fine‐tuned large language models (LLMs) trained on a human‐readable text description of the target crystal structure perform comparably to previous bespoke convolutional graph neural network methods, but better prediction quality can be achieved by training a positive‐unlabeled learning model on a text‐embedding representation of the structure. An LLM‐based workflow can then be used to generate human‐readable explanations for the types of factors governing synthesizability, extract the underlying physical rules, and assess the veracity of those rules. These explanations can guide chemists in modifying or optimizing non‐synthesizable hypothetical structures to make them more feasible for materials design.

Kim, Seongmin [Department of Chemical and Biologic↗
DOE OSTI2025

Explainable Synthesizability Prediction of Inorganic Crystal Polymorphs Using Large Language Models

Abstract We evaluate the ability of machine learning to predict whether a hypothetical crystal structure can be synthesized and explain those predictions to scientists. Fine‐tuned large language models (LLMs) trained on a human‐readable text description of the target crystal structure perform comparably to previous bespoke convolutional graph neural network methods, but better prediction quality can be achieved by training a positive‐unlabeled learning model on a text‐embedding representation of the structure. An LLM‐based workflow can then be used to generate human‐readable explanations for the types of factors governing synthesizability, extract the underlying physical rules, and assess the veracity of those rules. These explanations can guide chemists in modifying or optimizing non‐synthesizable hypothetical structures to make them more feasible for materials design.

Kim, Seongmin [Department of Chemical and Biologic↗
DOE OSTI2025

CAMFeND: Credibility-Aware Multimodal Fake News Detection with Rotational Attention

In the evolving digital landscape, fake news is a significant challenge, influencing public perception and decision-making. Traditional detection approaches focus on single-modal data or simple multimodal fusion, often overlooking deeper interactions and news credibility. We propose a novel model addressing these limitations by introducing rotational attention and news domain information as a feature. Unlike static attention mechanisms, our rotational attention dynamically shifts query, key, and value roles across text and image inputs, enabling richer cross-modal interaction. Incorporating news domain information further enhances the model’s reliability by associating news posts with top domains extracted from Google search results, reducing false detections. This approach assesses both the content and the broader web context in which the news is discussed. Our model outperforms existing state-of-the-art methods by providing deeper, layered multimodal integration and domain information analysis, resulting in a more robust and adaptive fake news detection system.

Gupta, Nidhi↗
DOE OSTI2025

The Evolution of Randomized Clinical Trial Designs to Assess Therapeutics in Alzheimer Disease

Importance The success of recent randomized clinical trials (RCTs) for Alzheimer disease (AD), particularly those focusing on anti-amyloid therapies, has been discussed at length. However, the evolution of RCT design features for AD that preceded this success remain underexplored. Objective To describe temporal changes in the features of RCT design for interventions in AD. Evidence Review PubMed, Scopus, and Web of Science databases were searched in January 2025 for phase 2 and 3 AD RCTs published between January 1992 and December 2024. RCTs that investigated an intervention for AD, with a placebo or standard-of-care control group, were included. Four assessors independently reviewed full-text articles to capture study characteristics. Main Outcomes and Measures The number of participants and the duration of RCTs as well as the target population, outcomes, and funding were extracted from published reports. These features were analyzed with respect to time using linear regression and χ 2 analyses. Results The study included 203 RCTs with 79 589 participants testing interventions in AD. From 1992 to 2024, the mean sample size increased by 464% for phase 2 RCTs (from 42 to 237), and 50% for phase 3 RCTs (from 632 to 951), while the mean trial duration increased by 188% (from 16 to 46 weeks) for phase 2, and 256% (from 20 to 71 weeks) for phase 3 RCTs. This longer duration of RCTs may be partially attributed by a greater share of disease-modifying rather than symptomatic treatments. Similarly, more recent trials required AD biomarker evidence for enrollment (from 1 of 36 [2.7%] before 2006 to 40 of 76 [52.6%] since 2019). A substantial difference in the type of therapeutics researched was observed, with anti-amyloid and anti-tau RCTs being more likely to be funded by the pharmaceutical industry compared with neurotransmitter or other RCTs (anti-amyloid or anti-tau, 68 of 71 [95.8%]; neurotransmitter, 52 of 69 [77.6%]; other, 33 of 52 [63.5%]). RCT transparency improved, with more frequent data accessibility statements, registered reports, and better reporting on race and ethnicity. Conclusions and Relevance This methodology research of AD RCTs highlights substantial changes in key features of AD clinical trials from 1992 to 2024. AD RCTs have become larger and longer, such that they are powered to detect smaller clinical differences. The increased sample sizes and duration should enable the detection of smaller and more slowly occurring outcomes, which may lead to successful RCTs of therapies with slower and more subtle efficacy.

General & Internal Medicine↗
DOE OSTI2025

TropiRoot 1.0: Database of tropical root characteristics across environments

Tropical ecosystems contain the world's largest biodiversity of vascular plants. Yet, our understanding of tropical functional diversity and its contribution to global diversity patterns is constrained by data availability. This discrepancy underscores an urgent need to bridge data gaps by incorporating comprehensive tropical root data into global datasets. Here, we provide a database of tropical root characteristics. This new database, TropiRoot 1.0, will be instrumental in evaluating an array of hypotheses pertaining to root functional ecology and plant biogeography, both within the tropics and relative to other global biomes. The data compilation was conducted by the TropiRoot Initiative, in partnership with the Fine-Root Ecology Database (FRED) and the Global Root Trait (GRooT) database, Colorado State University (CSU) and the Smithsonian Tropical Research Institute (STRI). Literature search and data extraction were conducted between 2020 and 2024. Literature was identified using Web of Science, Scopus, and complemented using the expert knowledge of members of TropiRoot. To provide broad environmental and geographical distributions, literature searches included root characteristics (traits) across global change drivers, natural gradients, and from different continents. We adopted FRED standardized data columns and streamlined the format to enhance accessibility for data extraction across various user groups. This optimized framework resulted in a smaller, yet comprehensive datasheet. To make the database compatible with other global root trait initiatives, column identification was standardized following the codes provided by FRED. These efforts culminated in data extracted from 104 new sources, resulting in more than 8000 rows of data (either species or community data). Most of the data in TropiRoot 1.0 include root characteristics such as root biomass, morphology, root dynamics, mass fraction, architecture, anatomy, physiology, and root chemistry. This initiative represents a 30% increase in the currently available data for tropical roots in FRED. TropiRoot 1.0 contains root characteristics from 25 different countries, where seven are located in Asia, six in South America, five in Central America and the Caribbean, four in Africa, two in North America, and 1 in Oceania. Due to the volume of data, when ancillary data were available, including soil data, these data were either extracted and included in the database or its availability was recorded in an additional column. Multiple contributors checked the entries for outliers during the collation process to ensure data quality. For text-based observations, we examined all cells to ensure that their content relates to their specific categories. For numerical observations, we ordered each numerical value from least to greatest and plotted the values, checking apparent outliers against the data in their respective sources and correcting or removing incorrect or impossible values. Some data (soil and aboveground) have different columns for the same variable presented in different units, including originally published units, but root characteristics data had units converted to match those reported in FRED. By filling a gap from global databases, TropiRoot 1.0 expands our knowledge of otherwise so far underrepresented regions and our ability to assess global trends. This advancement can be used to improve tropical forest representation in vegetation models. The data are freely available and should be cited when used.

FRED↗
DOE OSTI2025

Data from TropiRoot 1.0 database: tropical root characteristics across environments

TropiRoot 1.0 is a new tropical root database with root characteristics across environment gradients. It has data extracted from 104 new sources, resulting in more than 8000 rows of data (either species or community data). Most of the data in TropiRoot 1.0 includes root characteristics such as root biomass, morphology, root dynamics, mass fraction, architecture, anatomy, physiology and root chemistry. This initiative represents an approximately 30% increase in the currently available data for tropical roots in the Fine Root Ecology Database (FRED). TropiRoot 1.0, contains root characteristics from 25 different countries where seven are located in Asia, six in South America, five in Central America and the Caribbean, four in Africa, two in North America, and 1 in Oceania. Due to the volume of data, when ancillary data was available, including soil data, these data was either extracted and included in the database or their availability was recorded in an additional column. Multiple contributors checked the entries for outliers during the collation process to ensure data quality. For text-based observations, we examined all cells to ensure that their content relates to their specific categories. For numerical observations, we ordered each numerical value from least to greatest and plotted the values, checking apparent outliers against the data in their respective sources and correcting or removing incorrect or impossible values. Some data (soil and aboveground) have different columns for the same variable presented in different units, including originally published units, but root characteristics data had units converted to match the ones reported in FRED. By filling a gap from global databases, TropiRoot 1.0 expands our knowledge of otherwise so far underrepresented regions, and our ability to assess global trends. This advancement can be used to improve tropical forest representation in vegetation models.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2025

Old Woman Creek Wetland Sediment and Electrochemical Sensor Microbial Community, 2023

We are developing a technique to monitor microbiological activities referred to as zero resistance ammetry, which entails the deployment of graphite electrodes in sediments. Measurement of current between electrodes of contrasting redox regimes and/or predominant terminal electron accepting processes can be used as an indicator of the extents of microbiological activity. We deployed an electrode array at depths of 2 mm, 4 mm, 76 mm, 78 mm, 152 mm, 154 mm, 227 mm, and 229 mm below the wetland sediment water interface in the Old Woman Creek National Estuarine Research Center, Huron, OH, USA (Lat. = 41.380833, Long. = -82.508889). A core was collected from adjacent sediment and subsamples were collected from depth intervals of 0 – 25 mm, 25 – 127 mm, 127 – 128 mm, and below 178 mm. To determine if the microbial communities attached to the electrodes were reflective of the adjacent sediment-associated microbial community, we conducted a 16S rRNA gene-based (V4 region) survey of these respective materials. This data package contains the results of these surveys, including metadata on the depths from which samples were collected (samples.csv), DNA extraction and sequencing information (OWC_DEPTH_AMPLICON_SEQUENCING_METADATA), sequence processing information (OWC_DEPTH_BIOINFORMATIC_METADATA.csv), an operational taxonomic unit (OTU) table (OWC_DEPTH_97OTUS_TABLE.csv), and nucleotide sequences of OTUs (OWC_DEPTH_97OTUS_SEQS.fasta). All files can be opened using a text-editing application. The fasta file is compatible with bioinformatics applications.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2024

Multicriteria Measures to Assess the Sustainability of Diets: A Systematic Review

Abstract Context Assessing the overall sustainability of a diet is a challenging undertaking requiring a holistic approach capable of addressing the multicriteria nature of this concept. Objective The aim was to identify and summarize the multicriteria measures used to assess the sustainability characteristics of diets reported at the individual level by healthy adults. Data Sources Articles were identified via PubMed, Scopus, and Web of Science. The search strategy consisted of key words and MeSH terms, and was concluded in September 2022, covering references in English, Spanish, and Portuguese. Data Extraction This systematic review followed the PRISMA guidelines. The search identified 5663 references, from which 1794 were duplicates. Two reviewers independently screened the titles and abstracts of each of the 3869 records and the full-text of the 144 references selected. Of these, 7 studies met the inclusion criteria. Data Analysis A total of 6 multicriteria measures were identified: 3 different Sustainable Diet Indices, the Quality Environmental Costs of Diet, the Quality Financial Costs of Diet, and the Environmental Impact of Diet. All of these incorporated a health/nutrition dimension, while the environmental and economic dimensions were the second and the third most integrated, respectively. A sociocultural sustainability dimension was included in only 1 of the measures. Conclusion Despite some methodological concerns in the development and validation process of the identified measures, their inclusion is considered indispensable in assessing the transition towards sustainable diets in future studies. Systematic Review Registration PROSPERO registration no. CRD42022358824.

Rei, Mariana (ORCID:0000000189453708)↗
DOE OSTI2024

Climate change, malaria and neglected tropical diseases: a scoping review

Abstract To explore the effects of climate change on malaria and 20 neglected tropical diseases (NTDs), and potential effect amelioration through mitigation and adaptation, we searched for papers published from January 2010 to October 2023. We descriptively synthesised extracted data. We analysed numbers of papers meeting our inclusion criteria by country and national disease burden, healthcare access and quality index (HAQI), as well as by climate vulnerability score. From 42 693 retrieved records, 1543 full-text papers were assessed. Of 511 papers meeting the inclusion criteria, 185 studied malaria, 181 dengue and chikungunya and 53 leishmaniasis; other NTDs were relatively understudied. Mitigation was considered in 174 papers (34%) and adaption strategies in 24 (5%). Amplitude and direction of effects of climate change on malaria and NTDs are likely to vary by disease and location, be non-linear and evolve over time. Available analyses do not allow confident prediction of the overall global impact of climate change on these diseases. For dengue and chikungunya and the group of non-vector-borne NTDs, the literature privileged consideration of current low-burden countries with a high HAQI. No leishmaniasis papers considered outcomes in East Africa. Comprehensive, collaborative and standardised modelling efforts are needed to better understand how climate change will directly and indirectly affect malaria and NTDs.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2024

Topological Interpretability for Deep Learning

With the growing adoption of AI-based systems across everyday life, the need to understand their decision-making mechanisms is correspondingly increasing. The level at which we can trust the statistical inferences made from AI-based decision systems is an increasing concern, especially in high-risk systems such as criminal justice or medical diagnosis, where incorrect inferences may have tragic consequences. Despite their successes in providing solutions to problems involving real-world data, deep learning (DL) models cannot quantify the certainty of their predictions. These models are frequently quite confident, even when their solutions are incorrect. This work presents a method to infer prominent features in two DL classification models trained on clinical and non-clinical text by employing techniques from topological and geometric data analysis. We create a graph of a model's feature space and cluster the inputs into the graph's vertices by the similarity of features and prediction statistics. We then extract subgraphs demonstrating high-predictive accuracy for a given label. These subgraphs contain a wealth of information about features that the DL model has recognized as relevant to its decisions. We infer these features for a given label using a distance metric between probability measures, and demonstrate the stability of our method compared to the LIME and SHAP interpretability methods. This work establishes that we may gain insights into the decision mechanism of a DL model. This method allows us to ascertain if the model is making its decisions based on information germane to the problem or identifies extraneous patterns within the data.

Spannaus, Adam↗
DOE OSTI2025

Text-mined dataset of solid-state syntheses with impurity phases using Large Language Model

Solid-state synthesis is widely used to obtain various inorganic materials, such as battery materials and bulk thermoelectrics. Despite its prevalence, the process remains challenging due to the lack of a general theory and well-understood underlying reaction mechanisms. While prior works have successfully extracted structured datasets from literature, they often neglect product phase purity or yield. In this work, we construct a solid-state synthesis dataset consisting of 80,806 syntheses extracted with a large language model (LLM), including 18,869 reactions with impurity phase(s). Our dataset not only validates expected thermodynamic trends for impurity phase formation but also identifies challenging cases where impurity phases emerge even when the target phase is significantly more stable.

Lee, Sanghoon↗
DOE OSTI2025

Decoding substance use disorder severity from clinical notes using a large language model

Substance use disorder (SUD) poses a major concern due to its detrimental effects on health and society. SUD identification and treatment depend on a variety of factors such as severity, co-determinants (e.g., withdrawal symptoms), and social determinants of health. Existing diagnostic coding systems used by insurance providers, like the International Classification of Diseases (ICD-10), lack granularity for certain diagnoses, but American clinicians will add this granularity (as that found within the Diagnostic and Statistical Manual of Mental Disorders classification or DSM-5) as supplemental unstructured text in clinical notes. Traditional natural language processing (NLP) methods face limitations in accurately parsing such diverse clinical language. Large language models (LLMs) offer promise in overcoming these challenges by adapting to diverse language patterns. This study investigates the application of LLMs for extracting severity-related information for various SUD diagnoses from clinical notes. We propose a workflow employing zero-shot learning of LLMs with carefully crafted prompts and post-processing techniques. Through experimentation with Flan-T5, an open-source LLM, we demonstrate its superior recall compared to the rule-based approach. Focusing on 11 categories of SUD diagnoses, we show the effectiveness of LLMs in extracting severity information, contributing to improved risk assessment and treatment planning for SUD patients.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2025

Cosmological constraints from the Planck cluster catalogue with DES shear profiles and Chandra observations

We present cosmological constraints from the Planck PSZ2 cosmological cluster sample, using weak-lensing shear profiles from Dark Energy Survey (DES) data and X-ray observations from the Chandra telescope for the mass calibration. We compute hydrostatic mass estimates for all clusters in the PSZ2 sample with a scaling relation between their Sunyaev-Zeldovich signal and X-ray derived hydrostatic mass, calibrated with the Chandra data. We introduce a method to correct these masses with a hydrostatic mass bias using shear profiles from wide-field galaxy surveys. We simultaneously fit the number counts of the PSZ2 sample and the mass calibration with the DES data, finding $Ω_\text{m}=0.312^{+0.018}_{-0.024}$, $σ_8=0.777\pm 0.024$, $S_8\equiv σ_8 \sqrt{Ω_\text{m} / 0.3}=0.791^{+0.023}_{-0.021}$, and $(1-b)=0.844^{+0.055}_{-0.062}$ for our baseline analysis when combined with BAO data. When considering a hydrostatic mass bias evolving with mass, we find $Ω_\text{m}=0.353^{+0.025}_{-0.031}$, $σ_8=0.751\pm 0.023$, and $S_8=0.814^{+0.019}_{-0.020}$. We verify the robustness of our results by exploring a variety of analysis settings, with a particular focus on the definition of the halo centre used for the extraction of shear profiles. We compare our results with a number of other analyses, in particular two recent analyses of cluster samples obtained from SPT and eROSITA data that share the same mass calibration data set. We find that our results are in overall agreement with most late-time probes, in very mild tension with CMB results (1.6$σ$), and in significant tension with results from eROSITA clusters (2.9$σ$). We confirm that our mass calibration is consistent with the eROSITA analysis by comparing masses for clusters present in both Planck and eROSITA samples, eliminating it as a potential cause of tension.

Aymerich, G. [Orsay, IAS; AIM, Saclay] (ORCID:0009↗
DOE OSTI2024

The mass profiles of dwarf galaxies from Dark Energy Survey lensing

We present a novel approach to extracting dwarf galaxies from photometric data to measure their average halo mass profile with weak lensing. We characterize their stellar mass and redshift distributions with a spectroscopic calibration sample. By combining the ${\sim} 5000\,\mathrm{deg}^2$ multiband photometry from the Dark Energy Survey and redshifts from the Satellites Around Galactic Analogs Survey with an unsupervised machine learning method, we select a low-mass galaxy sample spanning redshifts $z\lt 0.3$ and divide it into three mass bins. From low to high median mass, the bins contain [146 420, 330 146, 275 028] galaxies and have median stellar masses of $\log _{10}(M_*/\text{M}_\odot)=\left[8.52\substack{+0.57 -0.76},\, 9.02\substack{+0.50 -0.64},\, 9.49\substack{+0.50 -0.58}\right]$ . We measure the stacked excess surface mass density profiles, $\Delta \Sigma (R)$, of these galaxies using galaxy–galaxy lensing with a signal-to-noise ratio of [14, 23, 28]. Through a simulation-based forward-modelling approach, we fit the measurements to constrain the stellar-to-halo mass relation and find the median halo mass of these samples to be $\log _{10}(M_{\rm halo}/\text{M}_\odot)$ = [$10.67\substack{+0.2 -0.4}$, $11.01\substack{+0.14 -0.27}$, $11.40\substack{+0.08 -0.15}$]. The cold dark matter profiles are consistent with NFW (Navarro, Frenk, and White) profiles over scales ${\lesssim} 0.15 \, {h}^{-1}$ Mpc. We find that ${\sim} 20$ per cent of the dwarf galaxy sample are satellites. This is the first measurement of the halo profiles and masses of such a comprehensive, low-mass galaxy sample. The techniques presented here pave the way for extracting and analysing even lower mass dwarf galaxies and for more finely splitting galaxies by their properties with future photometric and spectroscopic survey data.

dark matter↗
DOE OSTI2024

Verification of RESRAD-OFFSITE Code (V.4)

This report documents the verification of RESRAD-OFFSITE Version 4.0 and describes, where necessary, the verification of the following: • The data comprising the standard dose and risk coefficient libraries in the RESRAD database files Master_dcf_ICRP07.mdb and Master_dcf_2k.mdb. • The extraction and transfer of the data from the selected database file to the computational code by the RESRAD-OFFSITE 4.0 interface, ResOWin.exe. • The different processes that are modeled by the main computational code in RESRAD OFFSITE 4.0, ResOMain.exe. • The data displayed in the graphical and text reports. Many verifications were performed as part of the quality assurance quality control program associated with the development and release of RESRAD-OFFSITE 4.0, namely: • developer testing, • internal independent testing, and • release testing. Some were also performed in response to questions from users regarding the performance of the code. The main text of the report focuses on summarizing a subset of those tests, both independent and developer tests that verified the computations performed by the code. The verifications included in this report served as the basis for the development of the release tests of the computational executables and provided the quantitative results to be compared with the code output. The input and output interfaces and the data transfers between the various executables of the code were tested while performing the verification testing. They were tested intentionally during release testing. This report also provides some basic information to help in understanding the activities that were verified. The report: • outlines the components of RESRAD-OFFSITE 4.0 and the interconnections between these components, • outlines the processes modeled by the computational code, • provides summary figures and tables to offer confirmation of the verification of the computational components of the code, • reproduces the verifiers’ reports, if available, in individual appendices, • refers to the previous verification report (Yu et al. 2011) for more details about some of the verifications, and • reproduces the test cases and the testers’ reports from the release testing in individual appendices, when possible.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2024

Orbitrap LC-MS Analysis of Nanoparticle Composition at the EPCAPE Mount Soledad site between 04 18 2023 and 06 14 2023

Weekly peak lists containing m/z, intensity, and assigned formula for filter samples, size selected for sub-100 nm particles. Filters were collected daily between 4/18/23 and 6/14/23, grouped based on calendar week for extraction, and analyzed via Thermo Scientific Q Exactive Plus Orbitrap LC-MS. Formulas were assigned to background-corrected peak lists and restricted to CHONS/CHONSNa atoms for the negative and positive modes respectively. Filters were grouped into calendar weeks 0-8 with dates provided in README text file.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2024

Automating the Analysis of Large Language Models Responses through Zero-Shot Question Answering

Recent advancements in Large Language Models (LLMs) have shown significant potential in various applications, yet their evaluation, particularly in zero-shot question answering scenarios, remains a challenging task. In this study, our objective was to explore precision metrics for Large Language Models (LLM) and design and implement a software pipeline to automatically evaluate LLMs' outputs under zero-shot question answering. Zero-shot question answering involves a model providing answers to questions about topics it hasn't seen during training. It leverages the principles of zero-shot learning by relying on semantic understanding and generalization from related knowledge. The data used was metadata from medical databases on congenital heart disease. We explored eleven LLM metrics and selected three for our evaluation: BLEU, BERTScore, and MoverScore. BLEU calculates a score based on the overlap of n-grams (contiguous sequences of n items, typically words) between the machine-generated translation and the reference translations. Higher BLEU scores indicate better correspondence between the machine-generated and human-generated translations. BERTScore is a metric used to evaluate the quality of machine-generated text by measuring the similarity of token embeddings produced by BERT (Bidirectional Encoder Representations from Transformers) between the generated text and reference text. MoverScore is a metric that quantifies the dissimilarity between the distributions of word embeddings from machine-generated text and reference text, emphasizing semantic similarity over exact token overlap. We also introduced HBKI, a composite metric summarizing these approaches. We tested five models —GPT-3, Llama-2, Gemini 1.5 Pro, Solar 10.7B, and Mixtral-8x7b. Our software pipeline, designed and implemented using Object-Oriented Programming principles, allows users to customize the selection and extraction of features for topics of interest in their own research. Our results show that MoverScore delivered the most precise evaluation of the LLM's outputs, while Mixtral-8x7b achieved the best overall performance in extracting metadata from the databases.

97 MATHEMATICS AND COMPUTING↗