Search NASA⌕ Search

SEARCH · Search NASA

Results for “FAIR dataset”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 91 records · Page 5
All sourcesNASA NTRSDOE OSTI
DOE OSTI2026

Universal Workflow Language and Software Enable Geometric Learning and FAIR Scientific Protocol Reporting

Written language and conventional data structures for representing scientific procedures suffer from low process detail, often fail to accurately represent protocols, and lack universality. New strategies for the handling of experimental data are needed to provide viable process information for both humans and machines. In this work, we present the universal workflow language (UWL) and interface (UWLi). UWL is a findable, accessible, interoperable, and reusable (FAIR)-compatible, graph-based data architecture that can capture arbitrary scientific procedures through workflow representation, and UWLi is an accompanying software package for building, manipulating, and interpreting UWL entries. The UWL format was found to be highly effective in identifying deficiencies in the reported process details of high-impact, peer-reviewed scientific journals, and in simulated scenarios, the graph format was shown to be more effective than conventional methods in predictively modeling the outcome of diverse scientific protocols. Implementation of UWL could enable more accurate scientific communication and more impactful process datasets.

14 SOLAR ENERGY↗
DOE OSTI2023

Stratospheric temperature measurements from nanosatellite stellar occultation observations of refractive bending

Abstract. Stellar occultation observations from space can probe the stratosphere and mesosphere at a fine vertical scale around the globe. Unlike other measurement techniques like radiosondes and aircraft, stellar occultation has the potential to observe the atmosphere above 30 km, and unlike radio occultation, stellar occultation probes fine-scale phenomena with potential to observe atmospheric turbulence. We imaged the refractive bending angle of a star centroid for a series of occultations by the atmosphere. Atmospheric refractivity, density, and then temperature are retrieved from the bending observations with the Abel transformation and Edlén's law, the hydrostatic equation, and the ideal gas law. The retrieval technique is applied to data collected by two nanosatellites operated by Terran Orbital. Measurements were primarily taken by the GEOStare SV2 mission, with a dedicated imaging telescope, supplemented with images captured by spacecraft bus sensors, namely the star trackers on other Terran Orbital missions. The bending angle noise floor is 10 and 30 arcsec for the star tracker and GEOStare SV2 data, respectively. The most significant sources of uncertainty are due to centroiding errors due to the fairly low-resolution stellar images and telescope pointing knowledge derived from noisy satellite attitude sensors. The former mainly affects the star tracker data, while the latter limits the GEOStare SV2 accuracy, with both providing low vertical resolution. This translates to a temperature profile retrieval up to roughly 20 km for both star tracker and GEOStare SV2 datasets. In preparation of an upcoming 2023 mission designed to correct these deficiencies, SOHIP, we simulated bending angle measurements with varying magnitudes of error. The expected maximum altitude of retrieved temperature is 41 km on average for these simulated measurements with a noise floor of 0.39 arcsec. Our work highlights the capabilities of stellar occultation observations from nanosatellites for atmospheric sounding. Future work will investigate high-frequency observations of atmospheric gravity waves and turbulence, mitigating the major uncertainties observed in these datasets.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2024

Privacy-Preserving Federated Learning for Science: Challenges and Research Directions

This paper discusses the key challenges and future research directions for privacy-preserving federated learning (PPFL), with a focus on its application to large-scale scientific AI models, in particular, foundation models~(FMs). PPFL enables collaborative model training across distributed datasets while preserving privacy-- an important collaborative approach for science. We discuss the need for efficient and scalable algorithms to address the increasing complexity of FMs, particularly when dealing with heterogeneous clients. In addition, we underscore the need for developing advance privacy-preserving techniques, such as differential privacy, to balance privacy and utility in large FMs emphasizing fairness and incentive mechanisms to ensure equitable participation among heterogeneous clients. Finally, we emphasize the need for a robust software stack supporting scalable and secure PPFL deployments across multiple high-performance computing facilities. We envision that PPFL would play a crucial role to advance scientific discovery and enable large-scale, privacy-aware collaborations across science domains.

Kim, Kibaek [Argonne National Laboratory (ANL)]↗
DOE OSTI2021

Community-Driven Methods for Open and Reproducible Software Tools for Analyzing Datasets from Atom Probe Microscopy

Atom probe tomography, and related methods, probe the three-dimensional architecture of a material. The software tools that microscopists use, and how these tools are connected into workflows, makes a substantial contribution to the accuracy and precision of such a material characterization experiment. Typically, we adapt methods from other communities like mathematics, data science, computational geometry, artificial intelligence, or scientific computing. We also realize that improving on research data management is a challenge when it comes to align with the FAIR data stewardship principles. Faced with this global challenge, we are convinced that collaborating is useful. Here, we report the results and challenges with an inter-laboratory call for developing test cases for several types of atom probe software tools. The results support why defining detailed recipes of software workflows and sharing these recipes is necessary and rewarding: Open source tools and (meta)data exchange can help to make our day-to-day data processing tasks become more efficient, the training of new users and knowledge transfer become easier, and assist us with automated quantification of uncertainties to gain access to substantiated results.

36 MATERIALS SCIENCE↗
DOE OSTI2024

Invariant Features for Accurate Predictions of Quantum Chemical UV-vis Spectra of Organic Molecules

Including invariance of global properties of a phys-ical system as an intrinsic feature in graph neural networks (GNNs) enhances the model's robustness and generalizability and reduces the amount of training data required to obtain a desired accuracy for predictions of these properties. Existing open source GNN libraries construct invariant features only for specific GNN architectures. This precludes the generalization of invariant features to arbitrary message passing neural network (MPNN) layers which, in turn, precludes the use of these libraries for new, user-specified predictive tasks. To address this limitation, we implement invariant MPNNs into the flexible and scalable HydraGNN architecture. HydraGNN enables a seamless switch between various MPNNs in a unified layer sequence and allows for a fair comparison between the predictive performance of different MPNNs. We trained this enhanced HydraGNN archi-tecture on the ultraviolet-visible (UV-vis) spectrum of GDB-9 molecules, a feature that describes the molecule's electronic exci-tation modes, computed with time-dependent density functional tight binding (TD-DFTB) and available open source through the GDB-9-Ex dataset. We assess the robustness (i.e., accuracy and generalizability) of the predictions obtained using different invariant MPNNs with respect to different values of the full width at half maximum (FWHM) for the Gaussian smearing of the theoretical peaks. Our numerical results show that incorporating invariance in the HydraGNN architecture significantly enhances both accuracy and generalizability in predicting UV-vis spectra of organic molecules.

Baker, Justin↗
DOE OSTI2024

A Comparison between Invariant and Equivariant Classical and Quantum Graph Neural Networks

Machine learning algorithms are heavily relied on to understand the vast amounts of data from high-energy particle collisions at the CERN Large Hadron Collider (LHC). The data from such collision events can naturally be represented with graph structures. Therefore, deep geometric methods, such as graph neural networks (GNNs), have been leveraged for various data analysis tasks in high-energy physics. One typical task is jet tagging, where jets are viewed as point clouds with distinct features and edge connections between their constituent particles. The increasing size and complexity of the LHC particle datasets, as well as the computational models used for their analysis, have greatly motivated the development of alternative fast and efficient computational paradigms such as quantum computation. In addition, to enhance the validity and robustness of deep networks, we can leverage the fundamental symmetries present in the data through the use of invariant inputs and equivariant layers. In this paper, we provide a fair and comprehensive comparison of classical graph neural networks (GNNs) and equivariant graph neural networks (EGNNs) and their quantum counterparts: quantum graph neural networks (QGNNs) and equivariant quantum graph neural networks (EQGNN). The four architectures were benchmarked on a binary classification task to classify the parton-level particle initiating the jet. Based on their area under the curve (AUC) scores, the quantum networks were found to outperform the classical networks. However, seeing the computational advantage of quantum networks in practice may have to wait for the further development of quantum technology and its associated application programming interfaces (APIs).

Forestano, Roy T. (ORCID:0000000203552076)↗
DOE OSTI2022

Enabling FAIR data in Earth and environmental science with community-centric (meta)data reporting formats

Abstract Research can be more transparent and collaborative by using Findable, Accessible, Interoperable, and Reusable (FAIR) principles to publish Earth and environmental science data. Reporting formats—instructions, templates, and tools for consistently formatting data within a discipline—can help make data more accessible and reusable. However, the immense diversity of data types across Earth science disciplines makes development and adoption challenging. Here, we describe 11 community reporting formats for a diverse set of Earth science (meta)data including cross-domain metadata (dataset metadata, location metadata, sample metadata), file-formatting guidelines (file-level metadata, CSV files, terrestrial model data archiving), and domain-specific reporting formats for some biological, geochemical, and hydrological data (amplicon abundance tables, leaf-level gas exchange, soil respiration, water and sediment chemistry, sensor-based hydrologic measurements). More broadly, we provide guidelines that communities can use to create new (meta)data formats that integrate with their scientific workflows. Such reporting formats have the potential to accelerate scientific discovery and predictions by making it easier for data contributors to provide (meta)data that are more interoperable and reusable.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2022

Artificial Intelligence for Accelerating Nuclear Applications, Science, and Technology

Artificial intelligence (AI) and machine learning (ML) methods have had significant impacts in science and technology in recent years. These methods for generating models from datasets or logic-based algorithms that emulate aspects of human performance can similarly accelerate the fields of nuclear applications, science, and technology toward the IAEA goals of contributing to peace, health, and prosperity. In order to accomplish advances with AI in general and ML in particular across these fields, IAEA can play a significant role by establishing, hosting and curating centralised resources, including databases, adhering to FAIR (findable, accessible, interoperable and reusable) principles and Open Science best practices, providing stewardship of data sharing, supporting training efforts and development of relevant workforces, as well as enabling connections among the scientific, technology, mathematics, AI and ethics communities. Many areas can benefit from the use of AI in the realm of nuclear applications. In human health, these areas include clinical research, epidemiology, nutrition, medical imaging, radiotherapy and education of health professionals. AI-based tools are also being used to facilitate different clinical tasks in imaging, computer-assisted diagnosis in mammography and lung cancer screening programmes, and dose prediction in nuclear medicine procedures. ML methods in particular may also increase the efficiency and accuracy of the analysis of computerised tomography and dual-energy absorptiometry scans for body composition and bone analysis. The application of AI methods to nuclear and related technologies in food and agriculture can lead to significant advances and improved efficiency in the optimisation of agricultural production, food product development, management of supply chains, food safety and food authenticity control. In the water and environmental sector, AI can help inform policies to mitigate the world’s water problems. The application of AI techniques to hydrology and environmental sciences is expected to improve patterns identification and enable model predictions under a changing climate.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2020

Machine-learned impurity level prediction for semiconductors: the example of Cd-based chalcogenides

The ability to predict the likelihood of impurity incorporation and their electronic energy levels in semiconductors is crucial for controlling its conductivity, and thus the semiconductor's performance in solar cells, photodiodes, and optoelectronics. The difficulty and expense of experimental and computational determination of impurity levels makes a data-driven machine learning approach appropriate. In this work, we show that a density functional theory-generated dataset of impurities in Cd-based chalcogenides CdTe, CdSe, and CdS can lead to accurate and generalizable predictive models of defect properties. By converting any semiconductor + impurity system into a set of numerical descriptors, regression models are developed for the impurity formation enthalpy and charge transition levels. These regression models can subsequently predict impurity properties in mixed anion CdX compounds (where X is a combination of Te, Se and S) fairly accurately, proving that although trained only on the end points, they are applicable to intermediate compositions. We make machine-learned predictions of the Fermi-level-dependent formation energies of hundreds of possible impurities in 5 chalcogenide compounds, and we suggest a list of impurities which can shift the equilibrium Fermi level in the semiconductor as determined by the dominant intrinsic defects. Machine learning predictions for the dominating impurities compare well with DFT predictions, revealing the power of machine-learned models in the quick screening of impurities likely to affect the optoelectronic behavior of semiconductors.

36 MATERIALS SCIENCE↗
DOE OSTI2025

Application of PRIM for understanding patterns in carbon dioxide model-observation differences

Reducing uncertainties in regional carbon balances requires a better understanding of CO 2 transport in synoptic weather systems. Here, we apply the Patient Rule Induction Method (PRIM), a data-mining method to identify high-density regions for a target-class within an input parameter space, to airborne observations of potential temperature, wind speed, water vapor mixing ratio, and CO 2 dry mol fraction gathered during the Atmospheric Carbon and Transport (ACT)-America Summer 2016 and Winter 2017 campaigns. ACT observations were targeted at expert-designated cases of fair weather and near-frontal warm and cold sector air at atmospheric boundary-layer, lower-, and higher free tropospheric levels (ABL, LFT, and HFT, respectively). We investigate atmospheric characteristics of these pre-defined cases and associated CO 2 model-observation-differences in the mesoscale WRF-Chem model. PRIM results separate winter- and summertime observations as well as observations from ABL, LFT, and HFT with enrichment factors of 4.0–20.5 inside the PRIM box compared to the entire dataset but cannot distinguish between near-frontal warm and cold sector observations in the higher free troposphere. Analyzing of the parameter space constrained by PRIM, we find that large magnitude model observation differences preferentially associated with times when atmospheric conditions are less typical. This association suggests that PRIM could provide a useful tool for isolating atmospheric conditions with large-magnitude and non-Gaussian CO 2 -residuals for targeted transport model evaluation and to potentially improve inversion results during synoptically active periods.

Gerken, Tobias [James Madison Univ., Harrisonburg,↗
DOE OSTI2025

Overcoming sparse datasets with multi-task learning as applied to high entropy alloys

Abstract The design of novel High Entropy Alloys for use in high-temperature applications is an area of active interest due to their potential to provide exceptional properties compared to conventional alloys. Since the increased popularity of machine learning, an important cog in the design process has been training surrogate models on alloy properties. However, these Single-Task models are trained on individual mechanical properties and do not take advantage of the relatedness between properties. Multi-Task models can capture the interdependencies between tasks, leading to potentially more accurate predictions for all tasks. In this paper, we investigate if Multi-Task models can show improvement over Single-Task models when used for predicting the mechanical properties of these alloys. To ensure fair evaluation between the models, we apply L 0 regularization and skip connections to the models, which allows them to adjust the number of model parameters and depth for optimal performance. We find that the Multi-Task models can leverage task relationships to perform better than Single-Task models, especially for high amounts of missing data in the tasks. Furthermore, adding simple auxiliary targets can boost Multi-Task performance even further despite not being effective as input descriptors to single-task models themselves. We anticipate that the proposed strategies can achieve more accurate predictions and consequently enable better design capabilities for such data-constrained domains without incurring much additional computational cost.

Debnath, Arindam (ORCID:0000000194274499)↗
DOE OSTI2024

OrthoPhyl—streamlining large-scale, orthology-based phylogenomic studies of bacteria at broad evolutionary scales

Abstract There are a staggering number of publicly available bacterial genome sequences (at writing, 2.0 million assemblies in NCBI's GenBank alone), and the deposition rate continues to increase. This wealth of data begs for phylogenetic analyses to place these sequences within an evolutionary context. A phylogenetic placement not only aids in taxonomic classification but informs the evolution of novel phenotypes, targets of selection, and horizontal gene transfer. Building trees from multi-gene codon alignments is a laborious task that requires bioinformatic expertise, rigorous curation of orthologs, and heavy computation. Compounding the problem is the lack of tools that can streamline these processes for building trees from large-scale genomic data. Here we present OrthoPhyl, which takes bacterial genome assemblies and reconstructs trees from whole genome codon alignments. The analysis pipeline can analyze an arbitrarily large number of input genomes (>1200 tested here) by identifying a diversity-spanning subset of assemblies and using these genomes to build gene models to infer orthologs in the full dataset. To illustrate the versatility of OrthoPhyl, we show three use cases: E. coli/Shigella, Brucella/Ochrobactrum and the order Rickettsiales. We compare trees generated with OrthoPhyl to trees generated with kSNP3 and GToTree along with published trees using alternative methods. We show that OrthoPhyl trees are consistent with other methods while incorporating more data, allowing for greater numbers of input genomes, and more flexibility of analysis.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2021

The Atmospheric Carbon and Transport (ACT)-America Mission

The Atmospheric Carbon and Transport (ACT)-America NASA Earth Venture Suborbital Mission set out to improve regional atmospheric greenhouse gas (GHG) inversions by exploring the intersection of the strong GHG fluxes and vigorous atmospheric transport that occurs within the midlatitudes. In this study, two research aircraft instrumented with remote and in situ sensors to measure GHG mole fractions, associated trace gases, and atmospheric state variables collected 1,140.7 flight hours of research data, distributed across 305 individual aircraft sorties, coordinated within 121 research flight days, and spanning five 6-week seasonal flight campaigns in the central and eastern United States. Flights sampled 31 synoptic sequences, including fair-weather and frontal conditions, at altitudes ranging from the atmospheric boundary layer to the upper free troposphere. The observations were complemented with global and regional GHG flux and transport model ensembles. We found that midlatitude weather systems contain large spatial gradients in GHG mole fractions, in patterns that were consistent as a function of season and altitude. We attribute these patterns to a combination of regional terrestrial fluxes and inflow from the continental boundaries. These observations, when segregated according to altitude and air mass, provide a variety of quantitative insights into the realism of regional CO 2 and CH 4 fluxes and atmospheric GHG transport realizations. The ACT-America dataset and ensemble modeling methods provide benchmarks for the development of atmospheric inversion systems. As global and regional atmospheric inversions incorporate ACT-America’s findings and methods, we anticipate these systems will produce increasingly accurate and precise subcontinental GHG flux estimates.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2024

BindingDB in 2024: a FAIR knowledgebase of protein-small molecule binding data

Abstract BindingDB (bindingdb.org) is a public, web-accessible database of experimentally measured binding affinities between small molecules and proteins, which supports diverse applications including medicinal chemistry, biochemical pathway annotation, training of artificial intelligence models and computational chemistry methods development. This update reports significant growth and enhancements since our last review in 2016. Of note, the database now contains 2.9 million binding measurements spanning 1.3 million compounds and thousands of protein targets. This growth is largely attributable to our unique focus on curating data from US patents, which has yielded a substantial influx of novel binding data. Recent improvements include a remake of the website following responsive web design principles, enhanced search and filtering capabilities, new data download options and webservices and establishment of a long-term data archive replicated across dispersed sites. We also discuss BindingDB’s positioning relative to related resources, its open data sharing policies, insights gleaned from the dataset and plans for future growth and development.

Liu, Tiqing↗
DOE OSTI2020

Distribution Development for Residual Inventory at the New York West Valley Site - 20400

The New York State Energy Research and Development Authority (NYSERDA) is the owner of the Western New York Nuclear Service Center (WNYNSC), a 1,351 ha site located approximately 48 km south of Buffalo, New York. In 1962, Nuclear Fuel Services, Inc. (NFS) entered into Agreements with the Atomic Energy Commission and New York State to construct the first commercial reprocessing plant of nuclear fuel in the United States. NFS, a private company, built and operated the spent fuel reprocessing plant and waste disposal facilities, processing 640 Mg of spent nuclear fuel from 1966 to 1972 under an Atomic Energy Commission license. Nuclear fuel reprocessing operations ended in 1972 and never reopened, leaving behind radioactive and chemical wastes. Operations led to contamination in a number of facilities and locations. Some of that contamination has migrated from waste disposal zones to other layers, formations, and features on and off the WNYNSC. Phase I decommissioning activities are ongoing and involve the removal of a number of areas and structures that have been associated with contamination. The purpose of this work is to outline the approach for characterizing contamination not associated with disposed wastes, contaminated structures, or specific releases. In this work, the term, residual radiological activity, is used to describe environmental contamination that exists subsequent to the completion of Phase I decommissioning activities, that is not associated with disposed wastes, contaminated structures, or specific releases. Contamination from the Site was quantified relative to data that characterize the concentrations of radionuclides that exist in background. Background concentrations are those present in the area but having no influence from Site related activities. The existence of residual radiological activity that is elevated relative to background has the potential to contribute to future risks to human health and the environment. As a consequence, the residual inventory information is used to inform the West Valley Probabilistic Performance Assessment (PPA) model to characterize potential future risks to human health and the environment. The centralized West Valley Data Management System (DMS) was the source of information for the data assembled in this analysis. The DMS is a fairly large compilation consisting of thousands of records from investigation studies, with sample dates ranging from 1990 to present. Samples from monitoring wells, boreholes, geoprobe studies, surface water, surface soils, storm water outfalls, ventilation stack filters, plant and animal tissues, and more are included in the DMS. Results are typically reported in units of activity per unit volume. For the purpose of the analyses presented here, all results were converted into consistent units of pCi per unit volume. Since 1990, data have been collected from various locations across the WNYNSC at different times with varying frequency over the course of several decades. As a consequence, a number of potential issues can arise with respect to the assembly of a dataset that is deemed adequate for the characterization of residual radiological activity. These issues were assessed and resolved to the extent possible through careful consideration of the properties of the distributions. The intent was to use data which characterize the current state of the Site. Radionuclides can be designated to one of several groups depending on their origin. In this work the groups considered were 1) Naturally Occurring Radioactive Material (NORM), 2) fallout, and 3) Other (including power plant, medical research, etc). This grouping is a useful construct with respect to the interpretation of fixed laboratory results. For example, NORM radionuclides that exist within a decay chain should have approximately equivalent distributions of concentrations if they are representative of background conditions. Insights such as these can be used as a check to identify sample results that need to be further investigated or omitted due to issues associated with reported results from fixed laboratory analyses. This type of analysis provided a foundation for the assessment of the adequacy of sample results for use in subsequent components of an assessment. The general process for the assessment of residual radiological contamination at the Site consists of a sequence of several steps. First, for each analyte, several statistical tests were performed to assess the weight of evidence against the null hypothesis that the mean of the distribution of concentrations was equal to zero. If the mean of the distribution of concentrations for a given radionuclide was not found to be greater than zero, then it was removed from consideration as a component of the residual radiological contamination. If there was significant evidence to reject the hypothesis of the mean being equal to zero, the second step was to compare the distribution of the data from the Site to that of the corresponding background. A suite of tests was used to compare the distributions of the site and background data. The results of these tests were collectively used to determine if site data are elevated relative to background. The third step was to develop distributions using a Bayesian framework to characterize the distribution of mean of the increment present above background for each of the radionuclides. The Bayesian model implemented allowed for the comparison of site-specific records to background concentrations to better approximate contamination attributed to the Site. A final screening step was employed for radionuclides that exceed background. This screening step compared 95% upper confidence limits (UCLs) from the increment distribution developed in the previous step to the risk screening levels. This approach yields a list of analytes that were determined to be elevated relative to background.

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗
DOE OSTI2022

ESS-DIVE Reporting Format for Dataset Package Metadata

ESS-DIVE’s (Environmental Systems Science Data Infrastructure for a Virtual Ecosystem) dataset metadata reporting format is intended to compile information about a dataset (e.g., title, description, funding sources) that can enable reuse of data submitted to the ESS-DIVE data repository. The files contained in this dataset include instructions (dataset_metadata_guide.md and README.md) that can be used to understand the types of metadata ESS-DIVE collects. The data dictionary (dd.csv) follows ESS-DIVE’s file-level metadata reporting format and includes brief descriptions about each element of the dataset metadata reporting format. This dataset also includes a terminology crosswalk (dataset_metadata_crosswalk.csv) that shows how ESS-DIVE’s metadata reporting format maps onto other existing metadata standards and reporting formats.Data contributors to ESS-DIVE can provide this metadata by manual entry using a web form or programmatically via ESS-DIVE’s API (Application Programming Interface). A metadata template (dataset_metadata_template.docx or dataset_metadata_template.pdf) can be used to collaboratively compile metadata before providing it to ESS-DIVE.Since being incorporated into ESS-DIVE’s data submission user interface, ESS-DIVE’s dataset metadata reporting format, has enabled features like automated metadata quality checks, and dissemination of ESS-DIVE datasets onto other data platforms including Google Dataset Search and DataCite.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2020

waveSZ: A Hardware-Algorithm Co-Design of Efficient Lossy Compression for Scientific Data

Error-bounded lossy compression is critical to the success of extreme-scale scientific research because of ever-increasing volumes of data produced by today's high-performance computing (HPC) applications. Not only can error-controlled lossy compressors significantly reduce the I/O and storage burden but they can retain high data fidelity for post analysis. Existing state-of-the-art lossy compressors, however, generally suffer from relatively low compression and decompression throughput (up to hundreds of megabytes per second on a single CPU core), which considerably restrict the adoption of lossy compression by many HPC applications especially those with a fairly high data production rate. In this paper, we propose a highly efficient lossy compression approach based on field programmable gate arrays (FPGAs) under the state-of-the-art lossy compression model SZ. Our contributions are fourfold. (1) We adopt a wavefront memory layout to alleviate the data dependency during the prediction for higher-dimensional predictors, such as the Lorenzo predictor. (2) We propose a co-design framework named WAVESZ based on the wavefront memory layout and the characteristics of SZ algorithm and carefully implement it by using high-level synthesis. (3) We propose a hardware-algorithm co-optimization method to improve the performance. (4) We evaluate our proposed WAVESZ on three real-world HPC simulation datasets from the Scientific Data Reduction Benchmarks and compare it with other state-of-the-art methods on both CPUs and FPGAs. Experiments show that our WAVESZ can improve SZ's compression throughput by 6.9x similar to 8.7x over the production version running on a state-of-the-art CPU and improve the compression ratio and throughput by 2.1x and 5.8x on average, respectively, respectively, compared with the state-of-the-art FPGA design.

Compression Ratio↗
DOE OSTI2026

VirJenDB: a FAIR (meta)data and bioinformatics platform for all viruses

High-throughput sequencing has generated an unprecedented volume of data. However, researcher-submitted data in repositories requires extensive curation and quality control for reuse. These tasks are hindered by the multiplicity of repositories, the sheer volume of the data, and the complexity of virus (meta)data curation. To address these challenges, VirJenDB offers a user-friendly platform to facilitate versioned, community-driven curation, and ontology development. Virus sequences were ingested from 16 sources, including ~200 fields of metadata or standards, covering taxonomy, sample, and host information. Up to 85 metadata fields have undergone at least one round of curation, and are linked to 15.4 million virus sequences, with 88 % from those infecting eukaryotes and the remaining infecting prokaryotes. Subsets were created, including a novel collection of 0.91 million viral operational taxonomic unit (vOTU) sequences across all viruses, while keeping the original sequences from each vOTU to facilitate downstream analyses, e.g. sequence variation. The VirJenDB web portal (https://www.virjendb.org) provides HTTPS and Application Programming Interface (API) access to the sequence datasets and metadata, offering a search engine, filtering, download, visualizations, and documentation. VirJenDB aims to connect the phage and eukaryotic virus research communities by supporting webtool integration, meta-analyses, and metadata schema extensions.

Saghaei, Shahram↗