SEARCH · Search NASA

Results for “Ranking methods”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 541 records · Page 30

All sources NASA NTRS DOE OSTI

DOE OSTI2024

Patch2Self2: Self-supervised Denoising on Coresets via Matrix Sketching

Diffusion MRI (dMRI) non-invasively maps brain white matter yet necessitates denoising due to low signal-to-noise ratios. Patch2Self (P2S) employing self-supervised techniques and regression on a Casorati matrix effectively denoises dMRI images and has become the new de-facto standard in this field. P2S however is resource intensive both in terms of running time and memory usage as it uses all voxels (n) from all-but-one held-in volumes (d-1) to learn a linear mapping Phi : \mathbb R ^ n x(d-1) \mapsto \mathbb R ^ n for denoising the held-out volume. The increasing size and dimensionality of higher resolution dMRI acquisitions can make P2S infeasible for large-scale analyses. This work exploits the redundancy imposed by P2S to alleviate its performance issues and inspect regions that influence the noise disproportionately. Specifically this study makes a three-fold contribution: (1) We present Patch2Self2 (P2S2) a method that uses matrix sketching to perform self-supervised denoising. By solving a sub-problem on a smaller sub-space so called coreset we show how P2S2 can yield a significant speedup in training time while using less memory. (2) We present a theoretical analysis of P2S2 focusing on determining the optimal sketch size through rank estimation a key step in achieving a balance between denoising accuracy and computational efficiency. (3) We show how the so-called statistical leverage scores can be used to interpret the denoising of dMRI data a process that was traditionally treated as a black-box. Experimental results on both simulated and real data affirm that P2S2 maintains denoising quality while significantly enhancing speed and memory efficiency achieved by training on a reduced data subset.

Fadnavis, Shreyas↗

DOE OSTI2024

Circular Economy for Photovoltaics in Service of Energy Transition

The challenge of energy transition is immediate and immense; current projections target 75 TW of photovoltaics (PV) capacity by 2050. While any transition to renewable energy technology is preferable to the current fossil-based system, it is ideal to improve the sustainability of PV to minimize negative environmental and social impacts. Circular economy (CE) has been proposed as a method to improve the sustainability of PV, especially for emerging materials like perovskites. CE is a set of actions, principles, and systems which aim to design out waste and keep products and materials in use, to reduce environmental impacts and enable sustainable development. At the most basic level, CE is "reduce, reuse, recycle", the R-actions, in ranked order. CE of a PV technology can be metricized in a variety of ways, such as the Material Circularity Indicator (Smith and Jones, Ellen MacArthur Foundation, 2019) or recycling rates. Unfortunately, standard CE metrics have several shortcomings for measuring renewable energy technologies in the context of deployment for energy transition (Figge 2018, Saidani 2019): 1) Only measure mass flows; 2) De-prioritization of the use phase in favor of mass circularity when scoring; and 3) Tight focus on a single product scale The use phase and energy flows of PV are key to energy transition, and therefore need to be quantified. Additionally, correlating product-scale to system-scale is necessary for quantifying the environmental impacts of energy transition. Life Cycle Assessment (LCA) can address some of these concerns, but also focuses on a single product scale and has trouble capturing the dynamics of system-scale energy transition, such as the interaction of module lifetime with manufacturing demands for energy transition deployment schedules. Therefore, we developed an open-source Python-based system dynamics model to quantify the mass, energy and carbon impacts of CE R-actions for PV technologies in the energy transition; PV in the CE (PV ICE) (Ovaitt & Mirletz 2021). The tool captures supply chains from material extraction through end of life, incorporating 5 circular end of life pathways. PV ICE takes in any evolving bill of materials, module properties and deployment schedule to support researchers and decision makers with data-backed insights. In this work, we quantify and compare proposed CE sustainable PV module designs and lifecycle management strategies, spanning currently commercialized technologies, government and industry technology targets, and several low Technology Readiness Level (TRL) emerging PV technologies, including perovskites. Our analyses capture the projected evolutions of lifetime, efficiency and material circularity of these PV technologies, as well as their material supply chains. Our analyses emphasize the importance of examining a suite of metrics to identify priorities and tradeoffs, and inform design or lifecycle management decisions holistically. Previous analyses have demonstrated the central importance of PV module lifetime to support energy transition while minimizing impacts. High levels of material circularity (>90%) enable minimizing lifecycle wastes, can reduce virgin material demands if paired with improving efficiency, but demonstrate tradeoffs in energy return on investment. In the fervor of new material and technology development, it is important to remember that CE is not the end goal; decarbonization and energy transition are the end goal. CE should be used in service to improve the sustainability of PV, and R-actions evaluated for their usefulness and efficacy to this end.

carbon↗

NASA NTRS1979

Fine-structure lines and elemental abundances in the Orion Nebula

Infrared observations of the forbidden fine-structure lines of Ne II at 12.81 microns, S IV at 10.51 microns, and Ar III at 8.99 microns in the Orion Nebula are presented. These measurements are combined with optical observations of forbidden Ne III, S II, S III, O II, and O III to derive total elemental abundances of neon, sulfur, and oxygen, Ne/H = 7.91, S/H = 7.34, and O/H = 8.60, which are very similar to solar abundances. While the commonly employed ionization correction formula scheme is found to work satisfactorily for neon, the method overcompensates for the abundance of S(3+) and leads to a higher sulfur abundance than is actually present, though the net discrepancy in the elemental abundance of sulfur in Orion is small.

Lester, D. F.↗

DOE OSTI2022

Neural-network learning of SPOD latent dynamics

Here, we aim to reconstruct the latent space dynamics of high dimensional, quasi-stationary systems using model order reduction via the spectral proper orthogonal decomposition (SPOD). The proposed method is based on three fundamental steps: in the first, once that the mean flow field has been subtracted from the realizations (also referred to as snapshots), we compress the data from a high-dimensional representation to a lower dimensional one by constructing the SPOD latent space; in the second, we build the time-dependent coefficients by projecting the snapshots containing the fluctuations onto the SPOD basis and we learn their evolution in time with the aid of recurrent neural networks; in the third, we reconstruct the high-dimensional data from the learnt lower -dimensional representation. The proposed method is demonstrated on two different test cases, namely, a compressible jet flow, and a geophysical problem known as the Madden-Julian Oscillation. An extensive comparison between SPOD and the equivalent POD-based counterpart is provided and differences between the two approaches are highlighted. The numerical results suggest that the proposed model is able to provide low rank predictions of complex statistically stationary data and to provide insights into the evolution of phenomena characterized by specific range of frequencies. The comparison between POD and SPOD surrogate strategies highlights the need for further work on the characterization of the interplay of error between data reduction techniques and neural network forecasts.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2026

Steric Mapping, Ligand Dynamics, and Cycloisomerization Catalysis with Redox Robust Mn I/0/-I Dicarbenes

Manganese is perhaps the most electronically versatile element, yet the redox properties, reactivity, and catalytic applications of low-valent Mn 0 /Mn −I complexes remain underexplored due to the propensity for Mn 0 to dimerize, quenching highenergy metalloradicals. We report a series of redox-active monometallic Mn I , Mn 0 and Mn −I complexes containing a BH 2 - bridged dicarbene, characterized using a suite of experimental and cutting-edge computational (DFT) methods. Slow electron transfer kinetics at Mn I/0 are observed, with computations and electrochemical simulations in excellent agreement with experimental values. Despite the lack of steric bulk at the BH 2 -bridged Mn 0 , the t Bu groups at the dicarbene provide adequate steric protection to prevent dimerization, with percent buried volume (% V bur ) serving as a valuable steric ranking tool. We also show that a %V bur > 83% prevents dimerization for a diverse array of Mn 0 complexes from the literature. Ligand sterics of BPh 2 -and BH 2 -bridged complexes dictate reaction outcomes when Mn I and Mn −I are exposed to nucleophiles and electrophiles, respectively, while Mn0 facilitates the radical cycloisomerization catalysis of 6-iodo-1- hexene at room temperature. Furthermore, this work underscores the importance of ligand sterics in rationalizing reactivity patterns at Mn and provides valuable insights for designing chelating ligands that can selectively leverage Mn I/0/‑I states in redox-mediated catalytic reactions.

Ligands↗

NASA NTRS2007

NASA Tech Briefs, June 2007

Topics covered include: High-Accuracy, High-Dynamic-Range Phase-Measurement System; Simple, Compact, Safe Impact Tester; Multi-Antenna Radar Systems for Doppler Rain Measurements; 600-GHz Electronically Tunable Vector Measurement System; Modular Architecture for the Measurement of Space Radiation; VLSI Design of a Turbo Decoder; Architecture of an Autonomous Radio Receiver; Improved On-Chip Measurement of Delay in an FPGA or ASIC; Resource Selection and Ranking; Accident/Mishap Investigation System; Simplified Identification of mRNA or DNA in Whole Cells; Printed Multi-Turn Loop Antennas for RF Biotelemetry; Making Ternary Quantum Dots From Single-Source Precursors; Improved Single-Source Precursors for Solar-Cell Absorbers; Spray CVD for Making Solar-Cell Absorber Layers; Glass/BNNT Composite for Sealing Solid Oxide Fuel Cells; A Method of Assembling Compact Coherent Fiber-Optic Bundles; Manufacturing Diamond Under Very High Pressure; Ring-Resonator/Sol-Gel Interferometric Immunosensor; Compact Fuel-Cell System Would Consume Neat Methanol; Algorithm Would Enable Robots to Solve Problems Creatively; Hypothetical Scenario Generator for Fault-Tolerant Diagnosis; Smart Data Node in the Sky; Pseudo-Waypoint Guidance for Proximity Spacecraft Maneuvers; Update on Controlling Herds of Cooperative Robots; and Simulation and Testing of Maneuvering of a Planetary Rover.

Source record↗

NASA NTRS2011

Estimated Accuracy of Three Common Trajectory Statistical Methods

Three well-known trajectory statistical methods (TSMs), namely concentration field (CF), concentration weighted trajectory (CWT), and potential source contribution function (PSCF) methods were tested using known sources and artificially generated data sets to determine the ability of TSMs to reproduce spatial distribution of the sources. In the works by other authors, the accuracy of the trajectory statistical methods was estimated for particular species and at specified receptor locations. We have obtained a more general statistical estimation of the accuracy of source reconstruction and have found optimum conditions to reconstruct source distributions of atmospheric trace substances. Only virtual pollutants of the primary type were considered. In real world experiments, TSMs are intended for application to a priori unknown sources. Therefore, the accuracy of TSMs has to be tested with all possible spatial distributions of sources. An ensemble of geographical distributions of virtual sources was generated. Spearman s rank order correlation coefficient between spatial distributions of the known virtual and the reconstructed sources was taken to be a quantitative measure of the accuracy. Statistical estimates of the mean correlation coefficient and a range of the most probable values of correlation coefficients were obtained. All the TSMs that were considered here showed similar close results. The maximum of the ratio of the mean correlation to the width of the correlation interval containing the most probable correlation values determines the optimum conditions for reconstruction. An optimal geographical domain roughly coincides with the area supplying most of the substance to the receptor. The optimal domain s size is dependent on the substance decay time. Under optimum reconstruction conditions, the mean correlation coefficients can reach 0.70 0.75. The boundaries of the interval with the most probable correlation values are 0.6 0.9 for the decay time of 240 h and 0.5 0.95 for the decay time of 12 h. The best results of source reconstruction can be expected for the trace substances with a decay time on the order of several days. Although the methods considered in this paper do not guarantee high accuracy they are computationally simple and fast. Using the TSMs in optimum conditions and taking into account the range of uncertainties, one can obtain a first hint on potential source areas.

Kabashnikov, Vitaliy P.↗

DOE OSTI2023

Architector for high-throughput cross-periodic table 3D complex building

Abstract Rare-earth and actinide complexes are critical for a wealth of clean-energy applications. Three-dimensional (3D) structural generation and prediction for these organometallic systems remains a challenge, limiting opportunities for computational chemical discovery. Here, we introduce Architector , a high-throughput in-silico synthesis code for s-, p-, d-, and f-block mononuclear organometallic complexes capable of capturing nearly the full diversity of the known experimental chemical space. Beyond known chemical space, Architector performs in-silico design of new complexes including any chemically accessible metal-ligand combinations. Architector leverages metal-center symmetry, interatomic force fields, and tight binding methods to build many possible 3D conformers from minimal 2D inputs including metal oxidation and spin state. Over a set of more than 6,000 x-ray diffraction (XRD)-determined complexes spanning the periodic table, we demonstrate quantitative agreement between Architector-predicted and experimentally observed structures. Further, we demonstrate out-of-the box conformer generation and energetic rankings of non-minimum energy conformers produced from Architector , which are critical for exploring potential energy surfaces and training force fields. Overall, Architector represents a transformative step towards cross-periodic table computational design of metal complex chemistry.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

NASA NTRS2009

Active Learning with Irrelevant Examples

An improved active learning method has been devised for training data classifiers. One example of a data classifier is the algorithm used by the United States Postal Service since the 1960s to recognize scans of handwritten digits for processing zip codes. Active learning algorithms enable rapid training with minimal investment of time on the part of human experts to provide training examples consisting of correctly classified (labeled) input data. They function by identifying which examples would be most profitable for a human expert to label. The goal is to maximize classifier accuracy while minimizing the number of examples the expert must label. Although there are several well-established methods for active learning, they may not operate well when irrelevant examples are present in the data set. That is, they may select an item for labeling that the expert simply cannot assign to any of the valid classes. In the context of classifying handwritten digits, the irrelevant items may include stray marks, smudges, and mis-scans. Querying the expert about these items results in wasted time or erroneous labels, if the expert is forced to assign the item to one of the valid classes. In contrast, the new algorithm provides a specific mechanism for avoiding querying the irrelevant items. This algorithm has two components: an active learner (which could be a conventional active learning algorithm) and a relevance classifier. The combination of these components yields a method, denoted Relevance Bias, that enables the active learner to avoid querying irrelevant data so as to increase its learning rate and efficiency when irrelevant items are present. The algorithm collects irrelevant data in a set of rejected examples, then trains the relevance classifier to distinguish between labeled (relevant) training examples and the rejected ones. The active learner combines its ranking of the items with the probability that they are relevant to yield a final decision about which item to present to the expert for labeling. Experiments on several data sets have demonstrated that the Relevance Bias approach significantly decreases the number of irrelevant items queried and also accelerates learning speed.

Wagstaff, Kiri↗

DOE OSTI2026

SEED: Semantic Energy Exploration and Discovery

The Bioenergy Knowledge Discovery Framework (KDF) hosts a vast repository of specialized data, yet traditional keyword-based search methods often struggle to provide direct answers, requiring significant domain expertise and manual effort to filter through raw documents. To overcome these barriers, this software introduces a semantic search engine that enables both specialists and non-specialists to query the KDF using natural language. By shifting from rigid keyword matching to intent-based retrieval, the tool automatically identifies and ranks the most relevant sources within the database. The system functions by processing natural language queries to extract the most pertinent information, delivering an AI-generated plain-language summary alongside exact supporting quotes from retrieved documents. This integrated approach provides users with immediate, evidence-based answers while eliminating the need for exhaustive manual review. By surfacing direct insights and contextual evidence, the software enhances the usability of existing KDF resources and democratizes access to complex bioenergy data. Ultimately, this semantic search solution accelerates the discovery process and supports faster, more informed decision-making across the bioenergy sector.

Pan, Meiyu (Melrose) [Oak Ridge National Laborator↗

NASA NTRS2008

Rapid Cost Assessment of Space Mission Concepts Through Application of Complexity-Based Cost Indices

This slide presentation reviews the development of a rapid cost assessment models for evaluation of exploration missions through the application of complexity based cost indices. In Fall of 2004, NASA began developing 13 documents, known as "strategic roadmaps," intended to outline a strategy for space exploration over the next 30 years. The Third Strategic Roadmap, The Strategic Roadmap for Solar System Exploration, focused on strategy for robotic exploration of the Solar System. Development of the Strategic Roadmap for Solar System Exploration led to the investigation of a large variety of missions. However, the necessity of planning around scientific inquiry and budgetary constraints made it necessary for the roadmap development team to evaluate potential missions not only for scientific return but also cost. Performing detailed cost studies for each of the large number of missions was impractical given the time constraints involved and lack of detailed mission studies; so a method of rapid cost assessment was developed by us to allow preliminary analysis. It has been noted that there is a strong correlation between complexity and cost and schedule of planetary missions. While these correlations were made after missions had been built and flown (successfully or otherwise), it seemed likely that a similar approach could provide at least some relative cost ranking. Cost estimation relationships (CERs) have been developed based on subsystem design choices. These CERs required more detailed information than available, forcing the team to adopt a more high level approach. Costing by analogy has been developed for small satellites, however, planetary exploration missions provide such varying spacecraft requirements that there is a lack of adequately comparable missions that can be used for analogy.

cost estimation↗

NASA NTRS2013

The Impact of Model and Rainfall Forcing Errors on Characterizing Soil Moisture Uncertainty in Land Surface Modeling

The contribution of rainfall forcing errors relative to model (structural and parameter) uncertainty in the prediction of soil moisture is investigated by integrating the NASA Catchment Land Surface Model (CLSM), forced with hydro-meteorological data, in the Oklahoma region. Rainfall-forcing uncertainty is introduced using a stochastic error model that generates ensemble rainfall fields from satellite rainfall products. The ensemble satellite rain fields are propagated through CLSM to produce soil moisture ensembles. Errors in CLSM are modeled with two different approaches: either by perturbing model parameters (representing model parameter uncertainty) or by adding randomly generated noise (representing model structure and parameter uncertainty) to the model prognostic variables. Our findings highlight that the method currently used in the NASA GEOS-5 Land Data Assimilation System to perturb CLSM variables poorly describes the uncertainty in the predicted soil moisture, even when combined with rainfall model perturbations. On the other hand, by adding model parameter perturbations to rainfall forcing perturbations, a better characterization of uncertainty in soil moisture simulations is observed. Specifically, an analysis of the rank histograms shows that the most consistent ensemble of soil moisture is obtained by combining rainfall and model parameter perturbations. When rainfall forcing and model prognostic perturbations are added, the rank histogram shows a U-shape at the domain average scale, which corresponds to a lack of variability in the forecast ensemble. The more accurate estimation of the soil moisture prediction uncertainty obtained by combining rainfall and parameter perturbations is encouraging for the application of this approach in ensemble data assimilation systems.

Maggioni, V.↗

NASA NTRS1989

Methods for trend analysis: Examples with problem/failure data

Statistics are emphasized as an important role in quality control and reliability. Consequently, Trend Analysis Techniques recommended a variety of statistical methodologies that could be applied to time series data. The major goal of the working handbook, using data from the MSFC Problem Assessment System, is to illustrate some of the techniques in the NASA standard, some different techniques, and to notice patterns of data. Techniques for trend estimation used are: regression (exponential, power, reciprocal, straight line) and Kendall's rank correlation coefficient. The important details of a statistical strategy for estimating a trend component are covered in the examples. However, careful analysis and interpretation is necessary because of small samples and frequent zero problem reports in a given time period. Further investigations to deal with these issues are being conducted.

Church, Curtis K.↗

DOE OSTI2020

Density-Matrix Based Extended Lagrangian Born–Oppenheimer Molecular Dynamics

Extended Lagrangian Born–Oppenheimer molecular dynamics [ Phys. Rev. Lett. 2008, 100, 123004] is presented for Hartree–Fock theory, where the extended electronic degrees of freedom are represented by a density matrix, including fractional occupation numbers at elevated electronic temperatures. In contrast to regular direct Born–Oppenheimer molecular dynamics simulations, no iterative self-consistent field optimization is required prior to the force evaluations. To sample regions of the potential energy landscape where the gap is small or vanishing, which leads to particular convergence problems in regular direct Born–Oppenheimer molecular dynamics simulations, an adaptive integration scheme for the extended electronic degrees of freedom is presented. The integration scheme is based on a tunable, low-rank approximation of a fourth-order kernel, K, that determines the metric tensor, T ≡ K T K, used in the extended harmonic oscillator of the Lagrangian that generates the dynamics of the electronic degrees of freedom. Here, the formulation and algorithms provide a general guide to implement extended Lagrangian Born–Oppenheimer molecular dynamics for quantum chemistry, density functional theory, and semiempirical methods using a density matrix formalism.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

DOE OSTI2025

Rare species do not disproportionately contribute to phylogenetic diversity in a subalpine plant community

Abstract Premise Within plant communities, few species are abundant, and most are locally rare. Worldwide, 36% of plant species are exceedingly rare and often face high extinction risk. However, the community phylogenetic impact of the loss of rare plants is largely unknown in many systems. We address this gap by investigating how rare species contribute to phylogenetic diversity, considering multiple metrics of rarity and multiple elevations in a subalpine plant community. Methods We collected abundance data at three sites near the Rocky Mountain Biological Laboratory (Colorado, USA). We calculated each species' range size from public occurrence data. We calculated phylogenetic signal for abundance and range size, compared community phylogenetic metrics weighted by range size and abundance to unweighted metrics, and quantified the change in phylogenetic diversity when removing single species and groups of species ranked by rarity. Results We found phylogenetic signal for abundance, but not range size. There was no difference between rarity‐weighted and ‐unweighted phylogenetic diversity metrics. Finally, phylogenetic diversity did not decline more when we removed single rare species or groups of rare species than when we removed single common species and groups of common species. Conclusions We found that rare species, whether at low abundance or with a small range, do not disproportionately contribute to phylogenetic diversity in our subalpine plant community. These results were consistent across elevations. Instead, rare species might provide phylogenetic redundancy with common species. Deeper understanding of functional differentiation is needed to understand contributions of rare species to this system.

Plant Sciences↗

NASA NTRS2018

Item Retrieval as Utility Estimation

Retrieval systems have greatly improved over the last half century, estimating relevance to a latent user need in a wide variety of areas. One area that has not enjoyed such advancements is searching for items by attribute values, a common activity in e-commerce and science, particularly given numeric values. Existing item retrieval systems assume the user has a firm grasp of their own desires and can formulate a good Boolean or SQL-style query to retrieve items, as one would do with a database. A contrasting approach would be to estimate how well items match the user?s latent desires and return items ranked by this estimation. Towards this end, we present a retrieval model inspired by multi-criteria decision making theory, concentrating on numeric attributes. We evaluate our novel approach, the de-facto standard of Boolean retrieval, and several models proposed in the literature, in two user studies using Amazon Mechanical Turk. We use a competitive game to motivate test subjects and compare methods based on the results of the subjects? initial query and their success in the game. In our experiments, our new method signi cantly outperformed the others, whereas the Boolean approaches had the worst performance.

operations research↗

DOE OSTI2023

3D-equivariant graph neural networks for protein model quality assessment

Quality assessment (QA) of predicted protein tertiary structure models plays an important role in ranking and using them. With the recent development of deep learning end-to-end protein structure prediction techniques for generating highly confident tertiary structures for most proteins, it is important to explore corresponding QA strategies to evaluate and select the structural models predicted by them since these models have better quality and different properties than the models predicted by traditional tertiary structure prediction methods. We develop EnQA, a novel graph-based 3D-equivariant neural network method that is equivariant to rotation and translation of 3D objects to estimate the accuracy of protein structural models by leveraging the structural features acquired from the state-of-the-art tertiary structure prediction method—AlphaFold2. We train and test the method on both traditional model datasets (e.g. the datasets of the Critical Assessment of Techniques for Protein Structure Prediction) and a new dataset of high-quality structural models predicted only by AlphaFold2 for the proteins whose experimental structures were released recently. Our approach achieves state-of-the-art performance on protein structural models predicted by both traditional protein structure prediction methods and the latest end-to-end deep learning method—AlphaFold2. It performs even better than the model QA scores provided by AlphaFold2 itself. The results illustrate that the 3D-equivariant graph neural network is a promising approach to the evaluation of protein structural models. Integrating AlphaFold2 features with other complementary sequence and structural features is important for improving protein model QA.

59 BASIC BIOLOGICAL SCIENCES↗

DOE OSTI2022

Proximity Portability and in Transit , M-to-N Data Partitioning and Movement in SENSEI [Book Chapter]

In high-performance parallel in situ processing, the term in transit processing refers to those configurations where data must move from a producer to a consumer that runs on separate resources. In the context of parallel and distributed computing on an HPC platform one of the central challenges is to determine a mapping of data from producer ranks to consumer ranks. This problem is complicated by the heterogeneity that arises in producer-consumer pairs, such as when producer and consumer codes have different levels of concurrency, different scaling characteristics, or different data models. The resulting mapping and movement of data from M producer to N consumer ranks can have a significant impact on aggregate application performance, particularly when the data consumer requires only a subset of the overall data for its task. This chapter focuses on the design considerations that underlie SENSEI’s implementation to this challenging problem. These design considerations extend the core SENSEI architecture and include ideas like the need to accommodate flexibility in the choice of different partitioning methods, the ability for a data consumer to request and receive only the subset of data needed for its particular operation, and the ability to leverage any of several different data transport tools. The idea of proximity portability, being able to use different data transport methods as part of an in transit workflow, is illustrated through the use of three different transport layers where switching from one transport tool to another is accomplished with only a configuration file change. Here, the chapter also includes a performance analysis summary showing the performance gains that are possible in terms of multiple metrics, such as memory footprint, time to solution, and amount of data moved, when using optimized partitioners in an in transit setting, gains that are made possible by the implementation shaped by specific design considerations.

Bethel, E. Wes↗