Search NASA⌕ Search

SEARCH · Search NASA

Results for “multimodal data”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 235 records · Page 13

Retrieval of high time resolution growth factor probability density function from a humidity-controlled fast integrated mobility spectrometer

Hygroscopicity describes the tendency of aerosol particle to uptake water and is among the key parameters in determining the impact of atmospheric aerosols on global radiation and climate. A hygroscopicity tandem differential mobility analyzer (HTDMA) system is the most widely utilized instrument for determining the aerosol hygroscopic growth. Because of the time needed to scan the classifying voltage of the DMA, HTDMA measurement normally requires a minimum of 30 min to characterize the particle hygroscopic growth at a single relative humidity for five to six different sizes. This slow speed is often inadequate for measurements onboard mobile platforms or when aerosols evolve rapidly. Recently, a humidity-controlled fast integrated mobility spectrometer (HFIMS) was developed for measuring the hygroscopic growth of particles. The measurement speed of the HFIMS is about one order of magnitude faster than that of the conventional HTDMA. Here, a data inversion routine is developed to retrieve the growth factor probability density function (GF-PDF) of particles measured by the HFIMS. The inversion routine considers the transfer functions of the upstream DMA and the downstream water-based fast integrated mobility spectrometer (FIMS), and derives the GF-PDF that reproduces the measured responses of the HFIMS. The performance of the inversion routine is examined using ambient measurements with different assumptions for the spectral shape of the particle GF-PDF (multimodal lognormal or piecewise linear). The influences of the data inversion parameters and counting statistics on the inverted GF-PDFs were further investigated, and an approach to determine the optimized inversion parameters is presented.

47 OTHER INSTRUMENTATION↗

Picturing Data With Uncertainty

NASA is in the business of creating maps for scientific purposes to represent important biophysical or geophysical quantities over space and time. For example, maps of surface temperature over the globe tell scientists where and when the Earth is heating up; regional maps of the greenness of vegetation tell scientists where and when plants are photosynthesizing. There is always uncertainty associated with each value in any such map due to various factors. When uncertainty is fully modeled, instead of a single value at each map location, there is a distribution expressing a set of possible outcomes at each location. We consider such distribution data as multi-valued data since it consists of a collection of values about a single variable. Thus, a multi-valued data represents both the map and its uncertainty. We have been working on ways to visualize spatial multi-valued data sets effectively for fields with regularly spaced units or grid cells such as those in NASA's Earth science applications. A new way to display distributions at multiple grid locations is to project the distributions from an individual row, column or other user-selectable straight transect from the 2D domain. First at each grid cell in a given slice (row, column or transect), we compute a smooth density estimate from the underlying data. Such a density estimate for the probability density function (PDF) is generally more useful than a histogram, which is a classic density estimate. Then, the collection of PDFs along a given slice are presented vertically above the slice and form a wall. To minimize occlusion of intersecting slices, the corresponding walls are positioned at the far edges of the boundary. The PDF wall depicts the shapes of the distributions very dearly since peaks represent the modes (or bumps) in the PDFs. We've defined roughness as the number of peaks in the distribution. Roughness is another useful summary information for multimodal distributions. The uncertainty of the multi-valued data can also be interpreted by the number of peaks and the widths of the peaks as shown by the PDF walls.

Kao, David↗

Prediction of Cognitive States During Flight Simulation Using Multimodal Psychophysiological Sensing

The Commercial Aviation Safety Team found the majority of recent international commercial aviation accidents attributable to loss of control inflight involved flight crew loss of airplane state awareness (ASA), and distraction was involved in all of them. Research on attention-related human performance limiting states (AHPLS) such as channelized attention, diverted attention, startle/surprise, and confirmation bias, has been recommended in a Safety Enhancement (SE) entitled "Training for Attention Management." To accomplish the detection of such cognitive and psychophysiological states, a broad suite of sensors was implemented to simultaneously measure their physiological markers during a high fidelity flight simulation human subject study. Twenty-four pilot participants were asked to wear the sensors while they performed benchmark tasks and motion-based flight scenarios designed to induce AHPLS. Pattern classification was employed to predict the occurrence of AHPLS during flight simulation also designed to induce those states. Classifier training data were collected during performance of the benchmark tasks. Multimodal classification was performed, using pre-processed electroencephalography, galvanic skin response, electrocardiogram, and respiration signals as input features. A combination of one, some or all modalities were used. Extreme gradient boosting, random forest and two support vector machine classifiers were implemented. The best accuracy for each modality-classifier combination is reported. Results using a select set of features and using the full set of available features are presented. Further, results are presented for training one classifier with the combined features and for training multiple classifiers with features from each modality separately. Using the select set of features and combined training, multistate prediction accuracy averaged 0.64 +/- 0.14 across thirteen participants and was significantly higher than that for the separate training case. These results support the goal of demonstrating simultaneous real-time classification of multiple states using multiple sensing modalities in high fidelity flight simulators. This detection is intended to support and inform training methods under development to mitigate the loss of ASA and thus reduce accidents and incidents.

Harrivel, Angela R.↗

Poster — Thur Eve — 71: A 4D Multimodal Lung Phantom for Regmentation Evaluation

Segmentation and registration of medical imaging data are two processes that can be integrated (a process termed regmentation) to iteratively reinforce each other, potentially improving efficiency and overall accuracy. A significant challenge is presented when attempting to validate the joint process particularly with regards to minimizing geometric uncertainties associated with the ground truth while maintaining anatomical realism. This work demonstrates a 4D MRI, PET, and CT compatible tissue phantom with a known ground truth for evaluating registration and segmentation accuracy. The phantom consists of a preserved swine lung connected to an air pump via a PVC tube for inflation. Mock tumors were constructed from sea sponges contained within two vacuum-sealed compartments with catheters running into each one for injection of radiotracer solution. The phantom was scanned using a GE Discovery-ST PET/CT scanner and a 0.23T Phillips MRI, and resulted in anatomically realistic images. A bifurcation tracking algorithm was implemented to provide a ground truth for evaluating registration accuracy. This algorithm was validated using known deformations of up to 7.8 cm using a separate CT scan of a human thorax. Using the known deformation vectors to compare against, 76 bifurcation points were selected. The tracking accuracy was found to have maximum mean errors of −0.94, 0.79 and −0.57 voxels in the left-right, anterior-posterior and inferior-superior directions, respectively. A pneumatic control system is under development to match the respiratory profile of the lungs to a breathing trace from an individual patient.

60 APPLIED LIFE SCIENCES↗

How Advances in Imaging Will Affect Precision Radiation Oncology

Radiation oncology is 1 of the most structured disciplines in medicine. It is of a highly technical nature with reliance on robotic systems to deliver intervention, engagement of diverse expertise, and early adoption of digital approaches to optimize and execute the application of this highly effective cancer treatment. As a localized intervention, the dependence on sensitive, specific, and accurate imaging to define the extent of disease, its heterogeneity, and adjacency to normal tissues directly affects the therapeutic ratio. Image-based in vivo temporal monitoring of the response to treatment enables adaptation and further affects the therapeutic ratio. Thus, more precise intervention will enable fractionation schedules that better interoperate with advances such as immunotherapy. In the data set–rich era that promises precision and personalized medicine, the radiation oncology field will integrate these new data into highly protocoled pathways of care that begin with multimodality prediction and enable patient-specific adaptation of therapy based on quantitative measures of the individual's dose–volume temporal trajectory and midtherapy predictions of response. In addition to advancements in computed tomography imaging, emerging technologies, such as ultra-high-field magnetic resonance and molecular imaging will bring new information to the design of treatments. Next-generation image guided radiation therapy systems will inject high specificity and sensitivity data and stimulate adaptive replanning. In addition, a myriad of pre- and peritherapeutic markers derived from advances in molecular pathology (eg, tumor genomics), automated and comprehensive imaging analytics (eg, radiomics, tumor microenvironment), and many other emerging biomarkers (eg, circulating tumor cell assays) will need to be integrated to maximize the benefit of radiation therapy for an individual patient. We present a perspective on the promise and challenges of fully exploiting imaging data in the pursuit of personalized radiation therapy, drawing from the presentations and broader discussions at the 2016 American Society of Therapeutic Radiation Oncology–National Cancer Institute workshop on Precision Medicine in Radiation Oncology (Bethesda, MD).

62 RADIOLOGY AND NUCLEAR MEDICINE↗

TSDC: Transportation Secure Data Center: Real-World Data for Planning, Modeling, and Analysis

The Transportation Secure Data Center is a centralized repository for high-resolution transportation data from hundreds of travel and transit surveys and studies. It makes vital transportation data broadly available to users while preserving the privacy of survey participants. It houses surveys and studies conducted by state departments of transportation, metropolitan planning organizations, transit agencies, cities, and other public agencies. Meanwhile, the Livewire Data Platform empowers research, industry, and academic partners to easily and securely preserve, maintain, share, discover, and gain access to transportation and mobility data. Livewire accommodates a range of datasets, including behavioral, experimental, model, analytical, and raw data at the vehicle, traveler, and system levels. Datasets support mobility research and planning spanning urban science, connected and automated vehicles, fueling and charging infrastructure, mobility decision science, multimodal transportation, vehicle efficiency, and more.

33 ADVANCED PROPULSION SYSTEMS↗

Informed unsupervised machine learning analysis of dislocation microstructure from high-resolution differential aperture X-ray structural microscopy data

This study leverages high-resolution differential-aperture X-ray structural microscopy (DAXM) to probe the local dislocation structure in deformed 304L-stainless steel at small strain, by measuring the lattice rotation and deviatoric elastic strain with a sub-micron resolution. For a single grain in a polycrystalline specimen, the measured lattice rotation field over the measured volume exhibited a multimodal distribution while the deviatoric elastic strain showed a single-mode distribution. An unsupervised Cauchy mixture machine learning model was developed to resolve the multimodal distribution of the lattice rotation. By mapping the lattice rotation data associated with each Cauchy peak in the model back onto the measured volume, we identify contiguous regions of the crystal rotated near the average values corresponding to the peaks of the overall rotation distribution. These regions represent the grain subdivision in the microstructure. Finally, the dislocation density tensor was also computed and its norm was laid over the rotation field to detect the subgrain boundaries. This step provided a validation of the Cauchy mixture model for the analysis of the lattice rotation distribution. The current study highlights the integration of advanced X-ray microscopy techniques with data-driven analysis methods to uncover detailed microstructure scales in deformed crystals.

Machine learning; Lattice rotation; High-energy X-↗

Recent Developments in Femtosecond Laser-Enabled TriBeam Systems

Streams of multimodal three-dimensional (3D) and four-dimensional (4D) data are revolutionizing our ability to design and predict the behavior of a broad array of advanced materials systems. Over the last 10 years, a new 3D imaging platform consisting of a femtosecond (fs) pulsed laser coupled with a focused ion beam scanning electron microscope (FIB SEM) has been developed by UC Santa Barbara in collaboration with Thermo Fisher Scientific (formerly FEI). The femtosecond-laser-enabled FIB SEM, called the TriBeam, has become one of the only 3D serial sectioning methods available that can gather millimeter-scaled multimodal datasets at sub-μm voxel resolutions; these length scales are critical for many materials problems. Multimodal chemical, crystallographic, and morphological information can be gathered rapidly on a layer-by-layer basis and reconstructed in 3D. Large (gigabyte to terabyte scale) 3D datasets have been generated for a broad array of materials systems, including metallic alloys, ceramics, biomaterials, polymer- and ceramic-matrix composites, and semiconductors. The research tasks conducted have resulted in a completely new design, operating with a dual-wavelength femtosecond-pulsed laser on a plasma focused ion beam (PFIB) platform.

36 MATERIALS SCIENCE↗

Dose Summation Strategies for External Beam Radiation Therapy and Brachytherapy in Gynecologic Malignancy: A Review from the NRG Oncology and NCTN Medical Physics Subcommittees

Definitive, nonsurgical management of gynecologic malignancies involves external beam radiation therapy (EBRT) and/or brachytherapy (BT). Summation of the cumulative dose is critical to assess the total biologic effective dose to targets and organs at risk. Cumulative dose calculation from EBRT and BT can be performed with or without image registration (IR) and biologic dose summation. Among these dose summation strategies, linear addition of dose-volume histogram (DVH) parameters without IR is the global standard for composite dose reporting. This approach stems from an era without image guidance and simple external beam and brachytherapy treatment approaches. With technological advances, EBRT and high-dose-rate BT have evolved to allow for volume-based treatment planning and delivery. Modern conformal therapeutic radiation involves volumetric or intensity modulated EBRT, capable of simultaneously treating multiple targets at different specified dose levels. Therefore, given the complexity of modern radiation treatment, the linear addition of DVH parameters from EBRT and high-dose-rate BT is challenging to represent the combined dose distribution. Deformable image registration (DIR) between EBRT and image guided brachytherapy (IGBT) data sets may provide a more nuanced calculation of multimodal dose accumulation. However, DIR is still nascent in this regard, and needs further development for accuracy and efficiency for clinical use. Biologic dose summation can combine physical dose maps from EBRT and each IGBT fraction, thereby generating a composite DVH from the biologic effective dose. However, accurate radiobiologic parameters are tissue-dependent and not well characterized. A combination of voxel-based DIR and biologic weighted dose maps may be the best approximation of dose accumulation but remains invalidated. The purpose of this report is to review dose summation strategies for EBRT and BT, including conventional equivalent dose in 2-Gy fractions dose summation without image registration, physical dose summation using 3-dimensional rigid IR and DIR, and biologic dose summation. We also provide general clinical workflows for IGBT with a focus on cervical cancer.

62 RADIOLOGY AND NUCLEAR MEDICINE↗

Deep generative molecular design reshapes drug discovery

Recent advances and accomplishments of artificial intelligence (AI) and deep generative models have established their usefulness in medicinal applications, especially in drug discovery and development. To correctly apply AI, the developer and user face questions such as which protocols to consider, which factors to scrutinize, and how the deep generative models can integrate the relevant disciplines. This review summarizes classical and newly developed AI approaches, providing an updated and accessible guide to the broad computational drug discovery and development community. We introduce deep generative models from different standpoints and describe the theoretical frameworks for representing chemical and biological structures and their applications. We discuss the data and technical challenges and highlight future directions of multimodal deep generative models for accelerating drug discovery.

59 BASIC BIOLOGICAL SCIENCES↗

Numerical modelling of multimode fibre-optic communication lines

The results of numerical modelling of nonlinear propagation of an optical signal in multimode fibres with a small differential group delay are presented. It is found that the dependence of the error vector magnitude (EVM) on the differential group delay can be reduced by increasing the number of ADC samples per symbol in the numerical implementation of the differential group delay compensation algorithm in the receiver. The possibility of using multimode fibres with a small differential group delay for data transmission in modern digital communication systems is demonstrated. It is shown that with increasing number of modes the strong coupling regime provides a lower EVM level than the weak coupling one. (fibre-optic communication lines)

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

MTUQ: a framework for estimating moment tensors, point forces, and their uncertainties

SUMMARY We introduce MTUQ, an open-source Python package for seismic source estimation and uncertainty quantification, emphasizing flexibility and operational scalability. MTUQ provides MPI-parallelized grid search and global optimization capabilities, compatibility with 1-D and 3-D Green’s function database formats, customizable data processing, C-accelerated waveform and first-motion polarity misfit functions, and utilities for plotting seismic waveforms and visualizing misfit and likelihood surfaces. Applicability to a range of full- and constrained-moment tensor, point force, and centroid inversion problems is possible via a documented application programming interface, accompanied by example scripts and integration tests. We demonstrate the software using three different types of seismic events: (1) a 2009 intraslab earthquake near Anchorage, Alaska; (2) an episode of the 2021 Barry Arm landslide in Alaska; and (3) the 2017 Democratic People’s Republic of Korea underground nuclear test. With these events, we illustrate the well-known complementary character of body waves, surface waves, and polarities for constraining source parameters. We also convey the distinct misfit patterns that arise from each individual data type, the importance of uncertainty quantification for detecting multimodal or otherwise poorly constrained solutions, and the software’s flexible, modular design.

58 GEOSCIENCES↗

NASA Next-Generation Sensing Technologies for Earth & Planetary Science

Dr. Ved Chirayath's seminar will highlight new sensing technologies and airborne platforms he is developing for Earth & Space Science as director of the NASA Silicon Valley Laboratory for Advanced Sensing. His talk will feature recent work on Fluid Lensing, the first remote sensing technology capable of imaging through ocean waves in 3D at sub-cm resolutions, MiDAR, a next-generation active hyperspectral remote sensing and optical communications instrument, airborne gas sensing of multipollutant combustion sources, and a plasma actuated unmanned aerial vehicle (UAV) that utilized high-voltage dielectric discharge devices to achieve the first plasma controlled flight in history.Fluid Lensing and the NASA FluidCam instrument have been used extensively to provide distortion-free 3D multispectral imagery from UAVs of shallow marine systems around the world. MiDAR is being deployed on aircraft and underwater remotely operated vehicles (ROVs) as a new method to remotely sense living and nonliving structures in extreme environments as an analog for future Ocean Worlds robotic exploration missions. Finally, Chirayath will present preliminary results from NeMO-Net, a supercomputer-based neural network that uses a citizen science video game for global multimodal coral reef benthic habitat mapping, fusing remote sensing data from Fluid Lensing, MiDAR, NASA's Earth Observing System, and commercial satellites, to better understand the present and past dynamics of shallow marine systems. Together, these maturing technologies present promising new ways in which to explore terrestrial, marine, and aerial systems on Earth, and, ultimately, aid in the search for extraterrestrial life within our solar system and beyond.

Chirayath, Ved↗

Multimodal Few-Shot Segmentation of Electron Micrographs

Scanning transmission electron microscopy (STEM) is one of the most used methods of analyzing the chemistry and composition of materials. By analyzing microstructures, these microscopes can help scientists better understand the molecular underpinnings of microelectronics, batteries, and more. However, STEM data can be difficult to interpret, so recent developments have been made in applications of machine learning to analyze these images. The PNNL-developed pyCHIP Classifier has achieved results in segmenting STEM these images via few-shot learning, a method which requires little data and human input, perfect for quickly analysis. In my internship I (Eli Meyers) investigated a multimodal improvement of this classifier by incorporating energy dispersive x-ray spectroscopy (EDS) data into the classification process for a more accurate segmentation. Furthermore, I encoded the spectral data by training a mass spectrometry encoder on the EDS data to extract a more meaningful representation of the data.

36 MATERIALS SCIENCE↗

AI-Based Analytics and Energy Modeling Framework for Characterizing Urban Energy Systems

Developing location-specific district energy models is essential for understanding energy patterns and supporting efficient management and planning decisions. However, accurately characterizing these models remains challenging due to gaps in building characteristics and labor-intensive traditional modeling workflows. To address these challenges, we develop an AI-based framework that integrates top-down and bottom-up building energy data to automate urban energy model characterization. The framework trains multimodal deep learning models using heterogeneous ResStockTM datasets to infer missing building characteristics from varying levels of known information and generate simulation-ready inputs for district-scale energy modeling. It also employs a conditioning-based injection approach to generate ”what-if” scenarios, enabling users to explore retrofit, efficiency, and technology-upgrade pathways. Integrated within URBANoptTM, a bottom-up district energy modeling platform for simulating co-located buildings, the framework infers detailed building-level inputs required for bottom-up simulations. Both localized and generalized AI models are developed to learn relationships across categorical, numerical, and time-series data, enabling reconstruction of missing attributes and generation of targeted upgrade scenarios. We demonstrate this methodology on a residential neighborhood in Baltimore, MD, assessing internal consistency against ResStock reference data and URBANopt simulation, and comparing selected attributes against real-world building characteristics. Results show strong overall predictive accuracy in data completion and scenario generation, with localized and generalized models offering complementary trade-offs between precision and scalability. Overall, our automated framework streamlines energy modeling and provides a reliable framework for urban building energy characterization.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

Polynomial chaos representation of databases on manifolds

Characterizing the polynomial chaos expansion (PCE) of a vector-valued random variable with probability distribution concentrated on a manifold is a relevant problem in data-driven settings. The probability distribution of such random vectors is multimodal in general, leading to potentially very slow convergence of the PCE. In this paper, we build on a recent development for estimating and sampling from probabilities concentrated on a diffusion manifold. The proposed methodology constructs a PCE of the random vector together with an associated generator that samples from the target probability distribution which is estimated from data concentrated in the neighborhood of the manifold. The method is robust and remains efficient for high dimension and large datasets. The resulting polynomial chaos construction on manifolds permits the adaptation of many uncertainty quantification and statistical tools to emerging questions motivated by data-driven queries.

97 MATHEMATICS AND COMPUTING↗

Toward particle accelerator machine state embeddings as a modality for large language models

Understanding and diagnosing the state of a particle accelerator requires navigating high-dimensional control system data, often involving hundreds of interdependent parameters. We propose a novel multimodal embedding framework that jointly learns representations of machine states from both numerical control system readouts and natural language descriptions. This enables the translation of complex machine conditions into human-readable summaries while maintaining fidelity to the underlying physical system. The obtained embeddings are subsequently adapted to an open-weights large language model via cross-attention conditioning. We demonstrate a first implementation trained on European XFEL machine state data. This work covers the embedding model architecture, training methodology, and presents initial examples demonstrating the model's capabilities in action. Due to the general concept of machine state, the model can be easily adapted to other facilities and control system environments.

Accelerator Physics↗

A Hybrid Tour-Based Model For Energy Analysis Of Multi-Modal Intra-City Freight: A Case Study Of Autonomous Electric Vehicles: Preprint

With the emergence of cutting-edge transportation technologies, such as electric vehicles, connected and autonomous vehicles, and drones, the adoption of multimodal freight mobility has the potential to improve efficiency and save more energy. This paper proposes a hybrid tour-based model to evaluate the energy impact of multimodal intra-city freight movement for future scenarios. The model is built based on the traveling salesman problem model and clustering techniques. A case study using autonomous electric vehicles for package delivery is evaluated. The study was conducted using data analyzed from the Columbus, Ohio, metropolitan area. The initial results show that multimodal package delivery using autonomous electric vehicles reduced total travel time by 45% and saved total energy use by 19%, but increased total vehicle miles traveled by 15% when compared with the baseline scenario.

33 ADVANCED PROPULSION SYSTEMS↗