Search NASA⌕ Search

SEARCH · Search NASA

Results for “Language model”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 955 records · Page 53

Geometry and energy of non-Abelian vortices

We study pure Yang-Mills theory on {Sigma}xS{sup 2}, where {Sigma} is a compact Riemann surface, and invariance is assumed under rotations of S{sup 2}. It is well known that the self-duality equations in this setup reduce to vortex equations on {Sigma}. If the Yang-Mills gauge group is SU(2), the Bogomolny vortex equations of the Abelian Higgs model are obtained. For larger gauge groups, one generally finds vortex equations involving several matrix-valued Higgs fields. Here we focus on Yang-Mills theory with gauge group SU(N)/Z{sub N} and a special reduction which yields only one non-Abelian Higgs field. One of the new features of this reduction is the fact that while the instanton number of the theory in four dimensions is generally fractional with denominator N, we still obtain an integral vortex number in the reduced theory. We clarify the relation between these two topological charges at a bundle geometric level. Another striking feature is the emergence of nontrivial lower and upper bounds for the energy of the reduced theory on {Sigma}. These bounds are proportional to the area of {Sigma}. We give special solutions of the theory on {Sigma} by embedding solutions of the Abelian Higgs model into the non-Abelian theory, and we relate our work to the language of quiver bundles, which has recently proved fruitful in the study of dimensional reduction of Yang-Mills theory.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

GraQL: A Query Language for High-Performance Attributed Graph Databases

Graph databases have gained increasing interest in the last few years due to the emergence of data sources which are not easily analyzable in traditional relational models or for which a graph data model is the natural representation. In order to understand the design and implementation choices for an attributed graph database backend and query language, we have started to design our infrastructure for attributed graph databases. In this paper, we describe the design considerations of our in-memory attributed graph database system with a particular focus on the data definition and query language components.

attributed graphs↗

Remove Welding Residual Stress for CFETR Vacuum Vessel by Trailing Ultrasonic Impact Treatment

The China fusion engineering test reactor (CFETR) vacuum vessel is welded by narrow gap TIG (NG-TIG) welding, and the welding residual stress of the CFETR vacuum vessel can be redistributed by trailing welding ultrasonic impact treatment. In order to investigate the feasibility of the residual stress removing scheme, and to obtain the optimal trailing ultrasonic impact treatment technological parameters in the process of removing welding residual stress, a welding model that similar to vacuum vessel welding seam is established by using ABAQUS software, a NG-TIG welding heat source subroutine which is written in FORTRAN language used to simulate NG-TIG welding. According to the welding simulation results, a trailing welding ultrasonic impact treatment model is established, and the effects of the impact pin number, the impact method, the impact pin diameter and the impact frequency on welding residual stress are studied. The results show that the longitudinal residual stress in welding seam and its adjacent area and the lateral residual stress in the whole region have been obviously decreased by different trailing welding ultrasonic impact processes, and have made the tensile stress in the welding seam and its adjacent area has been changed into compressive stress, which can provide theoretical guidance and reference for actual production.

36 MATERIALS SCIENCE↗

Data imbalance in drug response prediction: multi-objective optimization approach in deep learning setting

Abstract Drug response prediction (DRP) methods tackle the complex task of associating the effectiveness of small molecules with the specific genetic makeup of the patient. Anti-cancer DRP is a particularly challenging task requiring costly experiments as underlying pathogenic mechanisms are broad and associated with multiple genomic pathways. The scientific community has exerted significant efforts to generate public drug screening datasets, giving a path to various machine learning models that attempt to reason over complex data space of small compounds and biological characteristics of tumors. However, the data depth is still lacking compared to application domains like computer vision or natural language processing domains, limiting current learning capabilities. To combat this issue and improves the generalizability of the DRP models, we are exploring strategies that explicitly address the imbalance in the DRP datasets. We reframe the problem as a multi-objective optimization across multiple drugs to maximize deep learning model performance. We implement this approach by constructing Multi-Objective Optimization Regularized by Loss Entropy loss function and plugging it into a Deep Learning model. We demonstrate the utility of proposed drug discovery methods and make suggestions for further potential application of the work to achieve desirable outcomes in the healthcare field.

Biochemistry & Molecular Biology↗

WORM (Write One, Read Many)

WORM (Write One, Run Many) is an easy to use, cross platform, embedded and extensible, functional programming language designed to facilitate the creation of input-decks for computer codes that use standard ASCII text files for input. WORM makes it easy to create generic (yet, complex and powerful) reusable models. Additionally its nature allows for complex calculations and routines to be coded once and easily reused, further simplifying the creation of input decks. WORM (Write One, Run Many) is a powerful and versatile tool designed to improve the efficiency of today’s criticality safety analyst by allowing: + input decks for parametric studies to be created quickly and easily, + calculations and variables to be imbedded into any input deck, thus allowing for meaningful parameter specifications, + problems to be specified using any combination of units, and + complex mathematically defined models to be created. A very simple syntax is employed, and therefore the WORM is easy to learn. A WORM model is essentially a standard input deck with some of its numerical values replaced by WORM code. WORM code may include and evaluate the following mathematical operators and functions: addition, subtraction, multiplication, division, exponentiation, modulus, sine, cosine, tangent, arcsine, arccosine, arctangent, the natural logarithm, logarithm base 10, integer truncation, absolute value, and random number. Several common constants, e.g., pi, e, and Avogadros’s Number (both as 6.022e23 and 0.6022), are predefined in WORM. Additionally, many unit conversion factors are also predefined: millimeters, meters, inches, feet, yards, and mils to centimeters; kilograms, pounds, and ounces to grams; liters, milliliters, gallons, and fluid ounces to cubic centimeters; and angular degrees to radians. For parametric studies, WORM supports various shorthand list specifications: the explicit step size, linear interpolation, and logarithmic interpolation. The list notation sequentially assigns multiple values to a name. WORM creates an input deck for each value of the name. If multiple lists are used, WORM steps through each list individually, i.e., WORM creates input decks corresponding to each and every permutation of the list values. Additionally, a library of standard material definitions and Perl subroutines are included. Any one of these files can be incorporated into the subject model with a simple WORM read command. WORM is completely written in Perl, the Practical Extraction and Reporting Language. Perl is one of the most portable programming languages available today. As such, the WORM works on practically any computer platform.

Sartor, Raymond↗

Digitizing Today’s Buildings in the Real World: Lessons from Field Demonstrations

Digital twins, created by generating a virtual replica of a building, enable safe evaluation of operational scenarios and applications like fault detection and diagnosis and advanced controls. However, a prerequisite is the creation of a machine-readable digital representation of a building, currently hindered by fragmented information scattered across mechanical drawings, point lists, and natural language sequences. As a result, digital twin development remains labor-intensive, error-prone, and difficult to validate. To address these challenges, two efforts from ASHRAE aim to support the digitalization of buildings. ASHRAE s223 establishes a semantic model of buildings, representing system components, configuration, and data sources. ASHRAE s231 defines a vendor-neutral programming language for expressing their control logic. As the industry evaluates implementing them in their products, understanding the challenges that vendors and implementers may face is crucial. In this paper, we present findings and lessons learned from field demonstrations in five buildings that implemented control applications using ASHRAE s223 and s231. The demonstrations highlight how semantic modeling and formalized control descriptions can significantly reduce software development time, manual point mapping, and hard-coding. Beyond time efficiency, they enable reliable automation by minimizing human interpretation and providing a means for consistency across projects. We describe the processes and best practices for model creation and model usage, from translating heterogeneous building documentation into semantic representations to implementing control logic in real-world systems. Finally, we discuss the challenges that persist, including integration with legacy software environments, gaps in interoperability, and the level of expertise still required to effectively leverage semantic models.

Prakash, Anand Krishnan↗

Pretraining Billion-Scale Geospatial Foundational Models on Frontier

As AI workloads increase in scope, generalization capability becomes challenging for small task-specific models and their demand for large amounts of labeled training samples increases. On the contrary, Foundation Models (FMs) are trained with internet-scale unlabeled data via self-supervised learning and have been shown to adapt to various tasks with minimal fine-tuning. Although large FMs have demonstrated significant impact in natural language processing and computer vision, efforts toward FMs for geospatial applications have been restricted to smaller size models, as pretraining larger models requires very large computing resources equipped with state-of-the-art hardware accelerators. Current satellite constellations collect 100+TBs of data a day, resulting in images that are billions of pixels and multimodal in nature. Such geospatial data poses unique challenges opening up new opportunities to develop FMs. We investigate billion scale FMs and HPC training profiles for geospatial applications by pretraining on publicly available data. We studied from end-to-end the performance and impact in the solution by scaling the model size. Our larger 3B parameter size model achieves up to 30% improvement in top1 scene classification accuracy when comparing a 100M parameter model. Moreover, we detail performance experiments on the Frontier supercomputer, America's first exascale system, where we study different model and data parallel approaches using PyTorch's Fully Sharded Data Parallel library. Specifically, we study variants of the Vision Transformer architecture (ViT), conducting performance analysis for ViT models with size up to 15B parameters. By discussing throughput and performance bottlenecks under different parallelism configurations, we offer insights on how to leverage such leadership-class HPC resources when developing large models for geospatial imagery applications.

Tsaris, Aristeidis (aris)↗

Retargetable Optimizing Compilers for Quantum Accelerators via a Multi-Level Intermediate Representation

In this work, we present a multi-level quantum-classical intermediate representation (IR) that enables an optimizing, retargetable compiler for available quantum languages. Our work builds upon the Multi-level Intermediate Representation (MLIR) framework and leverages its unique progressive lowering capabilities to map quantum languages to the LLVM machine-level IR. We provide both quantum and classical optimizations via the MLIR pattern rewriting sub-system and standard LLVM optimization passes, and demonstrate the programmability, compilation, and execution of our approach via standard benchmarks and test cases. In comparison to other standalone language and compiler efforts available today, our work results in compile times that are 1000x faster than standard Pythonic approaches, and 5-10x faster than comparative standalone quantum language compilers. Our compiler provides quantum resource optimizations via standard programming patterns that result in a 10x reduction in entangling operations, a common source of program noise. We see this work as a vehicle for rapid quantum compiler prototyping.

43 PARTICLE ACCELERATORS↗

Empirical validation and comparison of methodologies to simulate micro and macro-encapsulated PCMs in the building envelope

Thermal Energy Storage (TES) has the potential to shift peak electricity demand. Passive TES is usually implemented in building envelope as micro and macro encapsulated phase change materials (PCM) to shift electric energy demand and therefore requires careful heat transfer analysis. Whole building energy modelling with simplified heat transfer analysis has become extremely important for designers, architects, engineers, and researchers to predict energy performance of buildings. It is important to validate PCM modelling algorithms used in building energy programs to quantify their error and prove their capacity to model different PCM encapsulation types. This study uses data from a microencapsulated PCM and two macroencapsulated PCMs (Bio based PCM and hydrate salts) tested in full-scale using the Advanced Multiscale Building Energy Research (AMBER) Lab located at the Colorado School of Mines and is used to validate a numerical algorithm written in MATLAB language. To approximate the heat transfer through a wall assembly with macroencapsulated PCM pouches, several modelling techniques that can reduce 3D heat transfer characteristics to 1D are explored in this research. A parallel path heat transfer modelling approach is found to give the closest agreement with the experimental data for the pouched PCMs in building envelope applications.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

Matlab Based LOCO

The LOCO algorithm has been used by many accelerators around the world. Although the uses for LOCO vary, the most common use has been to find calibration errors and correct the optics functions. The light source community in particular has made extensive use of the LOCO algorithms to tightly control the beta function and coupling. Maintaining high quality beam parameters requires constant attention so a relatively large effort was put into software development for the LOCO application. The LOCO code was originally written in FORTRAN. This code worked fine but it was somewhat awkward to use. For instance, the FORTRAN code itself did not calculate the model response matrix. It required a separate modeling code such as MAD to calculate the model matrix then one manually loads the data into the LOCO code. As the number of people interested in LOCO grew, it required making it easier to use. The decision to port LOCO to Matlab was relatively easy. It's best to use a matrix programming language with good graphics capability; Matlab was also being used for high level machine control; and the accelerator modeling code AT, [5], was already developed for Matlab. Since LOCO requires collecting and processing a relative large amount of data, it is very helpful to have the LOCO code compatible with the high level machine control, [3]. A number of new features were added while porting the code from FORTRAN and new methods continue to evolve, [7][9]. Although Matlab LOCO was written with AT as the underlying tracking code, a mechanism to connect to other modeling codes has been provided.

43 PARTICLE ACCELERATORS↗

A Fine-grained Asynchronous Bulk Synchronous parallelism model for PGAS applications

The Partitioned Global Address Space (PGAS) model is well suited for executing irregular applications on cluster-based systems, due to its efficient support for short, one-sided messages. Separately, the actor model has been gaining popularity as a productive asynchronous message-passing approach for distributed objects in enterprise and cloud computing platforms, typically implemented in languages such as Erlang, Scala or Rust. To the best of our knowledge, there has been no past work on using the actor model to deliver both productivity and scalability to irregular PGAS applications with large number of small messages. In this paper, we introduce a new programming system for PGAS applications, in which point-to-point remote operations can be expressed as fine-grained asynchronous actor messages. In our approach, the programmer does not need to worry about programming complexities related to message aggregation and termination detection. Our approach can be viewed as extending the classical Bulk Synchronous Parallelism model with fine-grained asynchronous communications within a phase or superstep. Here, we believe that our approach offers a desirable point in the productivity-performance space for PGAS applications, with more scalable performance and higher productivity relative to past approaches. Specifically, for seven irregular mini-applications from the Bale Kernels and three graph kernels executed using 2048 cores in the NERSC Cori system, our approach shows geometric mean performance improvements of ≥ 20X relative to standard PGAS versions (UPC and OpenSHMEM) while maintaining comparable productivity to those versions.

97 MATHEMATICS AND COMPUTING↗

Symmetric tensor gauge theories on curved spaces

Fractons and other subdimensional particles are an exotic class of emergent quasi-particle excitations with severely restricted mobility. A wide class of models featuring these quasi-particles have a natural description in the language of symmetric tensor gauge theories, which feature conservation laws restricting the motion of particles to lower-dimensional sub-spaces, such as lines or points. In this work, we investigate the fate of symmetric tensor gauge theories in the presence of spatial curvature. We find that weak curvature can induce small (exponentially suppressed) violations on the mobility restrictions of charges, leaving a sense of asymptotic fractonic/sub-dimensional behaviour on generic manifolds. Nevertheless, we show that certain symmetric tensor gauge theories maintain sharp mobility restrictions and gauge invariance on certain special curved spaces, such as Einstein manifolds or spaces of constant curvature.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Tumor Control Probability of Radiosurgery and Fractionated Stereotactic Radiosurgery for Brain Metastases

As part of the American Association of Physicists in Medicine Working Group on Stereotactic Body Radiotherapy, tumor control probability (TCP) after stereotactic radiosurgery (SRS) and fractionated stereotactic radiosurgery (fSRS) for brain metastases was modeled based on pooled dosimetric and clinical data from published English-language literature.

62 RADIOLOGY AND NUCLEAR MEDICINE↗

Identifying COVID-19 cases and extracting patient reported symptoms from Reddit using natural language processing

We used social media data from “covid19positive” subreddit, from 03/2020 to 03/2022 to identify COVID-19 cases and extract their reported symptoms automatically using natural language processing (NLP). We trained a Bidirectional Encoder Representations from Transformers classification model with chunking to identify COVID-19 cases; also, we developed a novel QuadArm model, which incorporates Question-answering, dual-corpus expansion, Adaptive rotation clustering, and mapping, to extract symptoms. Our classification model achieved a 91.2% accuracy for the early period (03/2020-05/2020) and was applied to the Delta (07/2021–09/2021) and Omicron (12/2021–03/2022) periods for case identification. We identified 310, 8794, and 12,094 COVID-positive authors in the three periods, respectively. The top five common symptoms extracted in the early period were coughing (57%), fever (55%), loss of sense of smell (41%), headache (40%), and sore throat (40%). During the Delta period, these symptoms remained as the top five symptoms with percent authors reporting symptoms reduced to half or fewer than the early period. During the Omicron period, loss of sense of smell was reported less while sore throat was reported more. Our study demonstrated that NLP can be used to identify COVID-19 cases accurately and extracted symptoms efficiently.

60 APPLIED LIFE SCIENCES↗

Molecular property prediction for very large databases with natural language processing: a case study in ionic liquid design

The prospect of using artificial intelligence (AI) to accurately screen very large databases of compounds for multiple properties has yet to be realized. Here, we explore this possibility using ionic liquids (ILs) which offer unique physicochemical properties and excellent tunability, making them highly versatile solvents for various research applications. Screening millions of potential ILs for the best perfomance for use in specific tasks with experimental methods alone however, is impractical. Further, traditional’ physics-based computational chemistry is hindered by high computational cost. To address this challenge, we leverage a natural language processing (NLP)-based molecular embedding technique with advanced machine learning (ML) models to predict seven key IL properties: viscosity, density, ionic conductivity, surface tension, melting temperature, toxicity, and water solubility. Comprehensive datasets for these properties are obtained, then NLP featurization with Mol2vec is compared with other featurization techniques such as 2D Morgan fingerprints, and 3D quantum chemistry-derived sigma profiles. NLP-based featurization exhibited the best predictive performance, achieving the highest R 2 and lowest RMSE values for all the studied IL properties. Further, we present case studies of how ILs might be screened using combined property criteria for practical cases – lignocellulosic biomass processing, CO 2 capture, and optimal electrolytes for batteries – screening a novel database of ∼10.6 million generated feasible ILs. The results introduce NLP as a powerful tool for engineering many designer solvents with desirable properties for task specific applications.

Mohan, Mood [Oak Ridge National Laboratory (ORNL),↗

The application of neural networks with artificial intelligence technique in the modeling of industrial processes

Neural networks are a relatively new artificial intelligence technique that emulates the behavior of biological neural systems in digital software or hardware. These networks can 'learn', automatically, complex relationships among data. This feature makes the technique very useful in modeling processes for which mathematical modeling is difficult or impossible. The work described here outlines some examples of the application of neural networks with artificial intelligence technique in the modeling of industrial processes.

99 GENERAL AND MISCELLANEOUS↗

Watch and learn—a generalized approach for transferrable learning in deep neural networks via physical principles

Transfer learning refers to the use of knowledge gained while solving a machine learning task and applying it to the solution of a closely related problem. Such an approach has enabled scientific breakthroughs in computer vision and natural language processing where the weights learned in state-of-the-art models can be used to initialize models for other tasks which dramatically improve their performance and save computational time. Here we demonstrate an unsupervised learning approach augmented with basic physical principles that achieves fully transferrable learning for problems in statistical physics across different physical regimes. By coupling a sequence model based on a recurrent neural network to an extensive deep neural network, we are able to learn the equilibrium probability distributions and inter-particle interaction models of classical statistical mechanical systems. Our approach, distribution-consistent learning, DCL, is a general strategy that works for a variety of canonical statistical mechanical models (Ising and Potts) as well as disordered interaction potentials. Using data collected from a single set of observation conditions, DCL successfully extrapolates across all temperatures, thermodynamic phases, and can be applied to different length-scales. This constitutes a fully transferrable physics-based learning in a generalizable approach.

97 MATHEMATICS AND COMPUTING↗

SysCaps (Language Interfaces for Simulation Surrogates of Complex Systems) [SWR-24-97]

You've found the official code repository for the paper "SysCaps: Language Interfaces for Simulation Surrogates of Complex Systems," presented at the Foundation Models for Science: Progress, Opportunities, and Challenges workshop at NeurIPS 2024. Our paper conjectures that interfaces (both text templates as well as conversational) makes interacting with simulation surrogate models for complex systems more intuitive and accessible for both non-experts and experts. "System captions", or SysCaps, are text-based descriptions of systems based on information contained in simulation metadata. Our paper's goal is to train multimodal regression models that take text inputs (SysCaps) and timeseries inputs (exogenous system conditions such as hourly weather) and regress timeseries simulation outputs (e.g. hourly building energy consumption). The experiments in our paper with building and wind farm simulators, which can be reproduced using this codebase, aim to help us understand whether a) accurate regression in this setting is possible and b) if so, how well can we do it. Paper: https://arxiv.org/abs/2405.19653

Emami, Patrick↗