Search NASA⌕ Search
Home/Archive/DOE OSTI

DOE OSTI · 2349600

NEFTune: Noisy Embeddings Improve Instruction Finetuning

Jain, N·Chiang, P·Wen, Y·Kirchenbauer, J·Chu, H·Somepalli, G·Bartoldson, B R·Kailkhura, B·Schwarzschild, A·Saha, A·Goldblum, M·Geiping, J·Goldstein, T
View original source ↗Read source document ↗Export metadata ↓

Abstract

The source did not provide an abstract. Follow the original record for more information.

Explore related subjects

Mathematics and Computing

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Jain, N, Chiang, P, Wen, Y, Kirchenbauer, J, Chu, H, Somepalli, G, Bartoldson, B R, Kailkhura, B, Schwarzschild, A, Saha, A, Goldblum, M, Geiping, J, Goldstein, T. 2023-10-04. NEFTune: Noisy Embeddings Improve Instruction Finetuning. https://www.osti.gov/biblio/2349600

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related reports

DOE OSTI2026

High-Order Mesh r-Adaptivity with Tangential Relaxation and Guaranteed Mesh Validity

High-order meshes are crucial for achieving optimal convergence rates in curvilinear domains, preserving symmetry, and aligning with key flow features in moving mesh simulations [1], but their quality is challenging to control. In prior work, we have developed techniques based on Target-Matrix Optimization Paradigm (TMOP) to adapt a given high-order mesh to the geometry and solution of the partial differential equation (PDE) [2, 3]. Here, we extend this framework to address two key gaps in the literature for highorder mesh 𝑟-adaptivity. First, we introduce tangential relaxation on curved surfaces using solely the discrete mesh representation, eliminating the need for access to underlying geometry (e.g., CAD model). Second, we ensure a continuously positive Jacobian determinant throughout the domain. This determinant positivity is essential for using the high-order mesh resulting from 𝑟-adaptivity with arbitrary quadrature schemes in simulations. The proposed approach is demonstrated to be robust using a variety of numerical experiments.

Mathematics and Computing↗