Search NASA⌕ Search

SEARCH · Search NASA

Results for “data parallel”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,333 records · Page 74
All sourcesNASA NTRSDOE OSTI
DOE OSTI2016

The 3D-HST survey: Hubble Space Telescope WFC3/G141 GRISM spectra, Redshifts, and emission line measurements for ∼100,000 galaxies

We present reduced data and data products from the 3D-HST survey, a 248-orbit HST Treasury program. The survey obtained WFC3 G141 grism spectroscopy in four of the five CANDELS fields: AEGIS, COSMOS, GOODS-S, and UDS, along with WFC3 H {sub 140} imaging, parallel ACS G800L spectroscopy, and parallel I {sub 814} imaging. In a previous paper, we presented photometric catalogs in these four fields and in GOODS-N, the fifth CANDELS field. Here we describe and present the WFC3 G141 spectroscopic data, again augmented with data from GO-1600 in GOODS-N (PI: B. Weiner). We developed software to automatically and optimally extract interlaced two-dimensional (2D) and one-dimensional (1D) spectra for all objects in the Skelton et al. (2014) photometric catalogs. The 2D spectra and the multi-band photometry were fit simultaneously to determine redshifts and emission line strengths, taking the morphology of the galaxies explicitly into account. The resulting catalog has redshifts and line strengths (where available) for 22,548 unique objects down to JH{sub IR}⩽24 (79,609 unique objects down to JH{sub IR}⩽26). Of these, 5459 galaxies are at z>1.5 and 9621 are at 0.7<z<1.5, where Hα falls in the G141 wavelength coverage. The typical redshift error for JH{sub IR}⩽24 galaxies is σ{sub z}≈0.003×(1+z), i.e., one native WFC3 pixel. The 3σ limit for emission line fluxes of point sources is 2.1×10{sup −17} erg s{sup −1} cm{sup −2}. All 2D and 1D spectra, as well as redshifts, line fluxes, and other derived parameters, are publicly available.{sup 18}.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2018

Single-pass parallel prefix scan with dynamic look back

One embodiment of the present invention performs a parallel prefix scan in a single pass that incorporates variable look-back. A parallel processing unit (PPU) subdivides a list of inputs into sequentially-ordered segments and assigns each segment to a streaming multiprocessor (SM) included in the PPU. Notably, the SMs may operate in parallel. Each SM executes write operations on a segment descriptor that includes the status, aggregate, and inclusive-prefix associated with the assigned segment. Further, each SM may execute read operations on segment descriptors associated with other segments. In operation, each SM may perform reduction operations to determine a segment-wide aggregate, may perform look-back operations across multiple preceding segments to determine an exclusive-prefix, and may perform a scan seeded with the exclusive prefix to generate output data. Advantageously, the PPU performs one read operation per input, thereby reducing the time required to execute the prefix scan relative to prior-art parallel implementations.

Merrill, Duane↗
DOE OSTI2022

Optimizing Error-Bounded Lossy Compression for Scientific Data With Diverse Constraints

Vast volumes of data are produced by today's scientific simulations and advanced instruments. These data cannot be stored and transferred efficiently because of limited I/O bandwidth, network speed, and storage capacity. Error-bounded lossy compression can be an effective method for addressing these issues: not only can it significantly reduce data size, but it can also control the data distortion based on user-defined error bounds. In practice, many scientific applications have specific requirements or constraints for lossy compression, in order to guarantee that the reconstructed data are valid for post hoc analysis. For example, some datasets contain irrelevant data that should be isolated in particular and users often have intuition regarding value ranges, geospatial regions, and other data subsets that are crucial for subsequent analysis. Existing state-of-the-art error-bounded lossy compressors, however, do not consider these constraints during compression, resulting in inferior compression ratios with respect to user's post hoc analysis, due to the fact that the data itself provides little or no value for post hoc analysis. In this work we address this issue by proposing an optimized framework that can preserve diverse constraints during the error-bounded lossy compression, e.g., cleaning the irrelevant data, efficiently preserving different precision for multiple value intervals, and allowing users to set diverse precision over both regular and irregular regions. We perform our evaluation on a supercomputer with up to 2,100 cores. Experiments with six real-world applications show that our proposed diverse constraints based error-bounded lossy compressor can obtain a higher visual quality or data fidelity on reconstructed data with the same or even higher compression ratios compared with the traditional state-of-the-art compressor SZ. Furthermore, our experiments also demonstrate very good scalability in compression performance compared with the I/O throughput of the parallel file system.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2017

Lightweight Provenance Service for High-Performance Computing

Provenance describes detailed information about the history of a piece of data, containing the relationships among elements such as users, processes, jobs, and workflows that contribute to the existence of data. Provenance is key to supporting many data management functionalities that are increasingly important in operations such as identifying data sources, parameters, or assumptions behind a given result; auditing data usage; or understanding details about how inputs are transformed into outputs. Despite its importance, however, provenance support is largely underdeveloped in highly parallel architectures and systems. One major challenge is the demanding requirements of providing provenance service in situ. The need to remain lightweight and to be always on often conflicts with the need to be transparent and offer an accurate catalog of details regarding the applications and systems. To tackle this challenge, we introduce a lightweight provenance service, called LPS, for high-performance computing (HPC) systems. LPS leverages a kernel instrument mechanism to achieve transparency and introduces representative execution and flexible granularity to capture comprehensive provenance with controllable overhead. Extensive evaluations and use cases have confirmed its efficiency and usability. We believe that LPS can be integrated into current and future HPC systems to support a variety of data management needs.

Dai, Dong↗
DOE OSTI2019

Electron Energy Partition across Interplanetary Shocks. I. Methodology and Data Product

Analyses of 15,314 electron velocity distribution functions (VDFs) within ±2 hr of 52 interplanetary (IP) shocks observed by the Wind spacecraft near 1 au are introduced. The electron VDFs are fit to the sum of three model functions for the cold dense core, hot tenuous halo, and field-aligned beam/strahl component. The best results were found by modeling the core as either a bi-kappa or a symmetric (or asymmetric) bi-self-similar VDF, while both the halo and beam/strahl components were best fit to bi-kappa VDF. This is the first statistical study to show that the core electron distribution is better fit to a self-similar VDF than a bi-Maxwellian under all conditions. The self-similar distribution deviation from a Maxwellian is a measure of inelasticity in particle scattering from waves and/or turbulence. The ranges of values defined by the lower and upper quartiles for the kappa exponents are κ ec ~ 5.40–10.2 for the core, κ eh ~ 3.58–5.34 for the halo, and κ eb ~ 3.40–5.16 for the beam/strahl. The lower-to-upper quartile range of symmetric bi-self-similar core exponents is s ec ~ 2.00–2.04, and those of asymmetric bi-self-similar core exponents are p ec ~ 2.20–4.00 for the parallel exponent and q ec ~ 2.00–2.46 for the perpendicular exponent. The nuanced details of the fit procedure and description of resulting data product are also presented. The statistics and detailed analysis of the results are presented in Paper II and Paper III of this three-part study.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2018

About a method for compressing X-ray computed microtomography data

The management of scientific data is of high importance especially for experimental techniques that produce big data volumes. Such a technique is x-ray computed tomography (CT) and its community has introduced advanced data formats which allow for better management of experimental data. Rather than the organization of the data and the associated meta-data, the main topic on this work is data compression and its applicability to experimental data collected from a synchrotron-based CT beamline at the Elettra-Sincrotrone Trieste facility (Italy) and studies images acquired from various types of samples. This study covers parallel beam geometry, but it could be easily extended to a cone-beam one. The reconstruction workflow used is the one currently in operation at the beamline. Contrary to standard image compression studies, this manuscript proposes a systematic framework and workflow for the critical examination of different compression techniques and does so by applying it to experimental data. Beyond the methodology framework, this study presents and examines the use of JPEG-XR in combination with HDF5 and TIFF formats providing insights and strategies on data compression and image quality issues that can be used and implemented at other synchrotron facilities and laboratory systems. In conclusion, projection data compression using JPEG-XR appears as a promising, efficient method to reduce data file size and thus to facilitate data handling and image reconstruction

data compression↗
DOE OSTI2019

Avoiding Communication in Primal and Dual Block Coordinate Descent Methods

Primal and dual block coordinate descent methods are iterative methods for solving regularized and unregularized optimization problems. Distributed-memory parallel implementations of these methods have become popular in analyzing large machine learning datasets. However, existing implementations communicate at every iteration, which, on modern data center and supercomputing architectures, often dominates the cost of floating-point computation. Recent results on communication-avoiding Krylov subspace methods suggest that large speedups are possible by re-organizing iterative algorithms to avoid communication. We show how applying similar algorithmic transformations can lead to primal and dual block coordinate descent methods that only communicate every $s$ iterations---where $s$ is a tuning parameter---instead of every iteration for the regularized least-squares problem. We show that the communication-avoiding variants reduce the number of synchronizations by a factor of $s$ on distributed-memory parallel machines without altering the convergence rate and attain strong scaling speedups of up to $6.1\times$ over the “standard algorithm" on a Cray XC30 supercomputer.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2018

Symmetric Indefinite Linear Solver Using OpenMP Task on Multicore Architectures

Recently, the Open Multi-Processing (OpenMP) standard has incorporated task-based programming, where a function call with input and output data is treated as a task. At run time, OpenMP's superscalar scheduler tracks the data dependencies among the tasks and executes the tasks as their dependencies are resolved. On a shared-memory architecture with multiple cores, the independent tasks are executed on different cores in parallel, thereby enabling parallel execution of a seemingly sequential code. With the emergence of many-core architectures, this type of programming paradigm is gaining attention-not only because of its simplicity, but also because it breaks the artificial synchronization points of the program and improves its thread-level parallelization. In this paper, we use these new OpenMP features to develop a portable high-performance implementation of a dense symmetric indefinite linear solver. Obtaining high performance from this kind of solver is a challenge because the symmetric pivoting, which is required to maintain numerical stability, leads to data dependencies that prevent us from using some common performance-improving techniques. To fully utilize a large number of cores through tasking, while conforming to the OpenMP standard, we describe several techniques. Our performance results on current many-core architectures-including Intel's Broadwell, Intel's Knights Landing, IBM's Power8, and Arm's ARMv8-demonstrate the portable and superior performance of our implementation compared with the Linear Algebra PACKage (LAPACK). The resulting solver is now available as a part of the PLASMA software package.

Computer Science↗
DOE OSTI2018

Comparative demography elucidates the longevity of parasitic and symbiotic relationships

Parasitic and symbiotic relationships govern vast nutrient and energy flows, yet controversy surrounds their longevity. Enduring relationships may engender parallel phylogenies among hosts and parasites, but so may ephemeral relationships when parasites colonize related hosts. An understanding of whether symbiont and host populations have grown and contracted in concert would be useful when considering the temporal durability of these relationships. Here, we devised methods to compare demographic histories derived from genomic data. We compared the historical growth of the agent of severe human malaria, Plasmodium falciparum , and its mosquito vector, Anopheles gambiae , to human and primate histories, thereby discerning long-term parallels and anthropogenic population explosions. The growth history of Trichinella spiralis , a zoonotic parasite disseminated by swine, proved regionally specific, paralleling distinctive growth histories for wild boar in Asia and Europe. Parallel histories were inferred for an anemone and its algal symbiont ( Exaiptasia pallida and Symbiodinium minutum ). Concerted growth in potatoes and the agent of potato blight ( Solanum tuberosum and Phytophthora infestans ) did not commence until the age of potato domestication. Through these examples, we illustrate the utility of comparative historical demography as a new exploratory tool by which to interrogate the origins and durability of myriad ecological relationships. To facilitate future use of this approach, we introduce a tool called C-PSMC to align and evaluate the similarity of demographic history curves.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2025

I/O in Machine Learning Applications on HPC Systems: A 360-degree Survey

Growing interest in Artificial Intelligence (AI) has resulted in a surge in demand for faster methods of Machine Learning (ML) model training and inference. This demand for speed has prompted the use of high performance computing (HPC) systems that excel in managing distributed workloads. Because data is the main fuel for AI applications, the performance of the storage and I/O subsystem of HPC systems is critical. In the past, HPC applications accessed large portions of data written by simulations or experiments or ingested data for visualizations or analysis tasks. ML workloads perform small reads spread across a large number of random files. This shift of I/O access patterns poses several challenges to modern parallel storage systems. In this paper, we survey I/O in ML applications on HPC systems, and target literature within a 6-year time window from 2019 to 2024. We define the scope of the survey, provide an overview of the common phases of ML, review available profilers and benchmarks, examine the I/O patterns encountered during offline data preparation, training, and inference, and explore I/O optimizations utilized in modern ML frameworks and proposed in recent literature. Lastly, we seek to expose research gaps that could spawn further R&D.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2016

Towards Real-Time Detection and Tracking of Spatio-Temporal Features: Blob-Filaments in Fusion Plasma [Spatio-Temporal Features in Large Irregular Data: Blob-Filaments in Fusion Plasma]

This work was originally motivated the need to identify and track blob filaments, a feature often associated with instability in magnetically confined fusion plasma. Understanding and mitigating such instability would improve fusion reactors and make fusion a truly inexhaustible source of clean energy. Similar spatio-temporal features are important in many other applications, for example, ignition kernels in combustion and tumor cells in a medical image. This work presents an approach for extracting these features by dividing the overall task into three steps: local identification of feature cells, grouping feature cells into extended feature, and tracking movement of feature through overlapping in space. Through our extensive work in parallelization, we demonstrate that this approach can effectively make use of a large number of compute nodes to detect and track blobs in fusion plasma. On a set of 4.3TB data, we observed linear speedup on 1024 processes and completed blob detection in less than three milliseconds using Edison, a Cray XC30 system at NERSC.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2024

Performance on HPC Platforms Is Possible Without C++

Computing at large scales has become extremely challenging due to increasing heterogeneity in both hardware and software. More and more scientific workflows must tackle a range of scales and use machine learning and AI intertwined with more traditional numerical modeling methods, placing more demands on computational platforms. These constraints indicate a need to fundamentally rethink the way computational science is done and the tools that are needed to enable these complex workflows. The current set of C++-based solutions may not suffice, and relying exclusively upon C++ may not be the best option, especially because several newer languages and boutique solutions offer more robust design features to tackle the challenges of heterogeneity. In June 2023, we held a mini symposium that explored the use of newer languages and heterogeneity solutions that are not tied to C++ and that offer options beyond template metaprogramming and Parallel. For for performance and portability. In conclusion, we describe some of the presentations and discussion from the mini symposium in this article.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2016

On the isotropization of solar wind protons

Protons observed in the solar wind are characterized by temperature anisotropies whose upper and lower bounds can be partially explained by marginal instability conditions associated with various plasma instabilities. However, an outstanding problem is that the majority of data distributed in (β{sub ∥},T{sub ⊥}/T{sub ∥}) space are located away from the boundaries and occupy a broad region with the peak near isotropic condition. The present paper employs macroscopic-kinetic theory that includes adiabatic effects arising from various non-monotonic inhomogeneities of magnetic field and density, the influence from proton-cyclotron and parallel firehose instabilities, and collisional dissipation in order to explore the consequence of each effect. It is found that spatial inhomogeneities are the leading cause of the scattering of the data points away from the marginal stability boundaries in (β{sub ∥},T{sub ⊥}/T{sub ∥}) space, thus providing a potential explanation for observations.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2016

Revised Extended Grid Library

The Revised Eolus Grid Library (REGL) is a mesh-tracking library that was developed for use with the MCNP6TM computer code so that (radiation) particles can track on an unstructured mesh. The unstructured mesh is a finite element representation of any geometric solid model created with a state-of-the-art CAE/CAD tool. The mesh-tracking library is written using modern Fortran and programming standards; the library is Fortran 2003 compliant. The library was created with a defined application programmer interface (API) so that it could easily integrate with other particle tracking/transport codes. The library does not handle parallel processing via the message passing interface (mpi), but has been used successfully where the host code handles the mpi calls. The library is thread-safe and supports the OpenMP paradigm. As a library, all features are available through the API and overall a tight coupling between it and the host code is required. Features of the library are summarized with the following list: Can accommodate first and second order 4, 5, and 6-sided polyhedra; any combination of element types may appear in a single geometry model; parts may not contain tetrahedra mixed with other element types; pentahedra and hexahedra can be together in the same part; robust handling of overlaps and gaps; tracks element-to-element to produce path length results at the element level; finds element numbers for a given mesh location; finds intersection points on element faces for the particle tracks; produce a data file for post processing results analysis; reads Abaqus .inp input (ASCII) files to obtain information for the global mesh-model; supports parallel input processing via mpi; and support parallel particle transport by both mpi and OpenMP.

Martz, Roger L.↗
DOE OSTI2015

Lloviu virus VP24 and VP35 proteins function as innate immune antagonists in human and bat cells

Lloviu virus (LLOV) is a new member of the filovirus family that also includes Ebola virus (EBOV) and Marburg virus (MARV). LLOV has not been cultured; however, its genomic RNA sequence indicates the coding capacity to produce homologs of the EBOV and MARV VP24, VP35, and VP40 proteins. EBOV and MARV VP35 proteins inhibit interferon (IFN)-alpha/beta production and EBOV VP35 blocks activation of the antiviral kinase PKR. The EBOV VP24 and MARV VP40 proteins inhibit IFN signaling, albeit by different mechanisms. Here we demonstrate that LLOV VP35 suppresses Sendai virus induced IFN regulatory factor 3 (IRF3) phosphorylation, IFN-α/β production, and PKR phosphorylation. Additionally, LLOV VP24 blocks tyrosine phosphorylated STAT1 binding to karyopherin alpha 5 (KPNA5), STAT1 nuclear accumulation, and IFN-induced gene expression. LLOV VP40 lacks detectable IFN antagonist function. These activities parallel EBOV IFN inhibitory functions. EBOV and LLOV VP35 and VP24 proteins also inhibit IFN responses in bat cells. These data suggest that LLOV infection will block innate immune responses in a manner similar to EBOV. - Highlights: • Lloviu virus (LLOV) is a new member of the filovirus family. • LLOV VP35 blocks IRF3 phosphorylation, IFN-α/β production and PKR phosphorylation. • LLOV VP24 inhibits IFN responses by targeting phospho-STAT1 KPNA interaction. • Infection by LLOV may block innate immune responses in a manner similar to EBOV.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2020

One Structure, Two Elements—LuGe 2 Superconductor vs Ordinary Metallic Conductor LuSn 2 . A Case Study on How Site-Selective Germanium for Tin Atom Substitution Leads to Modulating of the Charge Distribution

The substitution of chemically similar elements in a given crystal structure is an effective way to enhance physical properties, but the understanding on such improvements is usually impeded because the substitutions are random, and the roles of the different atoms cannot be distinguished by crystallographic symmetry. Herein, we provide a detailed crystallographic analysis and property measurements for the continuous solid solutions LuGe x Sn 2–x (0 < x < 2). The results show that there is no apparent change of the global symmetry, with the end-members LuGe 2 and LuSn 2 , as well as the intermediate LuGe x Sn 2–x compositions adopting the ZrSi 2 type structure (space group Cmcm, Pearson index oC12). Yet, the refinements of the crystal structures from single-crystal X-ray diffraction data show that Ge–Sn atom substitutions are not random, but occur preferentially at the zig-zag chain. The patterned distribution of two group 14 elements leads to a significant variation in chemical bonding and charge ordering within the other structural fragment, the 2D square nets, thereby resulting in tuned electron transport. The enhancement is greater than the typical Bloch-Gruneisen model, and more akin to the parallel-resistor model. Magnetization measurements on single crystals show bulk superconductivity in all LuGe x Sn 2–x samples with shielding fractions as high as 90%. Specific heat data confirm the effect to originate from residual metallic tin in the material, indicating that Sn atom substitutions in the 2D square nets cause disruptions of the hypervalent bonding and local anisotropy, which ultimately leads to vanishing of the superconducting state in the end-member LuGe 2 . This work sheds light on how the complexity in chemical interactions by two different carbon congeners leads to changes in the physical properties and how they can be correlated with the induced charge distribution. Furthermore, these studies also provide a general approach to modulation of charge density, and thus, of emerging physical properties in other classes of intermetallic systems based on the main-group elements of groups 13 to 15.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2025

Characterizing Absolute Orientations in DNA Self-Assembly of Single Molecules

DNA self-assembly of single molecules (i.e., dyes) with deterministic orientations is a powerful approach for engineering mo-lecular excitons. However, current determination methods of the dye orientation relative to DNA do not account for the orien-tation of the dye plane, which is a missing degree of freedom needed to define absolute three-dimensional orientations. In this work, we combine fluorescence-detected linear dichroism, defocused dipole imaging, and DNA points accumulation for imaging in nanoscale topography (DNA-PAINT) super-resolution microscopy to determine the absolute three-dimensional orientations of single Cy5 dyes relative to host DNA duplexes which includes the dye plane orientation. The data revealed that the absorption and emission dipoles are perpendicular to the DNA duplex, and the mean dye plane is parallel to the DNA bases, which supports the notion that Cy5 dyes intercalate between DNA base pairs. The presented methodology will inspire the investigation of the dye plane orientation for controlling dye arrangement configurations beyond spontaneous π-stacking between dyes as well as achieving novel dye-DNA arrangements.

36 MATERIALS SCIENCE↗