Search NASA⌕ Search

SEARCH · Search NASA

Results for “large data”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 343 records · Page 19

Pre-commercial Demonstration of Direct Non-iterative State Estimator for Operational Use of Synchrophasor Management Systems

The objective of this project was to demonstrate functionality and performance of a Direct Non-iterative State Estimator (DNSE) integrated with NYPA’s Energy Management System (EMS) and with enhanced Real Time Dynamics Monitoring System (RTDMS) synchrophasor platform from Electric Power Group (EPG). DNSE is designed to overcome a major obstacle to operational use of Synchro-Phasor Management Systems (SPMS) by providing to synchrophasor management systems (SPMS) applications a consistent and a complete synchrophasor data foundation in the same way that a traditional EMS State Estimator (SE) provides to EMS applications. Specifically, DNSE is designed to use synchrophasor measurements collected by a central PDC, Supervisory Control and Data Acquisition (SCADA) measurements, and Energy Management System (EMS) network model, to obtain the complete state of the utility’s operating model at rates that are close to the synchrophasor data rates. In this way, the system is comprehensive in that it does not only cover the part of the network that is “visible” via synchrophasors, but also the part that is only “visible” through the SCADA measurements. Visualization needs associated with the use of DNSE results are fulfilled through suitably enhanced Real Time Dynamics Monitoring System (RTDMS), with the enhancements implemented by EPG. This project had the following goals in mind: To advance the deployment of commercial grade DNSE software application that relies on synchrophasor and SCADA data ; Apply DNSE at other utilities, to address a generic and fundamental need for “clean” operational data for synchrophasor applications; Provide means for “live” estimated data access by control system operators; Enhance potential for situational awareness through full system operational model coverage; Sub-second execution rate of the Direct Non-iterative State Estimator, eventually at a near-phasor data rate execution speed, i.e. < 0.1 sec. Anticipated benefits from this projects are: Enhanced reliability and improvements in the economic efficiency of bulk power system planning and operations; Providing “clean” data to other synchrophasor applications; Enhancement of situational awareness by providing the full operational model updated at near synchrophasor rate; A production-grade software tool that incorporate synchrophasor and SCADA data; Provides a basis for development of next generation monitoring and control applications, based on both SCADA and PMU data. Quanta Technology (QT) team worked in collaboration with Electric Power Group (EPG) whose team has enhanced its commercial Real Time Dynamics Monitoring System (RTDMS) to accommodate the requirements posed by DNSE application. EPG also provided its ePDC and Model-less Data Conditioning (PDVC) software for integration with DNSE+. QT developed the system requirements for DNSE; developed system architecture and defined interfaces between internal DNSE components. The core DNSE algorithm with all surrounding interfaces was named DNSE+. Since the DNSE development was done in a simulated system environment, QT used its PMU simulator that was enhanced during this project for development and factory acceptance testing (FAT). SCADA data in this stage was simulated by commercial PSS/e software. The output of DNSE are estimates of System states in C37.118-2 format, sent to RTDMS for further processing and display. As the number of these states is large, it was necessary to expand the C37.111-2 standard to accommodate large data sets. This enhancement was implemented in RTDMS. The demonstration of pre-production DNSE technology was done at NYPA using streaming field data from NYPA PMUs and from its RTUs through their SCADA system. NYPA provided ICCP interface as well as Common Information Model (CIM). The relevance of the DNSE+ application is that it provides state estimation of the power system based on hybrid set of data, consisting of both available PMU data and SCADA measurements. As this is a direct, non-iterative method of calculation of the system states, if does not suffer from convergence issues which is potential problem for conventional state estimators. Also, it can take any available PMU measurements, so it does not need to have a high percentage of PMU coverage needed in the case of Linear State Estimator. As the DNSE calculates synchrophasors of the system states (both phase and absolute value) as sub-second rate, this application can provide a basis for development of next generation of applications based both on SCADA and PMU data.

24 POWER TRANSMISSION AND DISTRIBUTION↗

MPI-Bash: Parallel scripting right from the Bourne-Again Shell (Bash)

Parallel, scientific applications running on massively parallel supercomputers commonly produce large numbers of large data files. While parallel filesystems improve the performance of file generation, postprocessing activities such as archiving and compressing the data are often run sequentially (and therefore slowly), squandering the supercomputer's vast performance. Consequently, data that take hours to generate may take days to postprocess. Writing and debugging parallel postprocessing programs is often too time-consuming to be worthwhile, especially for one-off postprocessing operations. What the community needs is a simple mechanism for cobbling together scripts that distribute postprocessing work across a supercomputer's many nodes. MPI-Bash attempts to supply that mechanism by augmenting the GNU project's command shell, Bash, with support for the Message Passing Interface (MPI) and, if available, the Libcircle library for work distribution and load balancing. MPI-Bash makes it easy to parallelize bash scripts that run a set of Linux commands independently over a large number of input files. Because MPI-Bash includes various MPI functions for data transfer and synchronization, it is not limited to embarrassingly parallel workloads but can incorporate phased operations (i.e., all workers must finish operation X before any worker is allowed to begin operation Y).

97 MATHEMATICS AND COMPUTING↗

Randomized low-rank decompositions of nuclear three-body interactions

First-principles simulations of many-fermion systems are commonly limited by the computational requirements of processing large data objects. As a remedy, we propose the use of low-rank approximations of three-body interactions, which are the dominant such limitation in nuclear physics. We introduce a randomized decomposition technique to handle the excessively large matrix dimensions and study the sensitivity of low-rank properties to interaction details. The developed low-rank three-nucleon interactions are benchmarked in ab initio simulations of few- and many-body systems. Exploiting low-rank properties provides a promising route to extend the microscopic description of atomic nuclei to large systems where storage requirements exceed the computational capacities of the most advanced high-performance computing facilities.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

Robust Online Sequential RVFLNs for Data Modeling of Dynamic Time-Varying Systems with Application of an Ironmaking Blast Furnace

In a world where the increasing complexity of modern industrial processes brings difficulties for accurate mathematical modeling, taking advantage of data has become an efficient solution to complex dynamic process modeling issue. In this paper, we develop a novel robust online sequential version of random vector functional-link networks (RVFLNs) for data-driven modeling of dynamic time-varying system and applied it in a blast furnace (BF) ironmaking process. First, to overcome the time-varying dynamics of process and to enable the RVFLNs to learn online with avoiding data saturation, an improved online sequential version of RVFLNs (OS-RFVLNs) is first presented by online sequential learning with forgetting factor. This improved OS-RVFLNs algorithm is not only suitable for the real-time and large data transfer situation, but also can adjust the sensitivity of the algorithm to different samples with the help of the introduced forgetting factor. Second, since the output weights of the improved OS-RVFLNs as well as other RVFLNs algorithms are obtained by the least squares approach, a robustness problem may occur when the training dataset is contaminated with various outliers. To solve this problem, a Cauchy distribution weighted M-estimator is introduced to improve the robustness of the improved OS- RVFLNs. For this proposed robust OS-RVFLNs (R-OS- RVFLNs), since the weights of different outlier data are properly determined by the Cauchy distribution function, their corresponding contribution on modeling can be properly distinguished. Thus robust and better modeling results can be achieved. Experiments using actual industrial data of BF ironmaking process and comparative studies have demonstrated that the proposed method produces a better estimation accuracy and stronger robustness than other methods.

Blast furnace (BF), Modelling, Dynamic systems↗

Patient-specific radiation dose and cancer risk estimation in CT: Part II. Application to patients

Purpose: Current methods for estimating and reporting radiation dose from CT examinations are largely patient-generic; the body size and hence dose variation from patient to patient is not reflected. Furthermore, the current protocol designs rely on dose as a surrogate for the risk of cancer incidence, neglecting the strong dependence of risk on age and gender. The purpose of this study was to develop a method for estimating patient-specific radiation dose and cancer risk from CT examinations. Methods: The study included two patients (a 5-week-old female patient and a 12-year-old male patient), who underwent 64-slice CT examinations (LightSpeed VCT, GE Healthcare) of the chest, abdomen, and pelvis at our institution in 2006. For each patient, a nonuniform rational B-spine (NURBS) based full-body computer model was created based on the patient's clinical CT data. Large organs and structures inside the image volume were individually segmented and modeled. Other organs were created by transforming an existing adult male or female full-body computer model (developed from visible human data) to match the framework defined by the segmented organs, referencing the organ volume and anthropometry data in ICRP Publication 89. A Monte Carlo program previously developed and validated for dose simulation on the LightSpeed VCT scanner was used to estimate patient-specific organ dose, from which effective dose and risks of cancer incidence were derived. Patient-specific organ dose and effective dose were compared with patient-generic CT dose quantities in current clinical use: the volume-weighted CT dose index (CTDI{sub vol}) and the effective dose derived from the dose-length product (DLP). Results: The effective dose for the CT examination of the newborn patient (5.7 mSv) was higher but comparable to that for the CT examination of the teenager patient (4.9 mSv) due to the size-based clinical CT protocols at our institution, which employ lower scan techniques for smaller patients. However, the overall risk of cancer incidence attributable to the CT examination was much higher for the newborn (2.4 in 1000) than for the teenager (0.7 in 1000). For the two pediatric-aged patients in our study, CTDI{sub vol} underestimated dose to large organs in the scan coverage by 30%-48%. The effective dose derived from DLP using published conversion coefficients differed from that calculated using patient-specific organ dose values by -57% to 13%, when the tissue weighting factors of ICRP 60 were used, and by -63% to 28%, when the tissue weighting factors of ICRP 103 were used. Conclusions: It is possible to estimate patient-specific radiation dose and cancer risk from CT examinations by combining a validated Monte Carlo program with patient-specific anatomical models that are derived from the patients' clinical CT data and supplemented by transformed models of reference adults. With the construction of a large library of patient-specific computer models encompassing patients of all ages and weight percentiles, dose and risk can be estimated for any patient prior to or after a CT examination. Such information may aid in decisions for image utilization and can further guide the design and optimization of CT technologies and scan protocols.

60 APPLIED LIFE SCIENCES↗

A Codesign Framework for Online Data Analysis and Reduction at the Exascale

In this paper we discuss our design of a toolset for automating performance studies of composed HPC applications that perform online data reduction and analysis. We describe Cheetah, a new framework for performing parametric studies on coupled applications. Cheetah facilitates understanding the impact of various factors such as process placement, synchronicity of algorithms, and storage vs. compute requirements for online analysis of large data. Ultimately, we aim to create a catalog of performance results that can help scientists understand tradeos when designing next-generation simulations that make use of online processing techniques. We illustrate the design choices of Cheetah by using a reaction-diusion simulation (Gray-Scott) paired with an analysis application to demonstrate initial results of fine-grained process placement on Summit, a pre-exascale supercomputer at Oak Ridge National Laboratory

Cheetah↗

Evaluating lossy data compression on climate simulation data within a large ensemble

High-resolution Earth system model simulations generate enormous data volumes, and retaining the data from these simulations often strains institutional storage resources. Further, these exceedingly large storage requirements negatively impact science objectives, for example, by forcing reductions in data output frequency, simulation length, or ensemble size. To lessen data volumes from the Community Earth System Model (CESM), we advocate the use of lossy data compression techniques. While lossy data compression does not exactly preserve the original data (as lossless compression does), lossy techniques have an advantage in terms of smaller storage requirements. To preserve the integrity of the scientific simulation data, the effects of lossy data compression on the original data should, at a minimum, not be statistically distinguishable from the natural variability of the climate system, and previous preliminary work with data from CESM has shown this goal to be attainable. However, to ultimately convince climate scientists that it is acceptable to use lossy data compression, we provide climate scientists with access to publicly available climate data that have undergone lossy data compression. In particular, we report on the results of a lossy data compression experiment with output from the CESM Large Ensemble (CESM-LE) Community Project, in which we challenge climate scientists to examine features of the data relevant to their interests, and attempt to identify which of the ensemble members have been compressed and reconstructed. We find that while detecting distinguishing features is certainly possible, the compression effects noticeable in these features are often unimportant or disappear in post-processing analyses. In addition, we perform several analyses that directly compare the original data to the reconstructed data to investigate the preservation, or lack thereof, of specific features critical to climate science. Overall, we conclude that applying lossy data compression to climate simulation data is both advantageous in terms of data reduction and generally acceptable in terms of effects on scientific results.

54 ENVIRONMENTAL SCIENCES↗

Mass Dependence of Galaxy–Halo Alignment in LOWZ and CMASS

We measure the galaxy-ellipticity (GI) correlations for the Sloan Digital Sky Survey Data Release 12 LOWZ and CMASS samples with the shape measurements from the DESI Legacy Imaging Surveys. We model the GI correlations in an N-body simulation with our recent accurate stellar–halo mass relation from the Photometric object Around Cosmic webs (PAC) method. The large data set and our accurate modeling turns out an accurate measurement of the alignment angle between central galaxies and their host halos. We find that the alignment of central elliptical galaxies with their host halos increases monotonically with galaxy stellar mass or host halo mass, which can be well described by a power law for the massive galaxies. We also find that central elliptical galaxies are more aligned with their host halos in LOWZ than in CMASS, which might indicate an evolution of galaxy–halo alignment, though future studies are needed to verify this is not induced by the sample selections. In contrast, central disk galaxies are aligned with their host halos about 10 times more weakly in the GI correlation. These results have important implications for intrinsic alignment (IA) correction in weak lensing studies, IA cosmology, and theory of massive galaxy formation.

79 ASTRONOMY AND ASTROPHYSICS↗

DEIMoS: An Open-Source Tool for Processing High-Dimensional Mass Spectrometry Data

We present DEIMoS: Data Extraction for Integrated Multidimensional Spectrometry, a Python application programming interface and command-line tool for high-dimensional mass spectrometry data analysis workflows, offering ease of development and access to efficient algorithmic implementations. Functionality includes feature detection, feature alignment, collision cross section calibration, isotope detection, and MS/MS spectral deconvolution, with the output comprising detected features aligned across study samples and characterized by mass, CCS, tandem mass spectra, and isotopic signature. Notably, DEIMoS operates on N-dimensional data, largely agnostic to acquisition instrumentation: algorithm implementations utilize all dimensions simultaneously to (i) offer greater separation between features, improving detection sensitivity, (ii) increase alignment/feature matching confidence among datasets, and (iii) mitigate convolution artifacts in tandem mass spectra. We demonstrate DEIMoS with LC-IMS-MS/MS data, demonstrating the advantages of a multidimensional approach in each data processing step.

Colby, Sean M↗

Super instruction architecture of petascale electronic structure software: the story

Theoretical methods in chemistry lead to algorithms for the computation of electronic energies and other properties of electronic wave functions that require large numbers of floating point operations and involve large data sets. Thus, computational chemists are very interested in using massively parallel computer systems and in particular the new petascale systems. In this paper we discuss a new programming paradigm that was developed at the Quantum Theory Project to construct electronic structure software that can scale to large numbers of cores of the order of 100,000 and beyond to solve problems in materials engineering relevant to the problems facing society today.

Chemistry↗

Data Preservation in High Energy Physics

Data from high-energy physics (HEP) experiments are collected with significant financial and human effort and are mostly unique. At the same time, HEP has no coherent strategy for data preservation and re-use. An inter-experimental Study Group on HEP data preservation and long-term analysis was convened at the end of 2008 and held two workshops, at DESY (January 2009) and SLAC (May 2009). This document is an intermediate report to the International Committee for Future Accelerators (ICFA) of the reflections of this Study Group. Large data sets accumulated during many years of detector operation at particle accelerators are the heritage of experimental HEP. These data sets offer unique opportunities for future scientific studies, sometimes long after the shut-down of the actual experiments: new theoretical input; new experimental results and analysis techniques; the quest for high-sensitivity combined analyses; the necessity of cross checks. In many cases, HEP data sets are unique; they cannot and most likely will not be superseded by data from newer generations of experiments. Once lost, or in an unusable state, HEP data samples cannot be reasonably recovered. The cost of conserving this heritage through a collaborative, target-oriented long-term data preservation program would be small, compared to the costs of past experimental projects or to the efforts to re-do experiments. However, this cost is not negligible, especially for collaborations close or past their end-date. The preservation of HEP data would provide today's collaborations with a secure way to complete their data analysis and enable them to seize new scientific opportunities in the coming years. The HEP community will benefit from preserved data samples through reanalysis, combination, education and outreach. Funding agencies would receive more scientific return, and a positive image, from their initial investment leading to the production and the first analysis of preserved data.

43 PARTICLE ACCELERATORS↗

Finite-time correlations boost large voltage angle fluctuations in electric power grids

Abstract Decarbonization in the energy sector has been accompanied by an increased penetration of new renewable energy sources in electric power systems. Such sources differ from traditional productions in that, first, they induce larger, undispatchable fluctuations in power generation and second, they lack inertia. Recent measurements have indeed reported long, non-Gaussian tails in the distribution of local voltage frequency data. Large frequency deviations may induce grid instabilities, leading in worst-case scenarios to cascading failures and large-scale blackouts. In this article, we investigate how correlated noise disturbances, characterized by the cumulants of their distribution, propagate through meshed, high-voltage power grids. For a single source of fluctuations, we show that long noise correlation times boost non-Gaussian voltage angle fluctuations so that they propagate similarly to Gaussian fluctuations over the entire network. However, they vanish faster, over short distances if the noise fluctuates rapidly. We furthermore demonstrate that a Berry–Esseen theorem leads to the vanishing of non-Gaussianities as the number of uncorrelated noise sources increases. Our predictions are corroborated by numerical simulations on realistic models of power grids.

24 POWER TRANSMISSION AND DISTRIBUTION↗

AUTONOMOUS GAUSSIAN DECOMPOSITION

We present a new algorithm, named Autonomous Gaussian Decomposition (AGD), for automatically decomposing spectra into Gaussian components. AGD uses derivative spectroscopy and machine learning to provide optimized guesses for the number of Gaussian components in the data, and also their locations, widths, and amplitudes. We test AGD and find that it produces results comparable to human-derived solutions on 21 cm absorption spectra from the 21 cm SPectral line Observations of Neutral Gas with the EVLA (21-SPONGE) survey. We use AGD with Monte Carlo methods to derive the H i line completeness as a function of peak optical depth and velocity width for the 21-SPONGE data, and also show that the results of AGD are stable against varying observational noise intensity. The autonomy and computational efficiency of the method over traditional manual Gaussian fits allow for truly unbiased comparisons between observations and simulations, and for the ability to scale up and interpret the very large data volumes from the upcoming Square Kilometer Array and pathfinder telescopes.

79 ASTRONOMY AND ASTROPHYSICS↗

Optimizing inference of segmentation on high-resolution images in MLExchange

MLExchange is a machine learning (ML) operations platform providing web user-interfaces (UIs) for data visualization and analysis pipelines at synchrotron facilities. Among these UIs is the segmentation app which helps synchrotron users utilize ML algorithms to automatically segment high-resolution scientific images with minimal manual annotation effort. In this work, we share code optimizations that significantly speed up the segmentation inference workflow of large data in short time. By optimizing the sequence of CPU-GPU data transfers and introducing CPU parallelization to key operations, we improve the per-device, per-image frame computational efficiency and observe close to 3×$$\times$$ speedup over the original segmentation inference workflow run time when utilizing a single GPU. Further adaptations enabling multi-GPU inference yield more than 40×$$\times$$ speedup with 100 GPUs compared to the optimized single GPU inference workflow. This acceleration of the segmentation inference workflow will provide MLExchange users with easy access to segmentation results with little wait time.

Lu, Shizhao↗

Materials informatics: From the atomic-level to the continuum

We report that in recent years materials informatics, which is the application of data science to problems in materials science and engineering, has emerged as a powerful tool for materials discovery and design. This relatively new field is already having a significant impact on the interpretation of data for a variety of materials systems, including those used in thermoelectrics, ferroelectrics, battery anodes and cathodes, hydrogen storage materials, polymer dielectrics, etc. Its practitioners employ the methods of multivariate statistics and machine learning in conjunction with standard computational tools (e.g., density-functional theory) to, for example, visualize and dimensionally reduce large data sets, identify patterns in hyperspectral data, parse microstructural images of polycrystals, characterize vortex structures in ferroelectrics, design batteries and, in general, establish correlations to extract important physics and infer structure-property-processing relationships. In this Overview, we critically examine the role of informatics in several important materials subfields, highlighting significant contributions to date and identifying known shortcomings. We specifically focus attention on the difference between the correlative approach of classical data science and the causative approach of physical sciences. From this perspective, we also outline some potential opportunities and challenges for informatics in the materials realm in this era of big data.

36 MATERIALS SCIENCE↗

Reverse Engineering Adverse Outcome Pathways

The toxicological effects of many stressors are mediated through unknown, or poorly characterized, mechanisms of action. We describe the application of reverse engineering complex interaction networks from high dimensional omics data (gene, protein, metabolic, signaling) to characterize adverse outcome pathways (AOPs) for chemicals that disrupt the hypothalamus-pituitary-gonadal endocrine axis in fathead minnows. Gene expression changes in fathead minnow ovaries in response to 7 different chemicals, over different times, doses, and in vivo versus in vitro conditions were captured in a large data set of 868 arrays. We examined potential AOPs of the antiandrogen flutamide using two mutual information theory methods, ARACNE and CLR to infer gene regulatory networks and potential adverse outcome pathways. Representative networks from these studies were used to predict a network path from stressor to adverse outcome as a candidate AOP. The relationship of individual chemicals to an adverse outcome can be determined by following perturbations through the network in response to chemical treatment leading to the nodes associated with the adverse outcome. Identification of candidate pathways allows for formation of testable hypotheses about key biologic processes, biomarkers or alternative endpoints, which could be used to monitor an adverse outcome pathway. Finally, we identify the unique challenges facing the application of this approach in ecotoxicology, and attempt to provide a road map for the utilization of these tools. Key Words: mechanism of action, toxicology, microarray, network inference

59 BASIC BIOLOGICAL SCIENCES↗

gPhoton: THE GALEX PHOTON DATA ARCHIVE

gPhoton is a new database product and software package that enables analysis of GALEX ultraviolet data at the photon level. The project’s stand-alone, pure-Python calibration pipeline reproduces the functionality of the original mission pipeline to reduce raw spacecraft data to lists of time-tagged, sky-projected photons, which are then hosted in a publicly available database by the Mikulski Archive at Space Telescope. This database contains approximately 130 terabytes of data describing approximately 1.1 trillion sky-projected events with a timestamp resolution of five milliseconds. A handful of Python and command-line modules serve as a front end to interact with the database and to generate calibrated light curves and images from the photon-level data at user-defined temporal and spatial scales. The gPhoton software and source code are in active development and publicly available under a permissive license. We describe the motivation, design, and implementation of the calibration pipeline, database, and tools, with emphasis on divergence from prior work, as well as challenges created by the large data volume. We summarize the astrometric and photometric performance of gPhoton relative to the original mission pipeline. For a brief example of short time-domain science capabilities enabled by gPhoton, we show new flares from the known M-dwarf flare star CR Draconis. The gPhoton software has permanent object identifiers with the ASCL (ascl:1603.004) and DOI (doi:10.17909/T9CC7G). This paper describes the software as of version v1.27.2.

79 ASTRONOMY AND ASTROPHYSICS↗