Search NASA⌕ Search

SEARCH · Search NASA

Results for “large data”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 559 records · Page 31

The Cabauw Intercomparison Campaign for Nitrogen Dioxide Measuring Instruments (CINDI): Design, Execution, and Early Results

From June to July 2009 more than thirty different in-situ and remote sensing instruments from all over the world participated in the Cabauw Intercomparison campaign for Nitrogen Dioxide measuring Instruments (CINDI). The campaign took place at KNMI's Cabauw Experimental Site for Atmospheric Research (CESAR) in the Netherlands. Its main objectives were to determine the accuracy of state-ofthe- art ground-based measurement techniques for the detection of atmospheric nitrogen dioxide (both in-situ and remote sensing), and to investigate their usability in satellite data validation. The expected outcomes are recommendations regarding the operation and calibration of such instruments, retrieval settings, and observation strategies for the use in ground-based networks for air quality monitoring and satellite data validation. Twenty-four optical spectrometers participated in the campaign, of which twenty-one had the capability to scan different elevation angles consecutively, the so-called Multi-axis DOAS systems, thereby collecting vertical profile information, in particular for nitrogen dioxide and aerosol. Various in-situ samplers and lidar instruments simultaneously characterized the variability of atmospheric trace gases and the physical properties of aerosol particles. A large data set of continuous measurements of these atmospheric constituents has been collected under various meteorological conditions and air pollution levels. Together with the permanent measurement capability at the CESAR site characterizing the meteorological state of the atmosphere, the CINDI campaign provided a comprehensive observational data set of atmospheric constituents in a highly polluted region of the world during summertime. First detailed comparisons performed with the CINDI data show that slant column measurements of NO2, O4 and HCHO with MAX-DOAS agree within 5 to 15%, vertical profiles of NO2 derived from several independent instruments agree within 25% of one another, and MAX-DOAS aerosol optical thickness agrees within 20-30% with AERONET data. For the in-situ NO2 instrument using a molybdenum converter, a bias was found as large as 5 ppbv during day time, when compared to the other in-situ instruments using photolytic converters.

Atmospheric composition↗

Satellite/rocket ozone comparisons at Natal, Brazil

Comparisons are presented of satellite, rocket, and balloon ozone profiles near Natal, Brazil (5.9 deg S, 35.2 deg W). The low variability of stratospheric ozone at Natal during March and April of 1985 has allowed intercomparisons of reasonably large data sets, rather than a small number of paired satellite/in situ comparisons. There are sharp differences between the profile from the SBUV instrument on Nimbus 7 and the in situ measurements. These results support the conclusions of the NASA Ozone Trends Panel that there is an instrumental cause for the very large changes in upper stratospheric ozone seen by SBUV. Along with other comparisons, these results are being used in a reassessment of the SBUV instrument and its data reduction procedures. The agreement between the ozone profiles from the SAGE II instrument on the ERBS satellite and the rocket values is excellent over the full range of comparisons. Both SAGE II and ROCOZ-A must convert from altitude to pressure for intercomparisons with SME and with SBUV-type instruments. The conversion between pressure and altitude is as important as the ozone measurements, especially in the upper stratosphere where the scale height for ozone is approximately half that for pressure.

Barnes, Robert A.↗

GPS FOM Chimney Analysis using Generalized Extreme Value Distribution

Many a time an objective of a statistical analysis is to estimate a limit value like 3-sigma 95% confidence upper limit from a data sample. The generalized Extreme Value Distribution method can be profitably employed in many situations for such an estimate. . .. It is well known that according to the Central Limit theorem the mean value of a large data set is normally distributed irrespective of the distribution of the data from which the mean value is derived. In a somewhat similar fashion it is observed that many times the extreme value of a data set has a distribution that can be formulated with a Generalized Distribution. In space shuttle entry with 3-string GPS navigation the Figure Of Merit (FOM) value gives a measure of GPS navigated state accuracy. A GPS navigated state with FOM of 6 or higher is deemed unacceptable and is said to form a FOM 6 or higher chimney. A FOM chimney is a period of time during which the FOM value stays higher than 5. A longer period of FOM of value 6 or higher causes navigated state to accumulate more error for a lack of state update. For an acceptable landing it is imperative that the state error remains low and hence at low altitude during entry GPS data of FOM greater than 5 must not last more than 138 seconds. I To test the GPS performAnce many entry test cases were simulated at the Avionics Development Laboratory. Only high value FoM chimneys are consequential. The extreme value statistical technique is applied to analyze high value FOM chimneys. The Maximum likelihood method is used to determine parameters that characterize the GEV distribution, and then the limit value statistics are estimated.

Ott, Rick↗

Modeling adsorption properties of structurally deformed metal–organic frameworks using structure–property map

Structural deformation and collapse in metal-organic frameworks (MOFs) can lead to loss of long-range order, making it a challenge to model these amorphous materials using conventional computational methods. In this work, we show that a structure–property map consisting of simulated data for crystalline MOFs can be used to indirectly obtain adsorption properties of structurally deformed MOFs. The structure–property map (with dimensions such as Henry coefficient, heat of adsorption, and pore volume) was constructed using a large data set of over 12000 crystalline MOFs from molecular simulations. By mapping the experimental data points of deformed SNU-200, MOF-5, and Ni-MOF-74 onto this structure–property map, we show that the experimentally deformed MOFs share similar adsorption properties with their nearest neighbor crystalline structures. Furthermore, once the nearest neighbor crystalline MOFs for a deformed MOF are selected from a structure–property map at a specific condition, then the adsorption properties of these MOFs can be successfully transformed onto the degraded MOFs, leading to a new way to obtain properties of materials whose structural information is lost.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Physical processes in comets

Post-Halley comets are known to be irregular objects with most nucleus activity very localized and with the dust coma capable of fragmentation and apparently being a source of gas. Older, 1-D strategies which assume steady isotropic outflow of material can give poor time-and-space averaged results, at best. With 2-D data, images through interference filters, it is hoped that: dust structures can be seen that give evidence of the proper geometry for data reduction; gradients along the axes of symmetry and evidence of fragmentation can be studied; and that evidence for gas abundance gradients associated with the dust can be found. High quality data from brighter comets can then be used to suggest improved data reduction procedures for fainter ones. To obtain such data, large image-quality interference filters were procured for use with a charge coupled device (CCD) camera at Lick Observatory, where the scale of the 1 m Nickel reflector is ideal for brighter comets. Whenever possible, data is taken simultaneously with other telescopes and equipment, especially spectroscopy at the Lick 3 m or infrared photometry at the Infrared Telescope Facility (IRTF) on Mauna Kea.

Newburn, Ray L., Jr.↗

A View from ORNL: Scientific Data Research Opportunities in the Big Data Age

One of the core issues across computer and computational science today is adapting to, managing, and learning from the influx of "Big Data". In the commercial space, this problem has led to a huge investment in new technologies and capabilities that are well adapted to dealing with the sorts of human-generated logs, videos, texts, and other large-data artifacts that are processed and resulted in an explosion of useful platforms and languages (Hadoop, Spark, Pandas, etc.). However, translating this work from the enterprise space to the computational science and HPC community has proven somewhat difficult, in part because of some of the fundamental differences in type and scale of data and timescales surrounding its generation and use. We describe a forward-looking research and development plan which centers around the concept of making Input/Output (I/O) intelligent for users in the scientific community, whether they are accessing scalable storage or performing in situ workflow tasks. Much of our work is based on our experience with the Adaptable I/O System (ADIOS 1.X), and our next generation version of the software ADIOS 2.X [1].

Klasky, Scott↗

Pre-commercial Demonstration of Direct Non-iterative State Estimator for Operational Use of Synchrophasor Management Systems

The objective of this project was to demonstrate functionality and performance of a Direct Non-iterative State Estimator (DNSE) integrated with NYPA’s Energy Management System (EMS) and with enhanced Real Time Dynamics Monitoring System (RTDMS) synchrophasor platform from Electric Power Group (EPG). DNSE is designed to overcome a major obstacle to operational use of Synchro-Phasor Management Systems (SPMS) by providing to synchrophasor management systems (SPMS) applications a consistent and a complete synchrophasor data foundation in the same way that a traditional EMS State Estimator (SE) provides to EMS applications. Specifically, DNSE is designed to use synchrophasor measurements collected by a central PDC, Supervisory Control and Data Acquisition (SCADA) measurements, and Energy Management System (EMS) network model, to obtain the complete state of the utility’s operating model at rates that are close to the synchrophasor data rates. In this way, the system is comprehensive in that it does not only cover the part of the network that is “visible” via synchrophasors, but also the part that is only “visible” through the SCADA measurements. Visualization needs associated with the use of DNSE results are fulfilled through suitably enhanced Real Time Dynamics Monitoring System (RTDMS), with the enhancements implemented by EPG. This project had the following goals in mind: To advance the deployment of commercial grade DNSE software application that relies on synchrophasor and SCADA data ; Apply DNSE at other utilities, to address a generic and fundamental need for “clean” operational data for synchrophasor applications; Provide means for “live” estimated data access by control system operators; Enhance potential for situational awareness through full system operational model coverage; Sub-second execution rate of the Direct Non-iterative State Estimator, eventually at a near-phasor data rate execution speed, i.e. < 0.1 sec. Anticipated benefits from this projects are: Enhanced reliability and improvements in the economic efficiency of bulk power system planning and operations; Providing “clean” data to other synchrophasor applications; Enhancement of situational awareness by providing the full operational model updated at near synchrophasor rate; A production-grade software tool that incorporate synchrophasor and SCADA data; Provides a basis for development of next generation monitoring and control applications, based on both SCADA and PMU data. Quanta Technology (QT) team worked in collaboration with Electric Power Group (EPG) whose team has enhanced its commercial Real Time Dynamics Monitoring System (RTDMS) to accommodate the requirements posed by DNSE application. EPG also provided its ePDC and Model-less Data Conditioning (PDVC) software for integration with DNSE+. QT developed the system requirements for DNSE; developed system architecture and defined interfaces between internal DNSE components. The core DNSE algorithm with all surrounding interfaces was named DNSE+. Since the DNSE development was done in a simulated system environment, QT used its PMU simulator that was enhanced during this project for development and factory acceptance testing (FAT). SCADA data in this stage was simulated by commercial PSS/e software. The output of DNSE are estimates of System states in C37.118-2 format, sent to RTDMS for further processing and display. As the number of these states is large, it was necessary to expand the C37.111-2 standard to accommodate large data sets. This enhancement was implemented in RTDMS. The demonstration of pre-production DNSE technology was done at NYPA using streaming field data from NYPA PMUs and from its RTUs through their SCADA system. NYPA provided ICCP interface as well as Common Information Model (CIM). The relevance of the DNSE+ application is that it provides state estimation of the power system based on hybrid set of data, consisting of both available PMU data and SCADA measurements. As this is a direct, non-iterative method of calculation of the system states, if does not suffer from convergence issues which is potential problem for conventional state estimators. Also, it can take any available PMU measurements, so it does not need to have a high percentage of PMU coverage needed in the case of Linear State Estimator. As the DNSE calculates synchrophasors of the system states (both phase and absolute value) as sub-second rate, this application can provide a basis for development of next generation of applications based both on SCADA and PMU data.

24 POWER TRANSMISSION AND DISTRIBUTION↗

MPI-Bash: Parallel scripting right from the Bourne-Again Shell (Bash)

Parallel, scientific applications running on massively parallel supercomputers commonly produce large numbers of large data files. While parallel filesystems improve the performance of file generation, postprocessing activities such as archiving and compressing the data are often run sequentially (and therefore slowly), squandering the supercomputer's vast performance. Consequently, data that take hours to generate may take days to postprocess. Writing and debugging parallel postprocessing programs is often too time-consuming to be worthwhile, especially for one-off postprocessing operations. What the community needs is a simple mechanism for cobbling together scripts that distribute postprocessing work across a supercomputer's many nodes. MPI-Bash attempts to supply that mechanism by augmenting the GNU project's command shell, Bash, with support for the Message Passing Interface (MPI) and, if available, the Libcircle library for work distribution and load balancing. MPI-Bash makes it easy to parallelize bash scripts that run a set of Linux commands independently over a large number of input files. Because MPI-Bash includes various MPI functions for data transfer and synchronization, it is not limited to embarrassingly parallel workloads but can incorporate phased operations (i.e., all workers must finish operation X before any worker is allowed to begin operation Y).

97 MATHEMATICS AND COMPUTING↗

Randomized low-rank decompositions of nuclear three-body interactions

First-principles simulations of many-fermion systems are commonly limited by the computational requirements of processing large data objects. As a remedy, we propose the use of low-rank approximations of three-body interactions, which are the dominant such limitation in nuclear physics. We introduce a randomized decomposition technique to handle the excessively large matrix dimensions and study the sensitivity of low-rank properties to interaction details. The developed low-rank three-nucleon interactions are benchmarked in ab initio simulations of few- and many-body systems. Exploiting low-rank properties provides a promising route to extend the microscopic description of atomic nuclei to large systems where storage requirements exceed the computational capacities of the most advanced high-performance computing facilities.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

Robust Online Sequential RVFLNs for Data Modeling of Dynamic Time-Varying Systems with Application of an Ironmaking Blast Furnace

In a world where the increasing complexity of modern industrial processes brings difficulties for accurate mathematical modeling, taking advantage of data has become an efficient solution to complex dynamic process modeling issue. In this paper, we develop a novel robust online sequential version of random vector functional-link networks (RVFLNs) for data-driven modeling of dynamic time-varying system and applied it in a blast furnace (BF) ironmaking process. First, to overcome the time-varying dynamics of process and to enable the RVFLNs to learn online with avoiding data saturation, an improved online sequential version of RVFLNs (OS-RFVLNs) is first presented by online sequential learning with forgetting factor. This improved OS-RVFLNs algorithm is not only suitable for the real-time and large data transfer situation, but also can adjust the sensitivity of the algorithm to different samples with the help of the introduced forgetting factor. Second, since the output weights of the improved OS-RVFLNs as well as other RVFLNs algorithms are obtained by the least squares approach, a robustness problem may occur when the training dataset is contaminated with various outliers. To solve this problem, a Cauchy distribution weighted M-estimator is introduced to improve the robustness of the improved OS- RVFLNs. For this proposed robust OS-RVFLNs (R-OS- RVFLNs), since the weights of different outlier data are properly determined by the Cauchy distribution function, their corresponding contribution on modeling can be properly distinguished. Thus robust and better modeling results can be achieved. Experiments using actual industrial data of BF ironmaking process and comparative studies have demonstrated that the proposed method produces a better estimation accuracy and stronger robustness than other methods.

Blast furnace (BF), Modelling, Dynamic systems↗

Distribution of atmospheric aerosols and CO2 lidar backscatter simulation

Development of a Global Wind Measurement Satellite System (WINDSAT) (NOAA, 1981) requires a knowledge of the global characteristics of the free tropospheric and lower stratospheric aerosol. In particular, there is a need to document the behavior of the aerosol backscattering function, at CO2 laser wavelengths, beta sub CO2, as a function of space and time. There is, however, a relative lack of data for the free troposphere, particularly over the remoter regions of the globe, as compared with that for the boundary layer and the stratosphere. Moreover, because of variations in concentration that occur as a function of space and time, large data sets are required to obtain meaningful averages. A recent study by Kent et al. (1985) uses three distinct tropospheric aerosol data sets in order to obtain an improved global model of the general aerosol characteristics, including variation of beta sub CO2 with latitude, season, and altitude. The more important findings are summarized.

Kent, G. S.↗

University of Washington Participation in SAFARI-2000

This report presents a summary, of the participation in the 6-week field study in southern Africa. During the field study there were flown 119.02 research hours (31 research flights). In these flights the researchers obtained many unique data sets for evaluating the effects of biomass burning and other sources of particles and gases on the climate of southern Africa, and obtained simultaneous measurements with NASA ER-2 and Terra overflights. They also analyzed portions of the large data sets acquired in SAFARI-2000. They attended several SAFARI-2000 workshops, national and international conferences to present SAFARI-2000 results.

Hobbs, Peter V.↗

NASA Global Satellite and Model Data Products and Services for Tropical Cyclone Research

The lack of observations over vast tropical oceans is a major challenge for tropical cyclone research. Satellite observations and model reanalysis data play an important role in filling these- gaps. Established in the mid-1980's, the Goddard Earth Sciences Data and Information Services Center (GES DISC), as one of the 12 NASA data centers, archives and distributes data from several Earth science disciplines such as precipitation, atmospheric dynamics, atmospheric composition, hydrology, including well-known NASA satellite missions (e.g. TRMM, GPM) and model assimilation projects (MERRA-2). Acquiring datasets suitable for tropical cyclone research in a large data archive is a challenge for many, especially for those who are not familiar with satellite or model data. Over the years, the GES DISC has developed user-friendly data services. For example, Giovanni is an online visualization and analysis tool, allowing users to visualize and analyze over 2000 satellite- and model-based variables with a Web browser, without downloading data and software. In this chapter, we will describe data and services at the GES DISC with emphasis on tropical cyclone research. We will also present two case studies and discuss future plans.

Liu, Zhong↗

Interactive access and management for four-dimensional environmental data sets using McIDAS

This grant has fundamentally changed the way that meteorologists look at the output of their atmospheric models, through the development and wide distribution of the Vis5D system. The Vis5D system is also gaining acceptance among oceanographers and atmospheric chemists. Vis5D gives these scientists an interactive three-dimensional movie of their very large data sets that they can use to understand physical mechanisms and to trace problems to their sources. This grant has also helped to define the future direction of scientific visualization through the development of the VisAD system and its lattice data model. The VisAD system can be used to interactively steer and visualize scientific computations. A key element of this capability is the flexibility of the system's data model to adapt to a wide variety of scientific data, including the integration of several forms of scientific metadata.

Hibbard, William L.↗

Patient-specific radiation dose and cancer risk estimation in CT: Part II. Application to patients

Purpose: Current methods for estimating and reporting radiation dose from CT examinations are largely patient-generic; the body size and hence dose variation from patient to patient is not reflected. Furthermore, the current protocol designs rely on dose as a surrogate for the risk of cancer incidence, neglecting the strong dependence of risk on age and gender. The purpose of this study was to develop a method for estimating patient-specific radiation dose and cancer risk from CT examinations. Methods: The study included two patients (a 5-week-old female patient and a 12-year-old male patient), who underwent 64-slice CT examinations (LightSpeed VCT, GE Healthcare) of the chest, abdomen, and pelvis at our institution in 2006. For each patient, a nonuniform rational B-spine (NURBS) based full-body computer model was created based on the patient's clinical CT data. Large organs and structures inside the image volume were individually segmented and modeled. Other organs were created by transforming an existing adult male or female full-body computer model (developed from visible human data) to match the framework defined by the segmented organs, referencing the organ volume and anthropometry data in ICRP Publication 89. A Monte Carlo program previously developed and validated for dose simulation on the LightSpeed VCT scanner was used to estimate patient-specific organ dose, from which effective dose and risks of cancer incidence were derived. Patient-specific organ dose and effective dose were compared with patient-generic CT dose quantities in current clinical use: the volume-weighted CT dose index (CTDI{sub vol}) and the effective dose derived from the dose-length product (DLP). Results: The effective dose for the CT examination of the newborn patient (5.7 mSv) was higher but comparable to that for the CT examination of the teenager patient (4.9 mSv) due to the size-based clinical CT protocols at our institution, which employ lower scan techniques for smaller patients. However, the overall risk of cancer incidence attributable to the CT examination was much higher for the newborn (2.4 in 1000) than for the teenager (0.7 in 1000). For the two pediatric-aged patients in our study, CTDI{sub vol} underestimated dose to large organs in the scan coverage by 30%-48%. The effective dose derived from DLP using published conversion coefficients differed from that calculated using patient-specific organ dose values by -57% to 13%, when the tissue weighting factors of ICRP 60 were used, and by -63% to 28%, when the tissue weighting factors of ICRP 103 were used. Conclusions: It is possible to estimate patient-specific radiation dose and cancer risk from CT examinations by combining a validated Monte Carlo program with patient-specific anatomical models that are derived from the patients' clinical CT data and supplemented by transformed models of reference adults. With the construction of a large library of patient-specific computer models encompassing patients of all ages and weight percentiles, dose and risk can be estimated for any patient prior to or after a CT examination. Such information may aid in decisions for image utilization and can further guide the design and optimization of CT technologies and scan protocols.

60 APPLIED LIFE SCIENCES↗

Seasonal to Decadal-Scale Variability in Satellite Ocean Color and Sea Surface Temperature for the California Current System

Support for this project was used to develop satellite ocean color and temperature indices (SOCTI) for the California Current System (CCS) using the historic record of CZCS West Coast Time Series (WCTS), OCTS, WiFS and AVHRR SST. The ocean color satellite data have been evaluated in relation to CalCOFI data sets for chlorophyll (CZCS) and ocean spectral reflectance and chlorophyll OCTS and SeaWiFS. New algorithms for the three missions have been implemented based on in-water algorithm data sets, or in the case of CZCS, by comparing retrieved pigments with ship-based observations. New algorithms for absorption coefficients, diffuse attenuation coefficients and primary production have also been evaluated. Satellite retrievals are being evaluated based on our large data set of pigments and optics from CalCOFI.

Mitchell, B. Greg↗

A Codesign Framework for Online Data Analysis and Reduction at the Exascale

In this paper we discuss our design of a toolset for automating performance studies of composed HPC applications that perform online data reduction and analysis. We describe Cheetah, a new framework for performing parametric studies on coupled applications. Cheetah facilitates understanding the impact of various factors such as process placement, synchronicity of algorithms, and storage vs. compute requirements for online analysis of large data. Ultimately, we aim to create a catalog of performance results that can help scientists understand tradeos when designing next-generation simulations that make use of online processing techniques. We illustrate the design choices of Cheetah by using a reaction-diusion simulation (Gray-Scott) paired with an analysis application to demonstrate initial results of fine-grained process placement on Summit, a pre-exascale supercomputer at Oak Ridge National Laboratory

Cheetah↗

Assimilation of scalar versus horizontal gradient information from the VAS into a mesoscale model

Comparisons are made between analyses and forecasts which incorporate VAS geopotential data as either scalar or horizontal gradient information for a case study on the AVE/VAS day of Mar. 6, 1982. On this day, incorporating the VAS information in analysis as a variational constraint on horizontal geopotential gradients significantly mitigated the effects of large data biases which made VAS assimilation by standard scalar methods very difficult. A subsequent forecast made from the gradient assimilation was superior to one made from the standard analysis and of comparable quality in geopotentials to a control forecast from synoptic data. Most impact was noted in the forecasts of vertical motion and precipitation in the gradient vs this control simulation.

Diak, George↗