Search NASA⌕ Search

SEARCH · Search NASA

Results for “large data”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,135 records · Page 63

The Medical Science DMZ

Objective We describe use cases and an institutional reference architecture for maintaining high-capacity, data-intensive network flows (e.g., 10, 40, 100 Gbpsþ) in a scientific, medical context while still adhering to security and privacy laws and regulations. Materials and Methods High-end networking, packet filter firewalls, network intrusion detection systems. Results We describe a “Medical Science DMZ” concept as an option for secure, high-volume transport of large, sensitive data sets between research institutions over national research networks. Discussion The exponentially increasing amounts of “omics” data, the rapid increase of high-quality imaging, and other rapidly growing clinical data sets have resulted in the rise of biomedical research “big data.” The storage, analysis, and network resources required to process these data and integrate them into patient diagnoses and treatments have grown to scales that strain the capabilities of academic health centers. Some data are not generated locally and cannot be sustained locally, and shared data repositories such as those provided by the National Library of Medicine, the National Cancer Institute, and international partners such as the European Bioinformatics Institute are rapidly growing. The ability to store and compute using these data must therefore be addressed by a combination of local, national, and industry resources that exchange large data sets. Maintaining data-intensive flows that comply with HIPAA and other regulations presents a new challenge for biomedical research. Recognizing this, we describe a strategy that marries performance and security by borrowing from and redefining the concept of a “Science DMZ”—a framework that is used in physical sciences and engineering research to manage high-capacity data flows. Conclusion By implementing a Medical Science DMZ architecture, biomedical researchers can leverage the scale provided by high-performance computer and cloud storage facilities and national high-speed research networks while preserving privacy and meeting regulatory requirements.

97 MATHEMATICS AND COMPUTING↗

CONSTRAINTS ON THE ORIGIN OF COSMIC RAYS ABOVE 10{sup 18} eV FROM LARGE-SCALE ANISOTROPY SEARCHES IN DATA OF THE PIERRE AUGER OBSERVATORY

A thorough search for large-scale anisotropies in the distribution of arrival directions of cosmic rays detected above 10{sup 18} eV at the Pierre Auger Observatory is reported. For the first time, these large-scale anisotropy searches are performed as a function of both the right ascension and the declination and expressed in terms of dipole and quadrupole moments. Within the systematic uncertainties, no significant deviation from isotropy is revealed. Upper limits on dipole and quadrupole amplitudes are derived under the hypothesis that any cosmic ray anisotropy is dominated by such moments in this energy range. These upper limits provide constraints on the production of cosmic rays above 10{sup 18} eV, since they allow us to challenge an origin from stationary galactic sources densely distributed in the galactic disk and emitting predominantly light particles in all directions.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

An open-access database and analysis tool for perovskite solar cells based on the FAIR data principles

Large datasets are now ubiquitous as technology enables higher-throughput experiments, but rarely can a research field truly benefit from the research data generated due to inconsistent formatting, undocumented storage or improper dissemination. Here we extract all the meaningful device data from peer-reviewed papers on metal-halide perovskite solar cells published so far and make them available in a database. We collect data from over 42,400 photovoltaic devices with up to 100 parameters per device. We then develop open-source and accessible procedures to analyse the data, providing examples of insights that can be gleaned from the analysis of a large dataset. The database, graphics and analysis tools are made available to the community and will continue to evolve as an open-source initiative. This approach of extensively capturing the progress of an entire field, including sorting, interactive exploration and graphical representation of the data, will be applicable to many fields in materials science, engineering and biosciences.

14 SOLAR ENERGY↗

Green Routing Fuel Saving Opportunity Assessment: A Case Study on California Large-Scale Real-World Travel Data

New technologies, such as connected and automated vehicles, have attracted more and more researchers for improving the energy efficiency and environmental impact of current transportation systems. The green routing strategy instructs a vehicle to select the most fuel-efficient route before the vehicle departs. It benefits the current transportation system with fuel saving opportunity through identifying the greenest route. This paper introduces an evaluation framework for estimating benefits of green routing based on large-scale, real-world travel data. The framework has the capability to quantify fuel savings by estimating the fuel consumption of actual routes and comparing to routes procured by navigation systems. A route-based fuel consumption estimation model, considering road traffic conditions, functional class, and road grade is proposed and used in the framework. An experiment using a large-scale data set from the California Household Travel Survey global positioning system trajectory data base indicates that 31% of actual routes have fuel savings potential with a cumulative estimated fuel savings of 12%.

33 ADVANCED PROPULSION SYSTEMS↗

Optical rogue waves in telecommunication data streams

Large broadening of short optical pulses due to fiber dispersion leads to a strong overlap in information data streams resulting in statistical deviations of the local power from its average. We present a theoretical analysis of rare events of high-intensity fluctuations--optical freak waves--that occur in fiber communication links using bit-overlapping transmission. Although the nature of the large fluctuations examined here is completely linear, as compared to commonly studied freak waves generated by nonlinear effects, the considered deviations inherit from rogue waves the key features of practical interest--random appearance of localized high-intensity pulses. We use the term ''rogue wave'' in an unusual context mostly to attract attention to both the possibility of purely linear statistical generation of huge amplitude waves and to the fact that in optics the occurrence of such pulses might be observable even with the standard Gaussian or even rarer-than-Gaussian statistics, without imposing the condition of an increased probability of extreme value events.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Tools for Analysis and Visualization of Large Time-Varying CFD Data Sets

In the second year, we continued to built upon and improve our scanline-based direct volume renderer that we developed in the first year of this grant. This extremely general rendering approach can handle regular or irregular grids, including overlapping multiple grids, and polygon mesh surfaces. It runs in parallel on multi-processors. It can also be used in conjunction with a k-d tree hierarchy, where approximate models and error terms are stored in the nodes of the tree, and approximate fast renderings can be created. We have extended our software to handle time-varying data where the data changes but the grid does not. We are now working on extending it to handle more general time-varying data. We have also developed a new extension of our direct volume renderer that uses automatic decimation of the 3D grid, as opposed to an explicit hierarchy. We explored this alternative approach as being more appropriate for very large data sets, where the extra expense of a tree may be unacceptable. We also describe a new approach to direct volume rendering using hardware 3D textures and incorporates lighting effects. Volume rendering using hardware 3D textures is extremely fast, and machines capable of using this technique are becoming more moderately priced. While this technique, at present, is limited to use with regular grids, we are pursuing possible algorithms extending the approach to more general grid types. We have also begun to explore a new method for determining the accuracy of approximate models based on the light field method described at ACM SIGGRAPH '96. In our initial implementation, we automatically image the volume from 32 equi-distant positions on the surface of an enclosing tessellated sphere. We then calculate differences between these images under different conditions of volume approximation or decimation. We are studying whether this will give a quantitative measure of the effects of approximation. We have created new tools for exploring the differences between images produced by various rendering methods. Images created by our software can be stored in the SGI RGB format. Our idtools software reads in pair of images and compares them using various metrics. The differences of the images using the RGB, HSV, and HSL color models can be calculated and shown. We can also calculate the auto-correlation function and the Fourier transform of the image and image differences. We will explore how these image differences compare in order to find useful metrics for quantifying the success of various visualization approaches. In general, progress was consistent with our research plan for the second year of the grant.

Wilhelms, Jane↗

Machine learning prediction of incidence of Alzheimer’s disease using large-scale administrative health data

Nationwide population-based cohort provides a new opportunity to build an automated risk prediction model based on individuals’ history of health and healthcare beyond existing risk prediction models. We tested the possibility of machine learning models to predict future incidence of Alzheimer’s disease (AD) using large-scale administrative health data. From the Korean National Health Insurance Service database between 2002 and 2010, we obtained de-identified health data in elders above 65 years (N = 40,736) containing 4,894 unique clinical features including ICD-10 codes, medication codes, laboratory values, history of personal and family illness and socio-demographics. To define incident AD we considered two operational definitions: “definite AD” with diagnostic codes and dementia medication (n = 614) and “probable AD” with only diagnosis (n = 2026). We trained and validated random forest, support vector machine and logistic regression to predict incident AD in 1, 2, 3, and 4 subsequent years. For predicting future incidence of AD in balanced samples (bootstrapping), the machine learning models showed reasonable performance in 1-year prediction with AUC of 0.775 and 0.759, based on “definite AD” and “probable AD” outcomes, respectively; in 2-year, 0.730 and 0.693; in 3-year, 0.677 and 0.644; in 4-year, 0.725 and 0.683. The results were similar when the entire (unbalanced) samples were used. Important clinical features selected in logistic regression included hemoglobin level, age and urine protein level. This study may shed a light on the utility of the data-driven machine learning model based on large-scale administrative health data in AD risk prediction, which may enable better selection of individuals at risk for AD in clinical trials or early detection in clinical settings.

97 MATHEMATICS AND COMPUTING↗

Investigation into Cloud Computing for More Robust Automated Bulk Image Geoprocessing

Geospatial resource assessments frequently require timely geospatial data processing that involves large multivariate remote sensing data sets. In particular, for disasters, response requires rapid access to large data volumes, substantial storage space and high performance processing capability. The processing and distribution of this data into usable information products requires a processing pipeline that can efficiently manage the required storage, computing utilities, and data handling requirements. In recent years, with the availability of cloud computing technology, cloud processing platforms have made available a powerful new computing infrastructure resource that can meet this need. To assess the utility of this resource, this project investigates cloud computing platforms for bulk, automated geoprocessing capabilities with respect to data handling and application development requirements. This presentation is of work being conducted by Applied Sciences Program Office at NASA-Stennis Space Center. A prototypical set of image manipulation and transformation processes that incorporate sample Unmanned Airborne System data were developed to create value-added products and tested for implementation on the "cloud". This project outlines the steps involved in creating and testing of open source software developed process code on a local prototype platform, and then transitioning this code with associated environment requirements into an analogous, but memory and processor enhanced cloud platform. A data processing cloud was used to store both standard digital camera panchromatic and multi-band image data, which were subsequently subjected to standard image processing functions such as NDVI (Normalized Difference Vegetation Index), NDMI (Normalized Difference Moisture Index), band stacking, reprojection, and other similar type data processes. Cloud infrastructure service providers were evaluated by taking these locally tested processing functions, and then applying them to a given cloud-enabled infrastructure to assesses and compare environment setup options and enabled technologies. This project reviews findings that were observed when cloud platforms were evaluated for bulk geoprocessing capabilities based on data handling and application development requirements.

Brown, Richard B.↗

Evaluation of the international reference ionosphere with the large AE-C and DE2 data bases

Empirical models such as the International Reference Ionosphere (IRI) are synthesized from large data bases. They can be viewed as analytical tools to facilitate accessing information stored in the data banks. However, in establishing the models, one has to apply smoothing and averaging procedures that in effect reduce the original information content. This study evaluates the agreement between the data base and the model at two opposite extremes of time resolution. Electron densities and temperatures in the altitude range of 300 to 400 km predicted by the IRI and measured by the AE-C and DE 2 satellites on the level of individual orbits as well as on the level of mission averages are compared. Whereas the averages show excellent agreement, the comparison for individual measurements indicates the limitations of empirical models.

Bilitza, D.↗

Spin Structure of the Nucleon - Status and Recent Results

After the initial discovery of the so-called "spin crisis in the parton model" in the 1980s, a large set of polarization data in deep inelastic lepton-nucleon scattering was collected at labs like SLAC, DESY and CERN. More recently, new high precision data at large $x$ and in the resonance region have come from experiments at Jefferson Lab. These data, in combination with the earlier ones, allow us to study in detail the polarized parton densities, the $Q^2$ dependence of various moments of spin structure functions, the duality between deep inelastic and resonance data, and the nucleon structure in the valence quark region. Together with complementary data from HERMES, RHIC and COMPASS, we can put new limits on the flavor decomposition and the gluon contribution to the nucleon spin. In this report, we provide an overview of our present knowledge of the nucleon spin structure and give an outlook on future experiments. We focus in particular on the spin structure functions $g_

43 PARTICLE ACCELERATORS↗

A Large-Grain Mapping Approach for Multiprocessor Systems Through Data Flow Model Ph.D. Thesis

A large-grain level mapping method is presented of numerical oriented applications onto multiprocessor systems. The method is based on the large-grain data flow representation of the input application and it assumes a general interconnection topology of the multiprocessor system. The large-grain data flow model was used because such representation best exhibits inherited parallelism in many important applications, e.g., CFD models based on partial differential equations can be presented in large-grain data flow format, very effectively. A generalized interconnection topology of the multiprocessor architecture is considered, including such architectural issues as interprocessor communication cost, with the aim to identify the 'best matching' between the application and the multiprocessor structure. The objective is to minimize the total execution time of the input algorithm running on the target system. The mapping strategy consists of the following: (1) large-grain data flow graph generation from the input application using compilation techniques; (2) data flow graph partitioning into basic computation blocks; and (3) physical mapping onto the target multiprocessor using a priority allocation scheme for the computation blocks.

Kim, Hwa-Soo↗

An interactive environment for the analysis of large Earth observation and model data sets

Envision is an interactive environment that provides researchers in the earth sciences convenient ways to manage, browse, and visualize large observed or model data sets. Its main features are support for the netCDF and HDF file formats, an easy to use X/Motif user interface, a client-server configuration, and portability to many UNIX workstations. The Envision package also provides new ways to view and change metadata in a set of data files. It permits a scientist to conveniently and efficiently manage large data sets consisting of many data files. It also provides links to popular visualization tools so that data can be quickly browsed. Envision is a public domain package, freely available to the scientific community. Envision software (binaries and source code) and documentation can be obtained from either of these servers: ftp://vista.atmos.uiuc.edu/pub/envision/ and ftp://csrp.tamu.edu/pub/envision/. Detailed descriptions of Envision capabilities and operations can be found in the User's Guide and Reference Manuals distributed with Envision software.

Bowman, Kenneth P.↗

Scalable Metadata Management for a Large Multi-Source Seismic Data Repository

In this work, we implemented the key metadata management components of a scalable seismic data ingestion framework to address limitations in our existing system, and to position it for anticipated growth in volume and complexity. We began the effort with an assessment of open source data flow tools from the Hadoop ecosystem. We then began the construction of a layered architecture that is specifically designed to address many of the scalability and data quality issues we experience with our current pipeline. This included implementing basic functionality in each of the layers, such as establishing a data lake, designing a unified metadata schema, tracking provenance, and calculating data quality metrics. Our original intent was to test and validate the new ingestion framework with data from a large-scale field deployment in a temporary network. This delivered somewhat unsatisfying results, since the new system immediately identified fatal flaws in the data relatively early in the pipeline. Although this is a correct result it did not allow us to sufficiently exercise the whole framework. We then widened our scope to process all available metadata from over a dozen online seismic data sources to further test the implementation and validate the design. This experiment also uncovered a higher than expected frequency of certain types of metadata issues that challenged us to further tune our data management strategy to handle them. Our result from this project is a greatly improved understanding of real world data issues, a validated design, and prototype implementations of major components of an eventual production framework. This successfully forms the basis of future development for the Geophysical Monitoring Program data pipeline, which is a critical asset supporting multiple programs. It also positions us very well to deliver valuable metadata management expertise to our sponsors, and has already resulted in an NNSA Office of Defense Nuclear Nonproliferation commitment to a multi-year project for follow-on work.

58 GEOSCIENCES↗

Modern sampled-data control theory: Design of the Large Space Telescope

This investigation was undertaken to determine the effect of varying the sampling period on the dynamic response of the sampled-data Large Space Telescope (LST) system. A range of sampling periods was recommended based on the criterion that self-sustained oscillations are to be avoided in the LST system. The step responses of the LST system were then investigated when various sampling periods are used. For small sampling periods, the dynamic behavior of the sampled-data system is very similar to that of the continuous-data system. When T is large (but less than 0.25 sec) the overshoot of the step response of the sampled-data becomes greater. However, the dynamic behavior of the sampled-data system may be improved by redesigning the controller. It appears that a sampling period as high as 0.1 second is feasible for the LST system. However, it should be noted that the conclusions are obtained with the existing system model. Other practical considerations such as noise, coupling effects and quantization errors, may restrict the sampling period to a lower value.

Kuo, B. C.↗

DOECGF 2010 Site Report

The Data group provides data analysis and visualization support to its customers. This consists primarily of the development and support of VisIt, a data analysis and visualization tool. Support ranges from answering questions about the tool, providing classes on how to use the tool, and performing data analysis and visualization for customers. The Information Management and Graphics Group supports and develops tools that enhance our ability to access, display, and understand large, complex data sets. Activities include applying visualization software for large scale data exploration; running video production labs on two networks; supporting graphics libraries and tools for end users; maintaining PowerWalls and assorted other displays; and developing software for searching and managing scientific data. Researchers in the Center for Applied Scientific Computing (CASC) work on various projects including the development of visualization techniques for large scale data exploration that are funded by the ASC program, among others. The researchers also have LDRD projects and collaborations with other lab researchers, academia, and industry. The IMG group is located in the Terascale Simulation Facility, home to Dawn, Atlas, BGL, and others, which includes both classified and unclassified visualization theaters, a visualization computer floor and deployment workshop, and video production labs. We continued to provide the traditional graphics group consulting and video production support. We maintained five PowerWalls and many other displays. We deployed a 576-node Opteron/IB cluster with 72 TB of memory providing a visualization production server on our classified network. We continue to support a 128-node Opteron/IB cluster providing a visualization production server for our unclassified systems and an older 256-node Opteron/IB cluster for the classified systems, as well as several smaller clusters to drive the PowerWalls. The visualization production systems includes NFS servers to provide dedicated storage for data analysis and visualization. The ASC projects have delivered new versions of visualization and scientific data management tools to end users and continue to refine them. VisIt had 4 releases during the past year, ending with VisIt 2.0. We released version 2.4 of Hopper, a Java application for managing and transferring files. This release included a graphical disk usage view which works on all types of connections and an aggregated copy feature for quickly transferring massive datasets quickly and efficiently to HPSS. We continue to use and develop Blockbuster and Telepath. Both the VisIt and IMG teams were engaged in a variety of movie production efforts during the past year in addition to the development tasks.

97 MATHEMATICS AND COMPUTING↗

Divide and Conquer (DC) BLAST: fast and easy BLAST execution within HPC environments

Bioinformatics is currently faced with very large-scale data sets that lead to computational jobs, especially sequence similarity searches, that can take absurdly long times to run. For example, the National Center for Biotechnology Information (NCBI) Basic Local Alignment Search Tool (BLAST and BLAST+) suite, which is by far the most widely used tool for rapid similarity searching among nucleic acid or amino acid sequences, is highly central processing unit (CPU) intensive. While the BLAST suite of programs perform searches very rapidly, they have the potential to be accelerated. In recent years, distributed computing environments have become more widely accessible and used due to the increasing availability of high-performance computing (HPC) systems. Therefore, simple solutions for data parallelization are needed to expedite BLAST and other sequence analysis tools. However, existing software for parallel sequence similarity searches often requires extensive computational experience and skill on the part of the user. In order to accelerate BLAST and other sequence analysis tools, Divide and Conquer BLAST (DCBLAST) was developed to perform NCBI BLAST searches within a cluster, grid, or HPC environment by using a query sequence distribution approach. Scaling from one (1) to 256 CPU cores resulted in significant improvements in processing speed. Thus, DCBLAST dramatically accelerates the execution of BLAST searches using a simple, accessible, robust, and parallel approach. DCBLAST works across multiple nodes automatically and it overcomes the speed limitation of single-node BLAST programs. DCBLAST can be used on any HPC system, can take advantage of hundreds of nodes, and has no output limitations. Thus, this freely available tool simplifies distributed computation pipelines to facilitate the rapid discovery of sequence similarities between very large data sets.

59 BASIC BIOLOGICAL SCIENCES↗

THE HIGH-DENSITY IONIZED GAS IN THE CENTRAL PARSEC OF THE GALAXY

We report a study of the H30{alpha} line emission at 1.3 mm from the region around Sgr A* made with the Submillimeter Array at a resolution of 2'' over a field of 60'' (2 pc) and a velocity range of -360 to +345 km s{sup -1}. This field encompasses most of the Galactic center's 'minispiral'. With an isothermal homogeneous H II model, we determined the physical conditions of the ionized gas at specific locations in the Northern and Eastern Arms from the H30{alpha} line data along with Very Large Array data from the H92{alpha} line at 3.6 cm and from the radio continuum emission at 1.3 cm. The typical electron density and kinetic temperature in the minispiral arms are 3-21x10{sup 4} cm{sup -3} and 5000-13,000 K, respectively. The H30{alpha} and H92{alpha} line profiles are broadened due to the large velocity shear within and along the beam produced by dynamical motions in the strong gravitational field near Sgr A*. We constructed a three-dimensional model of the minispiral using the orbital parameters derived under the assumptions that the gas flows are in Keplerian motion. The gas in the Eastern Arm appears to collide with the Northern Arm flow in the 'Bar' region, which is located 0.1-0.2 pc south of and behind Sgr A*. Finally, a total Lyman continuum flux of 3 x 10{sup 50} photons s{sup -1} is inferred from the assumption that the gas is photoionized and the ionizing photons for the high-density gas in the minispiral arms are from external sources, which is equivalent to {approx}250 O9-type zero-age-main-sequence stars.

79 ASTRONOMY AND ASTROPHYSICS↗

Development and performance evaluation of a novel γ-γ coincidence analysis software

Processing of large amounts of data from a γ-γ coincidence system is a practical challenge in trace-level radionuclide measurements. Given the benefits and growing number of multidimensional gamma spectrometers worldwide, this is a major restriction in wide-spread applicability of coincidence techniques. This study demonstrates a novel γ-γ coincidence analysis software developed at Pacific Northwest National Laboratory, USA. The software utilizes advanced search techniques and Python libraries to readily identify coincidence signatures in the large data. Further, using both synthetic and experimental data, the study uncovers various strategies/libraries employed during software’s development. This work aims to provide a reference document in handling large data routinely encountered in trace-level measurements.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗