Search NASA⌕ Search

SEARCH · Search NASA

Results for “large data”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,423 records · Page 79

DETERMINATION OF THE POINT-SPREAD FUNCTION FOR THE FERMI LARGE AREA TELESCOPE FROM ON-ORBIT DATA AND LIMITS ON PAIR HALOS OF ACTIVE GALACTIC NUCLEI

The Large Area Telescope (LAT) on the Fermi Gamma-ray Space Telescope is a pair-conversion telescope designed to detect photons with energies from Almost-Equal-To 20 MeV to >300 GeV. The pre-launch response functions of the LAT were determined through extensive Monte Carlo simulations and beam tests. The point-spread function (PSF) characterizing the angular distribution of reconstructed photons as a function of energy and geometry in the detector is determined here from two years of on-orbit data by examining the distributions of {gamma} rays from pulsars and active galactic nuclei (AGNs). Above 3 GeV, the PSF is found to be broader than the pre-launch PSF. We checked for dependence of the PSF on the class of {gamma}-ray source and observation epoch and found none. We also investigated several possible spatial models for pair-halo emission around BL Lac AGNs. We found no evidence for a component with spatial extension larger than the PSF and set upper limits on the amplitude of halo emission in stacked images of low- and high-redshift BL Lac AGNs and the TeV blazars 1ES0229+200 and 1ES0347-121.

79 ASTRONOMY AND ASTROPHYSICS↗

Using Temporal Information from Human Mobility Data to Detect Anchor Points

Spatiotemporal mobility data are available in massive quantities, but large quantities of data typically include fewer variables or data fields. Often, the only available fields are User ID, Longitude, Latitude, Timestamp (ULLT). This raises an important question: how much can we infer about human mobility patterns using only these four fields? With ULLT data, we do not know individuals' socioeconomic status information or when they are visiting their anchor points (AP) or locations (such as homes, places of employment, or schools), and it is a modern challenge to use this data to infer these characteristics. When detecting anchor locations with limited input information, verification and validation (VV) are significant challenges. This paper addresses the problem of identifying individuals' anchor locations using only temporal information from spatiotemporal datasets with limited attributes. Our approach does not explicitly use latitude and longitude during analysis. Locationbased information is only employed in the preprocessing stage to identify periods of movement (trips) and stops (dwelling). Beyond this step, all analysis is based on temporal patterns. In theory, if stops and dwell times could be detected through alternative means, our method could function entirely without location-based input. We demonstrate this methodology on the 2017 National Household Travel Survey (NHTS) data, because it includes a carefully designed and collected time use survey with representative sampling and labeled ground truth. The high-quality survey data allows us to test the accuracy of our methods because NHTS contains intended place labels and agent/user characteristics. We have also applied our validated AP identification algorithm on very large-scale GPS based trajectory data for Patterns-of-Life (PoL) assessment and other applications, but due to space limit that could not be presented here.

McBride, Liz [ORNL] (ORCID:0000000286925869)↗

Data-link alternatives for the NASA pilot data systems

There are several preliminary, or pilot, efforts for developing data systems for supporting NASA science research. The Pilot Data Systems consist of the Pilot Land Data System (PLDS), the Pilot Ocean Data System (PODS), the Pilot Climate Data System (PCDS), and the Pilot Planetary Data System (PPDS). The PLDS is likely to have the greatest communication needs of the various pilots. Communications issues and requirements are examined in the context of the mature PLDS as it may exist by 1990. The PLDS is seen as a distributed processing system linking resources at a number of NASA research centers and outside universities. Large image data sets, including LANDSAT scenes, are a major data type to be moved along the PLDS communications network. The unusually large size of these data files requires examining new technologies that may allow efficient and affordable communication at rates of up to 60 megabits per second. Potentially useful developments include portable satellite ground stations, and time division multiple access for sharing high-speed satellite links. Further pooling of communications for the various data systems into a single network would reduce costs and may make economic justification of access to the required data rates possible.

Likens, W. C.↗

Big Data Seismology

The discipline of seismology is based on observations of ground motion that are inherently undersampled in space and time. Our basic understanding of earthquake processes and our ability to resolve 4D Earth structure are fundamentally limited by data volume. Currently, Big Data Seismology is an emergent revolution involving the use of large, data-dense inquiries that is providing new opportunities to make fundamental advances in these areas. This article reviews recent scientific advances enabled by Big Data Seismology through the context of three major drivers: the development of new data-dense sensor systems, improvements in computing, and the development of new types of techniques and algorithms. Each driver is explored in the context of both global and exploration seismology, alongside collaborative opportunities that combine the features of long-duration data collections (common to global seismology) with dense networks of sensors (common to exploration seismology). The review explores some of the unique challenges and opportunities that Big Data Seismology presents, drawing on parallels from other fields facing similar issues. Finally, recent scientific findings enabled by dense seismic data sets are discussed, and we assess the opportunities for significant advances made possible with Big Data Seismology. This review is designed to be a primer for seismologists who are interested in getting up-to-speed with how the Big Data revolution is advancing the field of seismology.

58 GEOSCIENCES↗

Streaming Compression of Scientific Data via Weak-SINDy

Here, in this paper, a streaming weak-SINDy algorithm is developed specifically for compressing streaming scientific data. The production of scientific data, either via simulation or experiments, is undergoing a stage of exponential growth, which makes data compression important and often necessary for storing and utilizing large scientific data sets. As opposed to classical “offline” compression algorithms that perform compression on a readily available data set, streaming compression algorithms compress data “online” while the data generated from simulation or experiments is still flowing through the system. This feature makes streaming compression algorithms well suited for scientific data compression, where storing the full data set offline is often infeasible. This work proposes a new streaming compression algorithm, streaming weak-SINDy, which takes advantage of the underlying data characteristics during compression. The streaming weak-SINDy algorithm constructs feature matrices and target vectors in the online stage via a streaming integration method in a memory efficient manner. The feature matrices and target vectors are then used in the offline stage to build a model through a regression process that aims to recover equations that govern the evolution of the data. For compressing high-dimensional streaming data, we adopt a streaming proper orthogonal decomposition (POD) process to reduce the data dimension and then use the streaming weak-SINDy algorithm to compress the temporal data of the POD expansion. We propose modifications to the streaming weak-SINDy algorithm to accommodate the dynamically updated POD basis. By combining the built model from the streaming weak-SINDy algorithm and a small amount of data samples, the full data flow could be reconstructed accurately at a low memory cost, as shown in the numerical tests.

97 MATHEMATICS AND COMPUTING↗

Analyzing Mass Spectrometry Imaging Data of 13C-Labeled Phospholipids in Camelina sativa and Thlaspi arvense (Pennycress) Embryos

The combination of 13C-isotopic labeling and mass spectrometry imaging (MSI) offers an approach to analyze metabolic flux in situ. However, combining isotopic labeling and MSI presents technical challenges ranging from sample preparation, label incorporation, data collection, and analysis. Isotopic labeling and MSI individually create large, complex data sets, and this is compounded when both methods are combined. Therefore, analyzing isotopically labeled MSI data requires streamlined procedures to support biologically meaningful interpretations. Using currently available software and techniques, here we describe a workflow to analyze 13C-labeled isotopologues of the membrane lipid and storage oil lipid intermediate―phosphatidylcholine (PC). Our results with embryos of the oilseed crops, Camelina sativa and Thlaspi arvense (pennycress), demonstrated greater 13C-isotopic labeling in the cotyledons of developing embryos compared with the embryonic axis. Greater isotopic enrichment in PC molecular species with more saturated and longer chain fatty acids suggest different flux patterns related to fatty acid desaturation and elongation pathways. The ability to evaluate MSI data of isotopically labeled plant embryos will facilitate the potential to investigate spatial aspects of metabolic flux in situ.

metabolomics↗

Comprehensive Assessments of RNA-seq by the SEQC Consortium: FDA-Led Efforts Advance Precision Medicine

Studies on gene expression in response to therapy have led to the discovery of pharmacogenomics biomarkers and advances in precision medicine. Whole transcriptome sequencing (RNA-seq) is an emerging tool for profiling gene expression and has received wide adoption in the biomedical research community. However, its value in regulatory decision making requires rigorous assessment and consensus between various stakeholders, including the research community, regulatory agencies, and industry. The FDA-led SEquencing Quality Control (SEQC) consortium has made considerable progress in this direction, and is the subject of this review. Specifically, three RNA-seq platforms (Illumina HiSeq, Life Technologies SOLiD, and Roche 454) were extensively evaluated at multiple sites to assess cross-site and cross-platform reproducibility. The results demonstrated that relative gene expression measurements were consistently comparable across labs and platforms, but not so for the measurement of absolute expression levels. As part of the quality evaluation several studies were included to evaluate the utility of RNA-seq in clinical settings and safety assessment. The neuroblastoma study profiled tumor samples from 498 pediatric neuroblastoma patients by both microarray and RNA-seq. RNA-seq offers more utilities than microarray in determining the transcriptomic characteristics of cancer. However, RNA-seq and microarray-based models were comparable in clinical endpoint prediction, even when including additional features unique to RNA-seq beyond gene expression. The toxicogenomics study compared microarray and RNA-seq profiles of the liver samples from rats exposed to 27 different chemicals representing multiple toxicity modes of action. Cross-platform concordance was dependent on chemical treatment and transcript abundance. Though both RNA-seq and microarray are suitable for developing gene expression based predictive models with comparable prediction performance, RNA-seq offers advantages over microarray in profiling genes with low expression. The rat BodyMap study provided a comprehensive rat transcriptomic body map by performing RNA-Seq on 320 samples from 11 organs in either sex of juvenile, adolescent, adult and aged Fischer 344 rats. Lastly, the transferability study demonstrated that signature genes of predictive models are reciprocally transferable between microarray and RNA-seq data for model development using a comprehensive approach with two large clinical data sets. This result suggests continued usefulness of legacy microarray data in the coming RNA-seq era. In conclusion, the SEQC project enhances our understanding of RNA-seq and provides valuable guidelines for RNA-seq based clinical application and safety evaluation to advance precision medicine.

59 BASIC BIOLOGICAL SCIENCES↗

Remote sensing of wetlands

The concept of using remote sensing to inventory wetlands and the related topics of proper inventory design and data collection are discussed. The material presented shows that aerial photography is the form of remote sensing from which the greatest amount of wetlands information can be derived. For extensive, general-purpose wetlands inventories, however, the use of LANDSAT data may be more cost-effective. Airborne multispectral scanners and radar are, in the main, too expensive to use - unless the information that these sensors alone can gather remotely is absolutely required. Multistage sampling employing space and high altitude remote sensing data in the initial stages appears to be an efficient survey strategy for gathering non-point specific wetlands inventory data over large areas. The operational role of remote sensing insupplying inventory data for application to several typical wetlands management problems is illustrated by summary descriptions of past ERIM projects.

Roller, N. E. G.↗

Future trends in image processing software and hardware

JPL image processing applications are examined, considering future trends in fields such as planetary exploration, electronics, astronomy, computers, and Landsat. Attention is given to adaptive search and interrogation of large image data bases, the display of multispectral imagery recorded in many spectral channels, merging data acquired by a variety of sensors, and developing custom large scale integrated chips for high speed intelligent image processing user stations and future pipeline production processors.

Green, W. B.↗

Status of MTP Data Analysis for TCSP

Topics covered include: a) MTP temperature calibration and data analysis; b) Background for interpreting MTP data; c) Large amplitude temperature structure; d) Gravity waves (GWs) in MTP data; and e) Subsidence over hurricanes.

hurricanes↗

Assessment of High Resolution Commercial Satellite Geolocation Accuracy

Commercial companies such as Maxar, PlanetScope, and BlackSky have launched many satellites with revisit times ranging from two hours to one day and have built large archives of high resolution (1-3 m) Earth observing data. The high temporal resolution and global coverage of these satellites makes commercial satellite images ideal for scientists studying rapidly changing processes such as flooding and fires. A key step in assessing changes in these images is co-registration of various images to each other. High geolocation accuracy (at most 0.5 pixels of offset) allows for easy co-registration of images across different times and sensors. Here, we assess the latest PlanetScope imagery and evaluate it for geolocation accuracy. For our assessment, we compare the target image to a reference image with known geolocation accuracy (WorldView imagery) and determine the offset between these images by shifting them to maximize their Pearson Cross-Correlation (PCC) value. Offsets required to maximize the PCC give the geolocation accuracy of the target image relative to the reference image. Previously, our global assessment of Planet data revealed large variability in geolocation accuracy from one continent to another. The measured root mean squared errors (RMSEs) range from 5.4 m in North America to 14.9 m in Africa. We have updated this previous assessment to include both their newest SuperDove series as well as a more robust assessment of the temporal stability of PlanetScope's geolocation accuracy.

Alana G. Semple↗

2013 Progress Report -- DOE Joint Genome Institute

In October 2012, we introduced a 10-Year Strategic Vision [http://bit.ly/JGI-Vision] for the Institute. A central focus of this Strategic Vision is to bridge the gap between sequenced genomes and an understanding of biological functions at the organism and ecosystem level. This involves the continued massive-scale generation of sequence data, complemented by orthogonal new capabilities to functionally annotate these large sequence data sets. Our Strategic Vision lays out a path to guide our decisions and ensure that the evolving set of experimental and computational capabilities available to DOE JGI users will continue to enable groundbreaking science.

59 BASIC BIOLOGICAL SCIENCES↗

Reconciling Scratch Space Consumption, Exposure, and Volatility to Achieve Timely Staging of Job Input Data

Innovative scientific applications and emerging dense data sources are creating a data deluge for high-end computing systems. Processing such large input data typically involves copying (or staging) onto the supercomputer's specialized high-speed storage, scratch space, for sustained high I/O throughput. The current practice of conservatively staging data as early as possible makes the data vulnerable to storage failures, which may entail re-staging and consequently reduced job throughput. To address this, we present a timely staging framework that uses a combination of job startup time predictions, user-specified intermediate nodes, and decentralized data delivery to coincide input data staging with job start-up. By delaying staging to when it is necessary, the exposure to failures and its effects can be reduced. Evaluation using both PlanetLab and simulations based on three years of Jaguar (No. 1 in Top500) job logs show as much as 85.9% reduction in staging times compared to direct transfers, 75.2% reduction in wait time on scratch, and 2.4% reduction in usage/hour.

99 GENERAL AND MISCELLANEOUS↗

Earth Science Data Analytics: Preparing for Extracting Knowledge from Information

Data analytics is the process of examining large amounts of data of a variety of types to uncover hidden patterns, unknown correlations and other useful information. Data analytics is a broad term that includes data analysis, as well as an understanding of the cognitive processes an analyst uses to understand problems and explore data in meaningful ways. Analytics also include data extraction, transformation, and reduction, utilizing specific tools, techniques, and methods. Turning to data science, definitions of data science sound very similar to those of data analytics (which leads to a lot of the confusion between the two). But the skills needed for both, co-analyzing large amounts of heterogeneous data, understanding and utilizing relevant tools and techniques, and subject matter expertise, although similar, serve different purposes. Data Analytics takes on a practitioners approach to applying expertise and skills to solve issues and gain subject knowledge. Data Science, is more theoretical (research in itself) in nature, providing strategic actionable insights and new innovative methodologies. Earth Science Data Analytics (ESDA) is the process of examining, preparing, reducing, and analyzing large amounts of spatial (multi-dimensional), temporal, or spectral data using a variety of data types to uncover patterns, correlations and other information, to better understand our Earth. The large variety of datasets (temporal spatial differences, data types, formats, etc.) invite the need for data analytics skills that understand the science domain, and data preparation, reduction, and analysis techniques, from a practitioners point of view. The application of these skills to ESDA is the focus of this presentation. The Earth Science Information Partners (ESIP) Federation Earth Science Data Analytics (ESDA) Cluster was created in recognition of the practical need to facilitate the co-analysis of large amounts of data and information for Earth science. Thus, from a to advance science point of view: On the continuum of ever evolving data management systems, we need to understand and develop ways that allow for the variety of data relationships to be examined, and information to be manipulated, such that knowledge can be enhanced, to facilitate science. Recognizing the importance and potential impacts of the unlimited ways to co-analyze heterogeneous datasets, now and especially in the future, one of the objectives of the ESDA cluster is to facilitate the preparation of individuals to understand and apply needed skills to Earth science data analytics. Pinpointing and communicating the needed skills and expertise is new, and not easy. Information technology is just beginning to provide the tools for advancing the analysis of heterogeneous datasets in a big way, thus, providing opportunity to discover unobvious scientific relationships, previously invisible to the science eye. And it is not easy It takes individuals, or teams of individuals, with just the right combination of skills to understand the data and develop the methods to glean knowledge out of data and information. In addition, whereas definitions of data science and big data are (more or less) available (summarized in Reference 5), Earth science data analytics is virtually ignored in the literature, (barring a few excellent sources).

data analytics↗

Improving the Inventory of Large Lunar Basins: Using Lola Data to Test Previous Candidates and Search for New Ones

Topography and crustal thickness data from LOLA altimetry were used to test the validity of 98 candidate large lunar basins derived from photogeologic and earlier topographic and crustal thickness data, and to search for possible new candidates. We eliminate 23 previous candidates but find good evidence for 20 new candidates. The number of basins > 300 km diameter on the Moon is almost certainly a factor 2 (maybe 3?) larger than the number of named features having basin-like topography.

Frey, Herbert V.↗

Globus service enhancements for exascale applications and facilities

Many extreme-scale applications require the movement of large quantities of data to, from, and among leadership computing facilities, as well as other scientific facilities and the home institutions of facility users. These applications, particularly when leadership computing facilities are involved, can touch upon edge cases (e.g., terabyte files) that had not been a focus of previous Globus optimization work, which had emphasized rather the movement of many smaller (megabyte to gigabyte) files. We report here on how automated client-driven chunking can be used to accelerate both the movement of large files and the integrity checking operations that have proven to be essential for large data transfers. In conclusion, we present detailed performance studies that provide insights into the benefits of these modifications in a range of file transfer scenarios.

97 MATHEMATICS AND COMPUTING↗

ARM Climate Modeling Best Estimate Data - A new data product for climate modelers

This paper provides an overview of a new data product, named the Climate Modeling Best Estimate (CMBE) dataset, developed by the U.S. Department of Energy (DOE)’s Atmospheric Radiation Measurement (ARM) Program in order to better serve the need of climate model developers and encourage greater use of ARM data by modelers. The CMBE dataset contains those quantities that are often used in model evaluation and reflect unique ARM measurements of clouds and radiation (e.g., cloud occurrence, liquid water path, and surface radiative fluxes) from the highest quality data that ARM has for many years. The data are averaged over one hour period, which is comparable to a typical temporal resolution used in climate model output. They are currently available at five ARM Climate Research Facility (ACRF) sites located at the Southern Great Plains, North Slope of Alaska, and Tropic Western Pacific, and can be obtained from the ACRF data archive. The long-term continuous ARM data provide invaluable information to improve our understanding of the interaction between clouds and radiation and a solid observational basis for model validation and improvement. This paper shows some examples to demonstrate its unique values in studies of cloud processes, climate variability and change, and climate modeling. Plans for future enhancements of the CMBE product are also discussed.

54 ENVIRONMENTAL SCIENCES↗

Time-Series Forecast Modeling on High-Bandwidth Network Measurements

With the increasing number of geographically distributed scientific collaborations and the growing sizes of scientific data, it has become challenging for users to achieve the best possible network performance on a shared network. In this paper, we have developed a model to forecast expected bandwidth utilization on high-bandwidth wide area networks. The forecast model can improve the efficiency of the resource utilization and scheduling of data movements on high-bandwidth networks to accommodate ever increasing data volume for large-scale scientific data applications. A univariate time-series forecast model is developed with the Seasonal decomposition of Time series by Loess (STL) and the AutoRegressive Integrated Moving Average (ARIMA) on Simple Network Management Protocol (SNMP) path utilization measurement data. Compared with the traditional approach such as Box-Jenkins methodology to train the ARIMA model, our forecast model reduces computation time up to 92.6 %. It also shows resilience against abrupt network usage changes. Finally, our forecast model conducts the large number of multi-step forecast, and the forecast errors are within the mean absolute deviation (MAD) of the monitored measurements.

97 MATHEMATICS AND COMPUTING↗