Search NASA⌕ Search

SEARCH · Search NASA

Results for “large data”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 595 records · Page 33

An IDL-based analysis package for COBE and other skycube-formatted astronomical data

UIMAGE is a data analysis package written in IDL for the Cosmic Background Explorer (COBE) project. COBE has extraordinarily stringent accuracy requirements: 1 percent mid-infrared absolute photometry, 0.01 percent submillimeter absolute spectrometry, and 0.0001 percent submillimeter relative photometry. Thus, many of the transformations and image enhancements common to analysis of large data sets must be done with special care. UIMAGE is unusual in this sense in that it performs as many of its operations as possible on the data in its native format and projection, which in the case of COBE is the quadrilateralized sphereical cube ('skycube'). That is, after reprojecting the data, e.g., onto an Aitoff map, the user who performs an operation such as taking a crosscut or extracting data from a pixel is transparently acting upon the skycube data from which the projection was made, thereby preserving the accuracy of the result. Current plans call for formatting external data bases such as CO maps into the skycube format with a high-accuracy transformation, thereby allowing Guest Investigators to use UIMAGE for direct comparison of the COBE maps with those at other wavelengths from other instruments. It is completely menu-driven so that its use requires no knowledge of IDL. Its functionality includes I/O from the COBE archives, FITS files, and IDL save sets as well as standard analysis operations such as smoothing, reprojection, zooming, statistics of areas, spectral analysis, etc. One of UIMAGE's more advanced and attractive features is its terminal independence. Most of the operations (e.g., menu-item selection or pixel selection) that are driven by the mouse on an X-windows terminal are also available using arrow keys and keyboard entry (e.g., pixel coordinates) on VT200 and Tektronix-class terminals. Even limited grey scales of images are available this way. Obviously, image processing is very limited on this type of terminal, but it is nonetheless surprising how much analysis can be done on that medium. Such flexibility has the virtue of expanding the user community to those who must work remotely on non-image terminals, e.g., via modem.

Ewing, J. A.↗

GRIDVIEW: Recent Improvements in Research and Education Software for Exploring Mars Topography

GRIDVIEW is being developed by the GEODYNAMICS Branch at NASA's Goddard Space Flight Center and can be downloaded on the web at http://geodynamics.gsfc.nasa.gov/gridview/. The program is very mature and has been successfully used for more than four years, but is still under development as we add new features for data analysis and visualization. The software can run on any computer supported by the IDL virtual machine application supplied by RSI. The virtual machine application is currently available for recent versions of MS Windows, MacOS X, Red Hat Linux and UNIX. Minimum system memory requirement is 32 MB, however loading large data sets may require larger amounts of RAM to function adequately.

Roark, J. H.↗

Optimizing inference of segmentation on high-resolution images in MLExchange

MLExchange is a machine learning (ML) operations platform providing web user-interfaces (UIs) for data visualization and analysis pipelines at synchrotron facilities. Among these UIs is the segmentation app which helps synchrotron users utilize ML algorithms to automatically segment high-resolution scientific images with minimal manual annotation effort. In this work, we share code optimizations that significantly speed up the segmentation inference workflow of large data in short time. By optimizing the sequence of CPU-GPU data transfers and introducing CPU parallelization to key operations, we improve the per-device, per-image frame computational efficiency and observe close to 3×$$\times$$ speedup over the original segmentation inference workflow run time when utilizing a single GPU. Further adaptations enabling multi-GPU inference yield more than 40×$$\times$$ speedup with 100 GPUs compared to the optimized single GPU inference workflow. This acceleration of the segmentation inference workflow will provide MLExchange users with easy access to segmentation results with little wait time.

Lu, Shizhao↗

Materials informatics: From the atomic-level to the continuum

We report that in recent years materials informatics, which is the application of data science to problems in materials science and engineering, has emerged as a powerful tool for materials discovery and design. This relatively new field is already having a significant impact on the interpretation of data for a variety of materials systems, including those used in thermoelectrics, ferroelectrics, battery anodes and cathodes, hydrogen storage materials, polymer dielectrics, etc. Its practitioners employ the methods of multivariate statistics and machine learning in conjunction with standard computational tools (e.g., density-functional theory) to, for example, visualize and dimensionally reduce large data sets, identify patterns in hyperspectral data, parse microstructural images of polycrystals, characterize vortex structures in ferroelectrics, design batteries and, in general, establish correlations to extract important physics and infer structure-property-processing relationships. In this Overview, we critically examine the role of informatics in several important materials subfields, highlighting significant contributions to date and identifying known shortcomings. We specifically focus attention on the difference between the correlative approach of classical data science and the causative approach of physical sciences. From this perspective, we also outline some potential opportunities and challenges for informatics in the materials realm in this era of big data.

36 MATERIALS SCIENCE↗

Reverse Engineering Adverse Outcome Pathways

The toxicological effects of many stressors are mediated through unknown, or poorly characterized, mechanisms of action. We describe the application of reverse engineering complex interaction networks from high dimensional omics data (gene, protein, metabolic, signaling) to characterize adverse outcome pathways (AOPs) for chemicals that disrupt the hypothalamus-pituitary-gonadal endocrine axis in fathead minnows. Gene expression changes in fathead minnow ovaries in response to 7 different chemicals, over different times, doses, and in vivo versus in vitro conditions were captured in a large data set of 868 arrays. We examined potential AOPs of the antiandrogen flutamide using two mutual information theory methods, ARACNE and CLR to infer gene regulatory networks and potential adverse outcome pathways. Representative networks from these studies were used to predict a network path from stressor to adverse outcome as a candidate AOP. The relationship of individual chemicals to an adverse outcome can be determined by following perturbations through the network in response to chemical treatment leading to the nodes associated with the adverse outcome. Identification of candidate pathways allows for formation of testable hypotheses about key biologic processes, biomarkers or alternative endpoints, which could be used to monitor an adverse outcome pathway. Finally, we identify the unique challenges facing the application of this approach in ecotoxicology, and attempt to provide a road map for the utilization of these tools. Key Words: mechanism of action, toxicology, microarray, network inference

59 BASIC BIOLOGICAL SCIENCES↗

A land-surface Testbed for EOSDIS

The main objective of the Testbed project was to deliver satellite images via the Internet to scientific and educational users free of charge. The main method of operations was to store satellite images on a low cost tape library system, visually browse the raw satellite data, access the raw data filed, navigate the imagery through 'C' programming and X-Windows interface software, and deliver the finished image to the end user over the Internet by means of file transfer protocol methods. The conclusion is that the distribution of satellite imagery by means of the Internet is feasible, and the archiving of large data sets can be accomplished with low cost storage systems allowing multiple users.

Emery, William↗

A Multi-Band Analytical Algorithm for Deriving Absorption and Backscattering Coefficients from Remote-Sensing Reflectance of Optically Deep Waters

A multi-band analytical (MBA) algorithm is developed to retrieve absorption and backscattering coefficients for optically deep waters, which can be applied to data from past and current satellite sensors, as well as data from hyperspectral sensors. This MBA algorithm applies a remote-sensing reflectance model derived from the Radiative Transfer Equation, and values of absorption and backscattering coefficients are analytically calculated from values of remote-sensing reflectance. There are only limited empirical relationships involved in the algorithm, which implies that this MBA algorithm could be applied to a wide dynamic range of waters. Applying the algorithm to a simulated non-"Case 1" data set, which has no relation to the development of the algorithm, the percentage error for the total absorption coefficient at 440 nm a (sub 440) is approximately 12% for a range of 0.012 - 2.1 per meter (approximately 6% for a (sub 440) less than approximately 0.3 per meter), while a traditional band-ratio approach returns a percentage error of approximately 30%. Applying it to a field data set ranging from 0.025 to 2.0 per meter, the result for a (sub 440) is very close to that using a full spectrum optimization technique (9.6% difference). Compared to the optimization approach, the MBA algorithm cuts the computation time dramatically with only a small sacrifice in accuracy, making it suitable for processing large data sets such as satellite images. Significant improvements over empirical algorithms have also been achieved in retrieving the optical properties of optically deep waters.

Lee, Zhong-Ping↗

gPhoton: THE GALEX PHOTON DATA ARCHIVE

gPhoton is a new database product and software package that enables analysis of GALEX ultraviolet data at the photon level. The project’s stand-alone, pure-Python calibration pipeline reproduces the functionality of the original mission pipeline to reduce raw spacecraft data to lists of time-tagged, sky-projected photons, which are then hosted in a publicly available database by the Mikulski Archive at Space Telescope. This database contains approximately 130 terabytes of data describing approximately 1.1 trillion sky-projected events with a timestamp resolution of five milliseconds. A handful of Python and command-line modules serve as a front end to interact with the database and to generate calibrated light curves and images from the photon-level data at user-defined temporal and spatial scales. The gPhoton software and source code are in active development and publicly available under a permissive license. We describe the motivation, design, and implementation of the calibration pipeline, database, and tools, with emphasis on divergence from prior work, as well as challenges created by the large data volume. We summarize the astrometric and photometric performance of gPhoton relative to the original mission pipeline. For a brief example of short time-domain science capabilities enabled by gPhoton, we show new flares from the known M-dwarf flare star CR Draconis. The gPhoton software has permanent object identifiers with the ASCL (ascl:1603.004) and DOI (doi:10.17909/T9CC7G). This paper describes the software as of version v1.27.2.

79 ASTRONOMY AND ASTROPHYSICS↗

Analysis of Alternative Architectures for Cargo Lunar Landers

NASA’s Human Landing System (HLS) program has been working with commercial partners to develop human-class lunar landers to return the first American woman and next American man to the lunar surface in the mid 2020’s. In an effort to expand human presence beyond low Earth orbit, NASA’s Artemis program aims to facilitate a sustainable, long-term human presence in cis-lunar space. A component of this will require significant infrastructure to be delivered to the lunar surface. Delivering this infrastructure will require a significant lander capability that has yet to be developed. A thorough understanding of cargo lunar lander architectures is required such that select alternatives can be identified that best support the Artemis program’s objective of sustainability. The goal of this study is to aid NASA and its partners in the understanding of the cargo lunar lander trades space, as well as identify potential robust alternatives. The results will support NASA as it moves forward with key activities such as requirements formulation, agency strategic planning, and potential cargo lunar lander procurements. The study builds off of recent work performed by the Human Landing System program’s Architecture and Systems Analysis group to encompass a broad trade space of cargo lunar lander architecture alternatives. The current trade space as depicted by the morphological matrix and mission graph in Fig. 1 and Fig. 2, respectively, includes key alternative options that have become highly relevant due to current HLS activities and include on-orbit refueling, active cryogenic fluid management, Earth orbit aggregation, and global lunar access. The authors believe that there is also a statistically relevant impact of lander-payload configuration on the primary structure of the vehicle that could greatly impact alternative selection. Because of this, several conceptual lander-payload configurations will be evaluated to determine the level of impact. The current set of conceptual configurations are shown in Fig. 3 and Fig. 4. To aid the conceptual evaluation of these configurations, a catalogue of notional payloads has been developed that represent a wide range of masses and volumes that are expected to be delivered in support of a sustained human lunar presence, including pressurized and unpressurized rovers, surface habitats, power systems, and other support infrastructure. In order to execute this study in a timely fashion, a similar approach to that utilized in a similar 2019 study focused on 2024 human lunar sorties will be employed [1]. The team utilized a novel architecture synthesis framework currently being developed by NASA/MSFC to evaluate over 600,000 lunar lander architectures over a two month time frame [2]. From this large data set, varying ground rules and assumptions were applied as filters to explore the trade space to identify alternatives which exhibited robustness, as measured by launch vehicle payload margin, to absorb the natural growth that occurs during design maturation. The set of Earth-Moon system Delta-Vs assumed from the 2019 study, shown in Fig. 5, will be repurposed to accelerate model formulation for this effort. Additionally, current efforts in collaboration with the Georgia Institute of Technology’s Aerospace System Design Lab will be integrated to provide probabilistic modeling of the cargo lunar lander architectures to aid in identifying robust design alternatives [3]. The approach will help minimize potential impacts due to large levels of uncertainty inherent to pre phase-A conceptual design. By leveraging these past and present studies and partnerships, a highly detailed set of data can be generated in a short time period to aid NASA in the coming years to support the goal of a sustained human lunar presence.

Architectures↗

A new Monte Carlo generator for BSM physics in B → K*ℓ+ℓ− decays with an application to lepton non-universality in angular distributions

Abstract Within the widely used EvtGen framework, we have added a new event generator model forB → K * ℓ + ℓ − with improved standard model (SM) decay amplitudes and possible BSM physics contributions, which are implemented in the operator product expansion in terms of Wilson coefficients. This event generator can then be used to estimate the statistical sensitivity of a simulated experiment to the most general BSM signal resulting from dimension-six operators. We describe the advantages and potential of the newly developed ‘Sibidanov Physics Generator’ in improving the experimental sensitivity of searches for lepton non-universal BSM physics and clarifying signatures. The new generator can properly simulate BSM scenarios, interference between SM and BSM amplitudes, and correlations between different BSM observables as well as acceptance bias. We show that exploiting such correlations substantially improves experimental sensitivity. As a demonstration of the utility of the MC generator, we examine the prospects for improved measurements of lepton non-universality in angular distributions forB→K * ℓ + ℓ − decays from the expected 50 ab −1 data set of the Belle II experiment, using a four-dimensional unbinned maximum likelihood fit. We describe promising experimental signatures and correlations between observables. The use of lepton-universality violating ∆-observables significantly reduces uncertainties in the SM expectations due to QCD and resonance effects and is ideally suited for Belle II with the large data sets expected in the next decade. Thanks to the clean experimental environment of ane + e − machine, Belle II should be able to probe BSM physics in the Wilson coefficientsC 7 and$$ {C}_7^{\prime } $$ C 7 ′ , which appear at lowq 2 in the di-electron channel.

Physics↗

Evaluation of a Class of Simple and Effective Uncertainty Methods for Sparse Samples of Random Variables and Functions

When very few samples of a random quantity are available from a source distribution of unknown shape, it is usually not possible to accurately infer the exact distribution from which the data samples come. Under-estimation of important quantities such as response variance and failure probabilities can result. For many engineering purposes, including design and risk analysis, we attempt to avoid under-estimation with a strategy to conservatively estimate (bound) these types of quantities -- without being overly conservative -- when only a few samples of a random quantity are available from model predictions or replicate experiments. This report examines a class of related sparse-data uncertainty representation and inference approaches that are relatively simple, inexpensive, and effective. Tradeoffs between the methods' conservatism, reliability, and risk versus number of data samples (cost) are quantified with multi-attribute metrics use d to assess method performance for conservative estimation of two representative quantities: central 95% of response; and 10 -4 probability of exceeding a response threshold in a tail of the distribution. Each method's performance is characterized with 10,000 random trials on a large number of diverse and challenging distributions. The best method and number of samples to use in a given circumstance depends on the uncertainty quantity to be estimated, the PDF character, and the desired reliability of bounding the true value. On the basis of this large data base and study, a strategy is proposed for selecting the method and number of samples for attaining reasonable credibility levels in bounding these types of quantities when sparse samples of random variables or functions are available from experiments or simulations.

97 MATHEMATICS AND COMPUTING↗

Integrated remote sensing of the earth from low earth orbit in the 1990's

Trends in the areas of science, data systems, instruments and space technologies and their related infrastructures that will be experienced in earth remote sensing activities from LEO in the 1990s are discussed. Interdisciplinary studies will examine the flow of energy through the earth system, encompassing surveys of atmospheric and ocean circulations, the hydrologic cycle, and biogeochemical cycles. Mainframe computer assimilation of large data inputs will be associated with distributed microcomputer processing and analyses. Single satellite antennas will scan several microwave frequencies simultaneously, e.g., taking data on vegetation, water, and ice. Broadening the range of simultaneous observations will require multiinstrumented free-flying platforms, placed in orbit, repaired and/or retrieved by the Shuttle. Future instrumentation will include visible/IR imagers and the multifrequency microwave sounders with capabilities mainly limited by the availability of sufficient data-handling facilities.

Butler, D. M.↗

GeoDash: Assisting Visual Image Interpretation in Collect Earth Online by Leveraging Big Data on Google Earth Engine

Collect Earth Online (CEO) is a free and open online implementation of the FAO Collect Earth system for collaboratively collecting environmental data through the visual interpretation of Earth observation imagery. The primary collection mechanism in CEO is human interpretation of land surface characteristics in imagery served via Web Map Services (WMS). However, interpreters may not have enough contextual information to classify samples by only viewing the imagery served via WMS, be they high resolution or otherwise. To assist in the interpretation and collection processes in CEO, SERVIR, a joint NASA-USAID initiative that brings Earth observations to improve environmental decision making in developing countries, developed the GeoDash system, an embedded and critical component of CEO. GeoDash leverages Google Earth Engine (GEE) by allowing users to set up custom browser-based widgets that pull from GEE's massive public data catalog. These widgets can be quick looks of other satellite imagery, time series graphs of environmental variables, and statistics panels of the same. Users can customize widgets with any of GEE's image collections, such as the historical Landsat collection with data available since the 1970s, select date ranges, image stretch parameters, graph characteristics, and create custom layouts, all on-the-fly to support plot interpretation in CEO. This presentation focuses on the implementation and potential applications, including the back-end links to GEE and the user interface with custom widget building. GeoDash takes large data volumes and condenses them into meaningful, relevant information for interpreters. While designed initially with national and global forest resource assessments in mind, the system will complement disaster assessments, agriculture management, project monitoring and evaluation, and more.

SERVI↗

A Co-design Framework for Online Data Analysis and Reduction

Science applications preparing for the exascale era are increasingly exploring in situ computations comprising of simulation-analysis-reduction pipelines coupled in-memory. Efficient composition and execution of such complex pipelines for a target platform is a codesign process that evaluates the impact and tradeoffs of various application- and system-specific parameters. In this article, we describe a toolset for automating performance studies of composed HPC applications that perform online data reduction and analysis. We describe Cheetah, a new framework for composing parametric studies on coupled applications, and Savanna, a runtime engine for orchestrating and executing campaigns of codesign experiments. Furthermore, this toolset facilitates understanding the impact of various factors such as process placement, synchronicity of algorithms, and storage versus compute requirements for online analysis of large data. Ultimately, we aim to create a catalog of performance results that can help scientists understand tradeoffs when designing next-generation simulations that make use of online processing techniques. We illustrate the design of Cheetah and Savanna, and present application examples that use this framework to conduct codesign studies on small clusters as well as leadership class supercomputers.

97 MATHEMATICS AND COMPUTING↗

A unique, ring-like radio source with quadrilateral structure detected with machine learning

ABSTRACT We report the discovery of a unique object in the MeerKAT Galaxy Cluster Legacy Survey (MGCLS) using the machine learning anomaly detection framework astronomaly. This strange, ring-like source is 30′ from the MGCLS field centred on Abell 209, and is not readily explained by simple physical models. With an assumed host galaxy at redshift 0.55, the luminosity (1025 W Hz−1) is comparable to powerful radio galaxies. The source consists of a ring of emission 175 kpc across, quadrilateral enhanced brightness regions bearing resemblance to radio jets, two ‘ears’ separated by 368 kpc, and a diffuse envelope. All of the structures appear spectrally steep, ranging from −1.0 to −1.5. The ring has high polarization (25 per cent) except on the bright patches (<10 per cent). We compare this source to the Odd Radio Circles recently discovered in ASKAP data and discuss several possible physical models, including a termination shock from starburst activity, an end-on radio galaxy, and a supermassive black hole merger event. No simple model can easily explain the observed structure of the source. This work, as well as other recent discoveries, demonstrates the power of unsupervised machine learning in mining large data sets for scientifically interesting sources.

Astronomy & Astrophysics↗

THE IMPACT OF THE IONOSPHERE ON GROUND-BASED DETECTION OF THE GLOBAL EPOCH OF REIONIZATION SIGNAL

The redshifted 21 cm line of neutral hydrogen (H i), potentially observable at low radio frequencies (∼50–200 MHz), is a promising probe of the physical conditions of the intergalactic medium during Cosmic Dawn and the Epoch of Reionization (EoR). The sky-averaged H i signal is expected to be extremely weak (∼100 mK) in comparison to the Galactic foreground emission (∼10{sup 4} K). Moreover, the sky-averaged spectra measured by ground-based instruments are affected by chromatic propagation effects (∼tens of kelvin) originating in the ionosphere. We analyze data collected with the upgraded Broadband Instrument for Global Hydrogen Reionization Signal system deployed at the Murchison Radio-astronomy Observatory to assess the significance of ionospheric effects on the detection of the global EoR signal. The ionospheric effects identified in these data are, particularly during nighttime, dominated by absorption and emission. We measure some properties of the ionosphere, such as the electron temperature (T{sub e} ≈ 470 K at nighttime), magnitude, and variability of optical depth (τ{sub 100} {sub MHz} ≈ 0.01 and δτ ≈ 0.005 at nighttime). According to the results of a statistical test applied on a large data sample, very long integrations (∼100 hr collected over approximately 2 months) lead to increased signal-to-noise ratio even in the presence of ionospheric variability. This is further supported by the structure of the power spectrum of the sky temperature fluctuations, which has flicker noise characteristics at frequencies ≳10{sup −5} Hz, but becomes flat below ≈10{sup −5} Hz. Hence, we conclude that the stochastic error introduced by the chromatic ionospheric effects tends to zero in an average. Therefore, the ionospheric effects and fluctuations are not fundamental impediments preventing ground-based instruments from integrating down to the precision required by global EoR experiments, provided that the ionospheric contribution is properly accounted for in the data analysis.

79 ASTRONOMY AND ASTROPHYSICS↗

ParChain: a framework for parallel hierarchical agglomerative clustering using nearest-neighbor chain

This paper studies the hierarchical clustering problem, where the goal is to produce a dendrogram that represents clusters at varying scales of a data set. We propose the ParChain framework for designing parallel hierarchical agglomerative clustering (HAC) algorithms, and using the framework we obtain novel parallel algorithms for the complete linkage, average linkage, and Ward's linkage criteria. Compared to most previous parallel HAC algorithms, which require quadratic memory, our new algorithms require only linear memory, and are scalable to large data sets. ParChain is based on our parallelization of the nearest-neighbor chain algorithm, and enables multiple clusters to be merged on every round. We introduce two key optimizations that are critical for efficiency: a range query optimization that reduces the number of distance computations required when finding nearest neighbors of clusters, and a caching optimization that stores a subset of previously computed distances, which are likely to be reused. Experimentally, we show that our highly-optimized implementations using 48 cores with two-way hyper-threading achieve 5.8--110.1x speedup over state-of-the-art parallel HAC algorithms and achieve 13.75--54.23x self-relative speedup. Compared to state-of-the-art algorithms, our algorithms require up to 237.3x less space. Our algorithms are able to scale to data set sizes with tens of millions of points, which existing algorithms are not able to handle.

Computer Science↗

HVAC load Disaggregation using Low-resolution Smart Meter Data

Traditional non-intrusive load monitoring (NILM) methods are effective for load disaggregation using high resolution smart meter data collected by power quality meters. However, smart meter data collected and stored by utilities are normally 15-, 30- or 60-minute in granularity, making most NILM methods ineffective. This paper presents a novel sequential energy disaggregation (SED) algorithm for extracting heating, ventilation, and air conditioning (HVAC) energy consumptions from residential and small commercial building loads using 30 min smart meter data. Large, infrequently-used loads are first detected and removed from the total building energy consumption. Then, base energy consumption curves, defined as the energy consumption without heating and cooling loads, are identified using the mild-day method. After that, the heating and cooling loads are extracted using an average value subtracting method. Simulation results with real data show that the proposed SED method is computationally efficient, simple to implement and robust in performance across different types of buildings.

Liang, Ming↗