Search NASA⌕ Search

SEARCH · Search NASA

Results for “large data”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,153 records · Page 64

L-PBF High-Throughput Data Pipeline Approach for Multi-modal Integration

Abstract Metal-based additive manufacturing requires active monitoring solutions for assessing part quality. Multiple sensors and data streams, however, generate large heterogeneous data sets that are impractical for manual assessment and characterization. In this work, an automated pipeline is developed that enables feature extraction from high-speed camera video and multi-modal data analysis. The framework removes the need for manual assessment through the utilization of deep learning techniques and training models in a weakly supervised paradigm. We demonstrate this pipeline’s capability over 700,000 high-speed camera frames. The pipeline successfully extracts melt pool and spatter geometries and links them to corresponding pyrometry, radiography, and processparameter information. 715 individual prints are examined to reveal melt pool areas that exceeds 0.07 mm 2 and pyrometry signal over a threshold (375 pyrometry units) were more likely to have defects. These automated processes enable massive throughput of characterization techniques.

36 MATERIALS SCIENCE↗

Contribution of citizen science towards international biodiversity monitoring

To meet collective obligations towards biodiversity conservation and monitoring, it is essential that the world's governments and non-governmental organisations as well as the research community tap all possible sources of data and information, including new, fast-growing sources such as citizen science (CS), in which volunteers participate in some or all aspects of environmental assessments. Through compilation of a database on CS and community-based monitoring (CBM, a subset of CS) programs, we assess where contributions from CS and CBM are significant and where opportunities for growth exist. We use the Essential Biodiversity Variable framework to describe the range of biodiversity data needed to track progress towards global biodiversity targets, and we assess strengths and gaps in geographical and taxonomic coverage. Our results show that existing CS and CBM data particularly provide large-scale data on species distribution and population abundance, species traits such as phenology, and ecosystem function variables such as primary and secondary productivity. Only birds, Lepidoptera and plants are monitored at scale. Most CS schemes are found in Europe, North America, South Africa, India, and Australia. We then explore what can be learned from successful CS/CBM programs that would facilitate the scaling up of current efforts, how existing strengths in data coverage can be better exploited, and the strategies that could maximise the synergies between CS/CBM and other approaches for monitoring biodiversity, in particular from remote sensing. More and better targeted funding will be needed, if CS/CBM programs are to contribute further to international biodiversity monitoring.

59 BASIC BIOLOGICAL SCIENCES↗

Nuclear Science for the Manhattan Project and Comparison to Today’s ENDF Data

Nuclear physics advances in the United States and Britain from 1939 to 1945 are described. The Manhattan Project’s work led to an explosion in our knowledge of nuclear science. A conference in April 1943 at Los Alamos provided a simple formula used to compute critical masses and laid out the research program needed to determine the key nuclear constants. In short order, four university accelerators were disassembled and reassembled at Los Alamos, and methods were established to make measurements on extremely small samples owing to the initial lack of availability of enriched 235U and plutonium. I trace the program that measured fission cross sections, fission-emitted neutron multiplicities and their energy spectra, and transport cross sections, comparing the measurements with our best understanding today as embodied in the Evaluated Nuclear Data File ENDF/B-VIII.0. The large nuclear data uncertainties at the beginning of the project, which often exceeded 25% to 50%, were reduced by 1945 often to less than 5% to 10%. Uranium-235 and plutonium-239 fission cross-section assessments in the fast mega-electron-volt range were reduced following more accurate measurements, and the neutron multiplicity $\overline{v}$ increased. By a lucky coincidence of canceling errors, the initial critical mass estimates were close to the final estimated masses. Some images from historical documents from our Los Alamos archives are shown. Many of the original measurements from these early years have not previously been widely available. Through this work, these data have now been archived in the international experimental nuclear reaction data library (EXFOR) in a collaboration with the International Atomic Energy Agency and Brookhaven National Laboratory.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

DETECTION OF DIFFUSE NEUTRAL INTRAGROUP MEDIUM IN HICKSON COMPACT GROUPS

We present new Green Bank Telescope (GBT) 21 cm neutral hydrogen (H I) observations of a complete distance-limited sample of 22 Hickson Compact Groups (HCGs) with at least four true members. We detected an average H I mass of 8 x 10{sup 9} M{sub sun} (median = 6 x 10{sup 9} M{sub sun}), which is significantly larger than previous single-dish measurements. Consequently, the H I deficiencies for these HCGs have been reduced, although not completely eliminated. Spectral comparison of the GBT data with complementary Very Large Array data shows significant H I excess in the GBT spectra. The observed excess is primarily due to the high surface brightness (HSB) sensitivity of the GBT detecting diffuse, low column density H I in these groups. The excess gas forms a faint diffused neutral medium which is an intermediate stage in the evolution of HSB H I tidal debris in the intragroup medium (IGM) before it is fully ionized. The excess gas mass fraction, (M(H I){sub GBT} - M(H I){sub VLA})/M(H I){sub GBT}, for our complete sample varies from 5% to 81% with an average of 36% (median = 30%). The excess gas mass fraction is highest in slightly H I deficient groups where the tidal debris has had enough time to evolve. We also find the excess gas content increases with the evolutionary phase of the group described in Verdes-Montenegro et al. Theoretical calculations indicate that an H I cloud of radius >= 200 pc would survive in an IGM of 2 x 10{sup 6} K for more than the typical dynamical lifetime of a group. However, smaller clouds get evaporated and assimilated into the hot IGM in a much shorter timescale.

79 ASTRONOMY AND ASTROPHYSICS↗

AN INFRARED THROUGH RADIO STUDY OF THE PROPERTIES AND EVOLUTION OF IRDC CLUMPS

We examine the physical properties and evolutionary stages of a sample of 17 clumps within 8 Infrared Dark Clouds (IRDCs) by combining existing infrared, millimeter, and radio data with new Bolocam Galactic Plane Survey (BGPS) 1.1 mm data, Very Large Array radio continuum data, and Heinrich Hertz Telescope dense gas (HCO{sup +} and N{sub 2}H{sup +}) spectroscopic data. We combine literature studies of star formation tracers and dust temperatures within IRDCs with our search for ultracompact (UC) H II regions to discuss a possible evolutionary sequence for IRDC clumps. In addition, we perform an analysis of mass tracers in IRDCs and find that 8 {mu}m extinction masses and 1.1 mm BGPS masses are complementary mass tracers in IRDCs except for the most active clumps (notably those containing UC H II regions), for which both mass tracers suffer biases. We find that the measured virial masses in IRDC clumps are uniformly higher than the measured dust continuum masses on the scale of {approx}1 pc. We use {sup 13}CO, HCO{sup +}, and N{sub 2}H{sup +} to study the molecular gas properties of IRDCs and do not see any evidence of chemical differentiation between hot and cold clumps on the scale of {approx}1 pc. However, both HCO{sup +} and N{sub 2}H{sup +} are brighter in active clumps, due to an increase in temperature and/or density. We report the identification of four UC H II regions embedded within IRDC clumps and find that UC H II regions are associated with bright ({approx}>1 Jy) 24 {mu}m point sources, and that the brightest UC H II regions are associated with 'diffuse red clumps' (an extended enhancement at 8 {mu}m). The broad stages of the discussed evolutionary sequence (from a quiescent clump to an embedded H II region) are supported by literature dust temperature estimates; however, no sequential nature can be inferred between the individual star formation tracers.

79 ASTRONOMY AND ASTROPHYSICS↗

THE CHANDRA VIEW OF NGC 4178: THE LOWEST MASS BLACK HOLE IN A BULGELESS DISK GALAXY?

Using high-resolution Chandra data, we report the presence of a weak X-ray point source coincident with the nucleus of NGC 4178, a late-type bulgeless disk galaxy known to have high-ionization mid-infrared (mid-IR) lines typically associated with active galactic nuclei (AGNs). Although the faintness of this source precludes a direct spectral analysis, we are able to infer its basic spectral properties using hardness ratios. X-ray modeling, combined with the nuclear mid-IR characteristics, suggests that NGC 4178 may host a highly absorbed AGN accreting at a high rate with a bolometric luminosity on order of 10{sup 43} erg s{sup -1}. The black hole mass estimate, based on our Chandra data and archival Very Large Array data using the most recent fundamental plane relations, is {approx}10{sup 4}-10{sup 5} M{sub Sun }, possibly the lowest mass nuclear black hole currently known. There are also three off-nuclear sources, two with a similar brightness to the nuclear source at 36'' and 32'' from the center. As with the nuclear source, hardness ratios are used to estimate spectra for these two sources, and both are consistent with a simple power-law (PL) model with absorption. These two sources have X-ray luminosities of the order of {approx}10{sup 38} erg s{sup -1}, which place them at the threshold between X-ray binaries and ultraluminous X-ray sources (ULXs). The third off-nuclear source, located 49'' from the center, is the brightest source detected, with an X-ray luminosity of {approx}10{sup 40} erg s{sup -1}. Its spectrum is well fit with an absorbed PL model, suggesting that it is a ULX. We also fit its spectrum with the Bulk Motion Comptonization model and suggest that this source is consistent with an intermediate-mass black hole of mass (6 {+-} 2) Multiplication-Sign 10{sup 3} M{sub Sun }.

79 ASTRONOMY AND ASTROPHYSICS↗

Deep Learning on Operational Facility Data Related to Large-Scale Distributed Area Scientific Workflows

Abstract—Distributed computing platforms provide a robust mechanism to perform large-scale computations by splitting the task and data among multiple locations, possibly located thousands of miles apart geographically. This distribution of resources can lead to benefits such as redundancy of data, engagement with scientific teams that involve a diverse group of domain experts distributed spatially around the globe. It allows scientists to share their expertise and data with everyone and achieve insights by deploying collective intelligence of entire team. However, this distributed computing comes with its associated problems such as rampant duplication of file transfers increasing congestion, long job completion times, unexpected site crashing, suboptimal data transfer rates, unpredictable reliability in a time range, overshooting congestion, suboptimal usage of storage elements. In addition, each sub-system becomes a potential failure node that can trigger system wide disruptions.

Machine Learning, Provenance, Time Series, Semanti↗

A Study of the Trade-off Between Reducing Precision and Reducing Resolution for Data Analysis and Visualization

There presently exist two dominant strategies to reduce data sizes in analysis and visualization: reducing the precision of the data, e.g., through quantization, or reducing its resolution, e.g., by subsampling. Both have advantages and disadvantages and both face fundamental limits at which the reduced information ceases to be useful. The paper explores the additional gains that could be achieved by combining both strategies. In particular, we present a common framework that allows us to study the trade-off in reducing precision and/or resolution in a principled manner. We represent data reduction schemes as progressive streams of bits and study how various bit orderings such as by resolution, by precision, etc., impact the resulting approximation error across a variety of data sets as well as analysis tasks. Additionally, we compute streams that are optimized for different tasks to serve as lower bounds on the achievable error. Scientific data management systems can use the results proposed in this paper as guidance on how to store and stream data to make efficient use of the limited storage and bandwidth in practice.

97 MATHEMATICS AND COMPUTING↗

Advances in systems for interactive processing and display of meteorological data

Advances in systems for interactive processing and display of meteorological data are reviewed, with particular attention given to developments in hardware and software, meteorological data base, analysis and display, and systems availability. These developments include inexpensive minicomputers which give the user almost instantaneous results for many types of jobs; image terminals with the capability to enhance, quantify, animate, and compare image and graphical data; accessibility of a large meteorological data base and the capability of merging different types of data; and sophisticated analysis and multidimensional display techniques. Critical problems still to be solved include getting quick access to historical and real time data bases from any system and making it easy to transport software from one system to another.

Hasler, A. F.↗

A Southern Hemisphere radar meteor orbit survey

A meteor radar system has been operated on a routine basis near Christchurch, New Zealand, to determine the orbits of Earth-impacting interplanetary dust and meteoroids. The system sensitivity is +13 visual magnitude, corresponding to approximately 100 micron sized meteoroids. With an orbital precision of 2 degrees in angular elements and 10 percent in orbital energy (1/a), the operation yields an average of 1500 orbits daily with a total to date in excess of 10(exp 5). The use of pc's and automated data reduction permit the large orbital data sets we collect to be routinely reduced. Some illustrative examples are presented of the signal formats/processing and the results of data reduction, giving the individual orbital elements and hence the overall distributions. Current studies include the distribution of dust in the inner solar system; the influx of meteoroids associated with near-Earth asteroids; and the orbital structure existing in comet-produced streams.

Baggaley, W. Jack↗

Automated Induction Of Rule-Based Neural Networks

Prototype expert systems implemented in software and are functionally equivalent to neural networks set up automatically and placed into operation within minutes following information-theoretic approach to automated acquisition of knowledge from large example data bases. Approach based largely on use of ITRULE computer program.

Smyth, Padhraic J.↗

RAID Disk Arrays for High Bandwidth Applications

High bandwidth applications require large amounts of data transferred to/from storage devices at extremely high data rates. Further, these applications often are 'real time' in which access to the storage device must take place on the schedule of the data source, not the storage. A good example is a satellite downlink - the volume of data is quite large and the data rates quite high (dozens of MB/sec). Further, a telemetry downlink must take place while the satellite is overhead. A storage technology which is ideally suited to these types of applications is redundant arrays of independent discs (RAID). Raid storage technology, while offering differing methodologies for a variety of applications, supports the performance and redundancy required in real-time applications. Of the various RAID levels, RAID-3 is the only one which provides high data transfer rates under all operating conditions, including after a drive failure.

Moren, Bill↗

A Comparison of Ffowcs Williams-Hawkings Solvers for Airframe Noise Applications

This paper presents a comparison between two implementations of the Ffowcs Williams and Hawkings equation for airframe noise applications. Airframe systems are generally moving at constant speed and not rotating, so these conditions are used in the current investigation. Efficient and easily implemented forms of the equations applicable to subsonic, rectilinear motion of all acoustic sources are used. The assumptions allow the derivation of a simple form of the equations in the frequency-domain, and the time-domain method uses the restrictions on the motion to reduce the work required to find the emission time. The comparison between the frequency domain method and the retarded time formulation reveals some of the advantages of the different approaches. Both methods are still capable of predicting the far-field noise from nonlinear near-field flow quantities. Because of the large input data sets and potentially large numbers of observer positions of interest in three-dimensional problems, both codes utilize the message passing interface to divide the problem among different processors. Example problems are used to demonstrate the usefulness and efficiency of the two schemes.

Lockard, David P.↗

An Intelligent Archive Testbed Incorporating Data Mining

Many significant advances have occurred during the last two decades in remote sensing instrumentation, computation, storage, and communication technology. A series of Earth observing satellites have been launched by U.S. and international agencies and have been operating and collecting global data on a regular basis. These advances have created a data rich environment for scientific research and applications. NASA s Earth Observing System (EOS) Data and Information System (EOSDIS) has been operational since August 1994 with support for pre-EOS data. Currently, EOSDIS supports all the EOS missions including Terra (1999), Aqua (2002), ICESat (2002) and Aura (2004). EOSDIS has been effectively capturing, processing and archiving several terabytes of standard data products each day. It has also been distributing these data products at a rate of several terabytes per day to a diverse and globally distributed user community (Ramapriyan et al. 2009). There are other NASA-sponsored data system activities including measurement-based systems such as the Ocean Data Processing System and the Precipitation Processing system, and several projects under the Research, Education and Applications Solutions Network (REASoN), Making Earth Science Data Records for Use in Research Environments (MEaSUREs), and the Advancing Collaborative Connections for Earth-Sun System Science (ACCESS) programs. Together, these activities provide a rich set of resources constituting a value chain for users to obtain data at various levels ranging from raw radiances to interdisciplinary model outputs. The result has been a significant leap in our understanding of the Earth systems that all humans depend on for their enjoyment, livelihood, and survival. The trend in the community today is towards many distributed sets of providers of data and services. Despite this, visions for the future include users being able to locate, fuse and utilize data with location transparency and high degree of interoperability, and being able to convert data to information and usable knowledge in an efficient, convenient manner, aided significantly by automation (Ramapriyan et al. 2004; NASA 2005). We can look upon the distributed provider environment with capabilities to convert data to information and to knowledge as an Intelligent Archive in the Context of a Knowledge Building system (IA-KBS). Some of the key capabilities of an IA-KBS are: Virtual Product Generation, Significant Event Detection, Automated Data Quality Assessment, Large-Scale Data Mining, Dynamic Feedback Loop, and Data Discovery and Efficient Requesting (Ramapriyan et al. 2004).

Ramapriyan, H.↗

Potential impact of aerosols on convective clouds revealed by Himawari-8 observations over different terrain types in eastern China

Convective clouds are common and play a major role in Earth's water cycle and energy balance; they may even develop into storms and cause severe rainfall events. To understand the convective cloud development process, this study investigates the impact of aerosols on convective clouds by considering the influence of both topography and diurnal variation in radiation. By combining texture analysis, clustering, and thresholding methods, we identify all convective clouds in two warm seasons (May–September, 2016/17) in eastern China based on Himawari-8 Level 1 data. Having large diurnally resolved cloud data together with surface meteorological and environmental measurements, we investigate convective cloud properties and their variation, stratified by elevation and diurnal change. We then analyze the potential impact of aerosol on convective clouds under different meteorological conditions and topographies. In general, convective clouds tend to occur preferentially under polluted conditions in the morning, which reverses in the afternoon. Convective cloud fraction first increases then decreases with aerosol loading, which may contribute to this phenomenon. Topography and diurnal meteorological variations may affect the strength of aerosol microphysical and radiative effects. Updraft is always stronger along the windward slopes of mountains and plateaus, especially in northern China. The prevailing southerly wind near the foothills of mountains and plateaus is likely to contribute to this windward strengthening of updraft and to bring more pollutant into the mountains, thereby strengthening the microphysical effect, invigorating convective clouds. By comparison, over plain, aerosols decrease surface heating and suppress convection by blocking solar radiation reaching the surface.

clouds↗

Data for KETCHUP: Parameterizing of Large-Scale Kinetic Models Using Multiple Datasets with Different Reference States

Repository for Kinetic Estimation Tool Capturing Heterogeneous Datasets Using Pyomo (KETCHUP), a flexible parameter estimation tool that leverages a primal-dual interior-point algorithm to solve a nonlinear programming (NLP) problem that identifies a set of parameters capable of recapitulating the steady-state fluxes and concentrations in wild-type and perturbed metabolic networks. KETCHUP can use K-FIT [2] input files. Example K-FIT input files are located in the K-FIT repository at https://github.com/maranasgroup/K-FIT.

Metabolomics↗

Energy-efficient mapping of large-scale workflows under deadline constraints in big data computing systems

Large-scale workflows for big data analytics have become a main consumer of energy in data centers where moldable parallel computing models such as MapReduce are widely applied to meet high computational demands with time-varying computing resources. The granularity of task partitioning in each moldable job of such big data workflows has a significant impact on energy efficiency, which remains largely unexplored. In this paper, we analyze the properties of moldable jobs and formulate a workflow mapping problem to minimize the dynamic energy consumption of a given workflow request under a deadline constraint in big data systems. Since this problem is strongly NP-hard, we design a fully polynomial-time approximation scheme (FPTAS) for a special case with a pipeline-structured workflow on a homogeneous cluster and a heuristic for the generalized problem with an arbitrary workflow on a heterogeneous cluster. The performance superiority of the proposed solution in terms of dynamic energy saving and deadline missing rate is illustrated by extensive simulation results in comparison with existing algorithms, and further validated by real-life workflow implementation and experimental results in Hadoop/YARN systems.

97 MATHEMATICS AND COMPUTING↗

Optimizing Management of Persistent Data Structures in High-Performance Analytics

Large-scale data analytics workflows ingest massive input data into various data structures, including graphs and key-value datastores. These data structures undergo multiple transformations and computations and are typically reused in incremental and iterative analytics workflows. Persisting in-memory views of these data structures enables reusing them beyond the scope of a single program run while avoiding repetitive raw data ingestion overheads. Memory-mapped I/O enables persisting in-memory data structures without data serialization and deserialization overheads. However, memory-mapped I/O lacks the key feature of persisting consistent snapshots of these data structures for incremental ingestion and processing. The obstacles to efficient virtual memory snapshots using memory-mapped I/O include background writebacks outside the application’s control, and the significantly high storage footprint of such snapshots. To address these limitations, we present Privateer, a memory and storage management tool that enables storage-efficient virtual memory snapshotting while also optimizing snapshot I/O performance. Here, we integrated Privateer into Metall, a state-of-the-art persistent memory allocator for C++, and the Lightning Memory-Mapped Database (LMDB), a widely-used key-value datastore in data analytics and machine learning. Privateer optimized application performance by 1.22× when storing data structure snapshots to node-local storage, and up to 16.7× when storing snapshots to a parallel file system. Privateer also optimizes storage efficiency of incremental data structure snapshots by up to 11× using data deduplication and compression.

Computer science↗