Search NASA⌕ Search

SEARCH · Search NASA

Results for “commodity clusters”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 55 records · Page 3
All sourcesNASA NTRSDOE OSTI
NASA NTRS2004

The Development of Two Science Investigator-led Processing Systems (SIPS) for NASA's Earth Observation System (EOS)

In 2001, NASA Goddard Space Flight Center's Laboratory for Terrestrial Physics started the construction of a science Investigator-led Processing System (SIPS) for processing data from the Ozone Monitoring Instrument (OMI) which will launch on the Aura platform in mid 2004. The Ozone Monitoring Instrument (OMI) is a contribution of the Netherlands Agency for Aerospace Programs (NIVR) in collaboration with the Finnish Meteorological Institute (FMI) to the Earth Observing System (EOS) Aura mission. It will continue the Total Ozone Monitoring System (TOMS) record for total ozone and other atmospheric parameters related to ozone chemistry and climate. OMI measurements will be highly synergistic with the other instruments on the EOS Aura platform. The LTP previously developed the Moderate Resolution Imaging Spectrometer (MODIS) Data Processing System (MODAPS), which has been in full operations since the launches of the Terra and Aqua spacecrafts in December, 1999 and May, 2002 respectively. During that time, it has continually evolved to better support the needs of the MODIS team. We now run multiple instances of the system managing faster than real time reprocessings of the data as well as continuing forward processing. The new OMI Data Processing System (OMIDAPS) was adapted from the MODAPS. It will ingest raw data from the satellite ground station and process it to produce calibrated, geolocated higher level data products. These data products will be transmitted to the Goddard Distributed Active Archive Center (GDAAC) instance of the Earth Observing System (EOS) Data and Information System (EOSDIS) for long term archive and distribution to the public. The OMIDAPS will also provide data distribution to the OMI Science Team for quality assessment, algorithm improvement, calibration, etc. We have taken advantage of lessons learned from the MODIS experience and software already developed for MODIS. We made some changes in the hardware system organization, database and software to adapt the system for OMI. We replaced the fundamental database system, Sybase, with an Open Source RDBMS called PostgreSQL, and based the entire OMIDAPS on a cluster of Linux based commodity computers rather than the large SGI servers that MODAPS uses. Rather than relying on a central I/O server host, the new system distributes its data archive among multiple server hosts in the cluster. OMI is also customizing the graphical user interfaces and reporting structure to more closely meet the needs of the OMI Science Team. Prior to 2003, simulated OMI data and the science algorithms were not ready for production testing. We initially constructed a prototype system and tested using a 25 year dataset of Total Ozone Mapping Spectrometer (TOMS) and Solar Backscatter Ultraviolet Instrument (SBUV) data. This prototype system provided a platform to support the adaptation of the algorithms for OMI, and provided reprocessing of the historical data aiding in its analysis. In a recent reanalysis of the TOMS data, the OMIDAPS processed 108,000 full orbits of data through 4 processing steps per orbit, producing about 800,000 files (400 GiB) of level 2 and greater data files. More recently we have installed two instances of the OMIDAPS for integration and testing of OM1 science processes as they get delivered from the Science Team. A Test instance of the OMIDAPS has also supported a series of "Interface Confidence Tests" (ICTs) and End-to-End Ground System tests to ensure the launch readiness of the system. This paper will discuss the high-level hardware, software, and database organization of the OMIDAPS and how it builds on the MODAPS heritage system. It will also provide an overview of the testing and implementation of the production OMIDAPS.

Tilmes, Curt↗
NASA NTRS1999

Computing with Beowulf

Parallel computers built out of mass-market parts are cost-effectively performing data processing and simulation tasks. The Supercomputing (now known as "SC") series of conferences celebrated its 10th anniversary last November. While vendors have come and gone, the dominant paradigm for tackling big problems still is a shared-resource, commercial supercomputer. Growing numbers of users needing a cheaper or dedicated-access alternative are building their own supercomputers out of mass-market parts. Such machines are generally called Beowulf-class systems after the 11th century epic. This modern-day Beowulf story began in 1994 at NASA's Goddard Space Flight Center. A laboratory for the Earth and space sciences, computing managers there threw down a gauntlet to develop a $50,000 gigaFLOPS workstation for processing satellite data sets. Soon, Thomas Sterling and Don Becker were working on the Beowulf concept at the University Space Research Association (USRA)-run Center of Excellence in Space Data and Information Sciences (CESDIS). Beowulf clusters mix three primary ingredients: commodity personal computers or workstations, low-cost Ethernet networks, and the open-source Linux operating system. One of the larger Beowulfs is Goddard's Highly-parallel Integrated Virtual Environment, or HIVE for short.

Cohen, Jarrett↗
NASA NTRS2002

Using Clustering to Establish Climate Regimes from PCM Output

A multivariate statistical clustering technique--based on the k-means algorithm of Hartigan has been used to extract patterns of climatological significance from 200 years of general circulation model (GCM) output. Originally developed and implemented on a Beowulf-style parallel computer constructed by Hoffman and Hargrove from surplus commodity desktop PCs, the high performance parallel clustering algorithm was previously applied to the derivation of ecoregions from map stacks of 9 and 25 geophysical conditions or variables for the conterminous U.S. at a resolution of 1 sq km. Now applied both across space and through time, the clustering technique yields temporally-varying climate regimes predicted by transient runs of the Parallel Climate Model (PCM). Using a business-as-usual (BAU) scenario and clustering four fields of significance to the global water cycle (surface temperature, precipitation, soil moisture, and snow depth) from 1871 through 2098, the authors' analysis shows an increase in spatial area occupied by the cluster or climate regime which typifies desert regions (i.e., an increase in desertification) and a decrease in the spatial area occupied by the climate regime typifying winter-time high latitude perma-frost regions. The patterns of cluster changes have been analyzed to understand the predicted variability in the water cycle on global and continental scales. In addition, representative climate regimes were determined by taking three 10-year averages of the fields 100 years apart for northern hemisphere winter (December, January, and February) and summer (June, July, and August). The result is global maps of typical seasonal climate regimes for 100 years in the past, for the present, and for 100 years into the future. Using three-dimensional data or phase space representations of these climate regimes (i.e., the cluster centroids), the authors demonstrate the portion of this phase space occupied by the land surface at all points in space and time. Any single spot on the globe will exist in one of these climate regimes at any single point in time. By incrementing time, that same spot will trace out a trajectory or orbit between and among these climate regimes (or atmospheric states) in phase (or state) space. When a geographic region enters a state it never previously visited, a climatic change is said to have occurred. Tracing out the entire trajectory of a single spot on the globe yields a 'manifold' in state space representing the shape of its predicted climate occupancy. This sort of analysis enables a researcher to more easily grasp the multivariate behavior of the climate system.

Oglesby, Robert↗
NASA NTRS2005

A Parallel Processing Algorithm for Remote Sensing Classification

A current thread in parallel computation is the use of cluster computers created by networking a few to thousands of commodity general-purpose workstation-level commuters using the Linux operating system. For example on the Medusa cluster at NASA/GSFC, this provides for super computing performance, 130 G(sub flops) (Linpack Benchmark) at moderate cost, $370K. However, to be useful for scientific computing in the area of Earth science, issues of ease of programming, access to existing scientific libraries, and portability of existing code need to be considered. In this paper, I address these issues in the context of tools for rendering earth science remote sensing data into useful products. In particular, I focus on a problem that can be decomposed into a set of independent tasks, which on a serial computer would be performed sequentially, but with a cluster computer can be performed in parallel, giving an obvious speedup. To make the ideas concrete, I consider the problem of classifying hyperspectral imagery where some ground truth is available to train the classifier. In particular I will use the Support Vector Machine (SVM) approach as applied to hyperspectral imagery. The approach will be to introduce notions about parallel computation and then to restrict the development to the SVM problem. Pseudocode (an outline of the computation) will be described and then details specific to the implementation will be given. Then timing results will be reported to show what speedups are possible using parallel computation. The paper will close with a discussion of the results.

Gualtieri, J. Anthony↗
DOE OSTI2015

In-Memory Graph Databases for Web-Scale Data

RDF databases have emerged as one of the most relevant way for organizing, integrating, and managing expo- nentially growing, often heterogeneous, and not rigidly structured data for a variety of scientific and commercial fields. In this paper we discuss the solutions integrated in GEMS (Graph database Engine for Multithreaded Systems), a software framework for implementing RDF databases on commodity, distributed-memory high-performance clusters. Unlike the majority of current RDF databases, GEMS has been designed from the ground up to primarily employ graph-based methods. This is reflected in all the layers of its stack. The GEMS framework is composed of: a SPARQL-to-C++ compiler, a library of data structures and related methods to access and modify them, and a custom runtime providing lightweight software multithreading, network messages aggregation and a partitioned global address space. We provide an overview of the framework, detailing its component and how they have been closely designed and customized to address issues of graph methods applied to large-scale datasets on clusters. We discuss in details the principles that enable automatic translation of the queries (expressed in SPARQL, the query language of choice for RDF databases) to graph methods, and identify differences with respect to other RDF databases.

GEMS↗
DOE OSTI2014

A Scalable Solution Framework for Stochastic Transmission and Generation Planning Problems. Draft

Current commercial software tools for transmission and generation investment planning have limited stochastic modeling capabilities. Because of this limitation, electric power utilities generally rely on scenario planning heuristics to identify potentially robust and cost effective investment plans for a broad range of system, economic, and policy conditions. Several research studies have shown that stochastic models perform significantly better than deterministic or heuristic approaches, in terms of overall costs. However, there is a lack of practical solution approaches to solve such models. In this paper we propose a scalable decomposition algorithm to solve stochastic transmission and generation planning problems, respectively considering discrete and continuous decision variables for transmission and generation investments. Given stochasticity restricted to loads and wind, solar, and hydro power output, we develop a simple scenario reduction framework based on a clustering algorithm, to yield a more tractable model. The resulting stochastic optimization model is decomposed on a scenario basis and solved using a variant of the Progressive Hedging (PH) algorithm. We perform numerical experiments using a 240-bus network representation of the Western Electricity Coordinating Council in the US. Although convergence of PH to an optimal solution is not guaranteed for mixed-integer linear optimization models, we find that it is possible to obtain solutions with acceptable optimality gaps for practical applications. Our numerical simulations are performed both on a commodity workstation and on a high-performance cluster. The results indicate that large-scale problems can be solved to a high degree of accuracy in at most two hours of wall clock time.

24 POWER TRANSMISSION AND DISTRIBUTION↗
NASA NTRS2000

Message Passing vs. Shared Address Space on a Cluster of SMPs

The convergence of scalable computer architectures using clusters of PCs (or PC-SMPs) with commodity networking has become an attractive platform for high end scientific computing. Currently, message-passing and shared address space (SAS) are the two leading programming paradigms for these systems. Message-passing has been standardized with MPI, and is the most common and mature programming approach. However message-passing code development can be extremely difficult, especially for irregular structured computations. SAS offers substantial ease of programming, but may suffer from performance limitations due to poor spatial locality, and high protocol overhead. In this paper, we compare the performance of and programming effort, required for six applications under both programming models on a 32 CPU PC-SMP cluster. Our application suite consists of codes that typically do not exhibit high efficiency under shared memory programming. due to their high communication to computation ratios and complex communication patterns. Results indicate that SAS can achieve about half the parallel efficiency of MPI for most of our applications: however, on certain classes of problems SAS performance is competitive with MPI. We also present new algorithms for improving the PC cluster performance of MPI collective operations.

Shan, Hongzhang↗
DOE OSTI2015

Optimizing CMS build infrastructure via Apache Mesos

The Offline Software of the CMS Experiment at the Large Hadron Collider (LHC) at CERN consists of 6M lines of in-house code, developed over a decade by nearly 1000 physicists, as well as a comparable amount of general use open-source code. A critical ingredient to the success of the construction and early operation of the WLCG was the convergence, around the year 2000, on the use of a homogeneous environment of commodity x86-64 processors and Linux.Apache Mesos is a cluster manager that provides efficient resource isolation and sharing across distributed applications, or frameworks. It can run Hadoop, Jenkins, Spark, Aurora, and other applications on a dynamically shared pool of nodes. Lastly, we present how we migrated our continuous integration system to schedule jobs on a relatively small Apache Mesos enabled cluster and how this resulted in better resource usage, higher peak performance and lower latency thanks to the dynamic scheduling capabilities of Mesos.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
NASA NTRS2005

Performance Comparison of Mainframe, Workstations, Clusters, and Desktop Computers

A performance evaluation of a variety of computers frequently found in a scientific or engineering research environment was conducted using a synthetic and application program benchmarks. From a performance perspective, emerging commodity processors have superior performance relative to legacy mainframe computers. In many cases, the PC clusters exhibited comparable performance with traditional mainframe hardware when 8-12 processors were used. The main advantage of the PC clusters was related to their cost. Regardless of whether the clusters were built from new computers or whether they were created from retired computers their performance to cost ratio was superior to the legacy mainframe computers. Finally, the typical annual maintenance cost of legacy mainframe computers is several times the cost of new equipment such as multiprocessor PC workstations. The savings from eliminating the annual maintenance fee on legacy hardware can result in a yearly increase in total computational capability for an organization.

Farley, Douglas L.↗
DOE OSTI2014

Nanoscale Cluster Detection in Massive Atom Probe Tomography Data

Recent technological advances in atom probe tomography (APT) have led to unprecedented data acquisition capabilities that routinely generate data sets containing hundreds of millions of atoms. Detecting nanoscale clusters of different atom types present in these enormous amounts of data and analyzing their spatial correlations with one another are fundamental to understanding the structural properties of the material from which the data is derived. Extant algorithms for nanoscale cluster detection do not scale to large data sets. Here, a scalable, CUDA-based implementation of an autocorrelation algorithm is presented. It isolates spatial correlations amongst atomic clusters present in massive APT data sets in linear time using a linear amount of storage. Correctness of the algorithm is demonstrated using large synthetically generated data with known spatial distributions. Benefits and limitations of using GPU-acceleration for autocorrelation-based APT data analyses are presented with supporting performance results on data sets with up to billions of atoms. To our knowledge, this is the first nanoscale cluster detection algorithm that scales to massive APT data sets and executes on commodity hardware.

Seal, Sudip K↗
DOE OSTI2025

A Cocatalytic System for Electrooxidation of Primary, Secondary, and Benzyl Alcohols Based on a Triruthenium Oxo-Centered Cluster and NHPI

Increasing interest in alternative methods for fuel generation and chemical synthesis has resulted in an increased focus on the development of electrocatalysts for energy relevant small molecule transformations, such as the oxidation of methanol. Partial methanol oxidation is a crucial step in the generation of the commodity chemical formaldehyde, and its complete oxidation to carbon dioxide can also serve as the anodic reaction in direct methanol fuel cells. We report a coelectrocatalytic system comprised of an oxo-centered triruthenium cluster (Ru 3 O) as the catalyst, with the electro-generated N-phthalimido-N-oxyl (PINO) radical species acting as a redox mediator. Only a mild Brønsted base, 2,6-lutidine, is required to achieve an electrocatalytic response. The cocatalytic system demonstrates remarkable cooperativity, shifting the oxidation potential of MeOH (Ep) less positive by ca. 0.5 V compared to the intrinsic response of the Ru 3 O complex. Controlled potential electrolysis on a model substrate, 4-trifluoromethylbenzyl alcohol, demonstrates selective production of the two-electron, two-proton aldehyde product with a Faradaic efficiency of 79 ± 11% at a rate of 3.14 s –1 . The rate of cocatalysis is 50-fold greater than the intrinsic activity of Ru3O and 26-fold greater than that of PINO alone under otherwise identical conditions. Mechanistic studies reveal the oxidation of a Ru 3 O–alkoxide species as the potential-determining step, while two possible rate-determining steps are identified depending on the substrate. A preference for sterically uninhibited electron-rich benzyl alcohol substrates suggests that a H atom transfer from the Ru 3 O–alkoxide adduct to PINO is rate-determining, while the lack of an observed kinetic isotope effect using deuterated MeOH suggests the oxidation of the Ru 3 O–alkoxide species is both rate- and potential-determining for cocatalysis.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2017

Rapid Calculation of Max-Min Fair Rates for Multi-Commodity Flows in Fat-Tree Networks

Max-min fairness is often used in the performance modeling of interconnection networks. Existing methods to compute max-min fair rates for multi-commodity flows have high complexity and are computationally infeasible for large networks. In this paper, we show that by considering topological features, this problem can be solved efficiently for the fat-tree topology that is widely used in data centers and high performance compute clusters. Several efficient new algorithms are developed for this problem, including a parallel algorithm that can take advantage of multi-core and shared-memory architectures. Using these algorithms, we demonstrate that it is possible to find the max-min fair rate allocation for multi-commodity flows in fat-tree networks that support tens of thousands of nodes. We evaluate the run-time performance of the proposed algorithms and show improvement in orders of magnitude over the previously best known method. Finally, we further demonstrate a new application of max-min fair rate allocation that is only computationally feasible using our new algorithms.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS1998

An Examination of the Performance of Parallel Calculation of the Radiation Integral on a Beowulf-Class Computer

This paper uses the parallel calculation of the radiation integral for examination of performance and compiler issues on a Beowulf-class computer. This type of computer, built from mass-market, commodity, off-the-shelf components, has limited communications performance and therefore also has a limited regime of codes for which it is suitable.

Beowulf cluster pile of PCs parallel computation p↗
DOE OSTI2024

Catalyst design for small molecule activation of energy consequence

This project targets the conversion of ubiquitous small molecules (e.g. NO, CO, H 2 O) into viable precursors to synthetic fuels. Current state of the art catalyst design has not directly targeted transition metal complexes capable of mediating the multi-electron redox processes necessary to reduce the overpotential (energy loss) required achieve efficient activation of small molecule substrates. In this vein, a new strategy has been developed for the assembly of polynuclear architectures; allowing for the construction of tunable polymetallic centers that assemble easily within a pre-organized template (conferring stability, selectivity and tunability) that can effect multi-electron redox processes for reactions. Catalyst development has commenced with the following target design elements: (1) catalysts featuring multiple transition metal ions in the same reaction space to greatly expand accessible molecular redox capabilities; (2) catalysts are assembled in a polynucleating ligand framework that permits control over the cluster morphology as well as the local steric and electronic environment of the transition metal ions within the cluster. The high-tunability of the catalyst composition (metal content) and geometric flexibility has permitted a rigorous assessment of electronic-structure-to-function relationship to be developed, further guiding synthetic efforts to realize more potent catalysts. The numerous permutations possible showcase the high degree of generality to this approach with many synthetic handles to tune redox and reaction chemistry. Trinuclear complexes have been synthesized featuring homo- and hetero-trinuclear cores featuring a variety of first row transition metal ions (Cr→Ni). The molecular clusters have been shown to successfully mediate multi-electron redox processes in a cooperative fashion without requiring strong chemical reductants or oxidants. The reactive molecular complexes are being utilized to activate and breakdown the robust bonds within typical waste stream small molecules (e.g., greenhouse gases) and convert them into value-added commodity chemicals. Ultimately, the catalysts developed by this approach will be required to convert energy acquired via renewable resources (e.g., solar or wind) into synthetic fuels as an energy storage mechanism.

10 SYNTHETIC FUELS↗
DOE OSTI2019

Evaluation of Impact of Potential Extreme Rainfall Events on Mining in Peru

The impact that climate change may play in the future sustainability of mining projects has become increasingly important for the mining industry and its stakeholders. The most significant areas of concern are mine infrastructure, supply chains, health and safety conditions, environmental management, community relations and exploration. This is particularly relevant to mining in a country as climatically vulnerable as Peru. This study focuses on the identification of mining regions and main commodities in Peru that are potentially vulnerable to future extreme rainfall events associated with climate change. From a mine design and planning perspective, this study is a first step to illustrate the importance of considering the impacts of different climatic scenarios on mining in Peru. Based on HadGEM2-ES global climate model projections, mining regions across Peru were clustered into “super-regions” with differing potentials of extreme rainfall events during the next three decades. Five indices for precipitation extremes were used, and their variations between 1971–2000 and 2015–2034 were computed. Current and future metallic mining projects expected to take place across Peru in the next 30 years were retrieved from a mining database and subsequently exported into a geographical information systems software to represent their location and interpolate the variation for each precipitation extreme index. The results of this study point out at a decreasing trend in rainfall extremes intensity and frequency in regions of southern Peru. For copper projects located in these regions, a decrease in rainfall events could also imply an eventual decrease in total precipitation and consequently a deficit in water availability during the next three decades. Mining regions in central Peru, with significant number of zinc projects, are likely to experience a marked increase in overall annual precipitation, average daily precipitation intensity, consecutive days of precipitation and number of heavy precipitation days. At a lesser extent than in central Peru, gold projects in northern Peru are also likely to experience an overall increase in precipitation extremes. The approach used for this research could be extended to other mining regions around the world with extreme weather events.

58 GEOSCIENCES↗
DOE OSTI2019

MetaBAT 2: an adaptive binning algorithm for robust and efficient genome reconstruction from metagenome assemblies

We previously reported on MetaBAT, an automated metagenome binning software tool to reconstruct single genomes from microbial communities for subsequent analyses of uncultivated microbial species. MetaBAT has become one of the most popular binning tools largely due to its computational efficiency and ease of use, especially in binning experiments with a large number of samples and a large assembly. MetaBAT requires users to choose parameters to fine-tune its sensitivity and specificity. If those parameters are not chosen properly, binning accuracy can suffer, especially on assemblies of poor quality. Here, we developed MetaBAT 2 to overcome this problem. MetaBAT 2 uses a new adaptive binning algorithm to eliminate manual parameter tuning. We also performed extensive software engineering optimization to increase both computational and memory efficiency. Comparing MetaBAT 2 to alternative software tools on over 100 real world metagenome assemblies shows superior accuracy and computing speed. Binning a typical metagenome assembly takes only a few minutes on a single commodity workstation. We therefore recommend the community adopts MetaBAT 2 for their metagenome binning experiments. MetaBAT 2 is open source software and available at https://bitbucket.org/berkeleylab/metabat .

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2014

Identification and characterization of the polyketide synthase involved in ochratoxin A biosynthesis in Aspergillus carbonarius

Ochratoxin A (OTA) is a potent mycotoxin produced by Aspergillus and Penicillium species and is a common contaminant of a wide variety of food commodities, with Aspergillus carbonarius being the main producer of OTA contamination in grapes and wine. The molecular structure of OTA is composed of a dihydroisocoumarin ring linked to phenylalanine and, as shown in different producing fungal species, a polyketide synthase (PKS) is a component of the OTA biosynthetic pathway. Similar to observations in other filamentous ascomycetes, the genome sequence of A. carbonarius contains a large number of genes predicted to encode PKSs. In this work a pks gene identified within the putative OTA cluster of A. carbonarius, designated as AcOTApks, was inactivated and the resulting mutant strain was unable to produce OTA, confirming the role of AcOTApks in this biosynthetic pathway. AcOTApks protein is characteristic of the highly reduced (HR)-PKS family, and also contains a putative methyltransferase domain likely responsible for the addition of the methyl group to the OTA polyketide structure. AcOTApks is different from the ACpks protein that we previously described which showed an expression profile compatible with OTA production. We performed phylogenetic analyses of the β-ketosynthase and acyl-transferase domains of the OTA PKSs which had been identified and characterized in different OTA producing fungal species. The phylogenetic results were similar for both the two domains analyzed and showed that OTA PKS of A. carbonarius, Aspergillus niger, and Aspergillus ochraceus clustered in a monophyletic group with 100% bootstrap support suggesting a common origin, while the other OTA PKSs analyzed were phylogenetically distant. A qRT-PCR assay monitored AcOTApks expression during fungal growth and concomitant production of OTA by A. carbonarius in synthetic grape medium. A clear correlation between the expression profile of AcOTApks and kinetics of OTA production was observed with AcOTApks which reached its maximum level of transcription before OTA accumulation in mycelium reached its highest level, confirming the fact that gene transcription always precedes phenotypic production.

fungi, Aspergillus, mycotoxin, ochratoxin, carbona↗