Search NASA⌕ Search

SEARCH · Search NASA

Results for “HDFS”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records

A Monte Carlo Simulation Study of Interference Effect from Multiple HDFS Transmitters above 30 GHz

Commercial operators are now proposing to install thousands of High Density Fixed Services (HDFS) microwave transmitters in large urban centers, such as Los Angeles. These transmitters will share the same frequencies in the Ka band (32 GHz and 37 - 38 GHz) as some Space Research Service (SRS) receiving Earth stations. To face this challenge, Resolution 126 (WRC-97) has requested the International Telecommunications Union-Radiocommunications (ITU-R) to conduct, as a matter of urgency and in time for WRC-99, appropriate studies to determine sharing criteria between stations in the fixed service and stations in other services. The three DSN tracking stations worldwide utilize this frequency band and may become vulnerable to interference from the planned deployments of HDFS transmitters. These HDFS transmitters operate at a relatively strong signal power (up to -60 dBW/Hz). Thus, they will seriously interfere with the sensitive DSN receivers. It has become imperative to accurately predict the impact of HDFS transmitters on NASA's DSN receivers in the Ka band.

Ho, Christian↗

Offloading Calculations to Computational Storage Devices: Spark and HDFS [Slides]

The objective is to evaluate the capabilities of multiple CSDs (provided by NDG Systems) using Hadoop Filesystem and Apache Spark. The independent variables are: number of CSDs, 0, 1, 2, 4, or 6; size of dataset, 1 GB, 5 GB, 10 GB; type of dataset, one large file with all of the data, 10 files, 100 files. The dependent variables are: job time; execution time. the constants are operations on the dataset.

97 MATHEMATICS AND COMPUTING↗

Analysis of combined hydrogen, heat, and power as a bridge to a hydrogen transition.

Combined hydrogen, heat, and power (CHHP) technology is envisioned as a means to providing heat and electricity, generated on-site, to large end users, such as hospitals, hotels, and distribution centers, while simultaneously producing hydrogen as a by-product. The hydrogen can be stored for later conversion to electricity, used on-site (e.g., in forklifts), or dispensed to hydrogen-powered vehicles. Argonne has developed a complex-adaptive-system model, H2CAS, to simulate how vehicles and infrastructure can evolve in a transition to hydrogen. This study applies the H2CAS model to examine how CHHP technology can be used to aid the transition to hydrogen. It does not attempt to predict the future or provide one forecast of system development. Rather, the purpose of the model is to understand how the system works. The model uses a 50- by 100-mile rectangular grid of 1-square-mile cells centered on the Los Angeles metropolitan area. The major expressways are incorporated into the model, and local streets are considered to be ubiquitous, except where there are natural barriers. The model has two types of agents. Driver agents are characterized by a number of parameters: home and job locations, income, various types of 'personalities' reflective of marketing distinctions (e.g., innovators, early adopters), willingness to spend extra money on 'green' vehicles, etc. At the beginning of the simulations, almost all driver agents own conventional vehicles. They drive around the metropolitan area, commuting to and from work and traveling to various other destinations. As they do so, they observe the presence or absence of facilities selling hydrogen. If they find such facilities conveniently located along their routes, they are motivated to purchase a hydrogen-powered vehicle when it becomes time to replace their present vehicle. Conversely, if they find that they would be inconvenienced by having to purchase hydrogen earlier than necessary or if they become worried that they would run out of fuel before encountering a facility, their motivation to purchase a hydrogen-powered vehicle decreases. At vehicle purchase time, they weigh this experience, as well as other factors such as social influence by their peers, fuel cost, and capital cost of a hydrogen vehicle. Investor agents build full-service hydrogen fueling stations (HFSs) at different locations along the highway network. They base their decision to build or not build a station on their (imperfect) estimates of the sales the station would immediately generate (based on hydrogen-powered vehicle traffic past the location and other factors), as well as the growth in hydrogen sales they could expect throughout their investment horizon. The interaction between driver and investor agents provides the basis for growth in both the number of hydrogen vehicles and number of hydrogen stations. For the present report, we have added to this mix smaller, 'bare-bones' hydrogen dispensing facilities (HDFs) of the type that owners of CHHP facilities could provide to the public. The locations of these stations were chosen to match existing facilities that might reasonably incorporate CHHP plants in the future. Unlike the larger commercial stations, these facilities are built according to exogenously supplied timetables, and no attempt has been made to model the financial basis for the facilities. Rather, our objective is to understand how the presence of these additional stations might facilitate the petroleum-to-hydrogen transition. We discuss a base case in which the HDFs are not present, and then investigate the effects of introducing HDFs in various numbers; according to different timetables; with various production capacities; and with hydrogen selling at prices above, equal to, and below the commercial stations selling price. We conclude that HDFs can indeed be helpful in accelerating a petroleum-to-hydrogen transition. Placed in areas where investors might not be willing to install large for-profit HFSs, HDFs can serve as a bridge until demand for hydrogen increases to the point where larger stations are viable. For drivers to use them, however, HDFs must sell hydrogen at a price competitive with that of the larger HFSs. Because the marginal cost of producing hydrogen is expected to be higher for small capacities, HDFs may be uncompetitive without (possibly substantial) subsidies or other means to allow them to sell hydrogen at a price below the actual cost to produce and distribute it. Fortunately, HDFs do not need to be subsidized for very long. In our model, installing HDFs only as the demand for hydrogen increased was effective; then closing down all HDFs after five years into the transition made virtually no difference to the resultant accelerated outcome. Even installing much smaller numbers of HDFs only in the second and third years of the simulation and then closing them at the end of the fifth year was helpful.

08 HYDROGEN↗

The geospatial and economic viability of CO 2 storage in hydrocarbon depleted fractured shale formations

Hydrocarbon depleted fractured shale (HDFS) formations could be attractive for geologic carbon dioxide (CO 2 ) storage. Shale formations may be able to leverage existing infrastructure, have larger capacities, and be more secure than saline aquifers. We compared regional storage capacities and integrated CO 2 capture, transport, and storage systems that use HDFS with those that use saline aquifers in a region of the United States with extensive shale development that overlies prospective saline aquifers. We estimated HDFS storage capacities with a production-based method and costs by adapting methods developed for saline aquifers and found that HDFS formations in this region might be able to store with less cost an estimated ~14× more CO 2 on average than saline aquifers at the same location. The potential for smaller Areas of Review and less investment in infrastructure accounted for up to 84% of the difference in estimated storage costs. We implemented an engineering-economic geospatial optimization model to determine and compare the viability of storage capacity for these two storage resources. Across the state-specific and regional scenarios we investigated, our results for this region suggest that integrated CCS systems using HDFS could be more centralized, require less pipelines, prioritize different routes for trunklines, and be 6.4–6.8% ($5-10/tCO 2 ) cheaper than systems using saline aquifers. In conclusion, overall, CO 2 storage in HDFS could be technically and economically attractive and may lower barriers to large scale CO 2 storage if they can be permitted.

42 ENGINEERING↗

How fast can one resize a distributed file system?

Efficient resource utilization becomes a major concern as large-scale distributed computing infrastructures keep growing in size. Malleability, the possibility for resource managers to dynamically increase or decrease the amount of resources allocated to a job, is a promising way to save energy and costs. However, state-of-the-art parallel and distributed storage systems have not been designed with malleability in mind. The reason is mainly the supposedly high cost of data transfers required by resizing operations. Nevertheless, as network and storage technologies evolve, old assumptions about potential bottlenecks can be revisited. In this study, we evaluate the viability of malleability as a design principle for a distributed storage system. We specifically model the minimal duration of the commission and decommission operations. To show how our models can be used in practice, we evaluate the performance of these operations in HDFS, a relevant state-of-the-art distributed file system. We show that the existing decommission mechanism of HDFS is good when the network is the bottleneck, but can be accelerated by up to a factor 3 when storage is the limiting factor. We also show that the commission in HDFS can be substantially accelerated. With the highlights provided by our model, we suggest improvements to speed both operations in HDFS. We discuss how the proposed models can be generalized for distributed file systems with different assumptions and what perspectives are open for the design of efficient malleable distributed file systems.

97 MATHEMATICS AND COMPUTING↗

Circular RNA profiling reveals that circCOL3A1-859267 regulate type I collagen expression in photoaged human dermal fibroblasts

Production of type I collagen declines is a main characteristic during photoaging, but the mechanism is still not fully understood. Circular RNAs (circRNAs) are a class of newly identified non-coding RNAs with regulatory potency by sequestering miRNAs like a sponge. It's more stable than linear RNAs, and would be a useful tool for regulation of gene expression. However, the role of circRNAs in collagen expression during photoaging is still unclear. Here we performed deep sequencing of RNA generated from UVA irradiated and no irradiated human dermal fibroblasts (HDFs) and identified 29 significantly differentially expressed circRNAs (fold change ≥ 1.5, P < 0.05), 12 circRNAs were up-regulated and 17 circRNAs were down-regulated.3 most differentially expressed circRNAs were verified by qRT-PCR and the down-regulated circCOL3A1-859267 exhibited the most significantly altered in photoaged HDFs. Overexpression of circCOL3A1-859267 inhibited UVA-induced decrease of type I collagen expression and silencing of it reduced type I collagen intensity. Via a bioinformatic method, 44 miRNAs were predicted to binding with circCOL3A1-859267, 5 of them have been confirmed or predicted to interact with type I collagen. This study show that circCOL3A1-859267 regulate type I collagen expression in photoaged HDFs, suggesting it may be a novel target for interfering photoaging.

60 APPLIED LIFE SCIENCES↗

Statistical analysis of the horizontal divergent flow in emerging solar active regions

Solar active regions (ARs) are thought to be formed by magnetic fields from the convection zone. Our flux emergence simulations revealed that a strong horizontal divergent flow (HDF) of unmagnetized plasma appears at the photosphere before the flux begins to emerge. In our earlier study, we analyzed HMI data for a single AR and confirmed presence of this precursor plasma flow in the actual Sun. In this paper, as an extension of our earlier study, we conducted a statistical analysis of the HDFs to further investigate their characteristics and better determine the properties. From SDO/HMI data, we picked up 23 flux emergence events over a period of 14 months, the total flux of which ranges from 10{sup 20} to 10{sup 22} Mx. Out of 23 selected events, 6 clear HDFs were detected by the method we developed in our earlier study, and 7 HDFs detected by visual inspection were added to this statistic analysis. We found that the duration of the HDF is on average 61 minutes and the maximum HDF speed is on average 3.1 km s{sup –1}. We also estimated the rising speed of the subsurface magnetic flux to be 0.6-1.4 km s{sup –1}. These values are highly consistent with our previous one-event analysis as well as our simulation results. The observation results lead us to the conclusion that the HDF is a rather common feature in the earliest phase of AR emergence. Moreover, our HDF analysis has the capability of determining the subsurface properties of emerging fields that cannot be directly measured.

79 ASTRONOMY AND ASTROPHYSICS↗

MARIANE: MApReduce Implementation Adapted for HPC Environments

MapReduce is increasingly becoming a popular framework, and a potent programming model. The most popular open source implementation of MapReduce, Hadoop, is based on the Hadoop Distributed File System (HDFS). However, as HDFS is not POSIX compliant, it cannot be fully leveraged by applications running on a majority of existing HPC environments such as Teragrid and NERSC. These HPC environments typicallysupport globally shared file systems such as NFS and GPFS. On such resourceful HPC infrastructures, the use of Hadoop not only creates compatibility issues, but also affects overall performance due to the added overhead of the HDFS. This paper not only presents a MapReduce implementation directly suitable for HPC environments, but also exposes the design choices for better performance gains in those settings. By leveraging inherent distributed file systems&#x27; functions, and abstracting them away from its MapReduce framework, MARIANE (MApReduce Implementation Adapted for HPC Environments) not only allows for the use of the model in an expanding number of HPCenvironments, but also allows for better performance in such settings. This paper shows the applicability and high performance of the MapReduce paradigm through MARIANE, an implementation designed for clustered and shared-disk file systems and as such not dedicated to a specific MapReduce solution. The paper identifies the components and trade-offs necessary for this model, and quantifies the performance gains exhibited by our approach in distributed environments over Apache Hadoop in a data intensive setting, on the Magellan testbed at the National Energy Research Scientific Computing Center (NERSC).

97 MATHEMATICS AND COMPUTING↗

High Throughput WAN Data Transfer with Hadoop-Based Storage

Hadoop distributed file system (HDFS) is becoming more popular in recent years as a key building block of integrated grid storage solution in the field of scientific computing. Wide Area Network (WAN) data transfer is one of the important data operations for large high energy physics experiments to manage, share and process datasets of PetaBytes scale in a highly distributed grid computing environment. In this paper, we present the experience of high throughput WAN data transfer with HDFS-based Storage Element. Two protocols, GridFTP and fast data transfer (FDT), are used to characterize the network performance of WAN data transfer.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

Evaluating the Impact of Data Placement to Spark and SciDB with an Earth Science Use Case

We investigate the impact of data placement for two Big Data technologies, Spark and SciDB, with a use case from Earth Science where data arrays are multidimensional. Simultaneously, this investigation provides an opportunity to evaluate the performance of the technologies involved. Two datastores, HDFS and Cassandra, are used with Spark for our comparison. It is found that Spark with Cassandra performs better than with HDFS, but SciDB performs better yet than Spark with either datastore. The investigation also underscores the value of having data aligned for the most common analysis scenarios in advance on a shared nothing architecture. Otherwise, repartitioning needs to be carried out on the fly, degrading overall performance.

Spark↗

Diamondites: Evidence for a Distinct Tectono-Thermal Diamond-Forming Event Beneath the Kaapvaal Craton

The petrogenesis and relationship of diamondite to well-studied monocrystalline and fibrous diamonds are poorly understood yet would potentially reveal new aspects of how diamond-forming fluids are transported through the lithosphere and equilibrate with surrounding silicates. Of twenty-two silicate- and oxide-bearing diamondites investigated, most yielded garnet intergrowths (n = 15) with major element geochemistry (i.e. Ca-Cr) classifying these samples as low-Ca websteritic or eclogitic. The garnet REE patterns fit an equilibrium model suggesting the diamond-forming fluid shares an affinity with high-density fluids (HDF) observed in fibrous diamonds, specifically on the join between the saline–carbonate end- members. The δ13C values for the diamonds range from -5.27 to -22.48 ‰ (V-PDB) with δ18O values for websteritic garnets ranging from +7.6 to +5.9 ‰ (V-SMOW). The combined C-O stable isotope data support a model for a hydrothermally altered and organic carbon-bearing subducted crustal source(s) for the diamond- and garnet-forming media. The nitrogen aggregation states of the diamonds require that diamondite-formation event(s) pre-dates fibrous diamond-formation and post-dates most of the gem monocrystalline diamond-formation events at Orapa. The modelled fluid compositions responsible for the precipitation of diamondites match the fluid-poor and fluid-rich (fibrous) monocrystalline diamonds, where all grow from HDFs within the saline-silicic-carbonatitic ternary system. However, while the nature of the parental fluid(s) share a common lithophile element geochemical affinity, the origin(s) of the saline, silicic, and/or carbonatitic components of these HDFs do not always share a common origin. Therefore, it is wholly conceivable that the diamondites are evidence of a distinct and temporally unconstrained tectono-thermal diamond-forming event beneath the Kaapvaal craton.

S Mikhail↗

Deactivation and decommissioning web log analysis using big data technology - 15710

The D and D Knowledge Management Information Tool (D and D KM- IT) is a web-based knowledge management information tool built for the D and D user community. Big data is a massive volume of structured and unstructured datasets which is so large that it's difficult to process using traditional database techniques. Web logs are the repository of files which are generated automatically for any operation on the web site. Web log files generated from the D and D KM-IT will be processed using the Apache Hadoop Framework to extract meaningful data. Hadoop is an open-source software framework for storing and processing big data in a distributed environment on large clusters of commodity hardware. Hadoop framework consists of two main layers: Hadoop distributed file system (HDFS) and the Execution engine (MapReduce). Automatic parallelization and distribution; clean and simple programming abstraction. Implementation: Web log files from the D and D KM-IT server are fetched using Apache Flume and loaded into the Hadoop distributed file system. Using the Apache Hcatalog tool, the web log data is parsed and converted into a structured table format. A Pig program is developed to query the tables to retrieve and store the significant information onto the HDFS. The extracted data is fed to business visualization tools such as Microsoft Excel for analysis and reporting. Results The reports generated through the web log data processing will be analyzed to improve the usability and performance of the D and D KM-IT. Conclusion/Future Work: User keyword information can be explored to add new trending topics to the D and D KM-IT application. Statistics generated can be compared to analysis reports from existing analytical tools.

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗

HDF5 Roadmap 2019-2020

In this talk we will give an overview of the new features of the upcoming HDF5 release 1.12.0, and outline the HDF5 roadmap for the next year. We will demonstrate new open source file drivers to access HDF5 files via Amazon Simple Storage Service (Amazon S3) and on Hadoop Distributed File system (HDFS). We will use this presentation to get feedback on the HDF5 roadmap from the ESDIS users and application developers.

Cloud↗

Cell alignment by smectic liquid crystal elastomer coatings with nanogrooves

Control of cells behavior through topography of substrates is a vital theme in biomedical applications. Among many materials used as substrates, polymers show advantages since they can be tailored by chemical functionalization. Fabrication of polymer substrates with nano- and microscale topography requires processing by lithography, microprinting, etching, etc. In this work, we present a different approach based on anisotropic elastic properties of polymerized smectic A (SmA) liquid crystal elastomer (LCE). When the SmA liquid crystal coating is deposited onto a substrate with planar alignment of the molecules, it develops nanogrooves at its free surface. After photopolymerization, these nanogrooves show an excellent ability to align human dermal fibroblasts (hDFs) over large areas. The alignment quality is good for both bare SmA LCE substrates and for substrates coated with fibronectin. The SmA LCE nano-topographies show a high potential for tissue engineering.

36 MATERIALS SCIENCE↗

The Archive Solution for Distributed Workflow Management Agents of the CMS Experiment at LHC

The CMS experiment at the CERN LHC developed the Workflow Management Archive system to persistently store unstructured framework job report documents produced by distributed workflow management agents. In this paper we present its architecture, implementation, deployment, and integration with the CMS and CERN computing infrastructures, such as central HDFS and Hadoop Spark cluster. The system leverages modern technologies such as a document oriented database and the Hadoop eco-system to provide the necessary flexibility to reliably process, store, and aggregate $\mathcal{O}$(1M) documents on a daily basis. We describe the data transformation, the short and long term storage layers, the query language, along with the aggregation pipeline developed to visualize various performance metrics to assist CMS data operators in assessing the performance of the CMS computing system.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

Multifunctional Fibroblasts Enhanced via Thermal and Freeze-Drying Post-treatments of Aligned Electrospun Nanofiber Membranes

Parallel fibrous scaffolds play a critical role in controlling the morphology of cells to be more natural and biologically inspired. Among popular tissue engineering materials, poly(2-hydroxyethyl methacrylate) (pHEMA) has been widely investigated in conventional forms due to its biocompatibility, low toxicity, and hydrophilicity. However, the swelling of pHEMA in water remains a major concern. To address this issue, randomly oriented and aligned as-spun pHEMA nanofibrous scaffolds were first fabricated at speeds of 300 and 2000 rpm in this study, which were then post-treated using either a thermal or a freeze-drying method. In cell assays, human dermal fibroblasts (HDFs) adhered to the freeze-drying treated substrates at a significantly faster rate, whereas they had a higher cell growth rate on thermally-treated substrates. Results indicated that the structural properties of pHEMA nanofibrous scaffolds and subsequent cellular behaviors were largely dependent on post-treatment methods. Moreover, this study suggests that aligned pHEMA nanofibrous substrates tended to induce regular fibroblast orientation and unidirectionally oriented actin cytoskeletons over random pHEMA nanofibrous substrates. Such information has predictive power and provides insights into promising post-treatment methods for improving the properties of aligned pHEMA scaffolds for numerous tissue engineering applications.

36 MATERIALS SCIENCE↗