Search NASA⌕ Search

SEARCH · Search NASA

Results for “HDFS”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records

Offloading Calculations to Computational Storage Devices: Spark and HDFS [Slides]

The objective is to evaluate the capabilities of multiple CSDs (provided by NDG Systems) using Hadoop Filesystem and Apache Spark. The independent variables are: number of CSDs, 0, 1, 2, 4, or 6; size of dataset, 1 GB, 5 GB, 10 GB; type of dataset, one large file with all of the data, 10 files, 100 files. The dependent variables are: job time; execution time. the constants are operations on the dataset.

97 MATHEMATICS AND COMPUTING↗

Analysis of combined hydrogen, heat, and power as a bridge to a hydrogen transition.

Combined hydrogen, heat, and power (CHHP) technology is envisioned as a means to providing heat and electricity, generated on-site, to large end users, such as hospitals, hotels, and distribution centers, while simultaneously producing hydrogen as a by-product. The hydrogen can be stored for later conversion to electricity, used on-site (e.g., in forklifts), or dispensed to hydrogen-powered vehicles. Argonne has developed a complex-adaptive-system model, H2CAS, to simulate how vehicles and infrastructure can evolve in a transition to hydrogen. This study applies the H2CAS model to examine how CHHP technology can be used to aid the transition to hydrogen. It does not attempt to predict the future or provide one forecast of system development. Rather, the purpose of the model is to understand how the system works. The model uses a 50- by 100-mile rectangular grid of 1-square-mile cells centered on the Los Angeles metropolitan area. The major expressways are incorporated into the model, and local streets are considered to be ubiquitous, except where there are natural barriers. The model has two types of agents. Driver agents are characterized by a number of parameters: home and job locations, income, various types of 'personalities' reflective of marketing distinctions (e.g., innovators, early adopters), willingness to spend extra money on 'green' vehicles, etc. At the beginning of the simulations, almost all driver agents own conventional vehicles. They drive around the metropolitan area, commuting to and from work and traveling to various other destinations. As they do so, they observe the presence or absence of facilities selling hydrogen. If they find such facilities conveniently located along their routes, they are motivated to purchase a hydrogen-powered vehicle when it becomes time to replace their present vehicle. Conversely, if they find that they would be inconvenienced by having to purchase hydrogen earlier than necessary or if they become worried that they would run out of fuel before encountering a facility, their motivation to purchase a hydrogen-powered vehicle decreases. At vehicle purchase time, they weigh this experience, as well as other factors such as social influence by their peers, fuel cost, and capital cost of a hydrogen vehicle. Investor agents build full-service hydrogen fueling stations (HFSs) at different locations along the highway network. They base their decision to build or not build a station on their (imperfect) estimates of the sales the station would immediately generate (based on hydrogen-powered vehicle traffic past the location and other factors), as well as the growth in hydrogen sales they could expect throughout their investment horizon. The interaction between driver and investor agents provides the basis for growth in both the number of hydrogen vehicles and number of hydrogen stations. For the present report, we have added to this mix smaller, 'bare-bones' hydrogen dispensing facilities (HDFs) of the type that owners of CHHP facilities could provide to the public. The locations of these stations were chosen to match existing facilities that might reasonably incorporate CHHP plants in the future. Unlike the larger commercial stations, these facilities are built according to exogenously supplied timetables, and no attempt has been made to model the financial basis for the facilities. Rather, our objective is to understand how the presence of these additional stations might facilitate the petroleum-to-hydrogen transition. We discuss a base case in which the HDFs are not present, and then investigate the effects of introducing HDFs in various numbers; according to different timetables; with various production capacities; and with hydrogen selling at prices above, equal to, and below the commercial stations selling price. We conclude that HDFs can indeed be helpful in accelerating a petroleum-to-hydrogen transition. Placed in areas where investors might not be willing to install large for-profit HFSs, HDFs can serve as a bridge until demand for hydrogen increases to the point where larger stations are viable. For drivers to use them, however, HDFs must sell hydrogen at a price competitive with that of the larger HFSs. Because the marginal cost of producing hydrogen is expected to be higher for small capacities, HDFs may be uncompetitive without (possibly substantial) subsidies or other means to allow them to sell hydrogen at a price below the actual cost to produce and distribute it. Fortunately, HDFs do not need to be subsidized for very long. In our model, installing HDFs only as the demand for hydrogen increased was effective; then closing down all HDFs after five years into the transition made virtually no difference to the resultant accelerated outcome. Even installing much smaller numbers of HDFs only in the second and third years of the simulation and then closing them at the end of the fifth year was helpful.

08 HYDROGEN↗

The geospatial and economic viability of CO 2 storage in hydrocarbon depleted fractured shale formations

Hydrocarbon depleted fractured shale (HDFS) formations could be attractive for geologic carbon dioxide (CO 2 ) storage. Shale formations may be able to leverage existing infrastructure, have larger capacities, and be more secure than saline aquifers. We compared regional storage capacities and integrated CO 2 capture, transport, and storage systems that use HDFS with those that use saline aquifers in a region of the United States with extensive shale development that overlies prospective saline aquifers. We estimated HDFS storage capacities with a production-based method and costs by adapting methods developed for saline aquifers and found that HDFS formations in this region might be able to store with less cost an estimated ~14× more CO 2 on average than saline aquifers at the same location. The potential for smaller Areas of Review and less investment in infrastructure accounted for up to 84% of the difference in estimated storage costs. We implemented an engineering-economic geospatial optimization model to determine and compare the viability of storage capacity for these two storage resources. Across the state-specific and regional scenarios we investigated, our results for this region suggest that integrated CCS systems using HDFS could be more centralized, require less pipelines, prioritize different routes for trunklines, and be 6.4–6.8% ($5-10/tCO 2 ) cheaper than systems using saline aquifers. In conclusion, overall, CO 2 storage in HDFS could be technically and economically attractive and may lower barriers to large scale CO 2 storage if they can be permitted.

42 ENGINEERING↗

How fast can one resize a distributed file system?

Efficient resource utilization becomes a major concern as large-scale distributed computing infrastructures keep growing in size. Malleability, the possibility for resource managers to dynamically increase or decrease the amount of resources allocated to a job, is a promising way to save energy and costs. However, state-of-the-art parallel and distributed storage systems have not been designed with malleability in mind. The reason is mainly the supposedly high cost of data transfers required by resizing operations. Nevertheless, as network and storage technologies evolve, old assumptions about potential bottlenecks can be revisited. In this study, we evaluate the viability of malleability as a design principle for a distributed storage system. We specifically model the minimal duration of the commission and decommission operations. To show how our models can be used in practice, we evaluate the performance of these operations in HDFS, a relevant state-of-the-art distributed file system. We show that the existing decommission mechanism of HDFS is good when the network is the bottleneck, but can be accelerated by up to a factor 3 when storage is the limiting factor. We also show that the commission in HDFS can be substantially accelerated. With the highlights provided by our model, we suggest improvements to speed both operations in HDFS. We discuss how the proposed models can be generalized for distributed file systems with different assumptions and what perspectives are open for the design of efficient malleable distributed file systems.

97 MATHEMATICS AND COMPUTING↗

Circular RNA profiling reveals that circCOL3A1-859267 regulate type I collagen expression in photoaged human dermal fibroblasts

Production of type I collagen declines is a main characteristic during photoaging, but the mechanism is still not fully understood. Circular RNAs (circRNAs) are a class of newly identified non-coding RNAs with regulatory potency by sequestering miRNAs like a sponge. It's more stable than linear RNAs, and would be a useful tool for regulation of gene expression. However, the role of circRNAs in collagen expression during photoaging is still unclear. Here we performed deep sequencing of RNA generated from UVA irradiated and no irradiated human dermal fibroblasts (HDFs) and identified 29 significantly differentially expressed circRNAs (fold change ≥ 1.5, P < 0.05), 12 circRNAs were up-regulated and 17 circRNAs were down-regulated.3 most differentially expressed circRNAs were verified by qRT-PCR and the down-regulated circCOL3A1-859267 exhibited the most significantly altered in photoaged HDFs. Overexpression of circCOL3A1-859267 inhibited UVA-induced decrease of type I collagen expression and silencing of it reduced type I collagen intensity. Via a bioinformatic method, 44 miRNAs were predicted to binding with circCOL3A1-859267, 5 of them have been confirmed or predicted to interact with type I collagen. This study show that circCOL3A1-859267 regulate type I collagen expression in photoaged HDFs, suggesting it may be a novel target for interfering photoaging.

60 APPLIED LIFE SCIENCES↗

Statistical analysis of the horizontal divergent flow in emerging solar active regions

Solar active regions (ARs) are thought to be formed by magnetic fields from the convection zone. Our flux emergence simulations revealed that a strong horizontal divergent flow (HDF) of unmagnetized plasma appears at the photosphere before the flux begins to emerge. In our earlier study, we analyzed HMI data for a single AR and confirmed presence of this precursor plasma flow in the actual Sun. In this paper, as an extension of our earlier study, we conducted a statistical analysis of the HDFs to further investigate their characteristics and better determine the properties. From SDO/HMI data, we picked up 23 flux emergence events over a period of 14 months, the total flux of which ranges from 10{sup 20} to 10{sup 22} Mx. Out of 23 selected events, 6 clear HDFs were detected by the method we developed in our earlier study, and 7 HDFs detected by visual inspection were added to this statistic analysis. We found that the duration of the HDF is on average 61 minutes and the maximum HDF speed is on average 3.1 km s{sup –1}. We also estimated the rising speed of the subsurface magnetic flux to be 0.6-1.4 km s{sup –1}. These values are highly consistent with our previous one-event analysis as well as our simulation results. The observation results lead us to the conclusion that the HDF is a rather common feature in the earliest phase of AR emergence. Moreover, our HDF analysis has the capability of determining the subsurface properties of emerging fields that cannot be directly measured.

79 ASTRONOMY AND ASTROPHYSICS↗

MARIANE: MApReduce Implementation Adapted for HPC Environments

MapReduce is increasingly becoming a popular framework, and a potent programming model. The most popular open source implementation of MapReduce, Hadoop, is based on the Hadoop Distributed File System (HDFS). However, as HDFS is not POSIX compliant, it cannot be fully leveraged by applications running on a majority of existing HPC environments such as Teragrid and NERSC. These HPC environments typicallysupport globally shared file systems such as NFS and GPFS. On such resourceful HPC infrastructures, the use of Hadoop not only creates compatibility issues, but also affects overall performance due to the added overhead of the HDFS. This paper not only presents a MapReduce implementation directly suitable for HPC environments, but also exposes the design choices for better performance gains in those settings. By leveraging inherent distributed file systems&#x27; functions, and abstracting them away from its MapReduce framework, MARIANE (MApReduce Implementation Adapted for HPC Environments) not only allows for the use of the model in an expanding number of HPCenvironments, but also allows for better performance in such settings. This paper shows the applicability and high performance of the MapReduce paradigm through MARIANE, an implementation designed for clustered and shared-disk file systems and as such not dedicated to a specific MapReduce solution. The paper identifies the components and trade-offs necessary for this model, and quantifies the performance gains exhibited by our approach in distributed environments over Apache Hadoop in a data intensive setting, on the Magellan testbed at the National Energy Research Scientific Computing Center (NERSC).

97 MATHEMATICS AND COMPUTING↗

High Throughput WAN Data Transfer with Hadoop-Based Storage

Hadoop distributed file system (HDFS) is becoming more popular in recent years as a key building block of integrated grid storage solution in the field of scientific computing. Wide Area Network (WAN) data transfer is one of the important data operations for large high energy physics experiments to manage, share and process datasets of PetaBytes scale in a highly distributed grid computing environment. In this paper, we present the experience of high throughput WAN data transfer with HDFS-based Storage Element. Two protocols, GridFTP and fast data transfer (FDT), are used to characterize the network performance of WAN data transfer.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

Deactivation and decommissioning web log analysis using big data technology - 15710

The D and D Knowledge Management Information Tool (D and D KM- IT) is a web-based knowledge management information tool built for the D and D user community. Big data is a massive volume of structured and unstructured datasets which is so large that it's difficult to process using traditional database techniques. Web logs are the repository of files which are generated automatically for any operation on the web site. Web log files generated from the D and D KM-IT will be processed using the Apache Hadoop Framework to extract meaningful data. Hadoop is an open-source software framework for storing and processing big data in a distributed environment on large clusters of commodity hardware. Hadoop framework consists of two main layers: Hadoop distributed file system (HDFS) and the Execution engine (MapReduce). Automatic parallelization and distribution; clean and simple programming abstraction. Implementation: Web log files from the D and D KM-IT server are fetched using Apache Flume and loaded into the Hadoop distributed file system. Using the Apache Hcatalog tool, the web log data is parsed and converted into a structured table format. A Pig program is developed to query the tables to retrieve and store the significant information onto the HDFS. The extracted data is fed to business visualization tools such as Microsoft Excel for analysis and reporting. Results The reports generated through the web log data processing will be analyzed to improve the usability and performance of the D and D KM-IT. Conclusion/Future Work: User keyword information can be explored to add new trending topics to the D and D KM-IT application. Statistics generated can be compared to analysis reports from existing analytical tools.

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗

Cell alignment by smectic liquid crystal elastomer coatings with nanogrooves

Control of cells behavior through topography of substrates is a vital theme in biomedical applications. Among many materials used as substrates, polymers show advantages since they can be tailored by chemical functionalization. Fabrication of polymer substrates with nano- and microscale topography requires processing by lithography, microprinting, etching, etc. In this work, we present a different approach based on anisotropic elastic properties of polymerized smectic A (SmA) liquid crystal elastomer (LCE). When the SmA liquid crystal coating is deposited onto a substrate with planar alignment of the molecules, it develops nanogrooves at its free surface. After photopolymerization, these nanogrooves show an excellent ability to align human dermal fibroblasts (hDFs) over large areas. The alignment quality is good for both bare SmA LCE substrates and for substrates coated with fibronectin. The SmA LCE nano-topographies show a high potential for tissue engineering.

36 MATERIALS SCIENCE↗

The Archive Solution for Distributed Workflow Management Agents of the CMS Experiment at LHC

The CMS experiment at the CERN LHC developed the Workflow Management Archive system to persistently store unstructured framework job report documents produced by distributed workflow management agents. In this paper we present its architecture, implementation, deployment, and integration with the CMS and CERN computing infrastructures, such as central HDFS and Hadoop Spark cluster. The system leverages modern technologies such as a document oriented database and the Hadoop eco-system to provide the necessary flexibility to reliably process, store, and aggregate $\mathcal{O}$(1M) documents on a daily basis. We describe the data transformation, the short and long term storage layers, the query language, along with the aggregation pipeline developed to visualize various performance metrics to assist CMS data operators in assessing the performance of the CMS computing system.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

Multifunctional Fibroblasts Enhanced via Thermal and Freeze-Drying Post-treatments of Aligned Electrospun Nanofiber Membranes

Parallel fibrous scaffolds play a critical role in controlling the morphology of cells to be more natural and biologically inspired. Among popular tissue engineering materials, poly(2-hydroxyethyl methacrylate) (pHEMA) has been widely investigated in conventional forms due to its biocompatibility, low toxicity, and hydrophilicity. However, the swelling of pHEMA in water remains a major concern. To address this issue, randomly oriented and aligned as-spun pHEMA nanofibrous scaffolds were first fabricated at speeds of 300 and 2000 rpm in this study, which were then post-treated using either a thermal or a freeze-drying method. In cell assays, human dermal fibroblasts (HDFs) adhered to the freeze-drying treated substrates at a significantly faster rate, whereas they had a higher cell growth rate on thermally-treated substrates. Results indicated that the structural properties of pHEMA nanofibrous scaffolds and subsequent cellular behaviors were largely dependent on post-treatment methods. Moreover, this study suggests that aligned pHEMA nanofibrous substrates tended to induce regular fibroblast orientation and unidirectionally oriented actin cytoskeletons over random pHEMA nanofibrous substrates. Such information has predictive power and provides insights into promising post-treatment methods for improving the properties of aligned pHEMA scaffolds for numerous tissue engineering applications.

36 MATERIALS SCIENCE↗

Senescence-related functional nuclear barrier by down-regulation of nucleo-cytoplasmic trafficking gene expression

One of the characteristic natures of senescent cells is the hypo- or irresponsiveness not only to growth factors but also to apoptotic stress. In the present study, we confirmed the inhibition of nuclear translocation of activated p-ERK1/2 and NF-kB p50 in response to growth stimuli or LPS in the senescent human diploid fibroblasts. In order to elucidate the underlying mechanism for the senescence-associated hypo-responsiveness, we carried out the comparison study for gene expression profiles through microarray analysis. In consequence, we observed the vast reduction in expression of nucleo-cytoplasmic trafficking genes in senescent cells, when compared with those in young cells. Expression levels of several nucleoporins, karyopherin {alpha}, karyopherin {beta}, Ran, and Ran-regulating factors were confirmed to be down-regulated in senescent HDFs by using RT-PCR and Western blot methods. Taken together, these data suggest the operation of certain senescence-associated functional nuclear barriers by down-regulation of the nucleo-cytoplasmic trafficking genes in the senescent cells.

60 APPLIED LIFE SCIENCES↗

Cell-free therapy based on adipose tissue stem cell-derived exosomes promotes wound healing via the PI3K/Akt signaling pathway

Highlights: • ADSC-Exos are internalized into HDFs and regulate their biological behaviors and functions. • ADSC-Exos play an important role in accelerating wound healing via activating PI3K/Akt signaling pathway. • ADSC-Exos may serve as a cell-free therapy for the potential clinical treatment of wound healing. Adipose tissue-derived stem cells (ADSCs) have been shown to enhance wound healing via their paracrine function. Exosomes, as one of the most important paracrine factors, play an essential role in this process. However, the concrete mechanisms that underlie this effect are poorly understood. In this study, we aim to explore the potential roles and molecular mechanisms of exosomes derived from ADSCs in cutaneous wound healing.

60 APPLIED LIFE SCIENCES↗

Controlled Placement of Microparticles at the Water–Liquid Crystal Elastomer Interface

Control of cells behavior through topography of substrates is an important theme in biomedical applications. Among many materials used as substrates, polymers show advantages since they can be tailored by chemical functionalization. Fabrication of polymer substrates with nano- and microscale topography requires processing by lithography, microprinting, etching, etc. In this work, we introduce a different approach based on anisotropic elastic properties of polymerized smectic A (SmA) liquid crystal elastomer (LCE). When the SmA liquid crystal coating is deposited onto a substrate with planar alignment of the molecules, it develops nanogrooves at its free surface. After photopolymerization, these nanogrooves show an excellent ability to align human dermal fibroblasts (hDFs) over large areas. The alignment quality is good for both bare SmA LCE substrates and for substrates coated with fibronectin. The SmA LCE nano-topographies show a high potential for tissue engineering.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

OPTICAL SPECTROSCOPY OF DISTANT RED GALAXIES

We present optical spectroscopic follow-up of a sample of distant red galaxies (DRGs) with K {sup tot} {sub s,Vega} < 22.5, selected by (J - K){sub Vega}>2.3, in the Hubble Deep Field South (HDFS), the MS 1054-03 field, and the Chandra Deep Field South (CDFS). Spectroscopic redshifts were obtained for 15 DRGs. Only two out of 15 DRGs are located at z < 2, suggesting a high efficiency to select high-redshift sources. From other spectroscopic surveys in the CDFS targeting intermediate to high-redshift populations selected with different criteria, we find spectroscopic redshifts for a further 30 DRGs. We use the sample of spectroscopically confirmed DRGs to establish the high quality (scatter in DELTAz/(1 + z) of approx0.05) of their photometric redshifts in the considered deep fields, as derived with EAZY. Combining the spectroscopic and photometric redshifts, we find that 74% of DRGs with K {sup tot} {sub s,Vega} < 22.5 lie at z>2. The combined spectroscopic and photometric sample is used to analyze the distinct intrinsic and observed properties of DRGs at z < 2 and z>2. In our photometric sample to K {sup tot} {sub s,Vega} < 22.5, low-redshift DRGs are brighter in K{sub s} than high-redshift DRGs by 0.7 mag, and more extincted by 1.2 mag in A{sub V} . Our analysis shows that the DRG criterion selects galaxies with different properties at different redshifts. Such biases can be largely avoided by selecting galaxies based on their rest-frame properties, which requires very good multi-band photometry and high quality photometric redshifts.

79 ASTRONOMY AND ASTROPHYSICS↗

XRootd, disk-based, caching proxy for optimization of data access, data placement and data replication

Following the success of the XRootd-based US CMS data federation, the AAA project investigated extensions of the federation architecture by developing two sample implementations of an XRootd, disk-based, caching proxy. The first one simply starts fetching a whole file as soon as a file open request is received and is suitable when completely random file access is expected or it is already known that a whole file be read. The second implementation supports on-demand downloading of partial files. Extensions to the Hadoop Distributed File System have been developed to allow for an immediate fallback to network access when local HDFS storage fails to provide the requested block. Both cache implementations are in pre-production testing at UCSD.

Bauerdick, L. A.T.↗

Engineering the CernVM-Filesystem as a High Bandwidth Distributed Filesystem for Auxiliary Physics Data

A common use pattern in the computing models of particle physics experiments is running many distributed applications that read from a shared set of data files. We refer to this data is auxiliary data, to distinguish it from (a) event data from the detector (which tends to be different for every job), and (b) conditions data about the detector (which tends to be the same for each job in a batch of jobs). Relatively speaking, conditions data also tends to be relatively small per job where both event data and auxiliary data are larger per job. Unlike event data, auxiliary data comes from a limited working set of shared files. Since there is spatial locality of the auxiliary data access, the use case appears to be identical to that of the CernVM- Filesystem (CVMFS). However, we show that distributing auxiliary data through CVMFS causes the existing CVMFS infrastructure to perform poorly. We utilize a CVMFS client feature called 'alien cache' to cache data on existing local high-bandwidth data servers that were engineered for storing event data. This cache is shared between the worker nodes at a site and replaces caching CVMFS files on both the worker node local disks and on the site's local squids. We have tested this alien cache with the dCache NFSv4.1 interface, Lustre, and the Hadoop Distributed File System (HDFS) FUSE interface, and measured performance. In addition, we use high-bandwidth data servers at central sites to perform the CVMFS Stratum 1 function instead of the low-bandwidth web servers deployed for the CVMFS software distribution function. We have tested this using the dCache HTTP interface. As a result, we have a design for an end-to-end high-bandwidth distributed caching read-only filesystem, using existing client software already widely deployed to grid worker nodes and existing file servers already widely installed at grid sites. Files are published in a central place and are soon available on demand throughout the grid and cached locally on the site with a convenient POSIX interface. This paper discusses the details of the architecture and reports performance measurements.

Dykstra, D.↗