Search NASA⌕ Search

SEARCH · Search NASA

Results for “Data sharing”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 991 records · Page 55

Efficient I/O for Neural Network Training with Compressed Data

FanStore is a shared object store that enables efficient and scalable neural network training on supercomputers. By providing a global cache layer on node-local burst buffers using a compressed representation, it significantly enhances the processing capability of deep learning (DL) applications on existing hardware. In addition, FanStore allows POSIX-compliant file access to the compressed data in user space. We investigate the tradeoff between runtime overhead and data compression ratio using real-world datasets and applications, and propose a compressor selection algorithm to maximize storage capacity given performance constraints. We consider both asynchronous (i.e., with prefetching) and synchronous I/O strategies, and propose mechanisms for selecting compressors for both approaches. Using FanStore, the same storage hardware can host 2-13x more data for example applications without significant runtime overhead. Empirically, our experiments show that FanStore scales to 512 compute nodes with near linear performance scalability.

Zhang, Zhao↗

idaholab/cape2stix

This software allows for the conversion, extraction, and transformation of malware behavior data from "Malware Configuration And Payload Extraction" (CAPEv2) sandbox reports, to Structured Threat Information eXpression (STIX). This allows for further analysis to be performed, sharing of threat data, and transit to a graph database.

Cutshaw, Michael [Idaho National Laboratory (INL),↗

The North American Carbon Program Multi-scale Synthesis and Terrestrial Model Intercomparison Project – Part 2: Environmental driver data

Ecosystems are important and dynamic components of the global carbon cycle, and terrestrial biospheric models (TBMs) are crucial tools in further understanding of how terrestrial carbon is stored and exchanged with the atmosphere across a variety of spatial and temporal scales. Improving TBM skills, and quantifying and reducing their estimation uncertainties, pose significant challenges. The Multi-scale Synthesis and Terrestrial Model Intercomparison Project (MsTMIP) is a formal multi-scale and multi-model intercomparison effort set up to tackle these challenges. The MsTMIP protocol prescribes standardized environmental driver data that are shared among model teams to facilitate model model and model observation comparisons. In this article, we describe the global and North American environmental driver data sets prepared for the MsTMIP activity to both support their use in MsTMIP and make these data, along with the processes used in selecting/processing these data, accessible to a broader audience. Based on project needs and lessons learned from past model intercomparison activities, we compiled climate, atmospheric CO 2 concentrations, nitrogen deposition, land use and land cover change (LULCC), C3 / C4 grasses fractions, major crops, phenology and soil data into a standard format for global (0.5⁰ x 0.5⁰ resolution) and regional (North American: 0.25⁰ x 0.25⁰ resolution) simulations. In order to meet the needs of MsTMIP, improvements were made to several of the original environmental data sets, by improving the quality, and/or changing their spatial and temporal coverage, and resolution. The resulting standardized model driver data sets are being used by over 20 different models participating in MsTMIP. Lastly, the data are archived at the Oak Ridge National Laboratory Distributed Active Archive Center (ORNL DAAC, http://daac.ornl.gov) to provide long-term data management and distribution.

54 ENVIRONMENTAL SCIENCES↗

New SPDF Directions and Evolving Services Supporting Heliophysics Research

The next advances in Heliophysics science and its paradigm of a Great Observatory require an increasingly integrated and transparent data environment, where data can be easily accessed and used across the boundaries of both missions and traditional disciplines. The Space Physics Data Facility (SPDF) project includes uniquely important multi-mission data services with current data from most operating space physics missions. This paper reviews the capabilities of key services now available and the directions in which they are expected to evolve to enable future multi-mission correlative research. The Coordinated Data Analysis Web (CDAWeb) and Satellite Situation Center Web (SSCWeb), critically supported by the Common Data Format (CDF) effort and supplemented by more focused science services such as OMNIWeb and technical services such as data format translations are important operational capabilities serving the international community today (and cited last year by 20% of the papers published in JGR Space Physics). These services continue to add data from most current missions as SPDF works with new missions such as THEMIS to help enable their unique science goals and the meaningful sharing of their data in a multi-mission correlative context. Recent enhancements to CDF, our 3D Java interactive orbit viewer (TIPSOD), the CDAWeb Plus system, increasing automation of data service population, the new folding of the VSPO effort into SPDF and our continuing thrust towards fully-functional web services APIs to allow ready invocation from distributed external middleware and clients will be shown.

McGuire, Robert E.↗

Research Data Alliance: Understanding Big Data Analytics Applications in Earth Science

The Research Data Alliance (RDA) enables data to be shared across barriers through focused working groups and interest groups, formed of experts from around the world - from academia, industry and government. Its Big Data Analytics (BDA) interest groups seeks to develop community based recommendations on feasible data analytics approaches to address scientific community needs of utilizing large quantities of data. BDA seeks to analyze different scientific domain applications (e.g. earth science use cases) and their potential use of various big data analytics techniques. These techniques reach from hardware deployment models up to various different algorithms (e.g. machine learning algorithms such as support vector machines for classification). A systematic classification of feasible combinations of analysis algorithms, analytical tools, data and resource characteristics and scientific queries will be covered in these recommendations. This contribution will outline initial parts of such a classification and recommendations in the specific context of the field of Earth Sciences. Given lessons learned and experiences are based on a survey of use cases and also providing insights in a few use cases in detail.

Riedel, Morris↗

Development of a Technology Roadmap for Online Monitoring of Nuclear Power Plants

The nuclear power industry has recently recognized the potential cost saving that can be achieved from migrating the current manual and labor dependent surveillance and preventive maintenance activities to data-driven online monitoring methods. Since then, several efforts have been launched with various degrees of momentum to tackle specific surveillance or preventive maintenance activities. The light water reactor sustainability program (LWRS) program has anticipated this need and launched an effort that aimed to develop a technology roadmap for the nuclear power industry. Because the roadmap has to be focused towards the user or application’s value, and because nuclear power plants have different process efficiencies and deficiencies, the technology roadmap has to be plant independent. Achieving this objective required optimizing the extent of details described by a technology roadmap. The approach followed is for the roadmap to describe the sequence of activities needed to automate a process or equipment activity, but not describe the sequence of processes or equipment to target. For example, the steps required to migrate the inspection of a pump are plant independent. However, defining that the migration of feed water pumps inspections to online monitoring should be performed before inspections of feed water valves is dependent on multiple plant factors such as the plant requirements, labor work capacity, and process or equipment conditions. The technology roadmap was broken down into six elements, four of which have a sequential path of advancement, while the other two are supporting elements to the sequential elements. The sequential part of the roadmap analyzes data collection, analytics, visualization, and management. For these elements of the technology roadmap, the migration of an activity is defined in three states. The base state is defined as the most primitive, manual, and labor dependent process used by some nuclear power plants. The modern state is defined as the state that can be achieved if the activities are augmented or replaced by current technologies that are commercially available or are soon to be available. The state of the art is defined as using technologies of the future, i.e. concept and technologies that are being researched. The data collection part of the technology roadmap defines three states of every process performed in a nuclear power plant. However, the data analytics part is described based on equipment, because automating an activity performed on one equipment required considering multiple data sources. The visualization and data management elements are process or equipment independent. Visualization describes the means to use human factor science and technologies advancement in presenting the data collected and the data analytics results. The data management targets storage, communication, and computational power need, in addition to data integration and sharing. Since storage, communication, and computational power technologies and path for deployment are known to any information technology organization, this effort targeted the cost of data management. This has a direct impact on the feasibility of the automation effort. The supporting elements analyzes the value of automating the activity and the change challenges to tackle. For the support elements, the roadmap targeted describing activities that need to be performed as part of the roadmap development. The value element of the roadmap describes how to analyze the cost saving and the associated risk. The change management identifies multiple factors to consider such as regulation, cybersecurity, resource availability, supporting infrastructure availability, feasibility (power, fitness, environment tolerance, facilities), and cultural change.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗

Technology Performance Exchange

To address the need for accessible, high-quality data, the Department of Energy has developed the Technology Performance Exchange (TPEx). TPEx enables technology suppliers, third-party testing laboratories, and other entities to share product performance data. These data are automatically transformed into a format that technology evaluators can easily use in their energy modeling assessments to inform procurement decisions.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗

Value added data archiving

Researchers in the Molecular Sciences Research Center (MSRC) of Pacific Northwest Laboratory (PNL) currently generate massive amounts of scientific data. The amount of data that will need to be managed by the turn of the century is expected to increase significantly. Automated tools that support the management, maintenance, and sharing of this data are minimal. Researchers typically manage their own data by physically moving datasets to and from long term storage devices and recording a dataset's historical information in a laboratory notebook. Even though it is not the most efficient use of resources, researchers have tolerated the process. The solution to this problem will evolve over the next three years in three phases. PNL plans to add sophistication to existing multilevel file system (MLFS) software by integrating it with an object database management system (ODBMS). The first phase in the evolution is currently underway. A prototype system of limited scale is being used to gather information that will feed into the next two phases. This paper describes the prototype system, identifies the successes and problems/complications experienced to date, and outlines PNL's long term goals and objectives in providing a permanent solution.

Berard, Peter R.↗

Intelligent vehicle control: Opportunities for terrestrial-space system integration

For 11 years the Department of Defense has cooperated with a diverse array of other Federal agencies including the National Institute of Standards and Technology, the Jet Propulsion Laboratory, and the Department of Energy, to develop robotics technology for unmanned ground systems. These activities have addressed control system architectures supporting sharing of tasks between the system operator and various automated subsystems, man-machine interfaces to intelligent vehicles systems, video compression supporting vehicle driving in low data rate digital communication environments, multiple simultaneous vehicle control by a single operator, path planning and retrace, and automated obstacle detection and avoidance subsystem. Performance metrics and test facilities for robotic vehicles were developed permitting objective performance assessment of a variety of operator-automated vehicle control regimes. Progress in these areas will be described in the context of robotic vehicle testbeds specifically developed for automated vehicle research. These initiatives, particularly as regards the data compression, task sharing, and automated mobility topics, also have relevance in the space environment. The intersection of technology development interests between these two communities will be discussed in this paper.

Shoemaker, Charles↗

NASA Life Sciences Portal (NLSP): Supporting Scientific Transparency and Reproducibility

NASA’s Life Sciences Ports (NLSP) serves the scientific community by providing curated data from space life science experiment. The Human Research Program (HRP) with the help of NLSP is currently transforming their life sciences data archive systems and processes to improve compliance with the FAIR principles [1]. Some of these improvements will at the same time support the twin pillars of Open Science [2]: transparency of methods and reproducibility of results. Scientific transparency is marked by the easily intelligible communication of what has been investigated: what were the procedures for collecting sample and the characteristics of samples collected? what kinds of measurements were made, what were the environmental conditions of the measurements? What were the analysis techniques of the collected data? Reproducibility of the results and findings from the investigation requires a high level of transparency for all but the simplest investigations; the slightest deviation in communicating and replicating complex experimental procedures or data analyses can often yield quite different data and even findings, thwarting their validation. One of the ways the NLSP is aiming to improve the communication of scientific information is through the use of ontology-driven metadata. Ontologies are powerful, graph-based knowledge representation structures, which can be leveraged to increase data interoperability, the area of the FAIR principles in which many data systems most lack compliance. Over the past decade, there has been a concerted effort in the biomedical community to develop modular and narrowly focused domain and application-specific ontologies in a common, open-source framework, the Open Biological and Biomedical Ontology (OBO) Foundry [3]. The open sharing and modular nature of this effort promises huge increases in harmonized data sharing for systems that leverage these models. Which is in line with the FAIR Data Principles of Findability, Accessibility, Interoperability, and Reuse for scientific data management and stewardship. 1. Wilkinson, M.D., et al., The FAIR Guiding Principles for scientific data management and stewardship. Sci Data, 2016. 3: p. 160018. 2. National Academies of Sciences, E. and Medicine, Open Science by Design: Realizing a Vision for 21st Century Research. 2018, Washington, DC: The National Academies Press. 232. 3. Smith, B., et al., The OBO Foundry: coordinated evolution of ontologies to support biomedical data integration. Nat Biotechnol, 2007. 25(11): p. 1251-5.

Life Sciences data↗

NASA Life Sciences Portal (NLSP): Supporting Scientific Transparency and Reproducibility

NASA’s Life Sciences Ports (NLSP) serves the scientific community by providing curated data from space life science experiment. The Human Research Program (HRP) with the help of NLSP is currently transforming their life sciences data archive systems and processes to improve compliance with the FAIR principles [1]. Some of these improvements will at the same time support the twin pillars of Open Science [2]: transparency of methods and reproducibility of results. Scientific transparency is marked by the easily intelligible communication of what has been investigated: what were the procedures for collecting sample and the characteristics of samples collected? what kinds of measurements were made, what were the environmental conditions of the measurements? What were the analysis techniques of the collected data? Reproducibility of the results and findings from the investigation requires a high level of transparency for all but the simplest investigations; the slightest deviation in communicating and replicating complex experimental procedures or data analyses can often yield quite different data and even findings, thwarting their validation. One of the ways the NLSP is aiming to improve the communication of scientific information is through the use of ontology-driven metadata. Ontologies are powerful, graph-based knowledge representation structures, which can be leveraged to increase data interoperability, the area of the FAIR principles in which many data systems most lack compliance. Over the past decade, there has been a concerted effort in the biomedical community to develop modular and narrowly focused domain and application-specific ontologies in a common, open-source framework, the Open Biological and Biomedical Ontology (OBO) Foundry [3]. The open sharing and modular nature of this effort promises huge increases in harmonized data sharing for systems that leverage these models. Which is in line with the FAIR Data Principles of Findability, Accessibility, Interoperability, and Reuse for scientific data management and stewardship.

Life Sciences data↗

Genesis Mission Data cards

As data-intensive research and artificial intelligence become central to DOE mission science, the need for machine-actionable dataset documentation has grown accordingly. However, many DOE-aligned communities, including the Office of Science, NNSA, and cross-laboratory collaborations, have developed independent metadata practices. This fragmentation creates friction for discovery, federation, and reuse across programs. To address these challenges, this talk introduces the Genesis Data Card: a shared metadata artifact developed in collaboration with a broad DOE community (Jefferson Lab and the National Lab of the Rockies, Oak Ridge, Sandia, Idaho, Berkeley, and Los Alamos). The Genesis Data Card aims to standardize dataset documentation across DOE-aligned initiatives while remaining extensible to discipline-specific needs. This talk will describe the data card template and the supporting code to validate completed data cards, using a companion LinkML schema. I'll walk through the design decisions behind the template, its alignment with existing standards, its treatment of sensitivity and governance metadata, and the phased roadmap toward lifecycle-integrated "xCards" that support autonomous discovery and reuse. The talk closes with current gaps, ongoing work, and how others can contribute datasets and feedback to the shared repository.

McSpadden, Helen [Thomas Jefferson National Accele↗

In-Water Data Acquisition Tool Supports Four Marine Energy Projects

The NREL-developed Modular Ocean Data Acquisition (MODAQ) system is built from open-source hardware and software. The tool can both collect and store data and even share curated information through the cloud. Marine energy developers can work with NREL's engineers to design their own customized MODAQ and capture high-quality field data to help monitor and improve their technology designs. For example, users could assess how much power their device produces at sea, analyze the durability of a specific device component, or even control their device from a desk halfway around the world.

data acquisition↗

Effective Tooling for Linked Data Publishing in Scientific Research

Challenges that make it difficult to find, share, and combine published data, such as data heterogeneity and resource discovery, have led to increased adoption of semantic data standards and data publishing technologies. To make data more accessible, interconnected and discoverable, some domains are being encouraged to publish their data as Linked Data. Consequently, this trend greatly increases the amount of data that semantic web tools are required to process, store, and interconnect. In attempting to process and manipulate large data sets, tools–ranging from simple text editors to modern triplestores– eventually breakdown upon reaching undefined thresholds. This paper offers a systematic approach that data publishers can use to categorize suitable tools to meet their data publishing needs. We present a real-world use case, the Resource Discovery for Extreme Scale Collaboration (RDESC), which features a scientific dataset(maximum size of 1.4 billion triples) used to evaluate a toolbox for data publishing in climate research. This paper also introduces a semantic data publishing software suite developed for the RDESC project.

Purohit, Sumit↗

Memory Network For Distributed Data Processors

Universal Memory Network (UMN) is modular, digital data-communication system enabling computers with differing bus architectures to share 32-bit-wide data between locations up to 3 km apart with less than one millisecond of latency. Makes it possible to design sophisticated real-time and near-real-time data-processing systems without data-transfer "bottlenecks". This enterprise network permits transmission of volume of data equivalent to an encyclopedia each second. Facilities benefiting from Universal Memory Network include telemetry stations, simulation facilities, power-plants, and large laboratories or any facility sharing very large volumes of data. Main hub of UMN is reflection center including smaller hubs called Shared Memory Interfaces.

Bolen, David↗

Multivariate physiological recordings in an experimental hemorrhage model

In this paper we describe a data set of multivariate physiological measurements recorded from conscious sheep ( N = 8; 37.4 ± 1.1 kg) during hemorrhage. Hemorrhage was experimentally induced in each animal by withdrawing blood from a femoral artery at two different rates (fast: 1.25 mL/kg/min; and slow: 0.25 mL/kg/min). Data, including physiological waveforms and continuous/intermittent measurements, were transformed to digital file formats (European Data Format [EDF] for waveforms and Comma-Separated Values [CSV] for continuous and intermittent measurements) as a comprehensive data set and stored and publicly shared here (Appendix A). The data set comprises experimental information (e.g., hemorrhage rate, animal weight, event times), physiological waveforms (arterial and central venous blood pressure, electrocardiogram), time-series records of non-invasive physiological measurements (SpO 2 , tissue oximetry), intermittent arterial and venous blood gas analyses (e.g., hemoglobin, lactate, SaO 2 , SvO 2 ) and intermittent thermodilution cardiac output measurements. A detailed explanation of the hemodynamic and pulmonary changes during hemorrhage is available in a previous publication (Scully et al., 2016)

Yaghouby, Farid↗

Towards Composing Data Aware Systems Biology Workflows on Cloud Platforms: A MeDICi-based Approach

Cloud computing is being increasingly adopted for deploying systems biology scientific workflows. Scientists developing these workflows use a wide variety of fragmented and competing data sets and computational tools of all scales to support their research. To this end, the synergy of client side workflow tools with cloud platforms is a promising approach to share and reuse data and workflows. In such systems, the location of data and computation is essential consideration in terms of quality of service for composing a scientific workflow across remote cloud platforms. In this paper, we describe a cloud-based workflow for genome annotation processing that is underpinned by MeDICi - a middleware designed for data intensive scientific applications. The workflow implementation incorporates an execution layer for exploiting data locality that routes the workflow requests to the processing steps that are colocated with the data. We demonstrate our approach by composing two workflowswith the MeDICi pipelines.

59 BASIC BIOLOGICAL SCIENCES↗

Zed Context Driver (ZedCD) v1.0

The software takes the data stream generated using the ZedSDK from a Zed or Zed Mini (fancy camera that mimics human depth perception), and loads selected data streams into a rolling shared memory buffer for use in analysis codes. Essentially, the software streams data into a shared memory buffer to be analyzed only when the analysis software is triggered.

Quiter, BrianJ↗