Search NASA⌕ Search

SEARCH · Search NASA

Results for “research data”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records

Conquering Data Chaos: Research Data Management with Kubernetes

Managing massive volumes of data and effectively making it accessible to researchers poses significant challenges and is a barrier to scientific discovery. In many cases, critical data is locked up in unwieldy file formats or one-off databases and is too large to effectively process on a single machine. This talk explores the role of Kubernetes, an open-source container orchestration platform, in addressing research data management challenges. I will discuss how we are using a set of publicly available open-source and home-grown tools in the National Renewable Energy Lab (NREL) Data, Analysis, and Visualization (DAV) group to help researchers overcome data-related bottlenecks. The talk will begin by providing an overview of the data challenges faced in research data management, including data storage, processing, and analysis. I will highlight Kubernetes' ability to handle large-scale data by leveraging containerization and distributed computing, including distributed storage. Kubernetes allows researchers to encapsulate data processing infrastructure and workflows into portable containers, enabling reproducibility and ease of deployment. Kubernetes can then schedule and manage the resource allocation of these containers to enable efficient utilization of limited computing resources, leading to more efficient data processing and analysis. I will discuss some limitations of traditional, siloed approaches to dealing with data and emphasize the need for solutions which foster collaboration. I will highlight how we are using Kubernetes at NREL to facilitate data sharing and cooperation among research teams. Kubernetes' flexible architecture enables the deployment of shared computing environments, such as Apache Superset, where researchers can seamlessly access and analyze shared datasets. Providing the ability to have one research team easily consume data generated by another, utilizing Kubernetes' as a central data platform, is one of the major wins we've encountered by adopting the platform. Finally, I will showcase real-world use cases from NREL where we have used Kubernetes to solve some persistent data challenges involving large volumes of sensor and monitoring data. I will discuss the challenges we encountered when creating our cluster and making it available as a production-ready resource. I will also discuss the specific suite of tools, including Postgres and Apache Druid for columnar and timeseries data, and Redpanda Kafka for streaming data we have deployed in our infrastructure, and the process that went into the selection of these tools.

collaborative environment↗

The Modern Research Data Portal: A Design Pattern for Networked, Data-Intensive Science

Here we describe best practices for providing convenient, high-speed, secure access to large data via research data portals. We capture these best practices in a new design pattern, the Modern Research Data Portal, that disaggregates the traditional monolithic web-based data portal to achieve orders-of-magnitude increases in data transfer performance, support new deployment architectures that decouple control logic from data storage, and reduce development and operations costs. We introduce the design pattern; explain how it leverages high-performance Science DMZs and cloud-based data management services; review representative examples at research laboratories and universities, including both experimental facilities and supercomputer sites; describe how to leverage Python APIs for authentication, authorization, data transfer, and data sharing; and use coding examples to demonstrate how these APIs can be used to implement a range of research data portal capabilities. Sample code at a companion web site, https://docs.globus.org/mrdp, provides application skeletons that readers can adapt to realize their own research data portals.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗

The Modern Research Data Portal: a design pattern for networked, data-intensive science

We describe best practices for providing convenient, high-speed, secure access to large data via research data portals. Here, we capture these best practices in a new design pattern, the Modern Research Data Portal, that disaggregates the traditional monolithic web-based data portal to achieve orders-of-magnitude increases in data transfer performance, support new deployment architectures that decouple control logic from data storage, and reduce development and operations costs. We introduce the design pattern; explain how it leverages high-performance data enclaves and cloud-based data management services; review representative examples at research laboratories and universities, including both experimental facilities and supercomputer sites; describe how to leverage Python APIs for authentication, authorization, data transfer, and data sharing; and use coding examples to demonstrate how these APIs can be used to implement a range of research data portal capabilities. Sample code at a companion web site, https://docs.globus.org/mrdp , provides application skeletons that readers can adapt to realize their own research data portals.

97 MATHEMATICS AND COMPUTING↗

Whole-Organism Concentration Ratios for Plutonium in Wildlife from Past US Nuclear Research Data

Whole-organism concentration ratios (CR{sub wo-media}) for plutonium (Pu) in wildlife were calculated using data from the broad range of organism types and environmental settings of the US nuclear research program. Original sources included site-specific reports and scientific journal articles typically from 1960s to 80s research. Most of the calculated CR{sub wo-media} values are new to existing data sets, and, for some wildlife categories, serve to fill gaps or add to sparse data including those for terrestrial reptile; freshwater bird, crustacean and zooplankton; and marine crustacean and zooplankton. Ratios of Pu concentration in the whole-organism to that in specific tissues and organs are provided here for a range of freshwater and marine fish. The CR{sub wo-media} values in fish living in liquid discharge ponds were two orders of magnitude higher than those for similar species living in lakes receiving Pu from atmospheric fallout, suggesting the physico-chemical form of the source Pu can dominate over other factors related to transfer, such as organism size and feeding behavior. Small rodent data indicated one to two order of magnitude increases when carcass, pelt, and gastrointestinal tract were included together in the whole-organism calculation compared to that for carcass alone. Only 4% of Pu resided in the carcass of small rodents compared to 75% in the gastrointestinal tract and 21% in the pelt.

63 RADIATION, THERMAL, AND OTHER ENVIRON. POLLUTAN↗

Uses of earth resources data in the NASA/MSC Earth Resources Research Data Facility.

The operation of the Earth Resources Research Data Facility (ERRDF) in Clear Lake City, Texas, which contains both aircraft and spacecraft gathered information on remote sensing of earth resources, including geology, geography, agriculture, forestry, hydrology, and oceanography, is described in detail. Possible uses of ERRDF data for further studies and applications are discussed.

Zeitler, E. O.↗

The research data alliance photon and neutron science interest group

Scientific research data provides unique challenges that are distinct from classic “Big Data” sources. One common element in research data is that the experiment, observations, or simulation were designed, and data were specifically acquired, to shed light on an open scientific question. The data and methods are usually “owned” by the researcher(s) and the data itself might not be viewed to have long-term scientific significance after the results have been published. Often, the data volume was relatively low, with data sometimes easier to reproduce than to catalog and store. Some data and meta-data were not collected in a digital form, or were stored on antiquated or obsolete media. Generally speaking, policies, tools, and management of digital research data have reflected an ad hoc approach that varies domain by domain and research group by research group. This model, which treats research data as disposable, is proving to be a serious limitation as the volume and complexity of research data explodes. Changes are required at every level of scientific research: within the individual groups, and across scientific domains and interdisciplinary collaborations. Enabling researchers to learn about available tools, processes, and procedures should encourage a spirit of cooperation and collaboration, allowing researchers to come together for the common good. In conclusion, these community-oriented efforts provide the potential for targeted projects with high impact.

97 MATHEMATICS AND COMPUTING↗

Fundamental research data base

A fundamental research data base was created on a single 9-track 1600 BPI tape containing ground truth, image, and Badhwar profile feature data for 17 North Dakota, South Dakota, and Minnesota agricultural sites. Each site is 5x6 nm in area. Image data has been provided for a minimum of four acquisition dates for each site. All four images have been registered to one another. A list of the order of the files on tape and the dates of acquisition is provided.

Source record↗

Fundamental research data base

A fundamental research data base containing ground truth, image, and Badhwar profile feature data for 17 North Dakota, South Dakota, and Minnesota agricultural sites is described. Image data was provided for a minimum of four acquisition dates for each site and all four images were registered to one another.

Source record↗

University Data Management Pilot Utilizing the Nuclear Research Data System

Background In 2022, the Office of Science and Technology Policy (OSTP) issued a memo that significantly reshaped the landscape of access to federally funded research. The memo mandated that all taxpayer-funded research be made available to the public without delay upon publication, without an embargo period, superseding the 2013 OSTP public access policy. This public access policy promotes transparency and the democratization of knowledge, ensuring that the fruits of scientific endeavors funded by federal agencies could be immediately accessed and built upon by scientists, educators, students, and the public at large. To implement the requirements of the OSTP guidance and DOE Public Access Plan, the Office of Nuclear Energy (NE) has implemented public access plan guidance and has identified several areas where better data management practices would further expand public access to important nuclear energy related scientific data, reports, and other technical products. Significant NE supported efforts are already underway for data management and public access to important nuclear energy related data.1 2 To address gaps in data management practices, and improve retention and accessibility of data, NE is actively exploring enhanced data management options utilizing its high-performance computing resources administered by its Nuclear Scientific User Facility Program. A newly piloted system, the Nuclear Research Data System (NRDS) acts as a portal for data collection and dissemination. Nuclear Energy University Program Research and Development Portfolio According to Web of Science, NEUP has produced 2,345 journal publication that have been cited more than 61,000 times3 and countless conference proceedings. These publications are publicly available through OSTI.gov and in the open literature. Additional scientific and technical products including project milestones that are not publications and NEUP project final reports are vetted through OSTI.gov and released once reviewed and approved by DOE. Since 2009, NEUP has awarded close to 1,000 different R&D projects in technical areas across the NE research programs. As of June 2023, 512 NEUP reports are publicly available on OSTI. The underlying data for projects is still held at universities, and data transfer, co-location, and dissemination has not occurred in a systematic way. NEUP data is currently accessible through myriad university-based data repositories, or through direct requests to PIs. The program identified this patchwork of repositories, or often lack of publicly available data, as a significant barrier to an organized, accessible, and comprehensive solution to sharing data with the larger nuclear energy community. Approach The goal of this pilot project is to establish a pathway to a consolidated long-term repository for NEUP project data. To accomplish this goal, the pilot strives to accomplish the following objectives: Establish data collection standards, including a standard set of required supplementary information to contextualize and support raw data files. Work with the HPC group collect and upload information and to modify the NRDS system, as needed, to support a standardized approach. Resolve potential barriers to successful roll out of an expanded data collection strategy, including modifying data management plan guidelines and establishing a document and data release process that accounts for potential intellectual property and/or export control concerns. Results Overall, the pilot was successful in collecting 8,982 raw and processes data files, 220 reports, 56 calibration files, and 5,931 other supplementary documents. Supplementary documents included experimental plans, methods, journal publications and conference proceedings, milestone reports, and final reports. Figure 2 shows the number of data sets and supplementary project information provided by each project. Projects has significantly different input, depending on experimental data produced and completeness of the datasets provided.

Data collection↗

A View from ORNL: Scientific Data Research Opportunities in the Big Data Age

One of the core issues across computer and computational science today is adapting to, managing, and learning from the influx of "Big Data". In the commercial space, this problem has led to a huge investment in new technologies and capabilities that are well adapted to dealing with the sorts of human-generated logs, videos, texts, and other large-data artifacts that are processed and resulted in an explosion of useful platforms and languages (Hadoop, Spark, Pandas, etc.). However, translating this work from the enterprise space to the computational science and HPC community has proven somewhat difficult, in part because of some of the fundamental differences in type and scale of data and timescales surrounding its generation and use. We describe a forward-looking research and development plan which centers around the concept of making Input/Output (I/O) intelligent for users in the scientific community, whether they are accessing scalable storage or performing in situ workflow tasks. Much of our work is based on our experience with the Adaptable I/O System (ADIOS 1.X), and our next generation version of the software ADIOS 2.X [1].

Klasky, Scott↗

Board on Research Data and Information

The Board on Research Data and Information (BRDI) has planned and undertaken numerous activities regarding data citation, attribution, management, policy, publishing, centers, access, curation, sharing, and infrastructure; and international collaboration and cooperation. Some of these activities resulted in National Research Council reports (For Attribution: Developing Data Attribution and Citation Practices and Standards (2012), The Case for International Scientific Data Sharing: A Focus on Developing Countries (2012), and The Future of Scientific Knowledge Discovery in Open Networked Environments (2012); and a peer-reviewed paper (Out of Cite, Out of Mind: The Current State of Practice, Policy, and Technology for the Citation of Data, 2013). BRDI held symposia, workshops and sessions in the U.S. and abroad on diverse topics such as global scientific data infrastructures, discovery of data online, privacy in a big data world, and data citation principles, among other timely data-related subjects. In addition, BRDI effects the representation of the United States before the International Council for Science’s International Committee on Data for Science and Technology (CODATA).

99 GENERAL AND MISCELLANEOUS↗

Vegetation and soils field research data base: Experiment summaries

Understanding of the relationships between the optical, spectral characteristics and important biological-physical parameters of earth-surface features can best be obtained by carefully controlled studies over fields and plots where complete data describing the condition of targets are attainable and where frequent, timely spectral measurement can be obtained. Development of a vegetation and soils field research data base was initiated in 1972 at Purdue University's Laboratory for Applications of Remote Sensing and expanded in the fall of 1974 by NASA as part of LACIE. Since then, over 250,000 truck-mounted and helicopter-borne spectrometer/multiband radiometer observations have been obtained of more than 50 soil series and 20 species of crops, grasses, and trees. These data are supplemented by an extensive set of biophysical and meteorological data acquired during each mission. The field research data form one of the most complete and best-documented data sets acquired for agricultural remote sensing research. Thus, they are well-suited to serve as a data base for research to: (1) quantiatively determine the relationships of spectral and biophysical characteristics of vegetation, (2) define future sensor systems, and (3) develop advanced data analysis techniques.

Biehl, L. L.↗

Research Data Alliance: Understanding Big Data Analytics Applications in Earth Science

The Research Data Alliance (RDA) enables data to be shared across barriers through focused working groups and interest groups, formed of experts from around the world - from academia, industry and government. Its Big Data Analytics (BDA) interest groups seeks to develop community based recommendations on feasible data analytics approaches to address scientific community needs of utilizing large quantities of data. BDA seeks to analyze different scientific domain applications (e.g. earth science use cases) and their potential use of various big data analytics techniques. These techniques reach from hardware deployment models up to various different algorithms (e.g. machine learning algorithms such as support vector machines for classification). A systematic classification of feasible combinations of analysis algorithms, analytical tools, data and resource characteristics and scientific queries will be covered in these recommendations. This contribution will outline initial parts of such a classification and recommendations in the specific context of the field of Earth Sciences. Given lessons learned and experiences are based on a survey of use cases and also providing insights in a few use cases in detail.

Riedel, Morris↗

NCSP Related Nuclear Data Research at RPI [Slides]

This lecture is on NCSP related nuclear data research at the Rensselaer Polytechnic Institute (RPI). This presentation includes an overview of fiscal year 2022 activity. Additionally, it has a talk on the RPI Nuclear Data (ND) Group Research, an update on Linear Accelerators (LINAC) refurbishment, and the Enhanced Thermal Target & Cold Moderator (ETTC) Target System.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

Handling Flight-Research Data In Real Time

Researchers at widely separated locations able to participate in tests and analyze data immediately. Basic data-handling needs common: Communicates with vehicle, pilot, and test team; Acquires, computes, and displays data; knows exact location of research vehicle at all times. Continuing challenge for designers and operators of ground support facilities to perform tasks in real time and present integrated results to research team in real time. Paper presents several approaches to satisfaction of requirements of representative types of aircraft research programs at NASA Western Aeronautical Test Range of Ames Research Center.

Moore, Archie L.↗