Data access through digital Earth
Explore the source record for details and available documents.
data access Earth digital digital Earth↗
SEARCH · Search NASA
Results for “Data access”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Kamodo: Simplifying Model Data Access and Utilization
To address the lack of user-friendly software needed to simplify the utilization of model data across Heliophysics, the Community Coordinated Modeling Center (CCMC) at NASA’s Goddard Space Flight Center has developed a model-agnostic method via Kamodo for users to easily access and utilize model data in their workflows. By abstracting away the broad range of file formats and the intricacies of interpolation on specialized grids, this approach significantly lowers the barrier to model data access and utilization for the community while adding exciting new capabilities to their tool boxes. This paper describes the direct interfaces to the model data, called model readers, and a basic introduction on how to use them. Additionally, we detail the planned approach for including custom interpolation codes, and include current progress on specialized visualization developments. The CCMC is maintaining Kamodo as an official NASA open-sourced software to enable and encourage community collaboration.
Heliophysics↗
HPC Campaign Management: Remote data access with user-defined error bound using ADIOS and ZFP
Remote access to large-scale scientific datasets, like those generated by combustion simulations or other high-performance computing (HPC) applications, presents a significant challenge. Downloading entire datasets is often impractical due to their size and the bandwidth limitations of typical networks. To address this challenge, we propose a novel approach that enables efficient remote access to large datasets distributed across multiple facilities. Our method enables technologies to download only the data values of a select variable, in a select region of interest, to a user-defined accuracy. For this purpose, we extended the ADIOS IO library to provide read functions with user-defined accuracy, a remote data server that understands multidimensional selections of specific variables, steps and accuracy from an ADIOS dataset, and which uses lossy compression on the remote site to reduce the data to be transferred back to the client. In addition, our extension of the ADIOS library collects metadata from multiple datasets in small files called Campaign Archives, which can be shared among project participants on any HPC, cloud or laptop, and which can easily facilitate the discovery of content and pointers to the data location as well as remote access to the data by local tools as if data was local. This feature called Campaign Management, enables a group of scientists to manage related datasets stored in multiple files, across multiple facilities as if it was in a single file/database. We demonstrate the effectiveness of our approach using a 1.5 TB dataset from the S3D combustion simulation on Frontier at the Oak Ridge Leadership Facility. Even a single variable from this dataset, at 64 GB, is too large to be processed on a standard laptop. We show two different reading patterns for 2D plots and 3D visualization, with careful settings that a scientist studying combustion data would do and show that running the same Python scripts on Frontier directly takes comparable time than running them on the local laptop with remote access to the data on Frontier.
Podhorszki, Norbert [ORNL] (ORCID:000000019647542X↗
Man-computer Inactive Data Access System (McIDAS)
A technical description is given of the effort to design, develop, fabricate, and test the two dimensional data processing system, McIDAS. The system has three basic sections: an access and data archive section, a control section, and a display section. Areas reported include hardware, system software, and applications software.
Source record↗
Supporting Open Science through Cloud-Based and Interoperable APIs for Data Access and Visualization: The Multi-Mission Algorithm and Analysis Platform (MAAP) Data System
Explore the source record for details and available documents.
Aimee Barciauskas↗
Method and device for maximizing memory system bandwidth by accessing data in a dynamically determined order
A data processing system is disclosed which comprises a data processor and memory control device for controlling the access of information from the memory. The memory control device includes temporary storage and decision ability for determining what order to execute the memory accesses. The compiler detects the requirements of the data processor and selects the data to stream to the memory control device which determines a memory access order. The order in which to access said information is selected based on the location of information stored in the memory. The information is repeatedly accessed from memory and stored in the temporary storage until all streamed information is accessed. The information is stored until required by the data processor. The selection of the order in which to access information maximizes bandwidth and decreases the retrieval time.
Wulf, William A.↗
Next generation experimental data access at NSLS-II
The NSLS-II network and computing infrastructure has been significantly updated recently. The re-IP process in 2020-2021 enabled the NSLS-II network to be routable to the rest of the BNL campus. Then, standardization of the operating systems and deployment procedures helped to deliver a consistent environment to workstations and servers used by all NSLS-II beamlines. In particular, the RedHat Enterprise Linux 8 was deployed to 700+ machines using the RedHat Satellite infrastructure management product, and all critical services (IOCs, databases, etc.) were migrated to the new OS. NFS users’ home directories are consistent across all of the machines, which eliminates the need for the individual configuration of the user environment on each host. The standard suite of software packages is available to the beamline staff and users, which includes the system packages (deployed via RPM) as well as the conda environments for data acquisition and analysis. Security measures were implemented to comply with the industry standards, which include multi-factor authentication (using Duo), secure screen lock for the beamline machines, and advanced access control to the experimental data that is stored in shared central storage available on all hosts. These major enhancements facilitated sharing the experimental data (currently for a number of selected beamlines, with a plan to extend it to the whole facility in the nearest future) with the users via an externally facing JupyterHub instance. The beamlines keep using the Bluesky data acquisition framework to orchestrate their experiments, and the new infrastructure enabled them to use a next-generation data access library called tiled.
36 MATERIALS SCIENCE↗
Efforts and Innovations to Promote Data Sharing and Data Accessibility in the EGS Collab Experiments
Large-scale scientific research programs such as the EGS Collab experiments and the FORGE program play extremely important roles in advancing geothermal technologies. Such efforts involve a large number of researchers from multiple institutions, last multiple years, and generate large, complex datasets. The value of the research efforts is only realized when the datasets are used by a large community of researchers in the decades to come. A challenge is that due to the complexity of the data, it could require a user to devote a serious effort into understanding the data before the data can be effectively utilized. In the EGS Collab experiments, the team has devoted remarkable efforts and developed many innovative solutions to make the data more accessible to broader team members and future users. This paper documents the experience gained and lessons learned by the EGS Collab team in disseminating the data in the most informative and inspiring forms to maximize the value of this precious dataset. "Accessibility"in the title does not only mean making the raw data available for download. Particularly, we want to emphasize the importance of organizing, annotating, and presenting the data in ways to make it easy to digest by prospective consumers of the data.
circulation↗
A Network Based Approach to Data Access, Visualization, Interactive Analysis and Distribution
In today's heterogeneous computing environment of proliferating platforms and operating systems, the Internet, through the World Wide Web (WWW), is becoming the preferred interface to much of the world's archive of digital data.
data access visualization interactive analysis and↗
X10: A FORTRAN direct access data management system
The XIO system is a set of subroutines that provide generalized data management capability for FORTRAN programs using a direct access file. Arrays of integer, real, double precision, and character data may be stored, each logical group of data identified by a unique matrix number. A matrix may be organized and stored as batches to reduce core requirements. Batches may be accessed randomly or sequentially. The file may be checkpointed and retained, allowing for restarts with stored values. The XIO subroutines operate on either IBM 360-370/OS/VS or DEC PDP-11/RSX computing systems.
Roland, D. P.↗
Expedition Earth and Beyond: Engaging Classrooms in Student-Led Research Using NASA Data, Access to Scientists, and Integrated Educational Strategies
Classroom teachers are challenged with engaging and preparing today s students for the future. Activities are driven by state required skills, education standards, and high-stakes testing. Providing educators with standards-aligned, inquiry-based activities that will help them engage their students in student-led research in the classroom will help them teach required standards, essential skills, and help inspire their students to become motivated learners. The Astromaterials Research and Exploration Science (ARES) Education Program, classroom educators, and ARES scientists at the NASA Johnson Space Center created the Expedition Earth and Beyond education program to help teachers promote student-led research in their classrooms (grades 5-14) by using NASA data, providing access to scientists, and using integrated educational strategies.
Graff, P. V.↗
MODIS Technical Report Series. Volume 4: MODIS data access user's guide: Scan cube format
The software described in this document provides I/O functions to be used with Moderate Resolution Spectroradiometer (MODIS) level 1 and 2 data, and could be easily extended to other data sources. This data is in a scan cube data format: a 3-dimensional ragged array containing multiple bands which have resolutions ranging from 250 to 1000 meters. The complexity of the data structure is handled internally by the library. The I/O calls allow the user to access any pixel in any band through 'C' structure syntax. The high MODIS data volume (approaching half a terabyte per day) has been a driving factor in the library design. To avoid recopying data for user access, all I/O is performed through dynamic 'C' pointer manipulation. This manual contains background material on MODIS, several coding examples of library usage, in-depth discussions of each function, reference 'man' type pages, and several appendices with details of the included files used to customize a user's data product for use with the library.
Kalb, Virginia L.↗
NASA Earth Observations (NEO): Data Access for Informal Education and Outreach
The NEO (NASA Earth Observations) web space is currently under development with the goal of significantly increasing the demand for NASA remote sensing data while dramatically simplifying public access to georeferenced images. NEO will target the unsophisticated, nontraditional data users who are currently underserved by the existing data ordering systems. These users will include formal and informal educators, museum and science center personnel, professional communicators, and citizen scientists and amateur Earth observers. Users will be able to view and manipulate georeferenced browse imagery and, if they desire, download directly or order the source HDF data from the data provider (e.g., NASA DAAC or science team) via a single, integrated interface. NE0 will accomplish this goal by anticipating users expectations and knowledge level, thus providing an interface that presents material to users in a more simplified manner, without relying upon the jargon/technical terminology that make even the identification of the appropriate data set a significant hurdle. NEO will also act as a gateway that manages users expectations by providing specific details about images and data formats, developing tutorials regarding the manipulation of georeferenced imagery and raw data, links to software tools and ensuring that users are able to get the image they want in the format they want as easily as possible.
Ward, Kevin↗
Using OPeNDAP In The Cloud to Connect NASA Data Centers and Support Open Data Access
NASA DAACs (Distributed Active Archive Centers), including the Goddard Earth Sciences Data Information and Services Center (GES DISC), are currently transitioning from on-premises servers to a shared Earthdata Cloud in order to build more interoperability, cross-collaboration, and streamlined services between their data centers. Migrating its on-premises OPeNDAP service to the cloud is a critical component of making this interconnectedness between NASA DAACs possible. To improve their cloud services, GES DISC is leveraging open-source platforms like Github to collect user feedback, create use cases, and develop resources to enable real-time learning for users about OPeNDAP in the cloud. This presentation gives an overview of the OPeNDAP in the cloud, resources developed to access this service, and considerations for improved user guidance and experience to further support NASA's commitment to the Open-Source Science Initiative (OSSI).
Christopher Battisto↗
Making NASA Earth Observing System Satellite Data Accessible to the K-12 and Citizen Scientist Communities
The Atmospheric Sciences Data Center (ASDC) at NASA s Langley Research Center houses over 700 data sets related to Earth s radiation budget, clouds, aerosols and tropospheric chemistry. These data sets are produced to increase academic understanding of the natural and anthropogenic perturbations that influence global climate change. The Mentoring and inquirY using NASA Data on Atmospheric and earth science for Teachers and Amateurs (MY NASA DATA) project has been established to systematically support educational activities at all levels of formal and informal education by reducing these large data holdings to microsets that will be easily explored and understood by the K-12 and the amateur scientist communities
Moore, Susan W.↗
The NASA Tournament Laboratory (NTL): Improving Data Access at PDS while Spreading Joy and Engaging Students through 16 Micro-Contests
NASA PDS hosts terabytes of valuable data from hundreds of data sources and spans decades of research. Data is stored on flat-file systems regulated through careful meta dictionaries. PDS's data is available to the public through its website which supports data searches through drill-down navigation. While the system returns data quickly, result sets in response to identical input differ depending on the drill-down path a user follows. To correct this Issue, to allow custom searching, and to improve general accessibility, PDS sought to create a new data structure and API, and to use them to build applications that are a joy to use and showcase the value of the data to students, teachers and citizens. PDS engaged TopCoder and Harvard Business School through the NTL to pursue these objectives in a pilot effort. Scope was limited to Small Bodies Node data. NTL analyzed data, proposed a solution, and implemented it through a series of micro-contests. Contest focused on different segments of the problem; conceptualization, architectural design, implementation, testing, etc. To demonstrate the utility of the completed solution, NTL developed web-based and mobile applications that can compare targets, regardless of mission. To further explore the potential of the solution NTL hosted "Mash-up" challenges that integrated the API with other publically available assets, to produce consumer and teaching applications, including an Augmented Reality iPad tool. Two contests were also posted to middle and high school students via the NoNameSite.com platform, and as a result of these contests, PDS/SBN has initiated a Facebook program. These contests defined and implemented a data warehouse with the necessary migration tools to transform legacy data, produced a public web interface for the new search, developed a public API, and produced four mobile applications that we expect to appeal to users both within and, without the academic community.
LaMora, Andy↗
Support of Data Access for the IMP-8 UMD Experiment
This grant report provides information on data from the Interplanetary Monitoring Platform-8 (IMP-8). Topics covered include: (1) the science involved in the project; (2) the collection of data; (3) the processing of data; (4) the submission of data to other facilities; (5) the availability of data on the world wide web (WWW). Graphs are also included of data on the interstellar medium.
Ipavich, F. M.↗
Uniform Data Access Using GXD
This paper gives an overview of GXD, a framework facilitating publication and use of data from diverse data sources. GXD defines an object-oriented data model designed to represent a wide range of things including data, its metadata, resources and query results. GXD also defines a data transport language. a dialect of XML, for representing instances of the data model. This language allows for a wide range of data source implementations by supporting both the direct incorporation of data and the specification of data by various rules. The GXD software library, proto-typed in Java, includes client and server runtimes. The server runtime facilitates the generation of entities containing data encoded in the GXD transport language. The GXD client runtime interprets these entities (potentially from many data sources) to create an illusion of a globally interconnected data space, one that is independent of data source location and implementation.
Vanderbilt, Peter↗