Fulker, David | Search NASA

NASA NTRS2018

Accessing Data Stored in Amazon S3 Using the Hyrax OPeNDAP Server

For three years we have been investigating data storage and retrieval in the Amazon Cloud, striving to optimize both storage structures and (cache-enhanced retrieval processes. These optimizations are hidden from data users, who simply employ the Data Access Protocol (DAP), a popular, Web-based Application Programmer Interface (API) developed by OPeNDAP. We present our collected findings and their realization in the Hyrax data server. Specific techniques include caching data on spinning disk; accessing sharded data in place; optimizing the organizational structures for data within the flat key-value space of Simple Storage Service (S3); and employing an API extension that permits simultaneous operations on many datasets in a single request.

Data access↗

NASA NTRS2016

Intro and Recent Advances: Remote Data Access via OPeNDAP Web Services

During the upcoming Summer 2016 meeting of the ESIP Federation (July 19-22), OpenDAP will hold a Developers and Users Workshop. While a broad set of topics will be covered, a key focus is capitalizing on recent EOSDIS-sponsored advances in Hyrax, OPeNDAPs own software for server-side realization of the DAP2 and DAP4 protocols. These Hyrax advances are as important to data users as to data providers, and the workshop will include hands-on experiences of value to both. Specifically, a balanced set of presentations and hands-on tutorials will address advances in1.server installation,2.server configuration,3.Hyrax aggregation capabilities,4.support for data-access from clients that are HTTP-based, JSON-based or OGC-compliant (especially WCS and WMS),5.support for DAP4,6.use and extension of server-side computational capabilities, and7.several performance-affecting matters. Topics 2 through 7 will be relevant to data consumers, data providers and notably, due to the open-source nature of all OPeNDAP software to developers wishing to extend Hyrax, to build compatible clients and servers, and/or to employ Hyrax as middleware that enables interoperability across a variety of end-user and source-data contexts. A session for contributed talks will elaborate the topics listed above and embrace additional ones.

hyrax↗

NASA NTRS2016

Interoperability in the Labyrinth

During the upcoming Summer 2016 meeting of the ESIP Federation (July 19-22), OpenDAP will hold a Developers and Users Workshop. While a broad set of topics will be covered, a key focus is capitalizing on recent EOSDIS-sponsored advances in Hyrax, OPeNDAPs own software for server-side realization of the DAP2 and DAP4 protocols. These Hyrax advances are as important to data users as to data providers, and the workshop will include hands-on experiences of value to both.

ESIP↗