Improvement of Aerosol Optical Depth Data for Localized Solar Forecasting
This presentation provides an overview of the improvement of aerosol optical depth data for localized solar forecasting.
14 SOLAR ENERGY↗
SEARCH · Search NASA
Results for “data locality”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Multiple marker abundance profiling: combining selected reaction monitoring and data-dependent acquisition for rapid estimation of organelle abundance in subcellular samples
Measuring changes in protein or organelle abundance in the cell is an essential, but challenging aspect of cell biology. Frequently-used methods for determining organelle abundance typically rely on detection of a very few marker proteins, so are unsatisfactory. In silico estimates of protein abundances from publicly available protein spectra can provide useful standard abundance values but contain only data from tissue proteomes, and are not coupled to organelle localization data. A new protein abundance score, the normalized protein abundance scale (NPAS), expands on the number of scored proteins and the scoring accuracy of lower-abundance proteins in Arabidopsis. NPAS was combined with subcellular protein localization data, facilitating quantitative estimations of organelle abundance during routine experimental procedures. A suite of targeted proteomics markers for subcellular compartment markers was developed, enabling independent verification of in silico estimates for relative organelle abundance. Estimation of relative organelle abundance was found to be reproducible and consistent over a range of tissues and growth conditions. In silico abundance estimations and localization data have been combined into an online tool, multiple marker abundance profiling, available in the SUBA4 toolbox (http://suba.live).
59 BASIC BIOLOGICAL SCIENCES↗
Locating Seismic Events with Local-Distance Data
As the seismic monitoring community advances toward detecting, identifying, and locating ever-smaller natural and anthropogenic events, the need is constantly increasing for higher resolution, higher fidelity data, models, and methods for accurately characterizing events. Local-distance seismic data provide robust constraints on event locations, but also introduce complexity due to the significant geologic heterogeneity of the Earth’s crust and upper mantle, and the relative sparsity of data that often occurs with small events recorded on regional seismic networks. Identifying the critical characteristics for improving local-scale event locations and the factors that impact location accuracy and reliability is an ongoing challenge for the seismic community. Using Utah as a test case, we examine three data sets of varying duration, finesse, and magnitude to investigate the effects of local earth structure and modeling parameters on local-distance event location precision and accuracy. We observe that the most critical elements controlling relocation precision are azimuthal coverage and local-scale velocity structure, with tradeoffs based on event depth, type, location, and range.
42 ENGINEERING↗
A Heterogeneity-Aware Task Scheduler for Spark
Big data processing systems such as Spark are employed in an increasing number of diverse applications-such as machine learning, graph computation, and scientific computing-each with dynamic and different resource needs. These applications increasingly run on heterogeneous hardware, e.g., with out-of-core accelerators. However, big data platforms do not factor in the multi-dimensional heterogeneity of applications and hardware. This leads to a fundamental mismatch between the application and hardware characteristics, and the resource scheduling adopted in big data platforms. For example, Hadoop and Spark consider only data locality when assigning tasks to nodes, and typically disregard the hardware capabilities and suitability to specific application requirements. In this paper, we present RUPAM, a heterogeneity-aware task scheduling system for big data platforms, which considers both task-level resource characteristics and underlying hardware characteristics, as well as preserves data locality. RUPAM adopts a simple yet effective heuristic to decide the dominant scheduling factor (e.g., CPU, memory, or I/O), given a task in a particular stage. Our experiments show that RUPAM is able to improve the performance of representative applications by up to 62.3% compared to the standard Spark scheduler.
Computer Science↗
State & Local Government Data Centers
This fact sheet summarizes some of the key needs, barriers, and opportunities related to data center management and operations in the state and local government sector.
data center, data center management, local governm↗
Conical scan impact study. Volume 2: Small local user data processing facility
The impact of a conical scan versus a linear scan multispectral scanner (MSS) instrument on a small local-user data processing facility was studied. User data requirements were examined to determine the unique system rquirements for a low cost ground system (LCGS) compatible with the Earth Observatory Satellite (EOS) system. Candidate concepts were defined for the LCGS and preliminary designs were developed for selected concepts. The impact of a conical scan MSS versus a linear scan MSS was evaluated for the selected concepts. It was concluded that there are valid user requirements for the LCGS and, as a result of these requirements, the impact of the conical scanner is minimal, although some new hardware development for the LCGS is necessary to handle conical scan data.
Ebert, D. H.↗
Using Large Language Models to help customers monitor global threat data
Large Language Models have proven adept at answering general knowledge questions. To make these generative AI tools useful to our mission customers for monitoring global threats, the data sciences team at Sandia is utilizing retrieval augmented generation (RAG) techniques to customize these models with local data. The local data we use consists of data such as research articles and patent abstracts that we've collected over the last several years using automated pipelines.
Herzer, John Andrew [Sandia National Laboratories ↗
Conversion of Site-Specific Meteorological Data for use in CAP-88 PC
CAP-88 PC is a commonly used radiological atmospheric dispersion model. This US EPA-approved model is used to demonstrate compliance with atmospheric emission regulations for radionuclides. While the model includes a large library of meteorological data for use across the United States, there are applications when users may wish to use onsite meteorological data as an input to the CAP-88 PC model. Here, in this work, we present a work-around process for preparing and converting onsite data for use in CAP-88 PC. However, the use of local data should provide a more realistic estimate of doses to members of the public in the immediate vicinity of a facility, although the regulatory agency having jurisdiction may not accept the use of local data for compliance. Additionally, the historical meteorological records from 20+ years ago (at a site many kilometers away) might not be representative of current local weather patterns, highlighting another benefit of using local meteorological data.
54 ENVIRONMENTAL SCIENCES↗
Data catalog project—A browsable, searchable, metadata system
Modern experiments are typically conducted by large, extended groups, where researchers rely on other team members to produce much of the data they use. The experiments record very large numbers of measurements that can be difficult for users to find, access and understand. Here. we are developing a system for users to annotate their data products with structured metadata, providing data consumers with a discoverable, browsable data index. Machine understandable metadata captures the underlying semantics of the recorded data, which can then be consumed by both programs, and interactively by users. Collaborators can use these metadata to select and understand recorded measurements. The data catalog project is a data dictionary and index which enables users to record general descriptive metadata, use cases and rendering information as well as providing them a transparent data access mechanism (URI). Users describe their diagnostic including references, text descriptions, units, labels, example data instances, author contact information and data access URIs. The list of possible attribute labels is extensible, but limiting the vocabulary of names increases the utility of the system. The data catalog is focused on the data products and complements process-based systems like the Metadata Ontology Provenance project [Greenwald, 2012; Schissel, 2015]. This system can be coupled with MDSplus to provide a simple platform for data driven display and analysis programs. Sites which use MDSplus can describe tree branches, and if desired create ‘processed data trees’ with homogeneous node structures for measurements. Sites not currently using MDSplus can either use the database to reference local data stores, or construct an MDSplus tree whose leaves reference the local data store. A data catalog system can provide a useful roadmap of data acquired from experiments or simulations making it easier for researchers to find and access important data and understand the meaning of the data and how it was obtained. This is particularly useful in research facilities that study the results of many different experiments or simulations and may not know the intricacies of the data organization in use where the data was generated. It is also possible to store a local copy of key data items in local MDSplus trees and then add processed data to the local catalog.
70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
The Citizens and Remote Sensing Observational Network (CARSON) Guide: Merging NASA Remote Sensing Data with Local Environmental Awareness
"Citizen science" generally refers to observatoinal research and data collection conducted by non-professionals, commonly as volunteers. In the environmental science field, citizen scientists may be involved with local nad regional issues such as bird and wildlife populations, weather, urban sprawl, natural hazards, wetlands, lakes and rivers, estuaries, and a spectrum of public health concerns. Some citizen scientists may be primarily motivated by the intellectual challenge of scientific observations. Citizen scientists may now examine and utilize remote-sensing data related to their particular topics of interest with the easy-to-use NASA Web-based tools Giovanni and NEO, which allow exploration and investigation of a wide variety of Earth remote sensing data sets. The CARSON (Citizens and Remote Sensing Observational Network) Guide will be an online resource consisting of chapters each demonstrating how to utilize Giovanni and NEO to access and analyze specific remote-sensing data. Integrated in each chapter will be descriptions of methods that citizen scientists can employ to collect, monitor, analyze, and share data related to the chapter topic which pertain to environmental and ecological conditions in their local region. A workshop held in August 2008 initiated the development of prototype chapters on water quality, air quality, and precipitation. These will be the initial chapters in the first release of the CARSON Guide, which will be used in a pilot project at the Maryland Science Center in spring 2009. The goal of the CARSON Guide is to augment and enhance citizen scientist environmental research with NASA satellite data by creating a participatory network consisting of motivated individuals, environmental groups and organizations, and science-focused institutions such as museuma and nature centers. Members of the network could potentially interact with government programs, academic research projects, and not-for-profit organizations focused on environmental issues.
Acker, James↗
Calibration of a COTS Integration Cost Model Using Local Project Data
The software measures and estimation techniques appropriate to a Commercial Off the Shelf (COTS) integration project differ from those commonly used for custom software development. Labor and schedule estimation tools that model COTS integration are available. Like all estimation tools, they must be calibrated with the organization's local project data. This paper describes the calibration of a commercial model using data collected by the Flight Dynamics Division (FDD) of the NASA Goddard Spaceflight Center (GSFC). The model calibrated is SLIM Release 4.0 from Quantitative Software Management (QSM). By adopting the SLIM reuse model and by treating configuration parameters as lines of code, we were able to establish a consistent calibration for COTS integration projects. The paper summarizes the metrics, the calibration process and results, and the validation of the calibration.
Boland, Dillard↗
Toward higher-radix switches with co-packaged optics for improved network locality in data center and HPC networks [Invited]
In this work, we study the network locality improvements that can be achieved by using co-packaged optics in data center and high-performance computing (HPC) networks. The increased escape bandwidth offered by co-packaged optics can enable switches with speeds of 51.2 Tb/s and beyond. From a network architecture perspective, the key advantages of introducing co-packaged optics at the switch points include the implementation of large-scale topologies of >12,000 end points with 4× higher bisection bandwidth and the reduction of the required number of switches by >40% compared with state-of-the-art approaches. From a network operation perspective, improved network locality and faster operation can be achieved since the higher-radix switches can mitigate the impact of network contention. Placing applications under fewer leaf switches reduces the number of packets that cross the spine switches in a leaf-spine topology. The proposed scheme is evaluated via discrete-event simulations: we initially evaluate the network locality properties of the system by using virtual-machine traces from a production data center, and we subsequently quantify the performance improvements by simulating an all-to-all pattern for a variety of message sizes over a number of nodes. The results suggest that co-packaged optics form a promising solution for keeping up with bandwidth scaling in future networks. The virtual-machine analysis shows that large-scale applications can be placed under up to 50% fewer first-level switches, while the network analysis shows speedups of up to 7.1, which translates to execution time reductions of up to 26% and 42.7% for applications with communication ratios of 0.3 and 0.5, respectively.
99 GENERAL AND MISCELLANEOUS↗
Surface albedo spatial variability in North America: Gridded data vs. local measurements
Considering the current booming interest for the large-scale deployment of bifacial photovoltaic modules, the solar industry now requires accurate estimates of broadband surface albedo at high spatial resolution. In this context, the present study evaluates the adequacy and performance over North America of two Moderate Resolution Imaging Spectroradiometer (MODIS) white-sky albedo products (at 500-m and 1-km resolution) and the National Solar Radiation Database (NSRDB) product at 4-km resolution. Two variations of the 500-m MODIS product are also considered: black-sky albedo and all-sky albedo. Albedo observations from 36 radiometric stations during 2011–2015 are analyzed while considering the apparent homogeneity of the surface characteristics over the 4x4 km NSRDB pixel in which they are located. Even at sites where the albedo around the station has been found “homogeneous” in the literature, marked differences are found between the daily observations and the gridded estimates at any spatial resolution. Furthermore, differences in seasonal behavior and between the three different albedo types also impact the accuracy of the albedo estimates, with overtones caused by local specificities and inhomogeneities. All this precludes the desirable evaluation of the local albedo at a specific site of relatively small size compared to its corresponding 4x4-km pixel if only the mean albedo over that pixel is known. Significant discrepancies are also found at snow-impacted sites, most importantly in the case of the NSRDB albedo estimates, which are typically much too high.
14 SOLAR ENERGY↗
Maturation of Structural Health Management Systems for Solid Rocket Motors
Concepts of an autonomous and automated space-compliant diagnostic system were developed for conditioned-based maintenance (CBM) of rocket motors for space exploration vehicles. The diagnostic system will provide real-time information on the integrity of critical structures on launch vehicles, improve their performance, and greatly increase crew safety while decreasing inspection costs. Using the SMART Layer technology as a basis, detailed procedures and calibration techniques for implementation of the diagnostic system were developed. The diagnostic system is a distributed system, which consists of a sensor network, local data loggers, and a host central processor. The system detects external impact to the structure. The major functions of the system include an estimate of impact location, estimate of impact force at impacted location, and estimate of the structure damage at impacted location. This system consists of a large-area sensor network, dedicated multiple local data loggers with signal processing and data analysis software to allow for real-time, in situ monitoring, and longterm tracking of structural integrity of solid rocket motors. Specifically, the system could provide easy installation of large sensor networks, onboard operation under harsh environments and loading, inspection of inaccessible areas without disassembly, detection of impact events and impact damage in real-time, and monitoring of a large area with local data processing to reduce wiring.
Quing, Xinlin↗
Measuring and Modeling the Earth's Gravity - Introduction to Ground-Based Gravity Surveys and Analysis of Local Gravity Data
We can measure changes in gravity from place to place on the earth. These measurements require careful recording of location, elevation and time for each reading. These readings must be adjusted for known effects (such as elevation, latitude, tides) that can bias our data and mask the signal of interest. After making corrections to our data, we can remove regional trends to obtain local Bouguer anomalies. The Bouguer anomalies arise from variations in the subsurface density structure. We can build models to explain our observations, but these models must be consistent with what is known about the local geology. Combining gravity models with other information – geologic, seismic, electromagnetic, will improve confidence in the results.
71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
Accessing and Visualizing scientific spatiotemporal data
This paper discusses work done by JPL 's Parallel Applications Technologies Group in helping scientists access and visualize very large data sets through the use of multiple computing resources, such as parallel supercomputers, clusters, and grids These tools do one or more of the following tasks visualize local data sets for local users, visualize local data sets for remote users, and access and visualize remote data sets The tools are used for various types of data, including remotely sensed image data, digital elevation models, astronomical surveys, etc The paper attempts to pull some common elements out of these tools that may be useful for others who have to work with similarly large data sets.
data sets↗
OCEAN-PC and a distributed network for ocean data
The Intergovernmental Oceanographic Commission (IOC) wishes to develop an integrated software package for oceanographic data entry and access in developing countries. The software, called 'OCEAN-PC', would run on low cost PC microcomputers and would encourage and standardize: (1) entry of local ocean observations; (2) quality control of the local data; (3) merging local data with historical data; (4) improved display and analysis of the merged data; and (5) international data exchange. OCEAN-PC will link existing MS-DOS oceanographic programs and data sets with table-driven format conversions. Since many ocean data sets are now being distributed on optical discs (Compact Discs - Read Only Memory, CD-ROM, Mass et al. 1987), OCEAN-PC will emphasize access to CD-ROMs.
Mclain, Douglas R.↗
Leveraging Cloud Computing to Improve Storage Durability, Availability, and Cost for MER Maestro
The Maestro for MER (Mars Exploration Rover) software is the premiere operation and activity planning software for the Mars rovers, and it is required to deliver all of the processed image products to scientists on demand. These data span multiple storage arrays sized at 2 TB, and a backup scheme ensures data is not lost. In a catastrophe, these data would currently recover at 20 GB/hour, taking several days for a restoration. A seamless solution provides access to highly durable, highly available, scalable, and cost-effective storage capabilities. This approach also employs a novel technique that enables storage of the majority of data on the cloud and some data locally. This feature is used to store the most recent data locally in order to guarantee utmost reliability in case of an outage or disconnect from the Internet. This also obviates any changes to the software that generates the most recent data set as it still has the same interface to the file system as it did before updates
Chang, George W.↗