Search NASA⌕ Search

SEARCH · Search NASA

Results for “TRAINING PLATFORMS”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 145 records · Page 8
All sourcesNASA NTRSDOE OSTI
DOE OSTI2021

A Software Framework for Comparing Training Approaches for Spiking Neuromorphic Systems

There are a wide variety of training approaches for spiking neural networks for neuromorphic deployment. However, it is often not clear how these training algorithms perform or compare when applied across multiple neuromorphic hardware platforms and multiple datasets. In this work, we present a software framework for comparing performance across four neuromorphic training algorithms across three neuromorphic simulators and four simple classification tasks. We introduce an approach for training a spiking neural network using a decision tree, and we compare this approach to training algorithms based on evolutionary algorithms, back-propagation, and reservoir computing. We present a hyperparameter optimization approach to tune the hyperparameters of the algorithm, and show that these optimized hyperparameters depend on the processor, algorithm, and classification task. Finally, we compare the performance of the optimized algorithms across multiple metrics, including accuracy, training time, and resulting network size, and we show that there is not one best training algorithm across all datasets and performance metrics.

Schuman, Catherine↗
NASA NTRS2023

A Global Land Cover Training Dataset From 1984 to 2020

State-of-the-art cloud computing platforms such as Google Earth Engine (GEE) enable regional-to-global land cover and land cover change mapping with machine learning algorithms. However, collection of high-quality training data, which is necessary for accurate land cover mapping, remains costly and labor-intensive. To address this need, we created a global database of nearly 2 million training units spanning the period from 1984 to 2020 for seven primary and nine secondary land cover classes. Our training data collection approach leveraged GEE and machine learning algorithms to ensure data quality and biogeographic representation. We sampled the spectral-temporal feature space from Landsat imagery to efficiently allocate training data across global ecoregions and incorporated publicly available and collaborator-provided datasets to our database. To reflect the underlying regional class distribution and post-disturbance landscapes, we strategically augmented the database. We used a machine learning-based cross-validation procedure to remove potentially mis-labeled training units. Our training database is relevant for a wide array of studies such as land cover change, agriculture, forestry, hydrology, urban development, among many others.

Radost Stanimirova↗
DOE OSTI2014

Nuclear programs in India and Pakistan

India and Pakistan launched their respective nuclear programs in the 1940s and 1950s with considerable foreign technical support, especially from the United States Atoms for Peace Program. The technology and training that was acquired served as the platform for later nuclear weapon development efforts that included nuclear weapon testing in 1974 and in 1998 by India, and also in 1998 by Pakistan - which had illicitly acquired uranium enrichment technology especially from Europe and received assistance from China. As of 2013, both India and Pakistan were continuing to produce fissile material for weapons, in the case of India also for nuclear naval fuel, and were developing a diverse array of ballistic and cruise missiles. International efforts to restrain the South Asian nuclear build-up have been largely set aside over the past decade as Pakistani support became central for the U.S. war in Afghanistan and as U.S. geopolitical and economic interests in supporting the rise of India, in part as a counter to China, led to India being exempted both from U.S non-proliferation laws and international nuclear trade guidelines. In the absence of determined international action and with Pakistan blocking the start of talks on a fissile material cutoff treaty, nuclear weapon programs in South Asia are likely to keep growing for the foreseeable future.

98 NUCLEAR DISARMAMENT, SAFEGUARDS, AND PHYSICAL P↗
DOE OSTI2024

The Galaxy platform for accessible, reproducible, and collaborative data analyses: 2024 update

Galaxy (https://galaxyproject.org) is deployed globally, predominantly through free-to-use services, supporting user-driven research that broadens in scope each year. Users are attracted to public Galaxy services by platform stability, tool and reference dataset diversity, training, support and integration, which enables complex, reproducible, shareable data analysis. Applying the principles of user experience design (UXD), has driven improvements in accessibility, tool discoverability through Galaxy Labs/subdomains, and a redesigned Galaxy ToolShed. Galaxy tool capabilities are progressing in two strategic directions: integrating general purpose graphical processing units (GPGPU) access for cutting-edge methods, and licensed tool support. Engagement with global research consortia is being increased by developing more workflows in Galaxy and by resourcing the public Galaxy services to run them. The Galaxy Training Network (GTN) portfolio has grown in both size, and accessibility, through learning paths and direct integration with Galaxy tools that feature in training courses. Code development continues in line with the Galaxy Project roadmap, with improvements to job scheduling and the user interface. Environmental impact assessment is also helping engage users and developers, reminding them of their role in sustainability, by displaying estimated CO 2 emissions generated by each Galaxy job.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Cyber-Physical Event Emulation-Based Transmission-and-Distribution Co-Simulation for Situational Awareness of Grid Anomalies (SAGA)

Energy management of transmission and distribution networks (T&D) is becoming more challenging with the accelerated adoption of distributed energy resources (DERs)-such as distributed photovoltaic generation and battery energy storage systems (BESS)-on the electric grid. To better analyze the impacts of DERs on both transmission and distribution systems, a comprehensive T&D co-simulation platform is developed. Further, with DERs more actively participating in system operation-e.g., by providing real-time grid services-their cyber vulnerability needs to be better understood to maintain system reliability. This paper discusses a cyber-physical events emulation-based T&D co-simulation platform to perform comprehensive cyber events emulations, physical simulation, and analysis of interdependent impacts. Results from the case studies-which show how cyber events on a synthetic distribution network can impact operations on the transmission and distribution network-validate that the proposed T&D cosimulation platform can perform cyber-physical events emulation and produce response in near realtime; therefore, with extensive simulation using the proposed co-simulation platform, the system operators can accumulate adequate training data for system situational awareness of grid anomalies.

anomalies detection↗
DOE OSTI2022

Real-time Distribution Simulation and Application Development for Power Systems Education

To help bridge the gap between traditional power system engineering instruction and emerging industry needs, a new set of classroom and research tools are needed. Real-time simulation tools emulating power system control room software present an opportunity to introduce students to the array of operational considerations, technical challenges, and decision-making associated with power system operations. The GridAPPS-D platform is proposed to support coursework and academic research as it provides an open-source platform for simulation, application development, and software integration. The GridAPPS-D platform, simulation capabilities, development environment, and interactive training are discussed in the context of lessons-learned from implementation for undergraduate and graduate students in the US and India. A series of potential GridAPPS-D supported academic capabilities are introduced.

Active distribution networks, open educational res↗
DOE OSTI2018

CO 2 and Carbon Emissions from Cities: Linkages to Air Quality, Socioeconomic Activity, and Stakeholders in the Salt Lake City Urban Area

Urban areas are responsible for a substantial proportion of anthropogenic carbon emissions around the world. As global populations increasingly reside in cities, the role of urban emissions in determining the future trajectory of carbon emissions is magnified. Consequently, a number of research efforts have been started in the United States and beyond, focusing on observing atmospheric carbon dioxide (CO 2 ) and relating its variations to carbon emissions in cities. Because carbon emissions are intimately tied to socioeconomic activity through the combustion of fossil fuels, and many cities are actively adopting emission reduction plans, such urban carbon research efforts give rise to opportunities for stakeholder engagement and guidance on other environmental issues, such as air quality. This paper describes a research effort centered in the Salt Lake City, Utah, metropolitan region, which is the locus for one of the longest-running urban CO 2 networks in the world. The Salt Lake City area provides a rich environment for studying anthropogenic emissions and for understanding the relationship between emissions and socioeconomic activity when the CO 2 observations are enhanced with a) air quality observations, b) novel mobile observations from platforms on light-rail public transit trains and a news helicopter, c) dense meteorological observations, and d) modeling efforts that include atmospheric simulations and high-resolution emission inventories. Carbon dioxide and other atmospheric observations are presented herein, along with associated modeling work. Examples in which the work benefited from and contributed to the interests of multiple stakeholders (e.g., policymakers, air quality managers, municipal government, urban planners, industry, and the general public) are discussed.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
NASA NTRS2001

Space Station Human Factors: Designing a Human-Robot Interface

The experiments described in this paper are part of a larger joint MIT/NASA research effort and focus on the development of a methodology for designing and evaluating integrated interfaces for highly dexterous and multifunctional telerobot. Specifically, a telerobotic workstation is being designed for an Extravehicular Activity (EVA) anthropomorphic space station telerobot called Robonaut. Previous researchers have designed telerobotic workstations based upon performance of discrete subsets of tasks (for example, peg-in-hole, tracking, etc.) without regard for transitions that operators go through between tasks performed sequentially in the context of larger integrated tasks. The experiments presented here took an integrated approach to describing teleoperator performance and assessed how subjects operating a full-immersion telerobot perform during fine position and gross position tasks. In addition, a Robonaut simulation was also developed as part of this research effort, and experimentally tested against Robonaut itself to determine its utility. Results show that subject performance of teleoperated tasks using both Robonaut and the simulation are virtually identical, with no significant difference between the two. These results indicate that the simulation can be utilized as both a Robonaut training tool, and as a powerful design platform for telepresence displays and aids.

Rochlis, Jennifer L.↗
NASA NTRS2012

NASA's Space Launch System: Affordability for Sustainability

The National Aeronautics and Space Administration's (NASA) Space Launch System (SLS) Program, managed at the Marshall Space Flight Center, is charged with delivering a new capability for human exploration beyond Earth orbit in an austere economic climate. But the SLS value is clear and codified in United States (U.S.) budget law. The SLS Program knows that affordability is the key to sustainability and will provide an overview of initiatives designed to fit within the funding guidelines by using existing engine assets and hardware now in testing to meet a first launch by 2017 within the projected budget. It also has a long-range plan to keep the budget flat, yet evolve the 70-tonne (t) initial lift capability to 130-t lift capability after the first two flights. To achieve the evolved configuration, advanced technologies must offer appropriate return on investment to be selected through the competitive process. For context, the SLS will be larger than the Saturn V that took 12 men on 6 trips for a total of 11 days on the lunar surface some 40 years ago. Astronauts train for long-duration voyages on platforms such as the International Space Station, but have not had transportation to go beyond Earth orbit in modern times, until now. To arrive at the launch vehicle concept, the SLS Program conducted internal engineering and business studies that have been externally validated by industry and reviewed by independent assessment panels. In parallel with SLS concept studies, NASA is now refining its mission manifest, guided by U.S. space policy and the Global Exploration Roadmap, which reflects the mutual goals of a dozen member nations. This mission planning will converge with a flexible heavy-lift rocket that can carry international crews and the air, water, food, and equipment they need for extended trips to asteroids and Mars. In addition, the SLS capability will accommodate very large science instruments and other payloads, using a series of modular fairings and adapters to configure the rocket for the mission. The SLS affordability plan includes streamlining interfaces, applying risk-based insight into contracted work, centralizing systems engineering and integration, and nurturing a learning culture where the question Why? is often asked and the answer "Because we've always done it that way" is rarely heard. The SLS Program will deliver affordable space transportation solutions for the Orion Multi-Purpose Cargo Vehicle s first autonomous certification flight in 2017, followed by a crewed flight in 2021. As this briefing will show, the SLS will offer a global infrastructure asset for robotic and human scouts of all nations.

May, Todd A.↗
NASA NTRS2012

Application of Artificial Neural Networks to the Development of Improved Multi-Sensor Retrievals of Near-Surface Air Temperature and Humidity Over Ocean

Improved estimates of near-surface air temperature and air humidity are critical to the development of more accurate turbulent surface heat fluxes over the ocean. Recent progress in retrieving these parameters has been made through the application of artificial neural networks (ANN) and the use of multi-sensor passive microwave observations. Details are provided on the development of an improved retrieval algorithm that applies the nonlinear statistical ANN methodology to a set of observations from the Advanced Microwave Scanning Radiometer (AMSR-E) and the Advanced Microwave Sounding Unit (AMSU-A) that are currently available from the NASA AQUA satellite platform. Statistical inversion techniques require an adequate training dataset to properly capture embedded physical relationships. The development of multiple training datasets containing only in-situ observations, only synthetic observations produced using the Community Radiative Transfer Model (CRTM), or a mixture of each is discussed. An intercomparison of results using each training dataset is provided to highlight the relative advantages and disadvantages of each methodology. Particular emphasis will be placed on the development of retrievals in cloudy versus clear-sky conditions. Near-surface air temperature and humidity retrievals using the multi-sensor ANN algorithms are compared to previous linear and non-linear retrieval schemes.

Roberts, J. Brent↗
NASA NTRS2014

NASA 2014 The Hyperspectral Infrared Imager (HyspIRI) - Science Impact of Deploying Instruments on Separate Platforms

The Hyperspectral Infrared Imager (HyspIRI) mission was recommended for implementation by the 2007 report from the U.S. National Research Council Earth Science and Applications from Space: National Imperatives for the Next Decade and Beyond, also known as the Earth Science Decadal Survey. The HyspIRI mission is science driven and will address a set of science questions identified by the Decadal Survey and broader science community. The mission includes a visible shortwave infrared (VSWIR) imaging spectrometer, a multispectral thermal infrared (TIR) imager and an intelligent payload module (IPM). The IPM enables on-board processing and direct broadcast for those applications with short latency requirements. The science questions are organized as VSWIR-only, TIR-only and Combined science questions, the latter requiring data from both instruments. In order to prepare for the mission NASA is undertaking pre-phase A studies to determine the optimum mission implementation, in particular, cost and risk reduction activities. Each year the HyspIRI project is provided with feedback from NASA Headquarters on the pre-phase A activities in the form of a guidance letter which outlines the work that should be undertaken the subsequent year. The 2013 guidance letter included a recommendation to undertake a study to determine the science impact of deploying the instruments from separate spacecraft in sun synchronous orbits with various time separations and deploying both instruments on the International Space Station (ISS). This report summarizes the results from that study. The approach taken was to evaluate the impact on the combined science questions of time separations between the VSWIR and TIR data of <3 minutes, <1 week and a few months as well as deploying both instruments on the ISS. Note the impact was only evaluated for the combined science questions which require data from both instruments (VSWIR and TIR). The study concluded the impact of a separation of <3 minutes was minimal, e.g. if the instruments were on separate platforms that followed each other in a train. The impact of a separation of <1 week was strongly dependent on the question that was being addressed with no impact for some questions and a severe impact for others. The impact of a time separation of several months was severe and in many cases it was no longer possible to answer the sub-question. The impact of deploying the instruments on the ISS which is in a precessive (non-sun synchronous) orbit was also very question dependent, in some cases it was possible to go beyond the original question, e.g. to examine the impact of the diurnal cycle, whereas in other cases the question could not be addressed for example if the question required observations from the polar regions. As part of the study, the participants were asked to estimate, as a percentage, how completely a given sub-question could be answered with 100% indicating the question could be completely answered. These estimations should be treated with caution but nonetheless can be useful in assessing the impact. Averaging the estimates for each of the combined questions the results indicate that 97% of the questions could be answered with a separation of < 3 minutes. With a separation of < 1 week, 67% of the questions could be answered and with a separation of several months only 21% of the questions could be answered.

Turpie, Kevin↗
NASA NTRS2007

Exploring and Visualizing A-Train Instrument Data

The succession of US and international satellites that follow each other in close succession, known as the A-Train, affords an opportunity to atmospheric researchers that no single platform could provide: Increasing the number of observations at any given geographic location.. . a more complete "virtual science platform". However, vertically and horizontally, co-registering and regridding datasets from independently developed missions, Aqua, Calipso, Cloudsat, Parasol, and Aura, so that they can be inter-compared can be daunting to some, and may be repeated by many. Scientists will individually spend much of their time and resources acquiring A-Train datasets of interest residing at various locations, developing algorithms to match up and graph datasets along the A-Train track, and search through large amounts of data for areas and/or phenomena of interest. The aggregate amount of effort that can be expended on repeating pre-science tasks could climb into the tens of millions of dollars. The goal of the A-Train Data Depot (ATDD) is to enable free movement of remotely located A-Train data so that they are combined to create a consolidated vertical view of the Earth's Atmosphere along the A-Train tracks. The innovative approach of analyzing and visualizing atmospheric profiles along the platforms track (i.e., time) is accomplished by through the ATDDs Giovanni data analysis and visualization tool. Giovanni brings together data from Aqua (MODIS, AIRS, AMSR-E), Cloudsat (cloud profiling radar) and Calipso (CALIOP, IIR), as well as the Aura (OMI, MLS, HIRDLS, TES) to create a consolidated vertical view of the Earth's Atmosphere along the A-Train tracks. This easy to learn and use exploration tool will allow users to create vertical profiles of any desired A-Train dataset, for any given time of choice. This presentation shows the power of Giovanni by describing and illustrating how this tool facilitates and aids A-Train science and research. A web based display system Giovanni provides users with the capability of creating co-located profile images of temperature and humidity data from the MODIS, MLS and AIRS instruments for a user specified time and spatial area. In addition, Cloud and Aerosol profiles may also be displayed for the Cloudsat and Caliop instruments. The ability to modify horizontal and vertical axis range, data range and dynamic color range is also provided. Two dimensional strip plots of MODIS, AIRS, OM1 and POLDER parameters, co-located along the Cloudsat reference track, can also be plotted along with the Cloudsat cloud profiling data. Center swath pixels for the same parameters can also be shown as line plots overlaying the Cloudsat or Calipso profile images. Images and subsetted data produced in each analysis run may be downloaded. Users truly can explore and discover data specific to their needs prior to ever transferring data to their analysis tools.

Kempler, S.↗
NASA NTRS2012

Upset Simulation and Training Initiatives for U.S. Navy Commercial Derived Aircraft

Militarized versions of commercial platforms are growing in popularity due to many logistical benefits in the form of commercial off-the-shelf (COTS) parts, established production methods, and commonality for different certifications. Commercial data and best practices are often leveraged to reduce procurement and engineering development costs. While the developmental and cost reduction benefits are clear, these militarized aircraft are routinely operated in flight at significantly different conditions and in significantly different manners than for routine commercial flight. Therefore they are at a higher risk of flight envelope exceedance. This risk may lead to departure from controlled flight and/or aircraft loss1. Historically, the risk of departure from controlled flight for military aircraft has been mitigated by piloted simulation training and engineering analysis of typical aircraft response. High-agility military aircraft simulation databases are typically developed to include high angles of attack (AoA) and sideslip due to the dynamic nature of their missions and have been developed for many tactical configurations over the previous decades. These aircraft simulations allow for a more thorough understanding of the vehicle flight dynamics characteristics at high AoA and sideslip. In recent years, government sponsored research on transport airplane aerodynamic characteristics at high angles of attack has produced a growing understanding of stall/post-stall behavior. This research along with recent commercial airline training initiatives has resulted in improved understanding of simulator-based training requirements and simulator model fidelity.2-5 In addition, inflight training research over the past decade has produced a database of pilot performance and recurrency metrics6. Innovative solutions to aerodynamically model large commercial aircraft for upset conditions such as high AoA, high sideslip, and ballistic damage, as well as capability to accurately account for scaling factors, is necessary to develop realistic engineering and training simulations. Such simulations should significantly reduce the risk of departure from controlled flight, loss of aircraft, and ease the airworthiness certification process. The characteristics of commercial derivative aircraft are exemplified by the P-8A Multi-mission Maritime Aircraft (MMA) aircraft, and the largest benefits of initial investigation are likely to be yielded from this platform. The database produced would also be utilized by flight dynamics engineers as a means to further develop and investigate vehicle flight characteristics as mission tactics evolve through the years ahead. This paper will describe ongoing efforts by the U.S. Navy to develop a methodology for simulation and training for large commercial-derived transport aircraft at unusual attitudes, typically experienced during an aircraft upset. This methodology will be applied to a representative Navy aircraft (P-8A) and utilized to develop a robust simulation that should accurately represent aircraft response in these extremes. Simulation capabilities would then extend to flight dynamics analysis and simulation, as well as potential training applications. Recent evaluations of integrated academic, ground-based simulation, and in-flight upset training will be described along with important lessons learned, specific to military requirements.

Donaldson, Steven↗
DOE OSTI2024

chatHPC: Empowering HPC users with large language models

The ever-growing number of pre-trained large language models (LLMs) across scientific domains presents a challenge for application developers. While these models offer vast potential, fine-tuning them with custom data, aligning them for specific tasks, and evaluating their performance remain crucial steps for effective utilization. However, applying these techniques to models with tens of billions of parameters can take days or even weeks on modern workstations, making the cumulative cost of model comparison and evaluation a significant barrier to LLM-based application development. To address this challenge, we introduce an end-to-end pipeline specifically designed for building conversational and programmable AI agents on high performance computing (HPC) platforms. Our comprehensive pipeline encompasses: model pre-training, fine-tuning, web and API service deployment, along with crucial evaluations for lexical coherence, semantic accuracy, hallucination detection, and privacy considerations. Here, we demonstrate our pipeline through the development of chatHPC, a chatbot for HPC question answering and script generation. Leveraging our scalable pipeline, we achieve end-to-end LLM alignment in under an hour on the Frontier supercomputer. We propose a novel self-improved, self-instruction method for instruction set generation, investigate scaling and fine-tuning strategies, and conduct a systematic evaluation of model performance. The established practices within chatHPC will serve as a valuable guidance for future LLM-based application development on HPC platforms.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

VWC-BERT: Scaling Vulnerability–Weakness–Exploit Mapping on Modern AI Accelerators

Defending cybersystems needs accurate mapping of software and hardware vulnerabilities to generalized descriptions of weaknesses, and weaknesses to exploits. These mappings enable cyber defenders to build plans for effective defense and assessment of potential risks to a cybersystem. With close to 170k vulnerabilities, manual mapping is not a feasible option. However, automated mapping is challenging due to limited training data, computational intractability, and limitations in computational natural language processing. Tools based on breakthroughs in Transformer-based language models have been demonstrated to classify vulnerabilities with high accuracy. We make three key contributions in this paper: (1) We present a new framework, \VWCBERT, that augments the Transformer-based hierarchical multi-class classification framework of Das et al. (\textsc{V2W-BERT}) with the ability to map weaknesses to exploits. (2) We implement \VWCBERT~ on modern AI accelerator platforms using two data parallel techniques for the pre-training phase and demonstrate nearly linear speedups across NVIDIA and Graphcore accelerator platforms. We observe nearly linear speedups for up to 16 V100 and 8 A100 GPUs, and about 3.4$\times$ speedup for A100 relative to V100 GPUs. We also observe excellent speedups on Graphcore, with $5.7\times$ speedup on 128 IPUs relative to 16 IPUs. Enabled by scaling, we also demonstrate higher accuracy using a larger language model, RoBERTa-Large. We show up to 87\% accuracy for strict and up to 98\% accuracy for relaxed classification. (3) We develop a novel parallel link manager for the link prediction phase and demonstrate up to 21$\times$ speedup with 16 V100 GPUs relative to one V100 GPU, and thus reducing the runtime from 2.5 hours to 10 minutes. We believe that generalizability and scalability of \VWCBERT~ will benefit both the theoretical development and practical deployment of novel cyberdefense solutions and vulnerability classification.

Das, Siddhartha Shankar↗
NASA NTRS2018

NeMO-Net - The Neural Multi-Modal Observation & Training Network for Global Coral Reef Assessment

In the past decade, coral reefs worldwide have experienced unprecedented stresses due to climate change, ocean acidification, and anthropomorphic pressures, instigating massive bleaching and die-off of these fragile and diverse ecosystems. Furthermore, remote sensing of these shallow marine habitats is hindered by ocean wave distortion, refraction and optical attenuation, leading invariably to data products that are often of low resolution and signal-to-noise (SNR) ratio. However, recent advances in UAV and Fluid Lensing technology have allowed us to capture multispectral 3D imagery of these systems at sub-cm scales from above the water surface, giving us an unprecedented view of their growth and decay. By combining spatial and spectral information from varying resolutions, we seek to augment and improve the classification accuracy of previously low-resolution datasets at large temporal scales.NeMO-Net, the first open-source deep convolutional neural network (CNN) and interactive learning and training software, currently being developed at NASA Ames, is aimed at assessing the present and past dynamics of coral reef ecosystems through determination of percent living cover and morphology. The latest iteration uses fully convolutional networks to segment and identify coral imagery taken by UAVs and satellites, including WorldView-2 and Sentinel. We present results taken from the Indian Ocean where classification accuracy has exceeded 91% for 24 geomorphological classes given ample training data. In addition, we utilize deep Laplacian Pyramid Super-Resolution Networks (LapSRN) to reconstruct high resolution information from low resolution imagery, trained from various UAV and satellite datasets. Finally, in the case of insufficient training data, we have developed an interactive online platform that allows users to easily segment and submit their classifications, which has been integrated with the current NeMO-Net workflow. Specifically, we present results from the Fiji islands in which preliminary user data has allowed for the accurate identification of 9 separate classes, despite issues such as cloud shadowing and spectral variation. The project is being supported by NASA's Earth Science Technology Office (ESTO) Advanced Information Systems Technology (AIST-16) Program.

Neural↗
DOE OSTI2024

MDLoader: A Hybrid Model-Driven Data Loader for Distributed Graph Neural Network Training

Scalable data management is essential for processing large scientific dataset on HPC platforms for distributed deep learning. In-memory distributed storage is preferred for its speed, enabling rapid, random, and frequent data access required by stochastic optimizers. Processes use one-sided or collective communication to fetch remote data, with optimal performance depending on (i) dataset characteristics, (ii) training scale, and (iii) interconnection network. Empirical analysis shows collective communication excels with larger mini-batch sizes and/or fewer processes, whereas one-sided communication outperforms at larger scales. We propose MDLoader, a hybrid in-memory data loader for distributed graph neural network training. MDLoader features a model-driven performance estimator that dynamically selects between one-sided and collective communication at the beginning of training using Tree of Parzen Estimators (TPE). Evaluations on NERSC Perlmutter and OLCF Summit show MDLoader outperforms single-backend loaders by up to 2.83 × and predicts the suitable communication method with 96.3% (Perlmutter) and 94.3% (Summit) success rate.

Bae, Jonghyun↗
NASA NTRS2019

Accelerated Simulation of Air Pollution Using NVIDIA RAPIDS

Atmospheric chemistry models are a central tool to study and forecast the impact of air pollution on the environment, vegetation, and human health. However, the numerical simulation of chemical kinetics is computationally expensive due to the stiffness of the system of ordinary differential equations that describes atmospheric chemistry. Here we present an alternative approach to the computation of atmospheric chemistry based on machine learning. Our training data set is produced using the NASA Goddard Earth Observing System (GEOS) model with GEOS-Chem chemistry, run on the NASA Center for Climate Simulation (NCCS) Discover supercomputing cluster on 384 Intel Xeon Haswell cores. This model spends more than 50% of total run time on solving atmospheric chemistry. The data set contains as input features the air pollution concentrations before solving the differential equations, together with some key physical parameters such as temperature and sun intensity. As target variables we define the air pollution concentrations after solving the differential equations. Using Dask-cuDF and Dask-XGBoost on the NVIDIA RAPIDS platform on 8 Tesla V100 GPUs, we generate from this training set gradient boosted decision tree models that can reproduce the simulation of chemical kinetics. We do this on the NCCS Advanced Data Analytics Platform (ADAPT) science cloud environment. Our application takes full advantage of recent advances in Dask-XGBoost, such as multi-node and multi-GPU scaling for distributed training with large data sets. The increase in training data size enabled by this is critical to capture the full range of chemical environments encountered across the globe and all annual seasons.The boosted tree models offer good predictability and show many of the features of the full chemistry reference simulation. Further improvements can be achieved through mass balance considerations and by accounting for error correlations. We incorporate the boosted tree models into the GEOS reference model using XGBoost's C API. This enables a seamless integration of the GPU trained models into GEOS-Chem, which is written in Fortran and optimized for use in a massively parallel CPU environment. We show the benefits of this approach and discuss the potential speedup of this machine learning accelerated atmospheric chemistry model.

Keller, Christoph A.↗