Chandra Highlights
This talk will discuss the present status of Chandra's operations and pipeline processing and will present some science highlights.
Fabbiano, Giuseppina↗
SEARCH · Search NASA
Results for “Science Pipeline”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Packaging data products using data grid middleware for Deep Space Mission Systems
We are investigating a distributed computing infrastructure for releasing science products out of the mission pipeline. We are creating a DSMS Product Service and DSMS Data Packaging Service based on the OODT middleware, a Data Grid middleware technology that has proven successful in providing access to heterogeneous, disparate data, and federating that data using a common data model.
Deep Space Mission Systems↗
The Cumulus Ecosystem: Open Source and Beyond to Foster Collaboration
NASA’s Earth Observing System Data and Information System (EOSDIS) open source Cumulus software is designed as a common set of code and services that can be used to create a pipeline to deliver and manage earth science data in the cloud. Cumulus strives to create an ecosystem on the foundation of open source that unites those with shared problems and goals by encouraging users to contribute solutions back to the platform. Large parts of ingesting and managing data are common and much of what is created can be used by others. Our goal is to maximize collaboration and code reuse while allowing users to design a custom solution that meets their needs without having to take on extraneous functionality. In this talk we will describe how the Cumulus ecosystem works beyond just open source software. We will review the technology, the successes and challenges, and the evolution and future of Cumulus as an ecosystem.
Cumulus↗
Tuning a variational autoencoder for data accountability problem in the Mars Science Laboratory ground data system
The Mars Curiosity rover is frequently sending back engineering and science data that goes through a pipeline of systems before reaching its final destination at the mission operations center making it prone to volume loss and data corruption. A ground data system analysis (GDSA) team is charged with the monitoring of this flow of information and the detection of anomalies in that data in order to request a re-transmission when necessary. This work presents ∆-MADS, a derivative-free optimization method applied for tuning the architecture and hyperparameters of a variational autoencoder trained to detect the data with missing patches in order to assist the GDSA team in their mission.
Lakhmiri, Dounia↗
Aerospace Workforce Development: The Nebraska Proposal; and Native View Connections: A Multi-Consortium Workforce Development Proposal
The NASA Nebraska Space Grant Consortium (NSGC) continues to recognize the necessity of increasing the quantity and quality of highly skilled graduates and faculty involved with NASA. Through NASA Workforce Development funds awarded in 2002, NSGC spearheaded customer- focused workforce training and higher education, industry and community partnerships that are significantly impacting the state s workforce in the science, technology, engineering, and mathematics (STEM) competencies. NSGC proposes to build upon these accomplishments to meet the steadily increasing demand for STEM skills and to safeguard minority representation in these disciplines. A wide range of workforce development activities target NASA s need to establish stronger connections among higher education, industry, and community organizations. Participation in the National Student Satellite Program (NSSP), Community Internship Program, and Nebraska Science and Technology Recruitment Fair will extend the pipeline of employees benefiting NASA as well as Nebraska. The diversity component of this proposal catapults from the exceptional reputation NSGC has built by delivering geospatial science experiences to Nebraska s Native Americans. For 6 years, NSGC has fostered and sustained partnerships with the 2 tribal colleges and 4 reservation school districts in Nebraska to foster aeronautics education and outreach. This program, the Nebraska Native American Outreach Program (NNAOP), has grown to incorporate more than educational institutions and is now a partnership among tribal community leaders, academia, tribal schools, and industry. The content focus has broadened from aeronautics in the school systems to aerospace technology and earth science applications in tribal community decision-making and workforce training on the reservations. To date, participants include faculty and staff at 4 Nebraska tribal schools, 2 tribal colleges, approximately 1,000 Native American youth, and over 1,200 community members. This Native American Initiative of the NSGC addresses Nebraska workforce development and serves as a model to others. Following a structured evaluation process, NSGC proposes to sustain delivery of the training funded by NASA in 2002 to tribal entities through partnerships linking academic programs and industry leaders.
Bowen, Brent D.↗
Implementation and Comparison of Acoustic Travel-Time Measurement Procedures for the Solar Dynamics Observatory-Helioseismic and Magnetic Imager Time-Distance Helioseismology Pipeline
The Helioseismic and Magnetic Imager (HMI) instrument onboard the Solar Dynamics Observatory (SDO) satellite is designed to produce high-resolution Doppler-velocity maps of oscillations at the solar surface with high temporal cadence. To take advantage of these high-quality oscillation data, a time - distance helioseismology pipeline (Zhao et al., Solar Phys. submitted, 2010) has been implemented at the Joint Science Operations Center (JSOC) at Stanford University. The aim of this pipeline is to generate maps of acoustic travel times from oscillations on the solar surface, and to infer subsurface 3D flow velocities and sound-speed perturbations. The wave travel times are measured from cross-covariances of the observed solar oscillation signals. For implementation into the pipeline we have investigated three different travel-time definitions developed in time - distance helioseismology: a Gabor-wavelet fitting (Kosovichev and Duvall, SCORE'96: Solar Convection and Oscillations and Their Relationship, ASSL, Dordrecht, 241, 1997), a minimization relative to a reference cross-covariance function (Gizon and Birch, Astrophys. J. 571, 966, 2002), and a linearized version of the minimization method (Gizon and Birch, Astrophys. J. 614, 472, 2004). Using Doppler-velocity data from the Michelson Doppler Imager (MDI) instrument onboard SOHO, we tested and compared these definitions for the mean and difference traveltime perturbations measured from reciprocal signals. Although all three procedures return similar travel times in a quiet-Sun region, the method of Gizon and Birch (Astrophys. J. 614, 472, 2004) gives travel times that are significantly different from the others in a magnetic (active) region. Thus, for the pipeline implementation we chose the procedures of Kosovichev and Duvall (SCORE'96: Solar Convection and Oscillations and Their Relationship, ASSL, Dordrecht, 241, 1997) and Gizon and Birch (Astrophys. J. 571, 966, 2002). We investigated the relationships among these three travel-time definitions, their sensitivities to fitting parameters, and estimated the random errors that they produce.
HMI↗
E-PiPEline: Quick to Market Emulated-PPE using Commonly Available Materials
The Center for Disease Control has recommended the public to wear cloth face coverings in public settings that reduce potential exposure to COVID-19 where other social distancing measures are difficult to maintain (e.g., grocery stores and pharmacies) especially in areas of significant community based transmission. These face coverings and other Emulated-Personal Protective Equipment (EPPE) can be made by using Commonly Available Materials (CAMs). As part of the Sandia COVID-19 LDRD effort (funded under the Materials Science Investment Area), the Sandia E-PiPEline task evaluated E-PPE design options for face coverings and face shields considering their effectiveness, durability, build difficulty, build cost, and comfort. Observations from this investigation are presented here to provide guidelines for home construction of E-PPE. This executive summary includes a brief roadmap of the analysis methodology, two one-page handouts geared to be distributed to the public at large (one for E-PPE face coverings and one for E-PPE face shields), and additional observations regarding the potential solutions for E-PPE face coverings and face shields included to further support the one-page handouts.
60 APPLIED LIFE SCIENCES↗
The catalog-to-cosmology framework for weak lensing and galaxy clustering for LSST
We present TXPipe, a modular, automated and reproducible pipeline for ingesting catalog data and performing all the calculations required to obtain quality-assured two-point measurements of lensing and clustering, and their covariances, with the metadata necessary for parameter estimation. The pipeline is developed within the Rubin Observatory Legacy Survey of Space and Time (LSST) Dark Energy Science Collaboration (DESC), and designed for cosmology analyses using LSST data. In this paper, we present the pipeline for the so-called 3x2pt analysis -- a combination of three two-point functions that measure the auto- and cross-correlation between galaxy density and shapes. We perform the analysis both in real and harmonic space using TXPipe and other LSST-DESC tools. We validate the pipeline using Gaussian simulations and show that it accurately measures data vectors and recovers the input cosmology to the accuracy level required for the first year of LSST data under this simplified scenario. We also apply the pipeline to a realistic mock galaxy sample extracted from the CosmoDC2 simulation suite (Korytov et al. 2019). TXPipe establishes a baseline framework that can be built upon as the LSST survey proceeds. Furthermore, the pipeline is designed to be easily extended to science probes beyond the 3x2pt analysis.
79 ASTRONOMY AND ASTROPHYSICS↗
Optimal decision-making in high-throughput virtual screening pipelines
Screening large pools of molecular candidates to identify those with specific design criteria or targeted properties is demanding in various science and engineering domains. While a high-throughput virtual screening (HTVS) pipeline can provide efficient means to achieving this goal, its design and operation often rely on experts' intuition, potentially resulting in suboptimal performance. In this paper, we fill this critical gap by presenting a systematic framework that can maximize the return on computational investment (ROCI) of such HTVS campaigns. Based on various scenarios, we empirically validate the proposed framework and demonstrate its potential to accelerate scientific discoveries through optimal computational campaigns, especially in the context of virtual screening.
97 MATHEMATICS AND COMPUTING↗
The Mars 2020 Ground Data System Architecture
The Mars 2020 Mission’s primary objective is to collect 20 geographically unique samples during its prime mission of one and a quarter Martian years, or just over 2 Earth years. Mission planners determined the project needed to develop a system that would enable the operations team to analyze engineering and science data, make science decisions, select viable rover targets at a millimeter resolution and validate an uplink bundle for a car sized rover with more complex science instruments than any previous Mars surface mission. All this had to be done within a five hour time frame. Doing this with a small team would be a challenge, but this had to be accomplished by a large team of engineers and scientists located across North America and Europe. Achieving this level of operational efficiency was unheard of in the prime mission. In addition, the mission had another set of requirements that had nothing to do with surface operations; the Mars 2020 Ground Data System (GDS) was also expected to comply with a new set of security requirements to keep up with the ever changing cybersecurity landscape. The Mars 2020 Ground Data System (GDS) is a re-architected version of the Mars Science Laboratory GDS. The primary goal was to integrate the lessons learned from previous Mars surface missions, accommodate a set of new requirements and capabilities required to ensure mission success, and comply with a new set of cybersecurity controls. The new architecture includes several unique qualities including a data lake, language-agnostic system-wide event-based operations, containerization, automated deployment, network segmentation, infrastructure-as-code, API-driven interfaces, and the first Mars surface GDS to operate primarily in the cloud. The new architecture enabled greater access to the system’s data, tighter integration with the operations team, and a higher level of traceability. The availability of the data also enabled a new set of capabilities previously not possible on surface missions. These new capabilities include an autonomous data to information, pipeline for downlink analysis, horizontal scaling of science data processing capabilities, autonomous round trip data tracking of science and engineering data, integration of flight system state into the tactical planning cycle, high fidelity targeting utilizing kinematic data, and hierarchical image and 3d meshes data representations. This paper will introduce the requirements for the Mars 2020 Mission, the heritage architecture, and the rationale for the changes to achieve the new architecture. The paper will continue to describe the fundamental changes made to the GDS architecture, how these changes enabled a more tightly integrated GDS, and the new capabilities that were enabled by the new architecture. The paper will conclude with the lessons learned from the process of rearchitecting a heritage GDS system and from the first 200 days of operations supporting over 800 users from around the world.
Lopez-Roig, Reynaldo↗
DRAGraces: A Pipeline for the GRACES High-resolution Spectrograph at Gemini
This paper describes the software Data Reduction and Analysis for GRACES (DRAGraces), which is a pipeline reducing spectra from Gemini Remote Access to the CFHT ESPaDOnS Spectrograph (GRACES) at the Gemini North Telescope. The code is written in the IDL language. It is designed to find all the GRACES frames in a given directory, automatically determine the list of bias, flat, arc, and science frames, and perform the whole reduction and extraction within a few minutes. We compare the output from DRAGraces with that of the Open source Pipeline for ESPaDOnS Reduction and Analysis (OPERA), a pipeline developed at the Canada–France–Hawaii Telescope (CFHT) that also can extract GRACES spectra. Both pipelines were developed completely independently, yet they give very similar extracted spectra. They both have their advantages and disadvantages. For instance, DRAGraces is more straightforward and easy to use and is less likely to be derailed by a parameter that needs to be tweaked, while OPERA offers a more careful extraction that can be significantly superior when the highest resolution is required and when the signal-to-noise ratio is low. One should compare both before deciding which one to use for their science. Yet, both pipelines deliver a fairly comparable resolution power (R ~ 52.8k and 36.6k for DRAGraces and R ~ 58k and 40k for OPERA in high- and low-resolution spectral modes, respectively), wavelength solution, and signal-to-noise ratio per resolution element.
46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
Reviews and syntheses: The promise of big diverse soil data, moving current practices towards future potential
Abstract. In the age of big data, soil data are more available and richer than ever, but – outside of a few large soil survey resources – they remain largely unusable for informing soil management and understanding Earth system processes beyond the original study. Data science has promised a fully reusable research pipeline where data from past studies are used to contextualize new findings and reanalyzed for new insight. Yet synthesis projects encounter challenges at all steps of the data reuse pipeline, including unavailable data, labor-intensive transcription of datasets, incomplete metadata, and a lack of communication between collaborators. Here, using insights from a diversity of soil, data, and climate scientists, we summarize current practices in soil data synthesis across all stages of database creation: availability, input, harmonization, curation, and publication. We then suggest new soil-focused semantic tools to improve existing data pipelines, such as ontologies, vocabulary lists, and community practices. Our goal is to provide the soil data community with an overview of current practices in soil data and where we need to go to fully leverage big data to solve soil problems in the next century.
54 ENVIRONMENTAL SCIENCES↗
Undergraduate Women in STEM Internship Program
In collaboration with the Universities Research Association, Inc. (URA), establishment of the Undergraduate Women in STEM Internship pilot program assists Fermi Research Alliance, LLC (FRA) with honoring its commitment to increasing gender, racial and ethnic workforce diversity. This focused initiative is intended to invite, support and yield a qualified pipeline of alumnae underrepresented in FRA’s science and technology management and technical staff positions. Interns in this program spend 10-12 weeks at Fermilab working under the direct leadership, guidance and supervision of scientists, engineers, computing professionals, technicians and staff across all directorates performing hands-on research. In addition, the intern cohort meets with mentors and advisors to grow their professional networks, to take part in engaging and inclusive intellectual discussions, to directly benefit from workplace resources, and to participate in professional development and social engagement opportunities. The design of each research project and the supervision, mentorship and evaluation is conducted by a scientific, engineering, computing professional at Fermilab.
99 GENERAL AND MISCELLANEOUS↗
Comparative Study of Large Language Model Architectures on Frontier
Large language models (LLMs) have garnered significant attention in both the AI community and beyond. Among these, the Generative Pre-trained Transformer (GPT) has emerged as the dominant architecture, spawning numerous variants. However, these variants have undergone pre-training under diverse conditions, including variations in input data, data preprocessing, and training methodologies, resulting in a lack of controlled comparative studies. Here we meticulously examine two prominent open-sourced GPT architectures, GPT-NeoX and LLaMA, leveraging the computational power of Frontier, the world’s first Exascale supercomputer. Employing the same materials science text corpus and a comprehensive end-to-end pipeline, we conduct a comparative analysis of their training and downstream performance. Our efforts culminate in achieving state-of-the-art performance on a challenging materials science benchmark. Furthermore, we investigate the computation and energy efficiency, and propose a computationally efficient method for architecture design. To our knowledge, these pre-trained models represent the largest available for materials science. Our findings provide practical guidance for building LLMs on HPC platforms.
Yin, Junqi↗
Multi-Functional Distributed Fiber Sensors for Pipeline Monitoring and Methane Detections. Final Report
As an abundant and cheap fossil energy source, natural gas has become a significant energy supply to support the United States’ economy. However, the large-scale extraction and utilization of natural gas also impose significant challenges on methane leakage. This problem is exacerbated by aging gas utility delivery systems, including interstate high-pressure pipelines, storage, and transmission facilities. This project aims to develop a cost-effective fiber optical sensing method that can perform multi-parameter real-time measurements of natural gas pipelines across long interrogation distances up to 100 km with 1-meter spatial resolution. This sensing tool can evaluate overall pipeline efficiency and reduce methane emissions for mid-stream methane infrastructures. To accomplish this objective, research and development efforts funded by this project have resulted in the following accomplishments: This project successfully has developed new functional sensory polymer materials using Metal-Organic Frameworks (MOFs) that can be coated on optical fiber through the reel-to-reel coating process. Functional polymer-coated optical fibers can perform sensitive methane detection through evanescence wave interaction and strain-based measurements to achieve 1% detection sensitivities. The new sensors fibers support both distributed measurements and multiplexed fiber sensors array for multi-point measurements. This project developed and optimized a new multi-core optical fiber that supports simultaneous and distributed measurements of strain and temperatures with 1-meter spatial resolutions across up to 100-km interrogation distance. This new fiber, combined with sensory polymercoated fiber, could perform both distributed temperature and methane detections. This project developed a new artificial intelligence big data algorithm approach that can effectively analyze high-resolution data harnessed by distributed fiber sensors to protect natural gas pipelines against external threats and detect internal defects induced by corrosion. Working with our industry partner, this project developed new optical fibers that support fiber sensor fabrications through polymer coating after the fibers are drawn. These new fibers eliminate the need for direct sensor fabrication when the fiber is fabricated on a fiber draw tower, which drastically expands fiber sensors' applicability. This research project has significantly advanced the distributed fiber sensing technology. It will dramatically increase the applicability and adaptability of distributed fiber sensors for a wide array of applications in energy, sustainability, and environmental science, including structural health monitoring of natural gas pipelines, oil infrastructures, hydrogen facilities, and environmental monitoring of carbon storage sites, water supply systems, and others.
03 NATURAL GAS↗
A high-throughput structural system biology approach to increase structure representation of proteins from Clostridioides difficile
ABSTRACT Clostridioides difficile causes life-threatening gastrointestinal infections. It is a high-risk pathogen due to a lack of effective treatments, antimicrobial resistance, and a poorly conserved genomic core. Herein, we report 30 X-ray structures from a structure genomics pipeline spanning 13 years, representing 10.2% of the X-ray structures for this important pathogen.
59 BASIC BIOLOGICAL SCIENCES↗
Astro-H Data Analysis, Processing and Archive
Astro-H (Hitomi) is an X-ray Gamma-ray mission led by Japan with international participation, launched on February 17, 2016. The payload consists of four different instruments (SXS, SXI, HXI and SGD) that operate simultaneously to cover the energy range from 0.3 keV up to 600 keV. This paper presents the analysis software and the data processing pipeline created to calibrate and analyze the Hitomi science data along with the plan for the archive and user support.These activities have been a collaborative effort shared between scientists and software engineers working in several institutes in Japan and USA.
Archive↗
Reanalysis of Rat Data from Spacelab Life Sciences 2 (SLS-2) to Reveal Research Gaps in Spaceflight Data
Using and analyzing the legacy data obtained in space life sciences missions has the potential to provide researchers a complete picture of the molecular changes associated with space without further experimentation. This project’s objective is to extract, filter, organize, and analyze all Rattus norvegicus data and metadata obtained from Columbia’s Spacelab Life Sciences 2 (SLS-2, STS-58) mission to explore the ways that we can compile information from model organisms, in our case rats, to create a reliable model to understand biological mechanisms in response to these space flight changes. By reusing rare space legacy data coupled with data analysis techniques, we can combine individual preexisting datasets with current ones to gain new, comprehensive insights about the effects of spaceflight on our bodies. Our methods can also lead to the creation of a standardized pipeline that could be applied to other space life science datasets for analysis. In this review, every biological experiment conducted on rats in the SLS-2 Mission was studied with our pipeline to create a new biological library and model that could be used by scientists from around the world to make novel discoveries and develop new hypotheses from this priceless information without the limitation of the costs of spaceflight experimentation.
rats↗