Search NASA⌕ Search

SEARCH · Search NASA

Results for “pipeline data processing”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 289 records · Page 16

A bit-serial VLSI array processing chip for image processing

An array processing chip integrating 128 bit-serial processing elements (PEs) on a single die is discussed. Each PE has a 16-function logic unit, a single-bit adder, a 32-b variable-length shift register, and 1 kb of local RAM. Logic in each PE provides the capability to mask PEs individually. A modified grid interconnection scheme allows each PE to communicate with each of its eight nearest neighbors. A 32-b bus is used to transfer data to and from the array in a single cycle. Instruction execution is pipelined, enabling all instructions to be executed in a single cycle. The 1-micron CMOS design contains over 1.1 x 10 to the 6th transistors on an 11.0 x 11.7-mm die.

Heaton, Robert↗

Telecommunications issues of intelligent database management for ground processing systems in the EOS era

Future NASA earth science missions, including the Earth Observing System (EOS), will be generating vast amounts of data that must be processed and stored at various locations around the world. Here we present a stepwise-refinement of the intelligent database management (IDM) of the distributed active archive center (DAAC - one of seven regionally-located EOSDIS archive sites) architecture, to showcase the telecommunications issues involved. We develop this architecture into a general overall design. We show that the current evolution of protocols is sufficient to support IDM at Gbps rates over large distances. We also show that network design can accommodate a flexible data ingestion storage pipeline and a user extraction and visualization engine, without interference between the two.

Touch, Joseph D.↗

Time Series Analysis in the Search for Other Worlds Through Transit Photometry

The Kepler Mission launched in June 2009 to commence NASA's first mission to search for potentially habitable, Earth-size planets orbiting Sun-like stars. Kepler discovered explanets via the transit method: searching for minute (100 ppm) drops in brightness lasting 1 - 13 hours corresponding to occasions where the planet crosses the face of its host star from Kepler's point of view. The exquisite precision required to carry out the Kepler mission (20 ppm in 6.5 hours) pushed astronomical time series analysis to the limits, and motivated the development of novel algorithmic approaches. Transit signatures of rocky planets are often dwarfed by the intrinsic stellar variability, which is not white noise, and often is non-stationary, and by instrumental systematic effects, which can include transients and electronic artifacts. Surmounting this challenging regime of weak, temporally compact, periodic signals in observation noise with strong systematics and other sources of variability motivated the development of 1) an overcomplete, non-decimated, wavelet-based matched filter to jointly estimate the properties of the non-stationary, non-white observation noise process, and 2) a multi-scale, maximum a posteriori (msMAP) approach to identifying and removing instrumental systematic effects. After over nine years of observations, the Kepler spacecraft finally ran out of fuel in November 2018, ending its data collection activities. Over 2300 planets were discovered by Kepler in its primary mission, and over 355 have been discovered by K2, the repurposed mission that followed Kepler's primary mission after the loss of a second reaction wheel in May 2013. We have ported the Kepler science pipeline for the Transiting Exoplanet Survey Satellite (TESS) Mission, which began science observations in July 2019, and report initial results and performance of the modified science pipeline.The Kepler and TESS Missions are supported by NASA's Science Mission Directorate.

transit surveys↗

Web Exploration Tools for a Fast Federated Optical Survey Database

We implemented several new web-based tools to improve the efficiency and versatility of access to the APS Catalog of the POSS I (Palomar Observatory-National Geographic Sky Survey) and its associated image database. The most important addition was a federated database system to link the APS Catalog and image database into one Internet-accessible database. With the FDBS, the queries and transactions on the integrated database are performed as if it were a single database. We installed Myriad the FDBS developed by Professor Jaideep Srivastava and members of his group in the University of Minnesota Computer Science Department. It is the first system to provide schema integration, query processing and optimization, and transaction management capabilities in a single framework. The attached figure illustrates the Myriad architecture. The FDBS permits horizontal access to the data, not just vertical. For example, for the APS, queries can be made not only by sky position, but also by any parameter present in either of the databases. APS users will be able to produce an image of all the blue galaxies and stellar sources for comparison with x-ray source error ellipses from AXAF (X Ray Astrophysics Facility) (Chandra) for example. The FDBS is now available as a beta release with the appropriate query forms at our web site. While much of our time was occupied with adapting Myriad to the APS environment, we also made major changes in Star Base, our DBMS for the Catalog, at the web interface to improve its efficiency for issuing and processing queries. Star Base is now three times faster for large queries. Improvements were also made at the web end of the image database for faster access; although work still needs to be done to the image database itself for more efficient return with the FDBS. During the past few years, we made several improvements to the database pipeline that creates the individual plate databases queries by StarBase. The changes include improved positions especially for galaxies, using a new median centroider and integrated magnitudes for galaxies with an improved density-to-intensity calibration with a "sky" background subtraction. In the original version of StarBase the object classification fainter than 19.5-20.0 mag., was an extrapolation of the networks trained on brighter objects. We have used a new catalog of galaxies at the NGP to train a neural network on objects fainter than 20th mag. This improved classification is used in the new version of StarBase. We have also added a FITS table option for the returned data from queries on the object catalog. The APS image database includes images in both colors so we have added a tool for querying the image database in both colors simultaneously. The images can be displayed in parallel or blinked for comparison.

Humphreys, Roberta M.↗

Transiting Planet Search in the Kepler Pipeline

The Kepler Mission simultaneously measures the brightness of more than 160,000 stars every 29.4 minutes over a 3.5-year mission to search for transiting planets. Detecting transits is a signal-detection problem where the signal of interest is a periodic pulse train and the predominant noise source is non-white, non-stationary (1/f) type process of stellar variability. Many stars also exhibit coherent or quasi-coherent oscillations. The detection algorithm first identifies and removes strong oscillations followed by an adaptive, wavelet-based matched filter. We discuss how we obtain super-resolution detection statistics and the effectiveness of the algorithm for Kepler flight data.

Jenkins, Jon M.↗

Feed-forward volume rendering algorithm for moderately parallel MIMD machines

Algorithms for direct volume rendering on parallel and vector processors are investigated. Volumes are transformed efficiently on parallel processors by dividing the data into slices and beams of voxels. Equal sized sets of slices along one axis are distributed to processors. Parallelism is achieved at two levels. Because each slice can be transformed independently of others, processors transform their assigned slices with no communication, thus providing maximum possible parallelism at the first level. Within each slice, consecutive beams are incrementally transformed using coherency in the transformation computation. Also, coherency across slices can be exploited to further enhance performance. This coherency yields the second level of parallelism through the use of the vector processing or pipelining. Other ongoing efforts include investigations into image reconstruction techniques, load balancing strategies, and improving performance.

Yagel, Roni↗

Enabling and Enhancing Space Mission Success and Reduction of Risk through the Application of an Integrated Data Architecture

The engineering phases of design, development, test, and evaluation (DDT and E) and subsequent planning, preparation, and operation (Ops) of space vehicles in a complex and distributed environment requires massive and continuous flows of information across the enterprise and across temporal stages of the vehicle lifecycle. The resulting capabilities at each subsequent stage depend in part on the capture, preparation, storage, and subsequent provision of information from prior stages. The United States National Aeronautics and Space Administration (NASA) is currently designing a fleet of new vehicles that will replace the Space Shuttle and expand space operations and exploration capabilities. This includes the 2 stage human rated lift vehicle Ares 1 and its associated crew vehicle the Orion, and a service module; the heavy lift cargo vehicle, Ares 5, and an associated cargo stage known as the Earth Departure Stage; and a Lunar Lander vehicle that contains a descent stage, and ascent stage, and a habitation module. A variety of concurrent assorted ground operations infrastructure including software and facilities are also being developed, assorted technology and assembly designs and development for equipment such as EVA suits, life support systems, command and control technologies are also in the pipeline. The development is occurring in a distributed manner, with project deliverables being contributed by a large and diverse assortment of vendors and most space faring nations. Critical information about all of the components, software, and procedures must be shared during the DDT and E phases and then made readily available to the mission operations staff for access during the planning, preparation, and operations phases, and also need to be readily available for system to system interactions. The Constellation Data Systems Project (CxDS) is identifying the needs, and designing and deploying systems and processes to support these needs. This paper details the steps and processes that NASA is applying within the Constellation Program to manage this data and information, and to insure that the correct information is available, correctly annotated, and can be provisioned digitally to enhance response times, and support engineering analysis and anomaly resolution.

Brummett, Robert C.↗

New Techniques for High-Contrast Imaging with ADI: The ACORNS-ADI SEEDS Data Reduction Pipeline

We describe Algorithms for Calibration, Optimized Registration, and Nulling the Star in Angular Differential Imaging (ACORNS-ADI), a new, parallelized software package to reduce high-contrast imaging data, and its application to data from the Strategic Exploration of Exoplanets and Disks (SEEDS) survey. We implement seyeral new algorithms, includbg a method to centroid saturated images, a trimmed mean for combining an image sequence that reduces noise by up to approx 20%, and a robust and computationally fast method to compute the sensitivitv of a high-contrast obsen-ation everywhere on the field-of-view without introducing artificial sources. We also include a description of image processing steps to remove electronic artifacts specific to Hawaii2-RG detectors like the one used for SEEDS, and a detailed analysis of the Locally Optimized Combination of Images (LOCI) algorithm commonly used to reduce high-contrast imaging data. ACORNS-ADI is efficient and open-source, and includes several optional features which may improve performance on data from other instruments. ACORNS-ADI is freely available for download at www.github.com/t-brandt/acorns_-adi under a BSD license

Brandt, Timothy D.↗

MOC Automation with GMSEC and the Generic Extendable Message Utility (GEMU)

Automation has become critical for ground systems, improving efficiency and reliability while reducing costs across mission operations. The Goddard Mission Services Evolution Center (GMSEC) software suite has played a significant role in enabling this automation, leveraging its publish/subscribe paradigm through a message bus architecture to facilitate seamless communication and data flow. Historically, the GMSEC suite, through components like Criteria Action Table (CAT) has been pivotal in automating ground system capabilities. However, as technology advances, limitations in automation with CAT have emerged, creating an opportunity to enhance ground system automation through the introduction of GEMU. This new GMSEC component brings new capabilities and addresses specific automation constraints that CAT could not overcome, allowing for more sophisticated, flexible, and efficient message processing. GEMU, at its core, is designed to accelerate the development of custom GMSEC-compliant applications. It enables users to construct automated message processing pipelines quickly, supporting both drag-and-drop web-based configuration and scripting through a simple domain-specific language. This advancement not only simplifies the process but also reduces the time needed for implementing automated solutions. This presentation will outline GEMU’s potential value in improving mission operations automation. It will highlight the benefits of transitioning from CAT to GEMU and offer insights into how GEMU can drive operational efficiencies. We will also provide an overview of the automation capabilities of GEMU and its potential impact on mission operations centers (MOCs).

GMSEC↗

The Kepler End-to-End Data Pipeline: From Photons to Far Away Worlds

Launched by NASA on 6 March 2009, the Kepler Mission has been observing more than 100,000 targets in a single patch of sky between the constellations Cygnus and Lyra almost continuously for the last two years looking for planetary systems using the transit method. As of October 2011, the Kepler spacecraft has collected and returned to Earth just over 290 GB of data, identifying 1235 planet candidates with 25 of these candidates confirmed as planets via ground observation. Extracting the telltale signature of a planetary system from stellar photometry where valid signal transients can be small as a 40 ppm is a difficult and exacting task. The end-to end processing of determining planetary candidates from noisy, raw photometric measurements is discussed.

Cygnus constallation↗

Photometric Analysis in the Kepler Science Operations Center Pipeline

We describe the Photometric Analysis (PA) software component and its context in the Kepler Science Operations Center (SOC) Science Processing Pipeline. The primary tasks of this module are to compute the photometric flux and photocenters (centroids) for over 160,000 long cadence (~thirty minute) and 512 short cadence (~one minute) stellar targets from the calibrated pixels in their respective apertures. We discuss science algorithms for long and short cadence PA: cosmic ray cleaning; background estimation and removal; aperture photometry; and flux-weighted centroiding. We discuss the end-to-end propagation of uncertainties for the science algorithms. Finally, we present examples of photometric apertures, raw flux light curves, and centroid time series from Kepler flight data. PA light curves, centroid time series, and barycentric timestamp corrections are exported to the Multi-mission Archive at Space Telescope [Science Institute] (MAST) and are made available to the general public in accordance with the NASA/Kepler data release policy.

photocenter↗

MLtool: Universal Supervised Machine Learning Tool to Model Tabulated Data

Machine Learning (ML) is a subfield of Artificial Intelligence that gives computers the ability to learn from past data without being explicitly programmed. The predictive capabilities of ML models have already been used to facilitate several scientific breakthroughs. However, the practical application of ML is often limited due to the gaps in technical knowledge of its users. The common issue faced by many scientific researchers is the inability to choose the appropriate ML pipelines that are needed to treat real-world data, which is often sparse and noisy. To solve this problem, we have developed an automated Machine Learning tool (MLtool) that includes a set of ML algorithms and approaches to aid scientific researchers. The current version of MLtool is implemented as an object-oriented Python code that is easily extensible. It includes 44 different regression algorithms used to model data. MLtool helps users select the best model for their data, based on the scoring metrics used. Besides regression algorithms, MLtool also includes a suite of pre- and post-processing techniques such as missing value imputation, categorical variable encoding, input feature normalization, uncertainty quantification, exploratory data analysis (EDA), etc. MLtool was tested on several publicly available multi-dimensional data sets and was found capable of making accurate predictions.

Machine learning↗

MLtool Python Code

Machine Learning (ML) is a subfield of Artificial Intelligence that gives computers the ability to learn from past data without being explicitly programmed. The predictive capabilities of ML models have already been used to facilitate several scientific breakthroughs. However, the practical application of ML is often limited due to the gaps in technical knowledge of its users. The common issue faced by many scientific researchers is the inability to choose the appropriate ML pipelines that are needed to treat real-world data, which is often sparse and noisy. To solve this problem, we have developed an automated Machine Learning tool (MLtool) that includes a set of ML algorithms and approaches to aid scientific researchers. The current version of MLtool is implemented as an object-oriented Python code that is easily extensible. It includes 44 different regression algorithms used to model data. MLtool helps users select the best model for their data, based on the scoring metrics used. Besides regression algorithms, MLtool also includes a suite of pre- and post-processing techniques such as missing value imputation, categorical variable encoding, input feature normalization, uncertainty quantification, exploratory data analysis (EDA), etc. MLtool was tested on several publicly available multi-dimensional data sets and was found capable of making accurate predictions.

Machine Learning↗

NASA Tech Briefs, October 2012

Topics discussed include: Detection of Chemical Precursors of Explosives; Detecting Methane From Leaking Pipelines and as Greenhouse Gas in the Atmosphere; Onboard Sensor Data Qualification in Human-Rated Launch Vehicles; Rugged, Portable, Real-Time Optical Gaseous Analyzer for Hydrogen Fluoride; A Probabilistic Mass Estimation Algorithm for a Novel 7-Channel Capacitive Sample Verification Sensor; Low-Power Architecture for an Optical Life Gas Analyzer; Online Cable Tester and Rerouter; A Three-Frequency Feed for Millimeter-Wave Radiometry; Capacitance Probe Resonator for Multichannel Electrometer; Inverted Three-Junction Tandem Thermophotovoltaic Modules; Fabrication of Single Crystal MgO Capsules; Inflatable Hangar for Assembly of Large Structures in Space; Mars Aqueous Processing System; Hybrid Filter Membrane; Design for the Structure and the Mechanics of Moballs; Pressure Dome for High-Pressure Electrolyzer; Cascading Tesla Oscillating Flow Diode for Stirling Engine Gas Bearings; Compact, Low-Force, Low-Noise Linear Actuator; Ultra-Compact Motor Controller; Extreme Ionizing-Radiation-Resistant Bacterium; Wideband Single-Crystal Transducer for Bone Characterization; Fluorescence-Activated Cell Sorting of Live Versus Dead Bacterial Cells and Spores; Nonhazardous Urine Pretreatment Method; Laser-Ranging Transponders for Science Investigations of the Moon and Mars; Ka-Band Waveguide Three-Way Serial Combiner for MMIC Amplifiers; Structural Health Monitoring with Fiber Bragg Grating and Piezo Arrays; Low-Gain Circularly Polarized Antenna with Torus-Shaped Pattern; Stereo and IMU- Assisted Visual Odometry for Small Robots; Global Swath and Gridded Data Tiling; GOES-R: Satellite Insight; Aquarius iPhone Application; Monitoring of International Space Station Telemetry Using Shewhart Control Charts; Theory of a Traveling Wave Feed for a Planar Slot Array Antenna; Time Manager Software for a Flight Processor; Simulation of Oxygen Disintegration and Mixing With Hydrogen or Helium at Supercritical Pressure; A Superfluid Pulse Tube Refrigerator Without Moving Parts for Sub-Kelvin Cooling; Sapphire Viewports for a Venus Probe; The Mobile Chamber; Electric Propulsion Induced Secondary Mass Spectroscopy; and Radiation-Tolerant DC-DC Converters.

Source record↗

Using XML and Java Technologies for Astronomical Instrument Control

Traditionally, instrument command and control systems have been highly specialized, consisting mostly of custom code that is difficult to develop, maintain, and extend. Such solutions are initially very costly and are inflexible to subsequent engineering change requests, increasing software maintenance costs. Instrument description is too tightly coupled with details of implementation. NASA Goddard Space Flight Center, under the Instrument Remote Control (IRC) project, is developing a general and highly extensible framework that applies to any kind of instrument that can be controlled by a computer. The software architecture combines the platform independent processing capabilities of Java with the power of the Extensible Markup Language (XML), a human readable and machine understandable way to describe structured data. A key aspect of the object-oriented architecture is that the software is driven by an instrument description, written using the Instrument Markup Language (IML), a dialect of XML. IML is used to describe the command sets and command formats of the instrument, communication mechanisms, format of the data coming from the instrument, and characteristics of the graphical user interface to control and monitor the instrument. The IRC framework allows the users to define a data analysis pipeline which converts data coming out of the instrument. The data can be used in visualizations in order for the user to assess the data in real-time, if necessary. The data analysis pipeline algorithms can be supplied by the user in a variety of forms or programming languages. Although the current integration effort is targeted for the High-resolution Airborne Wideband Camera (HAWC) and the Submillimeter and Far Infrared Experiment (SAFIRE), first-light instruments of the Stratospheric Observatory for Infrared Astronomy (SOFIA), the framework is designed to be generic and extensible so that it can be applied to any instrument. Plans are underway to test the framework with other types of instruments, such as remote sensing earth science instruments.

Ames, Troy↗

The IPAC Image Subtraction and Discovery Pipeline for the Intermediate Palomar Transient Factory

We describe the near real-time transient-source discovery engine for the intermediate Palomar Transient Factory (iPTF), currently in operations at the Infrared Processing and Analysis Center (IPAC), Caltech. We coin this system the IPAC/iPTF Discovery Engine (or IDE). We review the algorithms used for PSF-matching, image subtraction, detection, photometry, and machine-learned (ML) vetting of extracted transient candidates. We also review the performance of our ML classifier. For a limiting signal-to-noise ratio of 4 in relatively unconfused regions, bogus candidates from processing artifacts and imperfect image subtractions outnumber real transients by approximately equal to 10:1. This can be considerably higher for image data with inaccurate astrometric and/or PSF-matching solutions. Despite this occasionally high contamination rate, the ML classifier is able to identify real transients with an efficiency (or completeness) of approximately equal to 97% for a maximum tolerable false-positive rate of 1% when classifying raw candidates. All subtraction-image metrics, source features, ML probability-based real-bogus scores, contextual metadata from other surveys, and possible associations with known Solar System objects are stored in a relational database for retrieval by the various science working groups. We review our efforts in mitigating false-positives and our experience in optimizing the overall system in response to the multitude of science projects underway with iPTF.

methods: analytical – methods: data analysis –↗

A Machine Learning Approach to Improve Air Traffic Management Initiatives

Collaborating closely with commercial air carriers and related organizations, the Federal Aviation Administration(FAA) regulates air traffic and ensures the safety and efficiency of air operations. Air traffic controllers make strategic decisions, such as delaying, rerouting, or canceling flights, partly based on guidance provided by the FAA’s Air TrafficControl System Command Center (ATCSCC). The guidance includes, among other things, control measures known asTraffic Management Initiatives (TMIs) designed to enhance safety and improve operational efficiency. TMIs play a crucial role in managing the demand and capacity within the U.S. National Airspace System (NAS). Two major TMIs that are routinely used (primarily to mitigate the adverse effects of bad weather) are Ground Delay Programs (GDPs) andGround Stops (GSs). In a GDP, flights destined for airports facing thunderstorm activity experience delays at their origin airports. This proactive approach minimizes the risk of routing aircraft through hazardous weather conditions and also replaces (fuel burning) airborne delays with ground delays. In a GS, a temporary restriction is imposed on the departure or arrival of aircraft at a specific airport or within a designated airspace. Although other TMIs (e.g., miles-in-trail) are also implemented as part of (air) traffic flow management in the NAS, the focus of this work is on GDPs and GSs. Since TMIs, by design, lead to flight delays or cancellations, it is crucial to put in place the right set of parameters(e.g., scope and duration of the GDP). For example, when the end time of a GDP extends beyond what is necessary, it imposes unnecessary delays on departing flights. This situation could occur as a result of inaccurate prediction of the(required) duration of the GDP based on the weather forecast. On the other hand, if a GDP ends prematurely before the underlying capacity constraints are resolved at the destination airport, it may result in airborne holding. The delicate balance lies in matching the termination of the GDP precisely with the resolution of capacity constraints, avoiding both the imposition of unnecessary ground delays and the need for airborne holding due to premature program termination.Failing to specify the right parameters for TMIs also leads to flight delays, creating a significant obstacle in managing the increasing traffic volumes causing increased work load for the controllers. To address this issue, we propose the integration of Machine Learning (ML) models in the traffic flow management(TFM) pipeline. In current operations, decisions are made by human experts based on extensive training, historical patterns, available traffic and weather data. Since we have an abundance of data from past events that tell us the likely impact of various TMIs, by ingesting historical data, properly trained ML models can offer valuable insights and aid human decision-making. With the FAA increasingly exploring advanced analytics, ML emerges as a focal point for enhancing TFM within the National Airspace System (NAS). As a first step, this study aims to provide traffic controllers with decision-making support for the issuance and adjustment of TMIs. Data analytics and machine learning have been previously employed to address some of the challenges associated with TMIs. Numerous studies have concentrated on various facets of TMI issuance, exploring factors influencing TMI parameters, including arrival rate, airport capacity, and delay prediction. For example, using weather forecasts, several statistical methods were used to produce probabilistic capacity profiles which in conjunction with deterministic models provided insights into the GDP planning process [1–4]. The downside of using deterministic models is that they rely on fixed inputs and predetermined rules, which lack the ability to account for the inherent uncertainty and variability present in real-world scenarios. In a separate series of studies, researchers aimed to predict the occurrences of GDPs and GSs. The majority of these studies utilized various supervised learning methods, including Decision Trees, Naive Bayes, Support VectorMachines, and Random Forests to analyze the influence of weather conditions and arrival demand on TMI incidents[5–8]. However, these studies primarily focused on predicting the incidence of TMIs without explicitly addressing the scope of TMIs, including their duration and their geographical coverage. Furthermore, the emphasis of these studies was largely on GDPs, given their higher frequency and longer duration when compared to GSs. A limited number of studies focused on predicting the parameters of TMIs, specifically addressing their duration and extent. In one such study focusing on optimizing the TMI parameters at San Francisco International Airport (SFO),the authors utilized a probabilistic forecast of fog [9]. They simulated various capacity scenarios based on the (fog)burn-off forecasts, selecting GDP parameters that minimized airborne and overall ground delays. However, this approach exclusively emphasizes stratus (fog) burn-off as the primary determinant of GDP and GS, neglecting other influential factors like severe weather events, runway closures, lower capacity than traffic demand, and other important variables. Given the complexity of predicting the TMI and determining its scope, we seek a more holistic approach. We aim to consider all significant factors that could impact TMIs and their parameters. What sets this research apart is the fusion of all data sources relevant to the issuance and adjustment of TMIs and it represents the first comprehensive attempt to optimize TMIs in this manner. Since this comprehensive solution involves various aspects, we break down the problem into smaller components and input all parameters into a unified model called the “TMI Adjuster”. Figure 1 shows the overall framework and the list of datasets used in each model. The objective of the TMI Adjuster module is to deliver reliable, consistent and expedited recommendations for the progression, adjustment, and termination of TMIs. The ML solution entails developing a pipeline capable of predicting the necessity of a TMI (e.g., GS or GDP) along with its various parameters. For example, in the case of a GS, this includes the scope of the GS either in terms of distance from the destination airport or based on pre-defined airspace sectors. Here, scope refers to those regions and departing airports that are subject to the GS. In this paper, we concentrate on the issuance of GSs in the three major airports in the New York area — LaGuardia(LGA), John F. Kennedy International (JFK), and Newark Liberty International (EWR). We fuse traffic, weather and other relevant aviation data from years 2017 to 2019 to train and validate the ML models. In particular, we use the following datasets: •Terminal Aerodrome Forecast (TAF): meteorological forecasts specific to each airport, issued four times a day, covering predefined time periods. •TMI data: includes all GSs and GDPs along with their respective parameters. •Aviation System Performance Metrics (ASPM): includes traffic related data such as aircraft delays, arrival, and departure rates. •Notices to Airmen (NOTAMs): utilized to extract runway closure data and manage interdependencies between terminals in close proximity. •Flight cancellation data •Airspace Flow Programs (AFP): includes information on flight airborne holdings caused by TMIs. The data preprocessing entails transforming ASPM, TMI, AFP, NOTAMs, and weather data into an hourly format and consolidating all datasets by merging them based on date and time as the primary key. The TMI Adjuster framework comprises two parallel models: one dedicated to GS and a second model focused on GDP. As previously mentioned, our specific focus is on the GS model as a multi-classification problem. In this framework, each data point of the GS model input summarizes ten hours of data. Specifically, the data loader for the GS model generates the input and output of the model as follows: at a given time step, the input includes the actual traffic, weather, and TMI data from the two-hour window before the time step, alongside the weather forecast and scheduled traffic for the next 8 hours starting from the time step. Based on this information, the output of the GS model for each time interval consists of three dimensions. The first dimension represents a binary decision on whether there should be a GS in place for the next hour or not. The second dimension is related to the scope of the GS in the United States, and the third dimension is related to the scope of the GS in Canada (i.e., to determine if the GS impacts airports in Canada).One of the challenges with TMI modeling is the sparsity of TMI events, particularly regarding its scope. To address this challenge in the scope of the GS model output, we implement grouping. The GS scope for the US region is defined based on a list of centers that should be included when the GS is in place. With 20 centers in the US, we utilized historical data to group them into 4 categories. In particular, we summarized our historical data in a graph format where nodes represent centers, and link weights are defined based on the co-occurrence of centers in the scope parameter ofTMIs. By identified strongly connected components in this graph, we were able to partition the centers into four groups. We consider two model structures for the GS Model. Firstly, a hierarchical classification model [10], where the human decision-making for a GS is of hierarchical nature. The decision-maker first decides whether there is a need fora GS, and if the answer is yes, determines the scope. A hierarchical classification model organizes the problem into a class hierarchy, typically a tree or a Directed Acyclic Graph (DAG) structure, and considers the dependency of the decision in the previous step to the next component [10]. Here, we employ the local classifier per level approach, which involves training one multi-class classifier for each level of the class hierarchy. The second structure is the independent structure. In this setting, as the name suggests, we do not consider the dependency of the decisions in the different dimensions of the output of the model. Instead, for each dimension, we train a multi-class classifier independently. Table 1 summarizes GS model statistics for training, validation and testing. The table documents the effect of limiting data to the time steps when there was actually a TMI in place or when a TMI had just terminated. This resulted in a more balanced distribution of the GS class(GS positive class)versus “No GS”(GS negative class), which might help the training process. While JFK and LGA follow very similar distributions, with 40% and 42% GS positive class respectively, EWR has proportionally fewer GS incidents at 28%. Our subsequent phase involves evaluating the performance of both hierarchical structure and independent structure using different state-of-the-art multi-class classifier models such as Random Forest, Decision Trees, K-nearest Neighbors, and Logistic Regression and forecast the duration and scope of the GSs.

Farzan Masrour Shalmani↗

Kepler Planet Detection Metrics: Robovetter Completeness and Effectiveness for Data Release 25

In general, the Kepler pipeline identifies a list of Threshold Crossing Events (TCEs), which are periodic flux decrements meeting certain criteria (Jenkins, 2017). These TCEs are reviewed and those that appear consistent with astrophysically transiting or eclipsing systems are classified as Kepler Objects of Interest (KOIs). Further review is given to KOIs, which are then dispositioned as Planet Candidates (PCs) or False Positive (FPs). FPs are further denoted by four major flags that indicate if the signal is Not Transit-Like (NTL), due to a Stellar Eclipse (SS; previously referred to as Significant Secondary), and/or due to contamination from a source other than the target as evidenced by a Centroid Offset (CO) oran Ephemeris Match (EM) with another object. This entire TCE review process is known as dispositioning or vetting.In the first five Kepler mission planet candidate catalogs (Borucki et al., 2011a,b; Batalha et al., 2013; Burke et al., 2014; Rowe et al., 2015), TCEs were manually examined on an individual basis and dispositioned using various plots and quantitative diagnostic tests (see e.g., Coughlin, 2017). In the sixth catalog, Mullally et al. (2015a) employed partial automation via simple parameter cuts to automatically disposition a large fraction of TCEs as not transit-like. Mullally et al. (2015a) also used an automated technique known as the centroid Robovetter (Mullally, 2017) to automatically identify some FP KOIs due to centroid offsets - a telltale signature of light contamination from another target. The remaining targets were manually dispositioned. In the seventh catalog, Coughlin et al. (2016) automated theentire dispositioning process using what is collectively known simply as the Robovetter.In the eighth and final mission catalog, Thompson et al. (2017) use a revised Robovetter to automate the dispositioning of all TCEs with an emphasis on creating a catalog suitable for accurately determining planet occurrence rates. In order to calculate accurate occurrence rates, the completeness and effectiveness of the Robovetter must be characterized. We define these terms as applied to the Robovetter, following Thompson et al. (2017), as:1. Completeness: The fraction of transiting planets detected by the pipeline that are classified as planet candidates by the Robovetter.2. Effectiveness: The fraction of false positives detected by the pipeline that are classified as false positives by the Robovetter.The remainder of this document describes products that can be used to quantitatively assess Robovetter completeness and effectiveness for an arbitrary set of Kepler stars.

Robovetter↗