DevOps: Images Scripts APIs Oh my!.
Abstract not provided.
Klein, Brandon Thorin (ORCID:0000000295031904)↗
SEARCH · Search NASA
Results for “script”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Data From: "Warming and snow loss increase reliance on old groundwater in a Colorado River headwater"
This repository contains the data and code associated with the paper titled "Warming and snow loss increase reliance on old groundwater in a Colorado River headwater," published in Nature Geoscience, 2026. This study seeks to answer how various ages of groundwater interact with mountainous streamflow in mountainous headwaters such as the East River. It includes various model-data processing scripts, primarily for ParFlow-CLM analysis of simulated water years 2015-2021, and two numerical warming experiments (+2.5 and +4.0 degrees C), including run scripts, forcing scripts, and post-processing, as well as comparison to observation datasets, detailed below. This data requires the use of R (.r, .rmd), Python (.py), Jupyter Notebook or Jupyter Lab (.ipynb), ParFLOW-CLM, EcoSLIM. Further information on the use of all file formats mentioned below (e.g. .tff. .nc) are provided within the associated scripts and directory where the files are located. Contents & Usage ASO/: Contains the bash and python scripts used to convert airborne snow observatory (ASO) data (ASO, 2023) in various data formats (georeferenced tiff file, NetCDF, UTM, and to latitude/longitude) then regrided to the ParFlow equivalent grid. Output data are in regrid_regll_data.zip and subsequently visualized and analyzed in plot_and_compare.py for Supplementary Figures A14 and A15. The wksht_ASO_comparison.xlsx spreadsheet is used to calculate the data for Supplementary Figure A16. EcoSLIM/: Contains the scripts and input files to run the EcoSLIM particle tracking simulations (/run_scripts) and the post-processing python script (/plot_scripts/eco_agedist_plots.ipynb). Jasechko et al./: Contains the jupyter notebook (Extract_Elevation.ipynb) to determine the outlet elevations of the 260 watersheds used in Jasechko et al. (2016), and the corresponding table, Table_S1_Watersheds_alt.csv. Used to create Supplementary Information Figure A2. PLM_Wells/: Contains the QA/QC-ed groundwater level time series of the PLM-1 and PLM-6 Monitoring Wells from Faybishenko et al. (2023), reformatted to water years used for Supplementary Figures A19 and and A20. ParFlow/: Contains the input files and run scripts to run ParFlow-CLM (/run_scripts), the python and tool command language (Tcl) scripts to create and distribute the ParFlow forcing simulation files (/forcing), and various scripts and intermediary files to analyze the model outputs (/post_process). SQUIRE/: Contains the processing scripts and intermediary files for the Surface QUantitatIve pRecipitation Estimation (SQUIRE) data (Grover, 2023) used to generate Supplementary Figure A18. USGS_Streamflow/: Contains the raw and gap-filled United States Geological Survey streamflow data (U.S. Geological Survey, 2026) used at the Almont station (site number 09112500). Gap-filling is performed in the R script with data from the Taylor station (site number 09110000). (/USGS_09112500_EAST_RIVER_AT_ALMONT_GAP_FILLED/code_almont_streamflow_gap_fill.Rmd). discharge/: Contains the gap-filled discharge data at the Watershed Function SFA East River pumphouse site (Newcomer et al., 2022) used to generate Supplementary Figure A13 and to compute hourly Nash-Sutcliffe model efficiency coefficients (NSE) in Table A4. snotel_and_flux_tower/: Contains the snow telemetry data (U.S. Department of Agriculture, 2024) from the Butte (site ID 380) and Schofield (site ID 737) stations, reformatted by water year, accessed with the snotelr R package. Used to create Supplementary Figure A17. Also contains the flux tower observational data (FluxTower_Pumphouse_ESS-DIVE.ET_only.h.txt) from Ryken et al. (2022) and sap flux transpiration data (MaxB_Transpiration_5Sites.daily_sums.h.txt) from Ryken (2021), used to create Supplementary Figures A22 and A23, respectively. Raw EcoSLIM model outputs are in excess of 24TB, and are stored on National Energy Research Scientific Computing Center (NERSC) and publicly available via the external link provided in the paper.
atmospheric warming↗
Automation of the CFD Process on Distributed Computing Systems
A script system was developed to automate and streamline portions of the CFD process. The system was designed to facilitate the use of CFD flow solvers on supercomputer and workstation platforms within a parametric design event. Integrating solver pre- and postprocessing phases, the fully automated ADTT script system marshalled the required input data, submitted the jobs to available computational resources, and processed the resulting output data. A number of codes were incorporated into the script system, which itself was part of a larger integrated design environment software package. The IDE and scripts were used in a design event involving a wind tunnel test. This experience highlighted the need for efficient data and resource management in all parts of the CFD process. To facilitate the use of CFD methods to perform parametric design studies, the script system was developed using UNIX shell and Perl languages. The goal of the work was to minimize the user interaction required to generate the data necessary to fill a parametric design space. The scripts wrote out the required input files for the user-specified flow solver, transferred all necessary input files to the computational resource, submitted and tracked the jobs using the resource queuing structure, and retrieved and post-processed the resulting dataset. For computational resources that did not run queueing software, the script system established its own simple first-in-first-out queueing structure to manage the workload. A variety of flow solvers were incorporated in the script system, including INS2D, PMARC, TIGER and GASP. Adapting the script system to a new flow solver was made easier through the use of object-oriented programming methods. The script system was incorporated into an ADTT integrated design environment and evaluated as part of a wind tunnel experiment. The system successfully generated the data required to fill the desired parametric design space. This stressed the computational resources required to compute and store the information. The scripts were continually modified to improve the utilization of the computational resources and reduce the likelihood of data loss due to failures. An ad-hoc file server was created to manage the large amount of data being generated as part of the design event. Files were stored and retrieved as needed to create new jobs and analyze the results. Additional information is contained in the original.
Tejnil, Ed↗
TURBOMAT-FR: Turbomachinery Aeroelastic Forced Response Analysis Automation Using MATLAB® Part 2
The primary aim of this effort is to develop a software tool named “Turbomachinery Aeroelastic Analysis Tool for Forced Response (TURBOMAT-FR)” using MATLAB® (The MathWorks, Inc.) scripts. The tool will aid in performing routine forced response analysis of turbomachinery fan blades with the aid of the modal summation method (MSUM). The MSUM approach requires a modal solution consisting of modal displacements, frequencies, and stresses. The MATLAB® scripts facilitate easy extraction of the required quantities from the computational fluid dynamics (CFD) and MSC Nastran (Hexagon AB) solution files. These quantities are subsequently utilized by the MSUM approach to perform a complete forced response analysis using the MATLAB® scripts. The MATLAB® scripts in TURBOMAT-FR are utilized in postprocessing of the MSC Nastran finite element analysis outputs in three types of solutions or steps, namely (1) stresses and displacements from static or steady response solution, (2) unsteady stresses and displacements from direct forced response solution, and (3) modal solution containing modal displacements and modal stresses. In step (4), separate MATLAB® scripts are also developed to read the processed outputs in step (3) and perform forced response analysis using MSUM. It should be noted that the direct forced response solution from step (2) will provide for a comparison with those obtained from the MSUM approach in steps (3) and (4). The TURBOMAT-FR MATLAB® scripts mentioned previously are presented in two reports designated Parts 1 and 2. Part 1 presents the scripts needed to accomplish steps (1) and (2). The current report, Part 2, presents the scripts needed to accomplish steps (3) and (4), noted previously. The TURBOMAT-FR MATLAB® scripts provided here in Part 2 facilitate the calculation of unsteady stresses using MSUM and provide an automated procedure associated with aeroelastic forced response analysis with different pressure files. The unsteady stresses calculated using MSUM can be compared with those presented in Part 1. A cantilevered blade-alone model subjected to a realistic pressure loading to a boundary was used for static and unsteady direct forced response analysis to demonstrate the effectiveness of the TURBOMATFR MATLAB® scripts.
T S R Reddy↗
TURBOMAT-FR: Turbomachinery Aeroelastic Forced Response Analysis Automation Using MATLAB®: PART 1
The primary aim of this effort is to develop a software tool named “Turbomachinery Aeroelastic Analysis Tool for Forced Response (TURBOMAT-FR)” using MATLAB® (The MathWorks, Inc.) scripts. The tool will aid in performing routine forced response analysis of turbomachinery fan blades with the aid of the modal summation method (MSUM). The MSUM approach requires a modal solution consisting of modal displacements, modal frequencies, and modal stresses. The MATLAB® scripts facilitate easy extraction of the required quantities from the computational fluid dynamics (CFD) and MSC Nastran (Hexagon AB) solution files. These quantities are subsequently utilized by the MSUM approach to perform a complete forced response analysis.The MATLAB® scripts in TURBOMAT-FR are utilized in postprocessing of the MSC Nastran finite element analysis outputs in three types of solutions or steps, namely (1) stresses and displacements from static and steady response solution, (2) unsteady stresses and displacements from direct forced response solution, and (3) modal solution containing modal displacements and modal stresses. In step (4), separate MATLAB® scripts are also developed to read the processed outputs in step (3) and perform forced response analysis using MSUM. The TURBOMAT-FR MATLAB® scripts mentioned previously are presented in two reports designated “Part 1” and “Part 2”. The current report, Part 1, presents the scripts needed to accomplish steps (1) and (2), noted previously. This will be exclusively useful for performing only the static and steady response analysis and the forced response analysis using the direct forced response solution approach. Part 2 contains the TURBOMAT-FR MATLAB® scripts needed to accomplish steps (3) and (4). The MATLAB® scripts given in Part 2 facilitate the calculation of unsteady stresses using MSUM and provide an automated procedure associated with aeroelastic forced response analysis with different pressure files for the same structural model. The unsteady stresses calculated using MSUM in Part 2 can be compared with those presented in Part 1. A cantilevered blade-alone model subjected to a realistic pressure loading to a boundary was used for static and unsteady direct forced response analysis to demonstrate the effectiveness of the TURBOMATFR MATLAB® scripts.
MATLAB-Scripts↗
A computerized, procedural aid for flight crews
A computer program for detecting pilot error is described. The program observes pilot actions through the aircraft controls and state, and the actions are compared to those of a procedural script which can be considered a prescriptive model of the procedural aspects of flight. Pilot error is a discrepancy between the pilot actions and the script. The script has a hierarchical structure consisting of a single script at the highest level, scripts for the major phases of flight at the next level, scripts for procedures to be executed during these phases, and procedure step scripts at the bottom. The program interprets the aircraft data base in terms of the script, detecting errors of commission from a set of allowed and disallowed changes stored with each script element.
Hammer, J. M.↗
cabbi-bio/ESM-R-script
This script is intended to provide a starting point for users to implement the equivalent soil mass (ESM) approach for calculating soil organic carbon (SOC) stocks and SOC mass percentages (commonly called “SOC concentration”). The user should read this readme file and understand how the script works before using it. The script contains some basic error checking algorithms, but it cannot catch all data entry mistakes. It is possible to misuse the script and produce invalid results. The script was designed to work on individual soil cores that have been divided into three or more contiguous segments, as the cubic spline function contains four coefficients. The script has not been tested on soil cores containing fewer than three contiguous segments. If only one segment is present, interpolation will necessarily be linear rather than cubic, which is expected to result in higher predictive error (Wendt & Hauser, 2013). Considering the wide range of potential applications, the script should not be expected to function correctly in all situations. The user is encouraged to modify or rewrite the script to meet their specific needs.
von Haden, Adam↗
Urban morphology and urban water demand evolution in the Los Angeles region
Detailed description of the dataset sources used in this study, the experimental workflow, and plotting for the paper figures provided at the associated GitHub Meta Repo: https://github.com/IMMM-SFA/Ferencz_et_al_2024_ERL The future water demand projections from this study are hypothetical future water demands that reflect the population and urban land cover changes represented by the scenarios considered. The intent and emphasis of this work is investigating the interactions between population change, evolution of urban morphology, and water demand. These projections are not meant to be likely future demands for specific water providers or the LA region and should not be interpreted as such. The folders contain input and output data for each step of the "Recreate my Experiment" workflow described in the associated GitHub meta-repository as well as data used for plotting Figures for the paper that this dataset supports. Description of each folder's contents and use: Step_1a: All necessary inputs to the associated python script provided on the GitHub repo. Step_1b: All necessary inputs (downscaled population rasters) used by the associated python script provided on the GitHub repo. Original 1-km squared rasters that were downscaled also provided. Step_1c: Urban growth projection rasters corresponding to SSP3 and SSP5 population scenarios are provided in separate subfolders as well as the water provider boundaries used for analysis. Outputs of data processing also provided. Associated python script provided on GitHub. Step_1d: Description of Inputs used by the QGIS Model Builder GUI that automates geospatial processing and clipping the of the high resolution land cover data for each urban land class footprint within a defined polygon boundary. The Model Builder is provided on the GitHub repo and can be used by QGIS. The outputs of this step are in "Clipped Provider Hi Res Landcover". If the user wants to use The Model Builder for different regions of LA or two test our outputs, they will need to download the hi resolution landcover raster listed in the Readme and in Ref [2] of the GitHub Page. Step_1e: All necessary inputs to generate average monthly demand for each water provider. Associated python script on GitHub. Step 2: Output data about land cover metrics (areas and fractions) for each urban land class for each water provider. Associated python script on GitHub. Uses outputs from Step 1d "Clipped Provider Hi Res Landcover" Step 3: Inputs for and Outputs from the urban projection raster analysis Python script on GitHub. The outputs are rasters of urban pixels that were converted to a higher land class and the number of land class units that changed (Values of 1, 2, or 3). For example, a value of 2 could be LC 21 -> 23 or LC 22 -> 24. These maps are label "intensification." The other outputs are "urban growth" rasters showing the conversion of non urban to urban land, which are indicated by pixel values of 1. Step 4: Output projections of indoor and outdoor annual and monthly demands for each water provider. These are used for Figures 4 - 7 of the paper. Figures: This folder has data used for plotting Figures 1 through 5. Data for Figures 6 and 7 are sourced directly from folders associated with the Processing and Analysis Steps 1 - 4 and the plotting scripts for Figures 6 and 7 are commented with what folder paths are needed to generate the figures. The GitHub page provides descriptions of how each figure was made and the associated plotting scripts used.
Los Angeles↗
Snorri Version 1.0
The software decomposes n-component time-series data into persistence vectors and provide visualizations of various summaries of this information. Persistence vectors are a type of vector, based on multidimensional persistent homology theory, which have magnitudes and directions that reflect dominant trends in the multi-component data. The software is a collection of MATLAB scripts, and thus MATLAB runs the scripts. The master script that calls other scripts is called Snorri. The purpose is thus analysis of time-series data to extract dominant trends or events and to classify the type of event. The software produces characteristic vectors (an average of the persistence vectors), which can have directions that reflect the overall dominant trend in a time-series data set. The MATLAB scripts have been partially converted to the PYTHON language and no do not have the full functionality of the MATLAB and are not fully tested; the python scripts are to be covered under this copyright assertion. Two versions of the PYTHON scripts are included, called: mph25.py and mph38.py, each with their own main.py file.
Callor, Nickolas↗
ROI Hide and Seek Protocol v1
1. Segmentation We provide scripts for the model definition of the U-net architecture adapted from: https://github.com/jvanvugt/pytorch-unet/blob/master/unet.py We developed scripts for preparing the lung segmentation data set. We developed scripts for training the U-Net architecture. We developed scripts for applying the trained U-Net model to perform the ROI Hide and Seek protocol on the classification dataset to create the modified dataset. 2. Classification We provide scripts for the training of the COVID-Net models provided by Linda Wang, this code is adapted from her github repository: https://github.com/lindawangg/COVID-Net/tree/d7b36831d854f57de5bc7557217f5439e86e016f. These scripts were modified to save training log information as well as to load the provided models in their github repo. We developed scripts for training standard Neural Network Models (resnet 50, vgg 11, Alexnet) on the COVID datsets along with the ROI Hide and Seek altered datasets.
Sadre, Robbie↗
Urban morphology and urban water demand evolution in the Los Angeles region
Detailed description of the dataset sources used in this study, the experimental workflow, and plotting for the paper figures provided at the associated GitHub Meta Repo: https://github.com/IMMM-SFA/Ferencz_et_al_2024_ERL The future water demand projections from this study are hypothetical future water demands that reflect the population and urban land cover changes represented by the scenarios considered. The intent and emphasis of this work is investigating the interactions between population change, evolution of urban morphology, and water demand. These projections are not meant to be likely future demands for specific water providers or the LA region and should not be interpreted as such. The folders contain input and output data for each step of the "Recreate my Experiment" workflow described in the associated GitHub meta-repository as well as data used for plotting Figures for the paper that this dataset supports. Description of each folder's contents and use: Step_1a: Inputs to the associated python script provided on the GitHub repo. Step_1b: Inputs (downscaled population rasters) used by the associated python script provided on the GitHub repo. Original 1-km squared rasters that were downscaled also provided. Step_1c: Urban growth projection rasters corresponding to SSP3 and SSP5 population scenarios are provided in separate subfolders as well as the water provider boundaries used for analysis. Outputs of data processing also provided. Associated python script provided on GitHub. Step_1d: Description of Inputs used by the QGIS Model Builder GUI that automates geospatial processing and clipping the of the high-resolution 60 cm land cover data for each urban land class footprint within a defined polygon boundary. The Model Builder is provided on the GitHub repo and can be used by QGIS. The outputs of this step are in "Clipped Provider Hi Res Landcover". If the user wants to use The Model Builder for different regions of LA or to test our outputs, they will need to download the hi resolution landcover raster listed in the Readme and in Ref [2] of the GitHub Page. Step_1e: All necessary inputs to generate average monthly demand over the 2017-2021 period and the minimum and maximum demands over the 2014-2021 for each water provider. Associated python scripts are on GitHub. Step 2: Output data about land cover metrics (areas and fractions) for each urban land class for each water provider. Associated python script on GitHub. Uses outputs from Step 1d "Clipped Provider Hi Res Landcover" Step 3: Both the Inputs for and Outputs from the urban projection raster analysis Python script on GitHub. The inputs are urban land class rasters for specific SSP and zoning scenarios (low, medium, high) from Step 1c. The outputs are rasters of urban pixels that were converted to a higher land class and the number of land class units that changed (Values of 1, 2, or 3). For example, a value of 2 could be LC 21 -> 23 or LC 22 -> 24. These maps are label "intensification." The other outputs are "urban growth" rasters showing the conversion of non urban to urban land, which are indicated by pixel values of 1. These are used for the urban growth change maps in Figure 3. Step 4: Output projections of indoor and outdoor annual and monthly demands for each water provider for the average, minimum, and maximum monthly demand scenarios for each of the four urban growth scenarios (SSP3 med, SSP5 low, SSP5 med, and SSP5 high). The outputs also include metrics on each water provider used for the demand sensitivity analysis presented in Figure 8. Outputs from Step 4 are used for Figures 4 - 8 of the paper. Figures: This folder has data used for plotting Figures 1 through 5, and 8. Data for Figures 6 and 7 are sourced directly from folders associated with the Processing and Analysis Steps 1 - 4. The GitHub meta repository provides descriptions of how each figure was made and the associated plotting scripts used.
Los Angeles↗
SCDU Testbed Automated In-Situ Alignment, Data Acquisition and Analysis
In the course of fulfilling its mandate, the Spectral Calibration Development Unit (SCDU) testbed for SIM-Lite produces copious amounts of raw data. To effectively spend time attempting to understand the science driving the data, the team devised computerized automations to limit the time spent bringing the testbed to a healthy state and commanding it, and instead focus on analyzing the processed results. We developed a multi-layered scripting language that emphasized the scientific experiments we conducted, which drastically shortened our experiment scripts, improved their readability, and all-but-eliminated testbed operator errors. In addition to scientific experiment functions, we also developed a set of automated alignments that bring the testbed up to a well-aligned state with little more than the push of a button. These scripts were written in the scripting language, and in Matlab via an interface library, allowing all members of the team to augment the existing scripting language with complex analysis scripts. To keep track of these results, we created an easily-parseable state log in which we logged both the state of the testbed and relevant metadata. Finally, we designed a distributed processing system that allowed us to farm lengthy analyses to a collection of client computers which reported their results in a central log. Since these logs were parseable, we wrote query scripts that gave us an effortless way to compare results collected under different conditions. This paper serves as a case-study, detailing the motivating requirements for the decisions we made and explaining the implementation process.
Automation↗
VeraRun
VeraRun provides users a command-line utility that determines the sequence of VERA codes and utilities to be executed, calculates and determines PBS directives and job parameters, creates a job script, and submits the script on behalf of the user. While hiding all the details, the script allows the user to set PBS parameters explicitly. VeraRun is implemented as a Python package with host-specific configurations and job script templates. Host configurations are stored in JavaScript Object Notation (JSON) files and specify machine characteristics (e.g., number of nodes, number of processors per node, memory available per node, location of VERA installations). Job templates are Bash scripts with parameters substituting for host-specific PBS directives and controls. These files are named by the hostname, thus making VeraRun extensible to other systems and environments. VeraRun reads the VERA input file to determine code and execution environment requirements, creating and optionally submitting a job script.
Lee, Ronaldw↗
ILI Code
This R code consists of three scripts, described below. 1. ILI_LHC_sample.R The first script generate 10,000 samples from the total parameter space formed by the ranges of the following five parameters inputted into the ILI model: transmission rate (beta), per capita rate of progress from exposed to infectious state (gamma_1), per capita rate of progress through initial infectious state (gamma_2), per capita rate of progress through hospitalized state (gamma_3), and per capita rate of progress through non-hospitalized infectious state (gamma_4). The output is LHS_samples.csv, for input into the global sensitivity script, ILI_SEIR. 2. ILI_SEIR.R The second script solves a standard Susceptible-Exposed-Infected- Recovered (SEIR) model (a system of Ordinary Differential Equations (ODEs)), runs global sensitivity analyses, and generates plots showing the impact of the input variables on the response variables for each virus. 3. virus_sims.R The third script solves a standard Susceptible-Exposed-Infected- Recovered (SEIR) model (a system of Ordinary Differential Equations (ODEs)), aimed at modeling outbreaks for five common upper respiratory viruses within a contained population. The output of this code is an epidemic curve to show timing and magnitude of the epidemic peak for each virus. Note: for all three scripts, the parameter values must be hard-coded by the user or the output will not be accurate.
Spencer, Julie↗
CMLM (Co-Optimized Machine-Learned Manifolds) [SWR-23-41]
Co-optimized Machine-Learned Manifolds (CMLM) is a data-driven approach for developing reduced-order manifold models for high-dimensional chemically reacting systems. It involves a specially designed neural network, the training of which simultaneously optimizes linear combinations of species that define the manifold, nonlinear mapping to outputs of interest such as reaction rates, and (optionally) subfilter closure for large eddy simulation. This software package provides an implementation of the CMLM approach in Python using the PyTorch machine learning library. A few example cases are included, showing how the tool can be applied to different types of data from 0D and 1D reacting simulations performed using Cantera. The neural networks can be saved in a format that is readable by the Pele suite of combustion solvers for use in reacting computational fluid dynamics simulations. This software repository contains several python scripts to perform various tasks associated with the Co-optimized Machine Learned Manifolds (CMLM) model, which is described in Perry, Henry de Frahan, and Yellapantula, CNF, 2022 (https://doi.org/10.1016/j.combustflame.2022.112286). This includes not only the code that defines the CMLM model, but also scripts to generate suitable training data, scripts to pre-process the data, scripts to train the CMLM model, and scripts to plot the output, as well as various other helper files. The scripts depend on several commonly used python libraries for data analysis and chemical reaction computations. The trained models that result from this tool are designed to work with the an interface being implemented in the Pele suite of reacting flow solvers (https://github.com/AMReX-Combustion).
Perry, Bruce↗
Data for "Quantifying the Propagation of Parametric Uncertainty on Flux Balance Analysis"
In the repository are example scripts that perform uncertainty injection and propagation to flux balance analysis with outputs for a small sample size (for demonstration purpose only). For proper analysis, user should download the scripts and run for a large sample size (e.g., 10,000 samples). If you use the scripts, please cite the following Metabolic Engineering article: “Quantifying the propagation of parametric uncertainty on flux balance analysis” (https://doi.org/10.1016/j.ymben.2021.10.012) There are two subdirectories: /uncFBA/uncBiom: injection of normally distributed noise to biomass precursor coeffcients and ATP maintenance (growth-associated ATP maintenance (GAM) and non-growth associated ATP maintenance (NGAM)) /uncFBA/uncRHS: departure from steady-state by adding noise drawn from normal distribution to the RHS terms of mass balance constraints
Metabolomics↗
Field and Model Data Associated with the Manuscript “Drivers of Streamflow Intermittency in Humid Regions: 1. Evaluating Above- and Below-ground Controls of Flow Persistence in a Forested Catchment”
This package contains field data, modeling files, and scripts supporting the investigation of the drivers of streamflow intermittency in a forested catchment. It includes the field data collected from electrical resistivity tomography (ERT) surveys, ground penetrating radar (GPR), continuous self-potential (SP) monitoring, electromagnetic (EM) imaging, groundwater and stilling well. In addition, it contains the data and results of the coupled water- and electrical-flow model developed using the COMSOL Multiphysics and Advanced Terrestrial Simulator (ATS), as well as software files and Jupyter notebooks used to process the data and generate figures in the manuscript submitted for peer review. The data archive is organized in the following directories: 1) Climate Includes hourly precipitation and daily evapotranspiration time series (2024 – 2025) provided as CSV files, alongside a text file detailing dataset units. 2) Coupled_model Contains two subfolders: Synthetic and Field_Application subfolder. Synthetic subfolder contains the ATS XML input script (can be opened using any code editor) for the four synthetic hydrological cases tested (Connected and gaining, Connected and losing, Disconnected and losing, and dry stream). It also includes other experimental cases to test the influence of precipitation and concentration gradient. For each synthetic case, the flow model simulation is executed using the ATS XML scripts and the included Python script (generate_data_set.py) to convert ATS output to COMSOL-ready input. COMSOL Multiphysics template (.mph can be opened with the commercial software COMSOL and requires a license) is executed using the ATS output data to simulate the potential field. It also includes the Synthetic_model_plot.ipynb (can be opened using any code editor) to visualize the SP result and generate manuscript figures. The data subfolder contains mesh files to run both the ATS (.exo and .stl files can be viewed using Paraview; .h5 files can be opened using HDFView software and h5py Python package) and COMSOL models. Field_Application subfolder contains two subfolders: ES_MDA_inversion and Final_Model. ES_MDA_inversion contains the Python script (.py can be opened using any code editor) and SP observation data used to run the Ensemble Smoother with Multiple Data Assimilation (ES-MDA) inversion sequence to get the optimal model parameters. The Final_model subfolder contains the ATS XML input scripts, data files, output data for the two SP sites. The same workflow steps outlined for the Synthetic subfolder apply here. It also contains the Jupyter notebook (Plot_final_calib.ipynb) to visualize the results of the modeled SP, stream-groundwater exchange and moisture content. 3) Discharge Includes the electrical conductivity (EC) time series (provided as CSV files) from salt slug injections. It also includes the Jupyter notebook (Discharge_process.ipynyb) used to estimate discharge. All discharge measurements collated into rating_curve_processed.csv 4) EM Contains the CSV file of the EM data from the DUALEM-42, including spatial coordinates (x, y, z), apparent conductivity, and in-phase measurements at 2 m coil separations for horizontal coplanar (HCP) and perpendicular (PRP) geometries. 5) ERT Contains raw resistivity data (provided as CSV files), spatial location of each of the electrodes (provided as CSV files), and files used for the resistivity inversion (.resipy can be opened with the open-source ResIPy software). 6) GPR Includes GPR field datasets collected at 100 MHz and 250 MHz antenna frequencies, along with the processing/interpretation project file (GPR_process.gpz can be viewed using EKKO_Project 6, a commercial software by Sensors & Software that requires a license). 7) Slug_test Includes the slug test data at all the groundwater wells provided as CSV files, as well as the Jupyter notebook (Slug_test.ipynb) for calculating hydraulic conductivity. 8) SP Contains the SP data collected in field at the two SP sites (one in the perennial reach and the other in the intermittent reach), provided as DAT files. 9) Well_data Contains two subfolders: 1) Raw, which provides unprocessed pressure, electrical conductivity and temperature timeseries downloaded from the loggers in all the groundwater and stilling wells, and 2) Processed, which contains sorted, QA/QC timeseries data for each well. The data archive also contains data_process.ipynb, a Jupyter notebook used for field data analysis and generating figures (plotting well, SP, climate, and discharge data, as well as calculating head gradient at sites with nested groundwater wells). It also includes DTW.ipynb, a Jupyter notebook containing the code for the dynamic time warping (DTW) with sliding window to evaluate SP signal synchronicity.
ATS↗
Automatic Testcase Generation for Flight Software
The TacSat3 project is applying Integrated Systems Health Management (ISHM) technologies to an Air Force spacecraft for operational evaluation in space. The experiment will demonstrate the effectiveness and cost of ISHM and vehicle systems management (VSM) technologies through onboard operation for extended periods. We present two approaches to automatic testcase generation for ISHM: 1) A blackbox approach that views the system as a blackbox, and uses a grammar-based specification of the system's inputs to automatically generate *all* inputs that satisfy the specifications (up to prespecified limits); these inputs are then used to exercise the system. 2) A whitebox approach that performs analysis and testcase generation directly on a representation of the internal behaviour of the system under test. The enabling technologies for both these approaches are model checking and symbolic execution, as implemented in the Ames' Java PathFinder (JPF) tool suite. Model checking is an automated technique for software verification. Unlike simulation and testing which check only some of the system executions and therefore may miss errors, model checking exhaustively explores all possible executions. Symbolic execution evaluates programs with symbolic rather than concrete values and represents variable values as symbolic expressions. We are applying the blackbox approach to generating input scripts for the Spacecraft Command Language (SCL) from Interface and Control Systems. SCL is an embedded interpreter for controlling spacecraft systems. TacSat3 will be using SCL as the controller for its ISHM systems. We translated the SCL grammar into a program that outputs scripts conforming to the grammars. Running JPF on this program generates all legal input scripts up to a prespecified size. Script generation can also be targeted to specific parts of the grammar of interest to the developers. These scripts are then fed to the SCL Executive. ICS's in-house coverage tools will be run to measure code coverage. Because the scripts exercise all parts of the grammar, we expect them to provide high code coverage. This blackbox approach is suitable for systems for which we do not have access to the source code. We are applying whitebox test generation to the Spacecraft Health INference Engine (SHINE) that is part of the ISHM system. In TacSat3, SHINE will execute an on-board knowledge base for fault detection and diagnosis. SHINE converts its knowledge base into optimized C code which runs onboard TacSat3. SHINE can translate its rules into an intermediate representation (Java) suitable for analysis with JPF. JPF will analyze SHINE's Java output using symbolic execution, producing testcases that can provide either complete or directed coverage of the code. Automatically generated test suites can provide full code coverage and be quickly regenerated when code changes. Because our tools analyze executable code, they fully cover the delivered code, not just models of the code. This approach also provides a way to generate tests that exercise specific sections of code under specific preconditions. This capability gives us more focused testing of specific sections of code.
Bushnell, David Henry↗