SEARCH · Search NASA

Results for “Batch processing”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

617 records · Page 35

All sources NASA NTRS DOE OSTI

NASA NTRSDate not supplied

Evaluation of Correction Methods for NASA GeneLab Transcriptomic Datasets

Conducting space biology experiments aboard the International Space Station, particularly those utilizing complex model organisms like mice, is expensive and difficult due to limited crew availability, hardware, and space. As a result, sample numbers from these studies are low, reducing the statistical power of any one experiment. Aggregating spaceflight datasets serves as a method to increase sample numbers, allowing for novel insights through bioinformatic analysis of ‘omics data from merged datasets. However, aggregating datasets can introduce unwanted variation including 1) differences in sample handling, processing, and sequencing platforms between datasets (technical variation) as well as 2) differences in experimental design between datasets. In the present study, NASA GeneLab-hosted RNAseq datasets from mouse liver tissues were used to evaluate several statistical methods to correct for this unwanted variation through two approaches, reference-based and standard. The following correction algorithms were applied with (reference-based) and/or without (standard) considering Universal Mouse RNA Reference samples: ComBat and ComBat_seq from the SVA package, median polish, empirical Bayes, and ANOVA-based algorithms from the MBatch package, and negative binomial regression normalization in the DESeq2 package. For each approach, after the correction algorithm was applied, differential gene expression (DGE) analysis of flight and ground control samples was performed with the combined data. The robustness of each tool was evaluated using BatchQC to determine statistical differences between datasets before and after correction, Principal Component Analysis to evaluate global gene expression in samples before and after correction, and by comparing DGE analysis of individual datasets and combined datasets before and after correction. The results showed that the reference-based approach introduced several additional (and likely artificial) DEGs when compared with the respective standard approach. Of the methods tested, standard ComBat and DESeq2 were identified as the most robust correction methods for combining spaceflight mouse liver RNAseq datasets hosted on GeneLab.

GeneLab↗

NASA NTRSDate not supplied

Evaluation of Correction Methods for NASA GeneLab Transcriptomic Datasets

Conducting space biology experiments aboard the International Space Station, particularly those utilizing complex model organisms like mice, is expensive and difficult due to limited crew availability, hardware, and space. As a result, sample numbers from these studies are low, reducing the statistical power of any one experiment. Aggregating spaceflight datasets serves as a method to increase sample numbers, allowing for novel insights through bioinformatic analysis of ‘omics data from merged datasets. However, aggregating datasets can introduce unwanted variation including 1) differences in sample handling, processing, and sequencing platforms between datasets (technical variation) as well as 2) differences in experimental design between datasets such as sex or age of the model organism used. In the present study, NASA GeneLab-hosted RNAseq datasets from rodent liver tissues were used to evaluate several statistical methods to correct for this unwanted variation through two approaches, reference-based and standard. The following correction algorithms were applied with (reference-based) and/or without (standard) considering Universal Mouse RNA Reference samples: ComBat and ComBat_seq from the SVA package, median polish, empirical Bayes, and ANOVA-based algorithms from the MBatch package, and negative binomial regression normalization in the DESeq2 package. For each approach, after the correction algorithm was applied, differential gene expression (DGE) analysis of flight and ground control samples was performed with the combined data. The robustness of each tool was evaluated using BatchQC, to determine statistical differences between datasets before and after correction, Principal Component Analysis, to evaluate global gene expression in samples before and after correction, and by comparing DGE analysis of individual datasets and combined datasets before and after correction. The results showed that the reference-based approach introduced several additional (and likely artificial) DEGs when compared with the standard approach. Thus, the most robust standard correction will be implemented in the GeneLab Visualization 2.0 platform when datasets are combined.

GeneLab, RNA-seq, Batch Correction↗

DOE OSTI2025

NLR HPC Kestrel Jobs Data

Overview: Anonymized job-level records from the Kestrel HPC system at the National Laboratory of the Rockies (NLR). Each record represents a Slurm batch job with scheduling metadata, resource requests, utilization, energy estimates, and efficiency metrics. Sensitive fields (user, account, job name, submit line, working directory, submit script, and job type) are replaced with 7-character cryptographic hashes. System & Timeframe: Kestrel is located at the NLR campus. Standard compute nodes have 104 cores and 256 GB RAM; bigmem nodes have 2,000 GB. GPU nodes (gpu-h100 partition) use NVIDIA H100 GPUs. Data covers jobs submitted August 2023 through December 2025. Funding provided by the U.S. Department of Energy, EERE. Files: esif.hpc.kestrel.job-anon.zip — Anonymized job records (Hive-partitioned Parquet) datacard.md — Full dataset documentation ~11 million rows, 50 variables. Readable with PyArrow, pandas, DuckDB, Apache Spark, or any Parquet-compatible tool. Data Collection: Jobs collected via sacct with timezone-aware export (SLURM_TIME_FORMAT="%Y-%m-%dT%H:%M:%S%z"), loaded into PostgreSQL. Calculated columns updated via database triggers and batch functions. All timestamps use timestamptz and correctly handle DST transitions. Preprocessing: Anonymization of name, user, account, submit_line, work_dir, submit_script, and job_type via 7-char hex hashes Derived columns: queue_wait, cpu_eff, max/min/avg_mem_eff, energy estimates Simplified job state mapping (e.g., "CANCELLED by 132357" → "CANCELLED") Boolean flags: python_job, reframe_job Temporal decomposition: year, month, day, day_of_week, hour, minute from submit_time Shared node tracking: shared_job_count, nodes_shared, jobs_shared Key Variables: Scheduling: job_id, partition, state_simple, submit_time, start_time, end_time, queue_wait Resources: nodes_req/used, processors_req/used, memory_req, wallclock_req/used, gpus_requested Efficiency: cpu_eff, max/min/avg_mem_eff Energy: cpu_energy_tdp_estimated_max/used_watt_hours, consumed_energy_raw_joules, consumed_energy_raw_watt_hours Sharing: shared_job_count, nodes_shared, jobs_shared Partitions: short, standard, debug, gpu-h100 Job States: CANCELLED, COMPLETED, FAILED, PENDING, RUNNING QoS Levels: normal, high Important Notes: Timestamps include timezone offsets; DST transitions are handled correctly, though adding intervals across DST boundaries requires offset adjustment shared_job_count reflects physical node co-residency, not use of the shared partition Job step records and raw Slurm JSONB fields are excluded Do not attempt to re-identify individuals from hashed fields

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2025

Machine learning modeling and model predictive control of a closed-circuit reverse osmosis system

Closed-circuit reverse osmosis (CCRO) offers a flexible and energy-efficient alternative to conventional reverse osmosis by operating in a semi-batch mode that recycles brine, enabling higher recovery rates and reduced specific energy consumption (SEC). However, developing accurate, system-level dynamic models for CCRO remains challenging due to its nonlinear, multi-phase operation and sensitivity to variable feed water conditions. Traditional modeling approaches, such as NARMAX (nonlinear autoregressive moving average with exogenous inputs), often struggle to generalize across varying inlet feed concentrations, necessitating frequent parameter re-estimation and limiting their utility for real-time control applications. To address these limitations, we developed a long short-term memory (LSTM) neural network model trained on an extensive experimental data set from a CCRO pilot plant. The model accepts three inputs, feed flow rate, recirculation flow rate, and initial feed conductivity, and predicts three key outputs: reject conductivity, feed pump power draw, and recirculation pump power draw. We validated the LSTM model against experimental data, demonstrating its ability to distinguish between different feed conductivities and adapt to variable flow rates. Subsequently, we incorporated the LSTM model within a nonlinear model predictive control (MPC) scheme and conducted closed-loop simulations to optimize the integrated SEC (iSEC). In conclusion, the results project up to a 6% reduction in iSEC by using MPC to optimize performance over the entire experiment duration, without requiring any random excitation for data collection or parameter re-estimation.

Desalination↗

DOE OSTI2024

Kansas City, Missouri, Streetlight Electric Vehicle Charging: Strategies and challenges for site selection of streetlight electric vehicle infrastructure in Kansas City, Missouri (Final Report)

Public streetlight charging, whether on streets in central business districts or residential areas, provides easy charging access for apartment residents and homeowners alike. While most electric vehicle (EV) drivers charge at home, they do so in garages or on driveways they own. For renters and residents of multifamily housing (MFH), however, this may not be an option. EVs have a lower cost of ownership compared to conventional vehicles, and a used EV may be an affordable option for a lower-income household. But without easy access to charging, even a low-cost used EV may not be an option for a prospective buyer. An affordable curbside charging network has the potential to expand EV adoption into neighborhoods that have to date seen minimal interest and uptake of the technology and associated charging infrastructure. Streetlight charging networks can provide an economical, scalable, and effective approach to providing equitable and convenient charging. Metropolitan Energy Center (MEC) is dedicated to the mission of creating resource efficiency, environmental health, and economic vitality in the Kansas City region and beyond. Since 1983, MEC has provided resources, outreach, and training to make alternative fuels and energy efficiency commonplace. MEC led a streetlight charging pilot project that installed limited EV charging infrastructure on the streetlight system in Kansas City, Missouri, to demonstrate and test the benefits of curbside charging for EVs at existing on-street parking locations. The project aimed to cost-effectively expand the charging network in Kansas City to support residential charging and provide infrastructure in one or more charging deserts throughout the city. This pilot evaluates the impact and overall success of streetlight charging based on community feedback, utilization of charging infrastructure, technical feasibility, and cost. The project has pursued a data- and community-driven site selection process designed to identify sites with high demand and high opportunity for EV charging. This project was funded by the U.S. Department of Energy (DOE) and awarded to MEC through a competitive proposal process. The novelty and complexity of this project required an organization that could facilitate collaboration across levels of government, community members, and industry partners. For the past 25 years, through Kansas City Regional Clean Cities, MEC has worked with numerous public and private fleets on a variety of projects to improve the environmental performance and efficiency of the regional vehicle fleet. To advance affordable, efficient, and clean transportation efforts, DOE Clean Cities and Communities coalitions create local networks of public and private sector stakeholders and engage communities. Rooted within their local communities, the coalitions serve as experts and ambassadors, bringing to bear the collective knowledge, experience, and practical know-how of the entire network from within DOE, its national laboratories, and diverse stakeholders in the field. MEC and its project partners made in-kind contributions to leverage federal dollars for the benefit of the Kansas City community. Findings from this project will help determine the best applications for streetlight charging technologies to maximize funding impact and serve community needs. The team evaluated locations based on expected charging demand, technical feasibility, safety considerations, and enhanced charging network siting needs. Throughout the project, the team gathered feedback and evaluated ways to make public charging for EVs available to all community members. The insights will help Kansas City and other communities streamline future efforts to support EV drivers through public charging in the city right-of-way. Furthermore, this project will inform citywide guidance for future installations. MEC is committed to a transparent and publicly accessible approach that encourages the collaborative evaluation of streetlight charging. The project has engaged the community to proactively identify and evaluate the benefits and impacts of streetlight charging. It was a priority for the project to ensure the benefits of this pilot are distributed equitably to all members of the Kansas City community and that new charging opportunities and associated resources are available in diverse neighborhoods across the city. The charging infrastructure supports an affordable curbside charging network that will enable more drivers to choose EVs and provide easy charging access for all community members interested in driving an EV. The community feedback received through this project informed future resources and opportunities to make EVs more accessible to all members of the Kansas City community. MEC worked with several community partners on this project, including Missouri University of Science and Technology (MST), Pennsylvania State University (Penn State), the National Renewable Energy Laboratory (NREL); the city of Kansas City, Missouri; Evergy; Black and McDonald (B&M); LilyPad EV; EVNoire; and Westside Housing Organization (WHO). Project partners contributed to the cost match required for DOE grants through capital expenditures, personnel, and other in-kind contributions. Detailed descriptions of project team organizations can be found in Appendix A. Project Partners. Analysts at NREL and MST/PennState developed site maps based on demand and equity considerations. MEC conducted outreach to community members to garner input on project design and site selection, and received approval from the Missouri Public Service Commission (PSC) for Evergy’s EV charging station ownership. MEC worked with all partners to gather additional siting criteria and developed a site selection evaluation checklist, and partners conducted site visits to proposed installation sites. Next, B&M, Evergy, and the city executed all site agreements, conducted site-specific engineering design, acquired associated permits, and issued notices to proceed site by site or in small batches. Finally, from January to April 2023, the project team installed 23 EV charging stations built on Kansas City’s streetlight system in six council districts. Evergy will own, operate, and monitor the stations for 10 years, sharing charging data with MEC for at least 1 year.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗