Search NASA⌕ Search

SEARCH · Search NASA

Results for “Data manipulation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 217 records · Page 12
All sourcesNASA NTRSDOE OSTI
NASA NTRS1998

A Geometry Based Infra-Structure for Computational Analysis and Design

The computational steps traditionally taken for most engineering analysis suites (computational fluid dynamics (CFD), structural analysis, heat transfer and etc.) are: (1) Surface Generation -- usually by employing a Computer Assisted Design (CAD) system; (2) Grid Generation -- preparing the volume for the simulation; (3) Flow Solver -- producing the results at the specified operational point; (4) Post-processing Visualization -- interactively attempting to understand the results. For structural analysis, integrated systems can be obtained from a number of commercial vendors. These vendors couple directly to a number of CAD systems and are executed from within the CAD Graphical User Interface (GUI). It should be noted that the structural analysis problem is more tractable than CFD; there are fewer mesh topologies used and the grids are not as fine (this problem space does not have the length scaling issues of fluids). For CFD, these steps have worked well in the past for simple steady-state simulations at the expense of much user interaction. The data was transmitted between phases via files. In most cases, the output from a CAD system could go to Initial Graphics Exchange Specification (IGES) or Standard Exchange Program (STEP) files. The output from Grid Generators and Solvers do not really have standards though there are a couple of file formats that can be used for a subset of the gridding (i.e. PLOT3D data formats). The user would have to patch up the data or translate from one format to another to move to the next step. Sometimes this could take days. Specifically the problems with this procedure are:(1) File based -- Information flows from one step to the next via data files with formats specified for that procedure. File standards, when they exist, are wholly inadequate. For example, geometry from CAD systems (transmitted via IGES files) is defined as disjoint surfaces and curves (as well as masses of other information of no interest for the Grid Generator). This is particularly onerous for modern CAD systems based on solid modeling. The part was a proper solid and in the translation to IGES has lost this important characteristic. STEP is another standard for CAD data that exists and supports the concept of a solid. The problem with STEP is that a solid modeling geometry kernel is required to query and manipulate the data within this type of file. (2) 'Good' Geometry. A bottleneck in getting results from a solver is the construction of proper geometry to be fed to the grid generator. With 'good' geometry a grid can be constructed in tens of minutes (even with a complex configuration) using unstructured techniques. Adroit multi-block methods are not far behind. This means that a million node steady-state solution can be computed on the order of hours (using current high performance computers) starting from this 'good' geometry. Unfortunately, the geometry usually transmitted from the CAD system is not 'good' in the grid generator sense. The grid generator needs smooth closed solid geometry. It can take a week (or more) of interaction with the CAD output (sometimes by hand) before the process can begin. One way Communication. (3) One-way Communication -- All information travels on from one phase to the next. This makes procedures like node adaptation difficult when attempting to add or move nodes that sit on bounding surfaces (when the actual surface data has been lost after the grid generation phase). Until this process can be automated, more complex problems such as multi-disciplinary analysis or using the above procedure for design becomes prohibitive. There is also no way to easily deal with this system in a modular manner. One can only replace the grid generator, for example, if the software reads and writes the same files. Instead of the serial approach to analysis as described above, CAPRI takes a geometry centric approach. This makes the actual geometry (not a discretized version) accessible to all phases of the analysis. The connection to the geometry is made through an Application Programming Interface (API) and NOT a file system. This API isolates the top-level applications (grid generators, solvers and visualization components) from the geometry engine. Also this allows the replacement of one geometry kernel with another, without effecting these top-level applications. For example, if UniGraphics is used as the CAD package then Parasolid (UG's own geometry engine) can be used for all geometric queries so that no solid geometry information is lost in a translation. This is much better than STEP because when the data is queried, the same software is executed as used in the CAD system. Therefore, one analyzes the exact part that is in the CAD system. CAPRI uses the same idea as the commercial structural analysis codes but does not specify control. Software components of the CAD system are used, but the analysis suite, not the CAD operator, specifies the control of the software session. This also means that the license issues (may be) minimized and individuals need not have to know how to operate a CAD system in order to run the suite.

Haimes, Robert↗
NASA NTRS2008

Potential Use of a Bayesian Network for Discriminating Flash Type from Future GOES-R Geostationary Lightning Mapper (GLM) data

Continuous monitoring of the ratio of cloud flashes to ground flashes may provide a better understanding of thunderstorm dynamics, intensification, and evolution, and it may be useful in severe weather warning. The National Lighting Detection Network TM (NLDN) senses ground flashes with exceptional detection efficiency and accuracy over most of the continental United States. A proposed Geostationary Lightning Mapper (GLM) aboard the Geostationary Operational Environmental Satellite (GOES-R) will look at the western hemisphere, and among the lightning data products to be made available will be the fundamental optical flash parameters for both cloud and ground flashes: radiance, area, duration, number of optical groups, and number of optical events. Previous studies have demonstrated that the optical flash parameter statistics of ground and cloud lightning, which are observable from space, are significantly different. This study investigates a Bayesian network methodology for discriminating lightning flash type (ground or cloud) using the lightning optical data and ancillary GOES-R data. A Directed Acyclic Graph (DAG) is set up with lightning as a "root" and data observed by GLM as the "leaves." This allows for a direct calculation of the joint probability distribution function for the lighting type and radiance, area, etc. Initially, the conditional probabilities that will be required can be estimated from the Lightning Imaging Sensor (LIS) and the Optical Transient Detector (OTD) together with NLDN data. Directly manipulating the joint distribution will yield the conditional probability that a lightning flash is a ground flash given the evidence, which consists of the observed lightning optical data [and possibly cloud data retrieved from the GOES-R Advanced Baseline Imager (ABI) in a more mature Bayesian network configuration]. Later, actual GLM and NLDN data can be used to refine the estimates of the conditional probabilities used in the model; i.e., the Bayesian network is a learning network. Methods for efficient calculation of the conditional probabilities (e.g., an algorithm using junction trees), finding data conflicts, goodness of fit, and dealing with missing data will also be addressed.

Solakiewiz, Richard↗
NASA NTRS1993

Global change information support: A north-south coalition

On a daily basis we become more aware that our planet, earth, exists in a delicate balance; we, its inhabitants, must be informed caretakers. Global change communities have emerged around the globe to address this multidisciplinary subject. Information systems that integrate text, bibliographic, numeric and visual data are needed to support these global change communities. No one information center can hope to collect all the relevant data. Rather, we must form a coalition, North and South, to collect and provide access to disparate, multidisciplinary sources of information, and to develop standardized tools for documenting and manipulating this data and information. International resources need to be mobilized in a coordinated manner to move us towards this goal. This paper looks at emerging information technologies that can be utilized to build such a system, and outlines some cooperative North/South strategies.

Blados, Walter R.↗
NASA NTRS1993

Global change information support - A north/south coalition

On a daily basis we become more aware that our planet, earth, exists in a delicate balance; we, its inhabitants, must be informed caretakers. Global change communities have emerged around the globe to address this multidisciplinary subject. Information systems that integrate text, bibliographic, numeric and visual data are needed to support these global change communities. No one information center can hope to collect all the relevant data. Rather, we must form a coalition, North and South, to collect and provide access to disparate, multidisciplinary sources of information, and to develop standardized tools for documenting and manipulating this data and information. International resources need to be mobilized in a coordinated manner to move us towards this goal. This paper looks at emerging information technologies that can be utilized to build such a system, and outlines some cooperative North/South strategies.

Blados, Walter R.↗
NASA NTRS1975

Data management, chapter 5, part C

The data management for a spacecraft radar was defined in terms of an end-to-end data system, which performs the following three functions: (1) sampling and compaction of data onboard the spacecraft, (2) manipulation of radar data on the ground and (3) conversion of radar measurements to geophysical quantities by means of pattern recognition and other machine techniques. Data processing for imaging radar onboard the spacecraft was examined with the conclusion that several techniques can be used to compact the data before storage. It is recommended that compaction techniques be studied further and that existing aircraft radars be modified to provide digital data so that these compaction techniques can be tested.

Source record↗
NASA NTRS2020

Development of a Display Tool to Quality Control Weather Balloon Data for Space Launch Vehicles Using Python

Continuous atmospheric data analysis is an important factor for space launch vehicle design and operations. The balloon quality control tool was developed by NASA’s Marshall Space Flight Center (MSFC) Natural Environments Branch (NEB) for monitoring quality control processes and verifying the automated flags created on the balloon data sets analyzed. The data sets currently analyzed are comprised of high-resolution and low-resolution balloon data from NASA Kennedy Space Center (KSC), co-located on the United States Air Force’s Eastern range (ER) at the Cape Canaveral Air Force Station. The NEB was tasked to perform a quality assessment of these data sets and needed a tool to confirm the quality control (QC) flags produced from an automated process and add additional QC flags if necessary. This Graphical User Interface (GUI) was developed to visualize all of the data from these balloon sets, display any flags from the automated QC process, and add additional flags to variables if necessary. The GUI was developed in Python 3.6 utilizing different packages available such as pandas for data analysis and manipulation, NumPy for high-performance multidimensional array and tools to compute with and manipulate arrays, Matplotlib for plotting data and Tkinter to build the GUI.

Jessica K Headley↗
NASA NTRS2020

Development of a Display Tool to Quality Control Weather Balloon Data for Space Launch Vehicles

Continuous atmospheric data analysis is an important factor for space launch vehicle design and operations. The balloon quality control tool was developed by NASA’s Marshall Space Flight Center (MSFC) Natural Environments Branch (NEB) for monitoring quality control processes and verifying the automated flags created on the balloon data sets analyzed. The data sets currently analyzed are comprised of high-resolution and low-resolution balloon data from NASA Kennedy Space Center (KSC), co-located on the United States Air Force’s Eastern range (ER) at the Cape Canaveral Air Force Station. The NEB was tasked to perform a quality assessment of these data sets and needed a tool to confirm the quality control (QC) flags produced from an automated process and add additional QC flags if necessary. This Graphical User Interface (GUI) was developed to visualize all of the data from these balloon sets, display any flags from the automated QC process, and add additional flags to variables if necessary. The GUI was developed in Python 3.6 utilizing different packages available such as pandas for data analysis and manipulation, NumPy for high-performance multidimensional array and tools to compute with and manipulate arrays, Matplotlib for plotting data and Tkinter to build the GUI.

Jessica K Headley↗
DOE OSTI2021

AbacusSummit: Cosmological N-body Halos, Light Cones, Particles, Merger Trees, Initial Conditions, and Power Spectra

This is the primary data release of the AbacusSummit, a suite of large, high-accuracy cosmological N-body simulations. Documentation describing the suite and code, including links to refereed publications and instructions for manipulating the data contained in this release, may be found at the following URL: https://abacussummit.readthedocs.io/ The primary data products presented in this release are halo catalogs, light cones, and particle data. Several supplemental products are also presented: power spectra, initial conditions, and merger trees. Cleaned halo catalogs are also provided for the subset of simulations that were used in refereed publications at the time of this DOI. This release also contains several large scale-free and high-redshift simulations run as part of the same allocation used to produce AbacusSummit.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2022

On the integration of molecular dynamics, data science, and experiments for studying solvent effects on catalysis

Computational workflows that combine molecular dynamics (MD) simulations and emerging data-centric (DC) methods can accelerate the screening and analysis of solvent systems experimentally and computationally. Here, MD simulations provide atomic positions and velocities of reactant, solvent, and catalyst materials that can be manipulated into data representations that in turn can be used by DC techniques to conduct predictive modeling, feature extraction, and experimental design. For liquid-phase catalytic applications, emerging DC techniques such as Convolutional and Graph Neural Networks (CNN/GNN), Topological Data Analysis (TDA), and Active Learning (AL) can leverage MD and experimental data to quickly predict solvent effects on reaction outcomes. For instance, in recent studies, 3D solvent environments obtained with MD have been exploited by CNNs to predict experimental reaction rates for homogeneous acid-catalyzed lignocellulosic processes. In this perspective, we discuss basic principles of DC methods and how these can be combined with MD to enable high-throughput screening of solvent selection for diverse catalysis applications.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
NASA NTRS2003

Moving NASA Remote Sensing Data to the GIS Environment for Health Studies

There has been an increasing demand by the health community for improved data on many different environmental factors relevant to the links between the environment and disease occurrence and transmission. These data are important for GIS-based monitoring, risk mapping, and surveillance of epidemiological parameters on a large number of different spatial, temporal, and spectral resolutions. Accordingly, NASA is developing new approaches to data collection and distribution in order to improve access to multiple sources of data streams to increase spatial and temporal coverage. Methods are being developed to incorporate different, scalable capabilities to handle multiple data sources by adding, deleting and replacing components as required as well as associated tools for their management. An approach has been to search for innovative solutions focused on the creation, use and manipulation of data stored in many different archives. These include data transformation and combination as well as data and information tools that can assist the public health and science community to use existing and anticipated products in new and flexible ways. This presentation will provide an inventory of geophysical parameters derived from satellite remote sensing sensors that are useful for GIS-based public health studies. The presentation will also discuss the physical and scientific limitations of access to and use of these data for health applications such as resolution and format differences, lack of software interoperability, data access problems. Finally, there will be a summary of the recent steps the NASA program has taken to bring NASA-generated satellite products to a wider range of users in the GIS community.

Vicente, Gilberto A.↗
NASA NTRS1997

Flood Management Enhancement Using Remotely Sensed Data

SENTAR, Inc., entered into a cooperative agreement with NASA Goddard Space Flight Center (GSFC) in December 1994. The intent of the NASA Cooperative Agreement was to stimulate broad public use, via the Internet, of the very large remote sensing databases maintained by NASA and other agencies, thus stimulating U.S. economic growth, improving the quality of life, and contributing to the implementation of a National Information Infrastructure. SENTAR headed a team of collaborating organizations in meeting the goals of this project. SENTAR's teammates were the NASA Marshall Space Flight Center (MSFC) Global Hydrology and Climate Center (GHCC), the U.S. Army Space and Strategic Defense Command (USASSDC), and the Alabama Emergency Management Agency (EMA). For this cooperative agreement, SENTAR and its teammates accessed remotely sensed data in the Distributed Active Archive Centers, and other available sources, for use in enhancing the present capabilities for flood disaster management by the Alabama EMA. The project developed a prototype software system for addressing prediction, warning, and damage assessment for floods, though it currently focuses on assessment. The objectives of the prototype system were to demonstrate the added value of remote sensing data for emergency management operations during floods and the ability of the Internet to provide the primary communications medium for the system. To help achieve these objectives, SENTAR developed an integrated interface for the emergency operations staff to simplify acquiring and manipulating source data and data products for use in generating new data products. The prototype system establishes a systems infrastructure designed to expand to include future flood-related data and models or to include other disasters with their associated remote sensing data requirements and distributed data sources. This report covers the specific work performed during the seventh, and final, milestone period of the project, which began on 1 October 1996 and ended on 31 January 1997. In addition, it provides a summary of the entire project.

Romanowski, Gregory J.↗
NASA NTRS1984

Application and enhancements of MOVIE.BYU

MOVIE.BYU (MOVIE.BRIGHAM YOUNG UNIVERSITY) is a system of programs for the display and manipulation of data representing mathematical, architectural, and topological models in which the geometry may be described in terms of panel (n-sided polygons) and solid elements or contour lines. The MOVIE.BYU system has been used in a series of applications of LaRC. One application has been the display, creation, and manipulation of finite element models in aeronautic/aerospace research. These models have been displayed on both vector and color raster devices, and the user has the option to modify color and shading parameters on these color raster devices. Another application involves the display of scalar functions (temperature, pressure, etc.) over the surface of a given model. This capability gives the researcher added flexibility in the analysis of the model and its accompanying data. Limited animation (frame-by-frame creation) has been another application of MOVIE.BYU in the modeling of kinematic processes in antenna structures.

Gates, R. L.↗
NASA NTRS1988

Requirements for ongoing development of the Pilot Land Data System (PLDS)

The Pilot Land Data System being developed to address the information processing needs of the NASA land sciences research community is presented. The objective of the pilot program is to establish a limited-scale, distributed information system for the archival, location, transfer, integration, and manipulation of data across multiple sites connected by a high-speed communications network. Functional capabilities required for users to create, access, and maintain local and distributed data bases containing various types of data in support of land sciences research are summarized.

Wharton, S. W.↗
DOE OSTI2026

Dataset for "A primer on forest structure measurement with lidar for ecologists"

This repository includes data and code accompanying the case study included in the manuscript "A primer on forest structure measurement with lidar for ecologists" (submitted to Ecosphere). We compiled lidar datasets from multiple platforms in a common area to: 1. Demonstrate how differences in sensor characteristics influence density and resolution of lidar data. 2. Provide open-source, co-located datasets for users to further inspect differences in lidar data. 3. Provide example code to perform basic lidar analysis. This case study is meant to allow readers to get hands-on experience with real-world data from different platforms. This case study is not meant to be a rigorous comparison of derived ecological metrics among all sensors; such comparisons can be found throughout other publications referenced throughout the main manuscript. Code includes basic functions in R commonly used to visualize and manipulate lidar data accessible with a normal laptop computer; more sophisticated algorithms for advanced users are also referenced throughout the main manuscript. Terrestrial laser scanning (TLS), mobile laser scanning (MLS), UAS laser scanning (ULS), airborne laser scanning (ALS), and spaceborne laser scanning (SLS) data were collected within the Smithsonian Environmental Research Center (SERC) forest dynamics plot in Maryland, USA. TLS, MLS, and ALS data were collected within 1 month of the 2021 growing season; ULS data were collected in November 2020 (“leaf-off” data) and July 2022 (“leaf-on” data).

54 ENVIRONMENTAL SCIENCES↗
NASA NTRSDate not supplied

The NASA Open Science Data Repository: Biomedical Fair Data, Analysis Tools, User Communities, Publications, and Discoveries for Deep Space Missions

Increased biomedical risks and challenges associated with deep space missions require new knowledge discovery, new health countermeasures, and development of novel ecosystems, life support, crop production, and biomedical support capabilities. To meet NASA’s Moon to Mars strategic program goals for Human and Biological Sciences, findable, accessible, interoperable, reusable (FAIR), and maximally open-access data is going to be required to enable humanity to thrive in deep space. Indeed, this cornerstone perspective on FAIR and maximally open access data was also recommended in the recent 2023-2032 Decadal Survey from the National Academies of Sciences, Engineering, and Medicine. The NASA Open Science Data Repository (OSDR) is a maximally open access and FAIR database, and meets various scientific, technical, and operational spaceflight needs. It offers public users and submitters the ability to upload, download, search, share, analyze, and visualize data across ‘omics, physiological, phenotypic, behavioral, bioimaging, video, and environmental monitoring telemetry datasets. OSDR includes NASA GeneLab, NASA Ames Life Sciences Data Archive, and the NASA Biological Institutional Scientific Collection. OSDR has >455 studies with datasets from model organisms and non-NASA human astronauts. There are ~12 datasets from the Inspiration 4 (I4) mission, spanning metagenomics, comprehensive metabolic panels, clonal hematopoiesis, spatial transcriptomics, proteomics, and cytokine panels. In the interest of data privacy, two I4 datasets have raw FASTQ and FASTA files relating to the epitranscriptome, and a new request feature is live in OSDR (with a backend review process established) which was developed based on industry norms. OSDR also recently began a collaboration with the European Space Agency (ESA) to scientifically curate and make available >200 terabytes of human and model organism space-relevant data. The OSDR submission portal is designed to ingest and curate ~25 ‘omics assay data types, and ~50 physiological-phenotypic-imaging assay data types, spanning ultrasonography, micro-computed tomography, histology, morphometric photography, rebound tonometry, gait analysis, optical coherence tomography, novel object recognition, flow cytometry, and immunohistochemistry. A suite of analysis tools are available for OSDR users including: 1) an Environmental Data Application to compare radiation, CO2, relative humidity, temperature, and other telemetry across missions and subjects, 2) the RadLab database, a collaboration between NASA, ESA, the German and Italian Space Agencies, and the Bulgarian Academy of Sciences, which compiles radiation measurements relevant to human spaceflight and provides tools for accessing and manipulating the data, and 3) a Multi-study visualization tool which enables users to look across and combine GeneLab’s omics datasets across different experiments and missions. There are ~600 volunteer OSDR Analysis Working Group (AWG) members who: 1) provide feedback on scientific standards for reuse (subject and assay metadata; processing pipelines; dataset formats and uniformed structures for machine-readability), and 2) collaborate to mine-reuse OSDR data conducting scientific analysis. OSDR has enabled 60 publications as of September 2023, many directly from AWG collaborations most notably the Cell Press package in 2020. Lastly, there are at least 15 articles which mine OSDR data part of a package of ~50 articles across Nature Portfolio with research stemming from I4, the Japan Aerospace Exploration Agency, NASA Space Biology, and the NASA Human Research Program.

space biology↗
NASA NTRSDate not supplied

NASA Open Science Data Repository: Biomedical FAIR Data, Analysis Tools, User Communities, and Discoveries for Deep Space Missions

Increased biomedical risks and challenges associated with deep space missions require new knowledge discovery, new health countermeasures, and development of novel ecosystems, life support, crop production, and biomedical support capabilities. To meet NASA’s Moon to Mars strategic program goals for Human and Biological Sciences, findable, accessible, interoperable, reusable (FAIR), and maximally open-access data is going to be required to enable humanity to thrive in deep space. Indeed, this cornerstone perspective on FAIR and maximally open access data was also recommended in the recent 2023-2032 Decadal Survey from the National Academies of Sciences, Engineering, and Medicine. The NASA Open Science Data Repository (OSDR) is a maximally open access and FAIR database, and meets various scientific, technical, and operational spaceflight needs. It offers public users and submitters the ability to upload, download, search, share, analyze, and visualize data across ‘omics, physiological, phenotypic, behavioral, bioimaging, video, and environmental monitoring telemetry datasets. OSDR includes NASA GeneLab, NASA Ames Life Sciences Data Archive, and the NASA Biological Institutional Scientific Collection. OSDR has >455 studies with datasets from model organisms and non-NASA human astronauts. There are ~12 datasets from the Inspiration 4 (I4) mission, spanning metagenomics, comprehensive metabolic panels, clonal hematopoiesis, spatial transcriptomics, proteomics, and cytokine panels. In the interest of data privacy, two I4 datasets have raw FASTQ and FASTA files relating to the epitranscriptome, and a new request feature is live in OSDR (with a backend review process established) which was developed based on industry norms. OSDR also recently began a collaboration with the European Space Agency (ESA) to scientifically curate and make available >200 terabytes of human and model organism space-relevant data. The OSDR submission portal is designed to ingest and curate ~25 ‘omics assay data types, and ~50 physiological-phenotypic-imaging assay data types, spanning ultrasonography, micro-computed tomography, histology, morphometric photography, rebound tonometry, gait analysis, optical coherence tomography, novel object recognition, flow cytometry, and immunohistochemistry. A suite of analysis tools are available for OSDR users including: 1) an Environmental Data Application to compare radiation, CO2, relative humidity, temperature, and other telemetry across missions and subjects, 2) the RadLab database, a collaboration between NASA, ESA, the German and Italian Space Agencies, and the Bulgarian Academy of Sciences, which compiles radiation measurements relevant to human spaceflight and provides tools for accessing and manipulating the data, and 3) a Multi-study visualization tool which enables users to look across and combine GeneLab’s omics datasets across different experiments and missions. There are ~600 volunteer OSDR Analysis Working Group (AWG) members who: 1) provide feedback on scientific standards for reuse (subject and assay metadata; processing pipelines; dataset formats and uniformed structures for machine-readability), and 2) collaborate to mine-reuse OSDR data conducting scientific analysis. OSDR has enabled 60 publications as of September 2023, many directly from AWG collaborations most notably the Cell Press package in 2020. Lastly, there are at least 15 articles which mine OSDR data part of a package of ~50 articles across Nature Portfolio with research stemming from I4, the Japan Aerospace Exploration Agency, NASA Space Biology, and the NASA Human Research Program.

open access↗
DOE OSTI2019

Robust, automated sleep scoring by a compact neural network with distributional shift correction

Studying the biology of sleep requires the accurate assessment of the state of experimental subjects, and manual analysis of relevant data is a major bottleneck. Recently, deep learning applied to electroencephalogram and electromyogram data has shown great promise as a sleep scoring method, approaching the limits of inter-rater reliability. As with any machine learning algorithm, the inputs to a sleep scoring classifier are typically standardized in order to remove distributional shift caused by variability in the signal collection process. However, in scientific data, experimental manipulations introduce variability that should not be removed. For example, in sleep scoring, the fraction of time spent in each arousal state can vary between control and experimental subjects. We introduce a standardization method, mixture z-scoring, that preserves this crucial form of distributional shift. Using both a simulated experiment and mouse in vivo data, we demonstrate that a common standardization method used by state-of-the-art sleep scoring algorithms introduces systematic bias, but that mixture z-scoring does not. We present a free, open-source user interface that uses a compact neural network and mixture z-scoring to allow for rapid sleep scoring with accuracy that compares well to contemporary methods. This work provides a set of computational tools for the robust automation of sleep scoring.

60 APPLIED LIFE SCIENCES↗