Search NASA⌕ Search

SEARCH · Search NASA

Results for “data streaming”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 127 records · Page 7
All sourcesNASA NTRSDOE OSTI
NASA NTRS1981

A study of high density bit transition requirements versus the effects on BCH error correcting coding

Several methods for increasing bit transition densities in a data stream are summarized, discussed in detail, and compared against constraints imposed by the 2 MHz data link of the space shuttle high rate multiplexer unit. These methods include use of alternate pulse code modulation waveforms, data stream modification by insertion, alternate bit inversion, differential encoding, error encoding, and use of bit scramblers. The psuedo-random cover sequence generator was chosen for application to the 2 MHz data link of the space shuttle high rate multiplexer unit. This method is fully analyzed and a design implementation proposed.

Ingels, F.↗
DOE OSTI2026

Towards a self-driving trigger at the LHC: adaptive response in real time

Real-time data filtering and selection—or trigger—systems at high-throughput scientific facilities such as the experiments at the Large Hadron Collider must process extremely high-rate data streams under stringent bandwidth, latency, and storage constraints. Yet these systems are typically designed as static, hand-tuned menus of selection criteria grounded in prior knowledge and simulation. In this work, we further explore the concept of a self-driving trigger, an autonomous data-filtering framework that reallocates resources and adjusts thresholds dynamically in real-time to optimize signal efficiency, rate stability, and computational cost as instrumentation and environmental conditions evolve. We introduce a benchmark ecosystem to emulate realistic collider scenarios and demonstrate real-time optimization of a menu including canonical energy sum triggers as well as modern anomaly-detection algorithms that target non-standard event topologies using machine learning. Using simulated data streams and publicly available collision data from the Compact Muon Solenoid experiment, we demonstrate the capability to dynamically and automatically optimize trigger performance under specific cost objectives without manual retuning. Our adaptive strategy shifts trigger design from static menus with heuristic tuning to intelligent, automated, data-driven control, unlocking greater flexibility and discovery potential in future high-energy physics analyses.

Emami, Shaghayegh [Michigan U.] (ORCID:00090007589↗
NASA NTRS1986

Ada and knowledge-based systems: A prototype combining the best of both worlds

A software architecture is described which facilitates the construction of distributed expert systems using Ada and selected knowledge based systems. This architecture was utilized in the development of a Knowledge-based Maintenance Expert System (KNOMES) prototype for the Space Station Mobile Service Center (MSC). The KNOMES prototype monitors a simulated data stream from MSC sensors and built-in test equipment. It detects anomalies in the data and performs diagnosis to determine the cause. The software architecture which supports the KNOMES prototype allows for the monitoring and diagnosis tasks to be performed concurrently. The basic concept of this software architecture is named ACTOR (Ada Cognitive Task ORganization Scheme). An individual ACTOR is a modular software unit which contains both standard data processing and artificial intelligence components. A generic ACTOR module contains Ada packages for communicating with other ACTORs and accessing various data sources. The knowledge based component of an ACTOR determines the role it will play in a system. In this prototype, an ACTOR will monitor the MSC data stream.

Brauer, David C.↗
NASA NTRS1990

RF/optical interference design for optical intersatellite links

A design approach for the RF/optical link interface for a data relay satellite is described. The flexibility of forward and return links in future data acquisition satellites in handling varying missions and data rates to 1 Gbit/s is considered. Attention is focused on requirements for the NASA Tracking and Data Acquisition System. System components are described including the return link multiplexer, the return link transmultiplexer, the forward link multiplexer, the forward link demultiplexer, and the frontside/backside switch. Ping-pong buffers, which provide rate buffering for each input data stream, are discussed and justification bits, which handle variations due to Doppler shift and local oscillator variation, are considered. The time-division multiplexed streams consist of a unique synchronization word for frame synchronization, and control words associated with each data burst to identify the presence or absence of a justification bit. Redundant data paths are described for both forward and return data streams.

Garlow, Ronald K.↗
NASA NTRS1998

Polar Hydra Data Analysis

The science activities are: 1) Hydra is still operating successfully on orbit. 2) A large amount of analysis and discovery has occurred with the Hydra ground data processing this past year. 3) Full interdetector calibration has been implemented and documented. This intercalibration was necessitated by the incorrect installation of bias resistors in the pre-acceleration stage to the electron channeltrons. This had the effect of making the counting efficiency for electrons energy dependent as well as channeltron specific. The nature of the error had no impact on the ion detection efficiency since they have a different bias arrangement. This intercalibration is so effective, that the electron and ion moment densities are routinely produced with a level of agreement better than 20%. 4) The data processing routinely removes glint in the sensors and produces public energy time spectrograms on the web overnight. 6) Routine, but more intensive computer processing codes are operational that determine for electrons and ions, the density, the flow vector, the pressure tensor and the heat flux by numerical integration. These codes use the magnetic field to sustain the quality of their output. To gain access to this high quality magnetic field within our data stream we have monitored Russell's web page for zero levels and timing files (since his data acquisition is not telemetry synchronous) and have a local reconstruction of B for our use. We have also detected a routine anomaly in the magnetometer data stream that we have documented to Chris Russell and developed an editing algorithm to intercept these "hits" and remove them from the geophysical analysis.

Scudder, J. D.↗
NASA NTRS1994

NbN A/D Conversion of IR Focal Plane Sensor Signal at 10 K

We are implementing a 12 bit SFQ counting ADC with parallel-to-serial readout using our established 10 K NbN capability. This circuit provides a key element of the analog signal processor (ASP) used in large infrared focal plane arrays. The circuit processes the signal data stream from a Si:As BIB detector array. A 10 mega samples per second (MSPS) pixel data stream flows from the chip at a 120 megabit bit rate in a format that is compatible with other superconductive time dependent processor (TDP) circuits being developed. We will discuss our planned ASP demonstration, the circuit design, and test results.

conversion↗
DOE OSTI2026

The LCLStream Ecosystem for Multi-Institutional Dataset Exploration

We describe a new end-to-end experimental data streaming framework designed from the ground up to support new types of applications – AI training, extremely high-rate X-ray time-of-flight analysis, crystal structure determination with distributed processing, and custom data science applications and visualizers yet to be created. Throughout, we use design choices merging cloud microservices with traditional HPC batch execution models for security and flexibility. This project makes a unique contribution to the DOE Integrated Research Infrastructure (IRI) landscape. By creating a flexible, API-driven data request service, we address a significant need for high-speed data streaming sources for the X-ray science data analysis community. With the combination of data request API, mutual authentication web security framework, job queue system, high-rate data buffer, and complementary nature to facility infrastructure, the LCLStreamer framework has prototyped and implemented several new paradigms critical for future generation experiments.

Rogers, David [ORNL] (ORCID:0000000251871768)↗
NASA NTRS1983

An Attached Payload Operations Center (APOC) at the Goddard Space Flight Center (GSFC), volume 2

An overview of the APOC is given. For Spacelab payloads channel 2 and 3 data are input via a Statistical Multiplexer (SM) to the various SIPS functions. These include recording of the data on High Density Recorders (HDR), DQM and demultiplexing of the composite data stream by the High Rate Demultiplexer (HRDM). This system performs the inverse functions of the onboard Spacelab High Rate Multiplexer (HRM) enabling access to the data streams as multiplexed onboard the Spacelab. The contents and characteristics of channels one, two and three data as downlinked by the Tracking and Data Relay Satellite System (TDRSS) ku-band are given.

Source record↗
NASA NTRS1989

Aeronautical mobile TDMA/MCTDMA system

A multiple carrier Time Division Multiple Access (TDMA) system capable of supporting voice, stream data, and packet data traffic between aircraft and ground terminals is presented. Demand assignment permits efficient resource sharing for voice and stream data. The bandwidth efficiency of uncoded A Quadrature Phase Shift Keying (AQPSK) is 1 bps/Hz. High time efficiency (approximately equals 83 percent) is achieved through the use of symbol synchronous TDMA. Demodulation is achieved without loop hang-up while requiring only a 16 symbol preamble each burst. A concatenated coding system provides reliable transmission under multipath conditions.

Magill, Thomas↗
NASA NTRS1996

Petabyte Class Storage at Jefferson Lab (CEBAF)

By 1997, the Thomas Jefferson National Accelerator Facility will collect over one Terabyte of raw information per day of Accelerator operation from three concurrently operating Experimental Halls. When post-processing is included, roughly 250 TB of raw and formatted experimental data will be generated each year. By the year 2000, a total of one Petabyte will be stored on-line. Critical to the experimental program at Jefferson Lab (JLab) is the networking and computational capability to collect, store, retrieve, and reconstruct data on this scale. The design criteria include support of a raw data stream of 10-12 MB/second from Experimental Hall B, which will operate the CEBAF (Continuous Electron Beam Accelerator Facility) Large Acceptance Spectrometer (CLAS). Keeping up with this data stream implies design strategies that provide storage guarantees during accelerator operation, minimize the number of times data is buffered allow seamless access to specific data sets for the researcher, synchronize data retrievals with the scheduling of postprocessing calculations on the data reconstruction CPU farms, as well as support the site capability to perform data reconstruction and reduction at the same overall rate at which new data is being collected. The current implementation employs state-of-the-art StorageTek Redwood tape drives and robotics library integrated with the Open Storage Manager (OSM) Hierarchical Storage Management software (Computer Associates, International), the use of Fibre Channel RAID disks dual-ported between Sun Microsystems SMP servers, and a network-based interface to a 10,000 SPECint92 data processing CPU farm. Issues of efficiency, scalability, and manageability will become critical to meet the year 2000 requirements for a Petabyte of near-line storage interfaced to over 30,000 SPECint92 of data processing power.

Chambers, Rita↗
NASA NTRS2008

Algorithms for Spectral Decomposition with Applications to Optical Plume Anomaly Detection

The analysis of spectral signals for features that represent physical phenomenon is ubiquitous in the science and engineering communities. There are two main approaches that can be taken to extract relevant features from these high-dimensional data streams. The first set of approaches relies on extracting features using a physics-based paradigm where the underlying physical mechanism that generates the spectra is used to infer the most important features in the data stream. We focus on a complementary methodology that uses a data-driven technique that is informed by the underlying physics but also has the ability to adapt to unmodeled system attributes and dynamics. We discuss the following four algorithms: Spectral Decomposition Algorithm (SDA), Non-Negative Matrix Factorization (NMF), Independent Component Analysis (ICA) and Principal Components Analysis (PCA) and compare their performance on a spectral emulator which we use to generate artificial data with known statistical properties. This spectral emulator mimics the real-world phenomena arising from the plume of the space shuttle main engine and can be used to validate the results that arise from various spectral decomposition algorithms and is very useful for situations where real-world systems have very low probabilities of fault or failure. Our results indicate that methods like SDA and NMF provide a straightforward way of incorporating prior physical knowledge while NMF with a tuning mechanism can give superior performance on some tests. We demonstrate these algorithms to detect potential system-health issues on data from a spectral emulator with tunable health parameters.

Srivastava, Askok N.↗
DOE OSTI2024

Arroyo Stream Processing Toolset (arroyopy) v0.1.0

Processing event or streaming data presents several technological challenges. A variety of technologies are often used by scientific user facilities. ZMQ is used to stream data and messages in a peer-to-peer fashion. Message brokers like Kafka, Redis Pubsub, EPICS PVA and RabbitMQ are often employed to route and pass messages from instruments to processing workflows. Arroyopy provides an API and structure to flexibly integrate with these tools and incorporate arbitrarily complex processing workflows, letting the hooks to the workflow code be independent of the connection code and hence reusable at a variety of instruments.

Chavez Esparza, Tanny Andrea [Lawrence Berkeley Na↗
NASA NTRS2002

Novel Advancements in Internet-Based Real Time Data Technologies

AZ Technology has been working with MSFC Ground Systems Department to find ways to make it easier for remote experimenters (RPI's) to monitor their International Space Station (ISS) payloads in real-time from anywhere using standard/familiar devices. AZ Technology was awarded an SBIR Phase I grant to research the technologies behind and advancements of distributing live ISS data across the Internet. That research resulted in a product called "EZStream" which is in use on several ISS-related projects. Although the initial implementation is geared toward ISS, the architecture and lessons learned are applicable to other space-related programs. This paper presents the high-level architecture and components that make up EZStream. A combination of commercial-off-the-shelf (COTS) and custom components were used and their interaction will be discussed. The server is powered by Apache's Jakarta-Tomcat web server/servlet engine. User accounts are maintained in a My SQL database. Both Tomcat and MySQL are Open Source products. When used for ISS, EZStream pulls the live data directly from NASA's Telescience Resource Kit (TReK) API. TReK parses the ISS data stream into individual measurement parameters and performs on-the- fly engineering unit conversion and range checking before passing the data to EZStream for distribution. TReK is provided by NASA at no charge to ISS experimenters. By using a combination of well established Open Source, NASA-supplied. and AZ Technology-developed components, operations using EZStream are robust and economical. Security over the Internet is a major concern on most space programs. This paper describes how EZStream provides for secure connection to and transmission of space- related data over the public Internet. Display pages that show sensitive data can be placed under access control by EZStream. Users are required to login before being allowed to pull up those web pages. To enhance security, the EZStream client/server data transmissions can be encrypted to preclude interception. EZStream was developed to make use of a host of standard platforms and protocols. Each are discussed in detail in this paper. The I3ZStream server is written as Java Servlets. This allows different platforms (i.e. Windows, Unix, Linux . Mac) to host the server portion. The EZStream client component is written in two different flavors: JavaBean and ActiveX. The JavaBean component is used to develop Java Applet displays. The ActiveX component is used for developing ActiveX-based displays. Remote user devices will be covered including web browsers on PC#s and scaled-down displays for PDA's and smart cell phones. As mentioned. the interaction between EZStream (web/data server) and TReK (data source) will be covered as related to ISS. EZStream is being enhanced to receive and parse binary data stream directly. This makes EZStream beneficial to both the ISS International Partners and non-NASA applications (i.e. factory floor monitoring). The options for developing client-side display web pages will be addressed along with the development of tools to allow creation of display web pages by non-programmers.

Myers, Gerry↗
NASA NTRS2012

Real-Time Cognitive Computing Architecture for Data Fusion in a Dynamic Environment

A novel cognitive computing architecture is conceptualized for processing multiple channels of multi-modal sensory data streams simultaneously, and fusing the information in real time to generate intelligent reaction sequences. This unique architecture is capable of assimilating parallel data streams that could be analog, digital, synchronous/asynchronous, and could be programmed to act as a knowledge synthesizer and/or an "intelligent perception" processor. In this architecture, the bio-inspired models of visual pathway and olfactory receptor processing are combined as processing components, to achieve the composite function of "searching for a source of food while avoiding the predator." The architecture is particularly suited for scene analysis from visual data and odorant.

Duong, Tuan A.↗
NASA NTRS2011

Binary-Signal Recovery

Binary communication through long cables, opto-isolators, isolating transformers, or repeaters can become distorted in characteristic ways. The usual solution is to slow the communication rate, change to a different method, or improve the communication media. It would help if the characteristic distortions could be accommodated at the receiving end to ease the communication problem. The distortions come from loss of the high-frequency content, which adds slopes to the transitions from ones to zeroes and zeroes to ones. This weakens the definition of the ones and zeroes in the time domain. The other major distortion is the reduction of low frequency, which causes the voltage that defines the ones or zeroes to drift out of recognizable range. This development describes a method for recovering a binary data stream from a signal that has been subjected to a loss of both higher-frequency content and low-frequency content that is essential to define the difference between ones and zeroes. The method makes use of the frequency structure of the waveform created by the data stream, and then enhances the characteristics related to the data to reconstruct the binary switching pattern. A major issue is simplicity. The approach taken here is to take the first derivative of the signal and then feed it to a hysteresis switch. This is equivalent in practice to using a non-resonant band pass filter feeding a Schmitt trigger. Obviously, the derivative signal needs to be offset to halfway between the thresholds of the hysteresis switch, and amplified so that the derivatives reliably exceed the thresholds. A transition from a zero to a one is the most substantial, fastest plus movement of voltage, and therefore will create the largest plus first derivative pulse. Since the quiet state of the derivative is sitting between the hysteresis thresholds, the plus pulse exceeds the plus threshold, switching the hysteresis switch plus, which re-establishes the data zero to one transition, except now at the logic levels of the receiving circuit. Similarly, a transition from a one to a zero will be the most substantial and fastest minus movement of voltage and therefore will create the largest minus first derivative pulse. The minus pulse exceeds the minus threshold, switching the hysteresis switch minus, which re-establishes the data one to zero transition. This innovation has a large increase in tolerance for the degradation of the binary pattern of ones and zeroes, and can reject the introduction of noise in the form of low frequencies that can cause the voltage pattern to drift up or down, and also higher frequencies that are beyond the recognizable content in the binary transitions.

Griebeler, Elmer L.↗
NASA NTRS2017

Interactive Visualization of High-Dimensional Petascale Ocean Data

We describe an application for interactive visualization of 5 petabytes of time-varying multivariate data from a high-resolution global ocean circulation model. The input data are 10311 hourly (ocean time) time steps of various 2D and 3D fields from a 22-billion point 1/48- degree “lat-lon cap” configuration of the MIT General Circulation Model (MITgcm). We map the global horizontal model domain onto our 128-screen (8x16) tiled display wall to produce a canonical tiling with approximately one MITgcm grid point per display pixel, and using this tiling we encode the entire time series for multiple native and computed scalar quantities at a collection of ocean depths. We reduce disk bandwidth requirements by converting the model’s floating point data to 16-bit fixed point values, and compressing those values with a lossless video encoder, which together allow synchronized playback at 24 time steps per second across all 128 displays. The application allows dynamic assignment of any two encoded tiles to any display, and has multiple interfaces for quickly specifying various orderly arrangements of tiles. All subsequent rendering is done on the fly, with run time control of colormaps, transfer functions, histogram equalization, and labeling. The two data streams on each screen can be rendered independently and combined in various ways, including blending, differencing, horizontal/ vertical wipes, and checkerboarding. The two data streams on any screen can optionally be displayed as a scatterplot in their joint attribute space. All scatterplots and map-view plots from the same x/y location and depth are linked so they all show the current brushable selection. Ocean scientists have used the system, and have found previously unidentified features in the data.

Interactive↗
NASA NTRS2012

Wireless Self-Acquistion of 12-Lead ECG via Android Smart Phone

Researchers at NASA s Johnson Space Center and at Orbital Research, Inc. (a NASA SBIR grant recipient) have recently developed a dry-electrode harness that allows for self-acquisition of resting 12-lead ECGs by minimally trained laypersons. When used in conjunction with commercial wireless (e.g., Bluetooth(TM) or 802.11-enabled) 12-lead ECG devices and custom smart phone-based software, the collected 12-lead ECG data can also immediately be forwarded from any geographic location within cellular range to the user s physician(s) of choice. The system can also be used to immediately forward to central receiving stations 12-lead ECG data collected during space flight or during activities in any remote terrestrial location supported by an internet or cellular phone infrastructure. The main novel aspects of the system are first, the dry-electrode 12-lead ECG harness itself, and second, an accompanying Android(TM) smart phone-based wireless 12-lead ECG capability. The ECG harness nominally employs dry electrodes manufactured by Orbital Research, Inc, recently cleared through the Food and Drug Administration (FDA). However, other dry electrodes that are not yet FDA cleared, for example those recently developed by Nanosonic, Inc as part of another NASA SBIR grant, can also be used. The various advantageous features of the harness include: 1) laypersons can be quickly instructed on its correct use, remotely if necessary; 2) all tangled "leadwire spaghetti" is eliminated, as is the common clinical problem of "leadwire reversal"; 3) all adhesives and disposables are also eliminated, the harness being fully reusable; if multiple individuals intend to use use the same harness, then standard antimicrobial wipes can be employed to sterilize the dry electrodes (and harness surface if needed) between users; 5) padded cushions at the lateral sides of the torso function to press the left arm (LA) and right arm (RA) dry electrodes mounted on the cushions against sideward or downward-rested arms of the subject; 6) sufficient distal placement of the arm electrodes achieves good electrode abutment to the arms without the need for adhesives, straps, bands, bracelets, or gloves; 7) padding over the sternum avoids "tenting" in the V1 through V3 (and, when present, the V3R) electrode positions; 8) easy-to-don, one-piece design with an adjustable, front-side single point of connection and an adjustable shoulder strap; and 9) Lund or "modified Lund" placement of the dry electrodes, the results of which more effectively reproduce results from "standard" 12-lead ECG placements than do results from Mason-Likar placements. The main limitation of the harness is that "one size does not fit all", meaning that an appropriately sized harness (small, medium, large, etc) must be chosen on the basis of an individual's size. To facilitate the use of the harness with inexpensive, commodity-grade cell phones and tablet devices, 12-lead ECG software is also being developed to accompany the harness for wireless use with Android. For this part of the project, NASA has teamed with TopCoder, Inc and the Harvard-affiliated NASA Tournament Lab in sponsoring java-based software programming contests through TopCoder. While ECG signals from the harness can already be wirelessly received and thoroughly processed (locally or remotely) by commercial-grade conventional (as well as advanced) 12-lead ECG software running on Microsoft Windows(TM), the Android-based software, once completed, will "cast a wider net" by allowing for a greater percentage of cell phone owners to participate in inexpensive, store-and-forward recordings of 12-lead ECGs worldwide, including for example Android cell phone users in many remote, third-world locations. At the time of writing, the Android 12-lead ECG software platform consists of a basic but expanding graphical user interface and accompanying software that: 1) wirelessly receives the 12-lead ECG data stream from a Bluetooth-based, FDA-cleared 12-leaCG device attached to the harness; 2) locally stores the same data in binary format to the SD card on the Android cell phone; and 3) makes the data stream in available in real time, for now to TopCoder's java programming contestants.

Schlegel, Todd T.↗
NASA NTRS1995

A packet data compressor

We are in the preliminary stages of creating an operational system for losslessly compressing packet data streams. The end goal is to reduce costs. Real world constraints include transmission in the presence of error, tradeoffs between the costs of compression and the costs of transmission and storage, and imperfect knowledge of the data streams to be transmitted. The overall method is to bring together packets of similar type, split the data into bit fields, and test a large number of compression algorithms. Preliminary results are very encouraging, typically offering compression factors substantially higher than those obtained with simpler generic byte stream compressors, such as Unix Compress and HA 0.98.

Grunes, Mitchell R.↗