DOE OSTI · code-175190
Arroyo Stream Processing Toolset (arroyopy) v0.1.0
Abstract
Processing event or streaming data presents several technological challenges. A variety of technologies are often used by scientific user facilities. ZMQ is used to stream data and messages in a peer-to-peer fashion. Message brokers like Kafka, Redis Pubsub, EPICS PVA and RabbitMQ are often employed to route and pass messages from instruments to processing workflows. Arroyopy provides an API and structure to flexibly integrate with these tools and incorporate arbitrarily complex processing workflows, letting the hooks to the workflow code be independent of the connection code and hence reusable at a variety of instruments.
Keep this discovery
Explore connections, maps & timelines
Chavez Esparza, Tanny Andrea [Lawrence Berkeley National Laboratory (LBNL), Berkeley, CA (United States)], McReynolds, Dylan [Lawrence Berkeley National Laboratory (LBNL), Berkeley, CA (United States)], Koepp, Wiebke [Lawrence Berkeley National Laboratory (LBNL), Berkeley, CA (United States)]. 2024-12-23. Arroyo Stream Processing Toolset (arroyopy) v0.1.0. https://doi.org/10.11578/dc.20260211.1
Cite the original work for its findings. Save a collection to share your selection of sources.