Related Researcher

Author's Photo

Jeon, Myeongjae
Research Interests
  • Parallel/distributed processing of deep learning workloads, Real-time stream data analytics at cloud/IoT scale, Public/private blockchain

StreamBox: Modern Stream Processing on a Multicore Machine

Cited 0 times inthomson ciCited 0 times inthomson ci
StreamBox: Modern Stream Processing on a Multicore Machine
Miao, HongyuPark, HeejinJeon, MyeongjaePekhimenko, GennadyMcKinley, Kathryn S.Lin, Felix Xiaozhu
Issue Date
USENIX Annual Technocal Conference
Stream analytics on real-time events has an insatiable demand for throughput and latency. Its performance on a single machine is central to meeting this demand, even in a distributed system. This paper presents a novel stream processing engine called StreamBox that exploits the parallelism and memory hierarchy of modern multicore hardware. StreamBox executes a pipeline of transforms over records that may arrive out-of-order. As records arrive, it groups the records into ordered epochs delineated by watermarks. A watermark guarantees no subsequent record’s event timestamp will precede it. Our contribution is to produce and manage abundant parallelism by generalizing out-of-order record processing within each epoch to out-of-order epoch processing and by dynamically prioritizing epochs to optimize latency. We introduce a data structure called cascading containers, which dynamically manages concurrency and dependences among epochs in the transform pipeline. StreamBox creates sequential memory layout of records in epochs and steers them to optimize NUMA locality. On a 56-core machine, StreamBox processes records up to 38 GB/sec (38M Records/sec) with 50 ms latency.
Appears in Collections:
CSE_Conference Papers
Files in This Item:
There are no files associated with this item.

find_unist can give you direct access to the published full text of this article. (UNISTARs only)

Show full item record


  • mendeley


Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.