ViStream : Distributed Streaming Framework using Vivaldi

Abstract

Advances in data acquisition techniques have resulted in a volume of data that is considerably larger than that accumulated in the past. The influx of data has given rise to a new trend of processing big data through the use of streaming framework. In this paper, we pro- pose a system called ViStream, which executes applications on out-of-core data in Vivaldi. Vivaldi is a domain specific language for heterogeneous distributed systems. We improved the language to enable the rendering of primitives through the use of OpenGL and tested applications that can be processed on out-of-core data. Specifically, three applications were validated. First, we executed an application that uses streaming, that is, histogram matching for 1.3 TB of zebrafish data. Second, we ran combined visualization with volume and primitive rendering. Finally, we executed an application that shows only projected volumetric data on a 3D skeletal representation of zebrafish.