Adaptive multi-sensor, multi-actuator media workflow system for interactive arts

We are presenting the Architecture for Interactive Arts (ARIA) middleware architecture for sensory/reactive environments, such as interactive performances. ARIA processes, filters, and fuses sensory inputs and actuates responses in real-time while providing various service guarantees. ARIA is deployed on a centralized stream processor that monitors and queries over distributed media sensors and outputs of the media workflows are sent to actuators with video and audio presentation capabilities. Due to rapid updates and the continuous nature of sensory data streams, approximation and adaptation techniques are necessary to optimize resource usage. We propose a mechanism that adapts media workflows by adjusting parameters of filters and fusion operators that dynamically control the speed of data streams and the sampling rate. In [1], we developed the ARIA middleware model, including the operators, performance models, and the optimization techniques for chain structured (single-sensor, single-actuator) media workflows. In this paper, we focus on development of flow optimization techniques in adaptive multi-sensor, multi-actuator workflow system. We experimentally evaluate the efficiency and the effectiveness of the algorithms.