Logical Optimisation and Cost Modelling of Stream-Processing Programs Written in a Purely-Functional Framework

Dowland, J; Watson, P; Cattermole, A

doi:10.1109/UCC56403.2022.00048

Logical Optimisation and Cost Modelling of Stream-Processing Programs Written in a Purely-Functional Framework

Lookup NU author(s): Jonathan Dowland ORCiD, Professor Paul Watson ORCiD, Dr Adam Cattermole

Downloads

Accepted version [.pdf]

Licence

This is the authors' accepted manuscript of a conference proceedings (inc. abstract) that has been published in its final definitive form by IEEE/ACM, 2022.

For re-use rights please refer to the publisher's terms and conditions.

Abstract

We present a vision for the automatic optimisation of distributed stream processing programs. StrIoT --- a distributed stream-processing framework built using purely-functional programming --- enables a set of validated logical optimisation rules to generate a set of possible deployment plans. A cost model then filters and ranks the plans before the best is automatically deployed across the cloud and edge devices. We describe StrIoT's functional operators for writing stream-processing programs; the design, implementation and performance of StrIoT's logical optimiser; and the cost model, which filters and ranks re-written programs and deployment plans in terms of two non-functional requirements: bandwidth and cost. The StrIoT vision is being explored through an open-source proof-of-concept implementation. We present our initial results with a motivating example before outlining the success criteria for future work in this area.