Survey of big data programming frameworks

Abstract

Advances in ICT today has made data more voluminous and multifarious and its being transferred at high speed. Applications in cloud like Yahoo weather, Facebook photo gallery, and Google search index are changing the IT landscape in a profound way. Reasons for these trend includes scientific organizations solving big problems related to high performance computing workloads, diverse public services being digitized; mobile devices, global positioning systems, sensors, social media, medical imaging, financial transaction logs and lots of them are all sources of massive data generating large sets of complex data. These applications are evolving to be data-intensive, processing very large volumes of data hence, require dynamically scalable, virtualized resources to handle them. This paper compared several of these big data programming frameworks used in data-intensive processes. Area of strengths and weaknesses of these frameworks were highlighted with the intention that future framework will harness all positives from this study for a better system.

Keywords: Big Data, Framework, Hadoop, Analytics, Workload, Resource Allocation