FENCE: Fast, ExteNsible, and ConsolidatEd Framework for Intelligent Big Data Processing

Abstract

The proliferation of smart devices and the advancement of data-intensive services has led to explosion of data, which uncovers massive opportunities as well as challenges related to real-time analysis of big data streams. The edge computing frameworks implemented over manycore systems can be considered as a promising solution to address these challenges. However, in spite of the availability of modern computing systems with a large number of processing cores and high memory capacity, the performance and scalability of manycore systems can be limited by the software and operating system (OS) level bottlenecks. In this work, we focus on these challenges, and discuss how accelerated communication, efficient caching, and high performance computation can be provisioned over manycore systems. The proposed Fast, ExteNsible, and ConsolidatEd (FENCE) framework leverages the availability of a large number of computing cores and overcomes the OS level bottlenecks to provide high performance and scalability for intelligent big data processing. We implemented a prototype of FENCE and the experiment results demonstrate that FENCE provides improved data reception throughput, read/write throughput, and application processing performance as compared to the baseline Linux system.