A scalable and flexible Open Source Big Data architecture for small and medium-sized enterprises

Abstract

The advancements of Big Data, Internet of Things and Artificial Intelligence are causing the industrial revolution known as Industry 4.0. For automated factories, adopting the necessary technologies for its implementation involves a series of challenges such as the lack of a proper infrastructure, financial limitations, coordination problems or a low understanding of Industry 4.0 implications. Additionally, many implementations focus on solving specific problems without taking other future or parallel projects into account, leading to continuous restructuring and increased complexity, that is, increasing costs. A lack of a global view when implementing Industry 4.0 solutions can cause difficulties in its adoption, leading to future problems that may be unaffordable for Small and Medium-sized Enterprises (SMEs). Traditional Big Data architectures offer remarkable solutions to complex data issues, but do not cover the complete flow of information that is required in Industry 4.0 applications. Therefore, there is a need to create solutions for the difficulties that this new digital transformation brings to avoid future problems, making it affordable also for SMEs. In this work we propose a flexible and scalable Big Data architecture that is well-suited for SMEs with automated factories, taking the aforementioned difficulties into account.