Network simulation in a TCP-enabled industrial internet of things environment - reproducibility issues for performance evaluation

Abstract

Network simulation is a tool used to analyse and predict the performance of Industrial Internet of Things deployments while dealing with the complexity of real testbeds. Large network deployments with complex protocols such as Transmission Control Protocol are subject to chaos-theory behaviour, i.e. small changes in the implementation of the protocol stack or simulator behaviour may result in large differences in the performance results. We present the results of simulating two different scenarios using three simulators. The first scenario focuses on the Incast phenomenon in a local area network where sensor data are collected. The second scenario focuses on a congested link traversed by the collected measurements. The performance metrics obtained from the simulators are compared among them and with ground-truth obtained from real network experiments. The results demonstrate how subtle implementation differences in network simulators impact performance results, and how network engineers must consider these differences.