Enhancing LSTM for sequential image classification by modifying data aggregation

Abstract

Recurrent Neural Networks (RNN) model sequential information and are commonly used for the analysis of time series. The most usual operation to fuse information in RNNs is the sum. In this work, we use a RNN extended type, Long Short-Term Memory (LSTM) and we use it for image classification, to which we give a sequential interpretation. Since the data used may not be independent to each other, we modify the sum operator of an LSTM unit using the n-dimensional Choquet integral, which considers possible data coalitions. We compare our methods to those based on usual aggregation functions, using the datasets Fashion-MNIST and MNIST.