Replacing pooling functions in convolutional neural networks by linear combinations of increasing functions
Fecha
2022Autor
Versión
Acceso abierto / Sarbide irekia
Tipo
Artículo / Artikulua
Versión
Versión publicada / Argitaratu den bertsioa
Identificador del proyecto
Impacto
|
10.1016/j.neunet.2022.04.028
Resumen
Traditionally, Convolutional Neural Networks make use of the maximum or arithmetic mean in order to reduce the features extracted by convolutional layers in a downsampling process known as pooling. However, there is no strong argument to settle upon one of the two functions and, in practice, this selection turns to be problem dependent. Further, both of these options ignore possible dependencies ...
[++]
Traditionally, Convolutional Neural Networks make use of the maximum or arithmetic mean in order to reduce the features extracted by convolutional layers in a downsampling process known as pooling. However, there is no strong argument to settle upon one of the two functions and, in practice, this selection turns to be problem dependent. Further, both of these options ignore possible dependencies among the data. We believe that a combination of both of these functions, as well as of additional ones which may retain different information, can benefit the feature extraction process. In this work, we replace traditional pooling by several alternative functions. In particular, we consider linear combinations of order statistics and generalizations of the Sugeno integral, extending the latter¿s domain to the whole real line and setting the theoretical base for their application. We present an alternative pooling layer based on this strategy which we name ¿CombPool¿ layer. We replace the pooling layers of three different architectures of increasing complexity by CombPool layers, and empirically prove over multiple datasets that linear combinations outperform traditional pooling functions in most cases. Further, combinations with either the Sugeno integral or one of its generalizations usually yield the best results, proving a strong candidate to apply in most architectures. [--]
Materias
Convolutional Neural Networks,
Pooling function,
Order statistic,
Generalized Sugeno integral
Editor
Elsevier
Publicado en
Neural Networks, 2022, 152, pp. 380-393
Departamento
Universidad Pública de Navarra. Departamento de Estadística, Informática y Matemáticas /
Nafarroako Unibertsitate Publikoa. Estatistika, Informatika eta Matematika Saila
Versión del editor
Entidades Financiadoras
The authors gratefully acknowledge the financial support of Tracasa Instrumental (iTRACASA), Spain and of the Gobierno de Navarra - Departamento de Universidad, Innovación y Transformación Digital, Spain, as well as that of the Spanish Ministry of Science, Spain (project PID2019-108392GB-I00 (AEI/10.13039/ 501100011033)) and the project PC095-096 FUSIPROD. F. Herrera is supported by the Andalusian Excellence project, Spain P18-FR-4961. G.P. Dimuro is supported by CNPq, Brazil (301618/2019-4) and FAPERGS, Brazil (19/2551-0001279-9).