Open Access
Generalizing max pooling via (a, b)-grouping functions for convolutional neural networks
(Elsevier, 2023) Rodríguez Martínez, Iosu; Da Cruz Asmus, Tiago; Pereira Dimuro, Graçaliz; Herrera, Francisco; Takáč, Zdenko; Bustince Sola, Humberto; Estadística, Informática y Matemáticas; Estatistika, Informatika eta Matematika; Institute of Smart Cities - ISC; Universidad Pública de Navarra / Nafarroako Unibertsitate Publikoa
Due to their high adaptability to varied settings and effective optimization algorithm, Convolutional Neural Networks (CNNs) have set the state-of-the-art on image processing jobs for the previous decade. CNNs work in a sequential fashion, alternating between extracting significant features from an input image and aggregating these features locally through ‘‘pooling" functions, in order to produce a more compact representation. Functions like the arithmetic mean or, more typically, the maximum are commonly used to perform this downsampling operation. Despite the fact that many studies have been devoted to the development of alternative pooling algorithms, in practice, ‘‘max-pooling" still equals or exceeds most of these possibilities, and has become the standard for CNN construction. In this paper we focus on the properties that make the maximum such an efficient solution in the context of CNN feature downsampling and propose its replacement by grouping functions, a family of functions that share those desirable properties. In order to adapt these functions to the context of CNNs, we present (𝑎, 𝑏)- grouping functions, an extension of grouping functions to work with real valued data. We present different construction methods for (𝑎, 𝑏)-grouping functions, and demonstrate their empirical applicability for replacing max-pooling by using them to replace the pooling function of many well-known CNN architectures, finding promising results.
Open Access
Generalizando el pooling maximo por funciones (a, b)-grouping en redes neuronales convolucionales
(CAEPIA, 2024) Rodríguez Martínez, Iosu; Da Cruz Asmus, Tiago; Pereira Dimuro, Graçaliz; Herrera, Francisco; Takáč, Zdenko; Bustince Sola, Humberto; Estadística, Informática y Matemáticas; Estatistika, Informatika eta Matematika; Institute of Smart Cities - ISC; Gobierno de Navarra / Nafarroako Gobernua
Este artículo es un resumen del trabajo publicado en la revista Information Fusion [1]. En este artículo explorábamos el reemplazo del operador de pooling máximo comunmente empleado en redes neuronales convolucionales (CNNs) por funciones (a, b)-grouping. Estas funciones extienden el concepto de función de grouping clásica [2] a un intervalo cerrado [a, b], siguiendo la filosofía de [3]. En el contexto del operador de pooling, estas nuevas funciones ayudan a la optimización de los modelos suavizando los gradientes en el proceso de retropropagación y obteniendo resultados competitivos con métodos más complejos
Open Access
Extensión multidimensional de la integral de Choquet discreta y su aplicación en redes neuronales recurrentes
(Universidad de Málaga, 2021) Ferrero Jaurrieta, Mikel; Rodríguez Martínez, Iosu; Pereira Dimuro, Graçaliz; Fernández Fernández, Francisco Javier; Bustince Sola, Humberto; Estadística, Informática y Matemáticas; Estatistika, Informatika eta Matematika
En este trabajo presentamos una definición de la integral de Choquet discreta n-dimensional, para fusionar datos vectoriales. Como aplicación, utilizamos estas nuevas integrales de Choquet discretas multidimensionales en la fusión de información secuencial en las redes neuronales recurrentes, mejorando los resultados obtenidos mediante el método de agregación tradicional.
Open Access
Replacing pooling functions in convolutional neural networks by linear combinations of increasing functions
(Elsevier, 2022) Rodríguez Martínez, Iosu; Lafuente López, Julio; Santiago, Regivan; Pereira Dimuro, Graçaliz; Herrera, Francisco; Bustince Sola, Humberto; Estadística, Informática y Matemáticas; Estatistika, Informatika eta Matematika; Gobierno de Navarra / Nafarroako Gobernua
Traditionally, Convolutional Neural Networks make use of the maximum or arithmetic mean in order to reduce the features extracted by convolutional layers in a downsampling process known as pooling. However, there is no strong argument to settle upon one of the two functions and, in practice, this selection turns to be problem dependent. Further, both of these options ignore possible dependencies among the data. We believe that a combination of both of these functions, as well as of additional ones which may retain different information, can benefit the feature extraction process. In this work, we replace traditional pooling by several alternative functions. In particular, we consider linear combinations of order statistics and generalizations of the Sugeno integral, extending the latter¿s domain to the whole real line and setting the theoretical base for their application. We present an alternative pooling layer based on this strategy which we name ¿CombPool¿ layer. We replace the pooling layers of three different architectures of increasing complexity by CombPool layers, and empirically prove over multiple datasets that linear combinations outperform traditional pooling functions in most cases. Further, combinations with either the Sugeno integral or one of its generalizations usually yield the best results, proving a strong candidate to apply in most architectures.
Open Access
On construction methods of (interval-valued) general grouping functions
(Springer, 2022) Pereira Dimuro, Graçaliz; Da Cruz Asmus, Tiago; Pinheiro, Jocivania; Santos, Helida; Borges, Eduardo N.; Lucca, Giancarlo; Rodríguez Martínez, Iosu; Mesiar, Radko; Bustince Sola, Humberto; Estadística, Informática y Matemáticas; Estatistika, Informatika eta Matematika
Recently, several theoretical and applied studies on grouping functions and overlap functions appeared in the literature, mainly because of their flexibility when comparing them with the popular aggregation operators t-conorms and t-norms, respectively. Additionally, they constitute richer classes of disjunction/conjunction operations than t-norms and t-conorms. In particular, grouping functions have been applied as the disjunction operator in several problems, like decision making based on fuzzy preference relations. In this case, when performing pairwise comparisons, grouping functions allow one to evaluate the measure of the amount of evidence in favor of either of two given alternatives. However, grouping functions are not associative. Then, in order to allow them to be applied in n-dimensional problems, such as the pooling layer of neural networks, some generalizations were introduced, namely, n-dimensional grouping functions and the more flexible general grouping functions, the latter for enlarging the scope of applications. Then, in order to h andle uncertainty on the definition of the membership functions in real-life problems, n-dimensional and general interval-valued grouping functions were proposed. This paper aims at providing new constructions methods of general (interval-valued) grouping functions, also providing some examples.