Open Access
Reemplazo de la función de pooling de redes neuronales convolucionales por combinaciones lineales de funciones crecientes
(Universidad de Málaga, 2021) Rodríguez Martínez, Iosu; Lafuente López, Julio; Sesma Sara, Mikel; Herrera, Francisco; Ursúa Medrano, Pablo; Bustince Sola, Humberto; Estadística, Informática y Matemáticas; Estatistika, Informatika eta Matematika
Las redes convolucionales llevan a cabo un proceso automatico de extracción y fusión de características mediante el cual obtienen la información más relevante de una imagen dada. El proceso de submuestreo mediante el cual se fusionan características localmente próximas, conocido como ‘pooling’, se lleva a cabo tradicionalmente con funciones sencillas como el máximo o la media aritmética, ignorando otras opciones muy populares en el campo de la teoría de agregaciones. En este trabajo proponemos reemplazar dichas funciones por otra serie de ordenes estadísticos, así como por la integral de Sugeno y una nueva generalización de la misma. Además, basándonos en trabajos que emplean la combinación convexa del máximo y la media, presentamos una nueva capa que permite combinar varias de las nuevas agregaciones, mejorando sus resultados individuales.
Open Access
Generalizing max pooling via (a, b)-grouping functions for convolutional neural networks
(Elsevier, 2023) Rodríguez Martínez, Iosu; Da Cruz Asmus, Tiago; Pereira Dimuro, Graçaliz; Herrera, Francisco; Takáč, Zdenko; Bustince Sola, Humberto; Estadística, Informática y Matemáticas; Estatistika, Informatika eta Matematika; Institute of Smart Cities - ISC; Universidad Pública de Navarra / Nafarroako Unibertsitate Publikoa
Due to their high adaptability to varied settings and effective optimization algorithm, Convolutional Neural Networks (CNNs) have set the state-of-the-art on image processing jobs for the previous decade. CNNs work in a sequential fashion, alternating between extracting significant features from an input image and aggregating these features locally through ‘‘pooling" functions, in order to produce a more compact representation. Functions like the arithmetic mean or, more typically, the maximum are commonly used to perform this downsampling operation. Despite the fact that many studies have been devoted to the development of alternative pooling algorithms, in practice, ‘‘max-pooling" still equals or exceeds most of these possibilities, and has become the standard for CNN construction. In this paper we focus on the properties that make the maximum such an efficient solution in the context of CNN feature downsampling and propose its replacement by grouping functions, a family of functions that share those desirable properties. In order to adapt these functions to the context of CNNs, we present (𝑎, 𝑏)- grouping functions, an extension of grouping functions to work with real valued data. We present different construction methods for (𝑎, 𝑏)-grouping functions, and demonstrate their empirical applicability for replacing max-pooling by using them to replace the pooling function of many well-known CNN architectures, finding promising results.
Open Access
Generalizando el pooling maximo por funciones (a, b)-grouping en redes neuronales convolucionales
(CAEPIA, 2024) Rodríguez Martínez, Iosu; Da Cruz Asmus, Tiago; Pereira Dimuro, Graçaliz; Herrera, Francisco; Takáč, Zdenko; Bustince Sola, Humberto; Estadística, Informática y Matemáticas; Estatistika, Informatika eta Matematika; Institute of Smart Cities - ISC; Gobierno de Navarra / Nafarroako Gobernua
Este artículo es un resumen del trabajo publicado en la revista Information Fusion [1]. En este artículo explorábamos el reemplazo del operador de pooling máximo comunmente empleado en redes neuronales convolucionales (CNNs) por funciones (a, b)-grouping. Estas funciones extienden el concepto de función de grouping clásica [2] a un intervalo cerrado [a, b], siguiendo la filosofía de [3]. En el contexto del operador de pooling, estas nuevas funciones ayudan a la optimización de los modelos suavizando los gradientes en el proceso de retropropagación y obteniendo resultados competitivos con métodos más complejos
Embargo
Extremal values-based aggregation functions
(Elsevier, 2024-10-01) Halaš, Radomír; Mesiar, Radko; Kolesárová, Anna; Saadati, Reza; Herrera, Francisco; Rodríguez Martínez, Iosu; Bustince Sola, Humberto; Estadística, Informática y Matemáticas; Estatistika, Informatika eta Matematika; Institute of Smart Cities - ISC
We introduce and study aggregation functions based on extremal values, namely extended (𝑙, 𝑢)- aggregation functions whose outputs only depend on a fixed number 𝑙 of extremal lower input values and a fixed number 𝑢 of extremal upper input values, independently of the arity of the input 𝑛-tuples (𝑛 ≥ 𝑙 + 𝑢). We discuss several general properties of (𝑙, 𝑢)-aggregation functions and we study special (𝑙, 𝑢)-aggregation functions with neutral element, including t-conorms, t-norms and uninorms. We also study (𝑙, 𝑢)-aggregation functions defined by means of integrals with respect to discrete fuzzy measures, as well as (𝑙, 𝑢)-ordered weighted quasi-arithmetic means based on appropriate weighting vectors. We also stress some generalizations based on recently introduced new types of monotonicity. Some possible applications are sketched, too.
Open Access
Replacing pooling functions in convolutional neural networks by linear combinations of increasing functions
(Elsevier, 2022) Rodríguez Martínez, Iosu; Lafuente López, Julio; Santiago, Regivan; Pereira Dimuro, Graçaliz; Herrera, Francisco; Bustince Sola, Humberto; Estadística, Informática y Matemáticas; Estatistika, Informatika eta Matematika; Gobierno de Navarra / Nafarroako Gobernua
Traditionally, Convolutional Neural Networks make use of the maximum or arithmetic mean in order to reduce the features extracted by convolutional layers in a downsampling process known as pooling. However, there is no strong argument to settle upon one of the two functions and, in practice, this selection turns to be problem dependent. Further, both of these options ignore possible dependencies among the data. We believe that a combination of both of these functions, as well as of additional ones which may retain different information, can benefit the feature extraction process. In this work, we replace traditional pooling by several alternative functions. In particular, we consider linear combinations of order statistics and generalizations of the Sugeno integral, extending the latter¿s domain to the whole real line and setting the theoretical base for their application. We present an alternative pooling layer based on this strategy which we name ¿CombPool¿ layer. We replace the pooling layers of three different architectures of increasing complexity by CombPool layers, and empirically prove over multiple datasets that linear combinations outperform traditional pooling functions in most cases. Further, combinations with either the Sugeno integral or one of its generalizations usually yield the best results, proving a strong candidate to apply in most architectures.