Non-symmetric over-time pooling using pseudo-grouping functions for convolutional neural networks
| dc.contributor.author | Ferrero Jaurrieta, Mikel | |
| dc.contributor.author | Paiva, Rui | |
| dc.contributor.author | Cruz, Anderson | |
| dc.contributor.author | Bedregal, Benjamin | |
| dc.contributor.author | Miguel Turullols, Laura de | |
| dc.contributor.author | Takáč, Zdenko | |
| dc.contributor.author | López Molina, Carlos | |
| dc.contributor.author | Bustince Sola, Humberto | |
| dc.contributor.department | Estadística, Informática y Matemáticas | es_ES |
| dc.contributor.department | Estatistika, Informatika eta Matematika | eu |
| dc.contributor.department | Institute of Smart Cities - ISC | en |
| dc.date.accessioned | 2024-05-07T09:11:27Z | |
| dc.date.issued | 2024-07-01 | |
| dc.date.updated | 2024-05-07T08:57:26Z | |
| dc.description.abstract | Convolutional Neural Networks (CNNs) are a family of networks that have become state-of-the-art in several fields of artificial intelligence due to their ability to extract spatial features. In the context of natural language processing, they can be used to build text classification models based on textual features between words. These networks fuse local features to generate global features in their over-time pooling layers. These layers have been traditionally built using the maximum function or other symmetric functions such as the arithmetic mean. It is important to note that the order of input local features is significant (i.e. the symmetry is not an inherent characteristic of the model). While this characteristic is appropriate for image-oriented CNNs, where symmetry might make the network robust to image rigid transformations, it seems counter-productive for text processing, where the order of the words is certainly important. Our proposal is, hence, to use non-symmetric pooling operators to replace the maximum or average functions. Specifically, we propose to perform over-time pooling using pseudo-grouping functions, a family of non-symmetric aggregation operators that generalize the maximum function. We present a construction method for pseudo-grouping functions and apply different examples of this family to over-time pooling layers in text-oriented CNNs. Our proposal is tested on seven different models and six different datasets in the context of engineering applications, e.g. text classification. The results show an overall improvement of the models when using non-symmetric pseudo-grouping functions over the traditional pooling function. | en |
| dc.description.sponsorship | The authors acknowledge with thanks to their universities. Furthermore, this work was supported by the Brazilian funding agency CNPq (Brazilian Research Council) under Projects 311429/2020-3 and 200282/2022-0, by the project PID2022-136627NB-I00 founded by MCIN/AEI/10.13039/501100011033/FEDER, UE, of the Spanish Government, Project VEGA 1/0193/22 and by Tracasa Instrumental and the Immigration Policy and Justice Department of the Government of Navarre. | en |
| dc.embargo.inicio | 2024-05-07 | |
| dc.embargo.lift | 2026-07-01 | |
| dc.embargo.terms | 2026-07-01 | |
| dc.format.mimetype | application/pdf | en |
| dc.identifier.citation | Ferrero-Jaurrieta, M., Paiva, R., Cruz, A., Bedregal, B., De Miguel, L., Takac, Z., Lopez-Molina, C., Bustince, H. (2024) Non-symmetric over-time pooling using pseudo-grouping functions for convolutional neural networks. Engineering Applications of Artificial Intelligence, 133(E), 108470-108470. https://doi.org/10.1016/j.engappai.2024.108470. | es_ES |
| dc.identifier.doi | 10.1016/j.engappai.2024.108470 | |
| dc.identifier.issn | 0952-1976 | |
| dc.identifier.uri | https://academica-e.unavarra.es/handle/2454/48073 | |
| dc.language.iso | eng | en |
| dc.publisher | Elsevier | en |
| dc.relation.ispartof | Engineering Applications of Artificial Intelligence (2024), vol. 133(E), 108470 | es_ES |
| dc.relation.projectID | info:eu-repo/grantAgreement/AEI/Plan Estatal de Investigación Científica y Técnica y de Innovación 2021-2023/PID2022-136627NB-I00/ES/ | |
| dc.relation.projectID | info:eu-repo/grantAgreement/Gobierno de Navarra//1%2F0193%2F22/ | |
| dc.relation.publisherversion | https://doi.org/10.1016/j.engappai.2024.108470 | |
| dc.rights | © 2024 Elsevier Ltd. This manuscript version is made available under the CC-BY-NC-ND 4.0 | es_ES |
| dc.rights.accessRights | info:eu-repo/semantics/embargoedAccess | |
| dc.rights.uri | https://creativecommons.org/licenses/by-nc-nd/4.0/ | |
| dc.subject | Pseudo-grouping function | en |
| dc.subject | Over-time pooling | en |
| dc.subject | Text classification | en |
| dc.subject | Feature fusion | en |
| dc.subject | Aggregation function | en |
| dc.title | Non-symmetric over-time pooling using pseudo-grouping functions for convolutional neural networks | en |
| dc.type | info:eu-repo/semantics/article | |
| dc.type.version | info:eu-repo/semantics/acceptedVersion | |
| dspace.entity.type | Publication | |
| relation.isAuthorOfPublication | 3e9cb4ee-2d64-47ff-9f40-d7519dc6cb0d | |
| relation.isAuthorOfPublication | e3f0cca3-0e2f-46b7-b476-5cda361e0efe | |
| relation.isAuthorOfPublication | b6673ff1-b94d-4852-be93-90b30a9c8555 | |
| relation.isAuthorOfPublication | b1df82f9-2ce4-488f-afe0-98e7f27ece58 | |
| relation.isAuthorOfPublication | 1bdd7a0e-704f-48e5-8d27-4486444f82c9 | |
| relation.isAuthorOfPublication.latestForDiscovery | 3e9cb4ee-2d64-47ff-9f40-d7519dc6cb0d |