Articles producció científica> Enginyeria Informàtica i Matemàtiques

Grouped Pointwise Convolutions Significantly Reduces Parameters in EfficientNet

  • Dades identificatives

    Identificador: imarina:9380779
    Autors:
    Schwarz Schuler, Joao PauloRomani, SantiagoAbdel-Nasser, MohamedRashwan, HatemPuig, Domenec
    Resum:
    EfficientNet is a recent Deep Convolutional Neural Network (DCNN) architecture intended to be proportionally extendible in depth, width and resolution. Through its variants, it can achieve state of the art accuracy on the ImageNet classification task as well as on other classical challenges. Although its name refers to its efficiency with respect to the ratio between outcome (accuracy) and needed resources (number of parameters, flops), we are studying a method to reduce the original number of trainable parameters by more than 84% while keeping a very similar degree of accuracy. Our proposal is to improve the pointwise (1x1) convolutions, whose number of parameters rapidly grows due to the multiplication of the number of filters by the number of input channels that come from the previous layer. Basically, our tweak consists in grouping filters into parallel branches, where each branch processes a fraction of the input channels. However, by doing so, the learning capability of the DCNN is degraded. To avoid this effect, we suggest inter-leaving the output of filters from different branches at intermediate layers of consecutive pointwise convolutions. Our experiments with the CIFAR-10 dataset show that our optimized EfficientNet has similar learning capacity to the original layout when training from scratch.
  • Altres:

    Autor segons l'article: Schwarz Schuler, Joao Paulo; Romani, Santiago; Abdel-Nasser, Mohamed; Rashwan, Hatem; Puig, Domenec
    Departament: Enginyeria Informàtica i Matemàtiques
    Autor/s de la URV: Abdellatif Fatahallah Ibrahim Mahmoud, Hatem / Abdelnasser Mohamed Mahmoud, Mohamed / Puig Valls, Domènec Savi / Romaní Also, Santiago / Schwarz Schuler, Joao Paulo
    Paraules clau: Efficientnet Deep learning Dcnn Dcn Computer vision Cnn Classification
    Resum: EfficientNet is a recent Deep Convolutional Neural Network (DCNN) architecture intended to be proportionally extendible in depth, width and resolution. Through its variants, it can achieve state of the art accuracy on the ImageNet classification task as well as on other classical challenges. Although its name refers to its efficiency with respect to the ratio between outcome (accuracy) and needed resources (number of parameters, flops), we are studying a method to reduce the original number of trainable parameters by more than 84% while keeping a very similar degree of accuracy. Our proposal is to improve the pointwise (1x1) convolutions, whose number of parameters rapidly grows due to the multiplication of the number of filters by the number of input channels that come from the previous layer. Basically, our tweak consists in grouping filters into parallel branches, where each branch processes a fraction of the input channels. However, by doing so, the learning capability of the DCNN is degraded. To avoid this effect, we suggest inter-leaving the output of filters from different branches at intermediate layers of consecutive pointwise convolutions. Our experiments with the CIFAR-10 dataset show that our optimized EfficientNet has similar learning capacity to the original layout when training from scratch.
    Àrees temàtiques: Medicina ii Interdisciplinar Información y documentación General o multidisciplinar Engenharias iv Engenharias iii Comunicació i informació Ciências agrárias i Artificial intelligence
    Accès a la llicència d'ús: https://creativecommons.org/licenses/by/3.0/es/
    Adreça de correu electrònic de l'autor: mohamed.abdelnasser@urv.cat hatem.abdellatif@urv.cat joaopaulo.schwarz@estudiants.urv.cat santiago.romani@urv.cat domenec.puig@urv.cat
    Identificador de l'autor: 0000-0002-1074-2441 0000-0001-5421-1637 0000-0002-7582-0711 0000-0001-6673-9615 0000-0002-0562-4205
    Data d'alta del registre: 2025-02-18
    Versió de l'article dipositat: info:eu-repo/semantics/publishedVersion
    URL Document de llicència: https://repositori.urv.cat/ca/proteccio-de-dades/
    Referència a l'article segons font original: Frontiers In Artificial Intelligence And Applications. 339 383-391
    Referència de l'ítem segons les normes APA: Schwarz Schuler, Joao Paulo; Romani, Santiago; Abdel-Nasser, Mohamed; Rashwan, Hatem; Puig, Domenec (2021). Grouped Pointwise Convolutions Significantly Reduces Parameters in EfficientNet. Amsterdam: IOS Press
    Entitat: Universitat Rovira i Virgili
    Any de publicació de la revista: 2021
    Tipus de publicació: Proceedings Paper
  • Paraules clau:

    Artificial Intelligence
    Efficientnet
    Deep learning
    Dcnn
    Dcn
    Computer vision
    Cnn
    Classification
    Medicina ii
    Interdisciplinar
    Información y documentación
    General o multidisciplinar
    Engenharias iv
    Engenharias iii
    Comunicació i informació
    Ciências agrárias i
    Artificial intelligence
  • Documents:

  • Cerca a google

    Search to google scholar