Articles producció científica> Enginyeria Informàtica i Matemàtiques

Explaining predictions and attacks in federated learning via random forests

  • Dades identificatives

    Identificador: imarina:9261618
    Autors:
    Haffar, RamiSanchez, DavidDomingo-Ferrer, Josep
    Resum:
    Artificial intelligence (AI) is used for various purposes that are critical to human life. However, most state-of-the-art AI algorithms are black-box models, which means that humans cannot understand how such models make decisions. To forestall an algorithm-based authoritarian society, decisions based on machine learning ought to inspire trust by being explainable. For AI explainability to be practical, it must be feasible to obtain explanations systematically and automatically. A usual methodology to explain predictions made by a (black-box) deep learning model is to build a surrogate model based on a less difficult, more understandable decision algorithm. In this work, we focus on explaining by means of model surrogates the (mis)behavior of black-box models trained via federated learning. Federated learning is a decentralized machine learning technique that aggregates partial models trained by a set of peers on their own private data to obtain a global model. Due to its decentralized nature, federated learning offers some privacy protection to the participating peers. Nonetheless, it remains vulnerable to a variety of security attacks and even to sophisticated privacy attacks. To mitigate the effects of such attacks, we turn to the causes underlying misclassification by the federated model, which may indicate manipulations of the model. Our approach is to use random forests containing decision trees of restricted depth as surrogates of the federated black-box model. Then, we leverage decision trees in the forest to compute the importance of the features involved in the wrong predictions. We have applied our method to detect security and privacy attacks that malicious peers or the model manager may orchestrate in federated learning scenarios. Empirical results show tha
  • Altres:

    Autor segons l'article: Haffar, Rami; Sanchez, David; Domingo-Ferrer, Josep
    Departament: Enginyeria Informàtica i Matemàtiques
    Autor/s de la URV: Domingo Ferrer, Josep / Haffar, Rami / Sánchez Ruenes, David
    Paraules clau: Surrogate model Random decision forests Machine learning Federated learning Explainability Attack detection
    Resum: Artificial intelligence (AI) is used for various purposes that are critical to human life. However, most state-of-the-art AI algorithms are black-box models, which means that humans cannot understand how such models make decisions. To forestall an algorithm-based authoritarian society, decisions based on machine learning ought to inspire trust by being explainable. For AI explainability to be practical, it must be feasible to obtain explanations systematically and automatically. A usual methodology to explain predictions made by a (black-box) deep learning model is to build a surrogate model based on a less difficult, more understandable decision algorithm. In this work, we focus on explaining by means of model surrogates the (mis)behavior of black-box models trained via federated learning. Federated learning is a decentralized machine learning technique that aggregates partial models trained by a set of peers on their own private data to obtain a global model. Due to its decentralized nature, federated learning offers some privacy protection to the participating peers. Nonetheless, it remains vulnerable to a variety of security attacks and even to sophisticated privacy attacks. To mitigate the effects of such attacks, we turn to the causes underlying misclassification by the federated model, which may indicate manipulations of the model. Our approach is to use random forests containing decision trees of restricted depth as surrogates of the federated black-box model. Then, we leverage decision trees in the forest to compute the importance of the features involved in the wrong predictions. We have applied our method to detect security and privacy attacks that malicious peers or the model manager may orchestrate in federated learning scenarios. Empirical results show that our method can detect attacks with high accuracy and, unlike other attack detection mechanisms, it can also explain the operation of such attacks at the peers’ side.
    Àrees temàtiques: Matemática / probabilidade e estatística Interdisciplinar Engenharias iv Engenharias iii Computer science, artificial intelligence Ciências ambientais Ciências agrárias i Ciência da computação Biotecnología Artificial intelligence Administração, ciências contábeis e turismo
    Accès a la llicència d'ús: https://creativecommons.org/licenses/by/3.0/es/
    Adreça de correu electrònic de l'autor: rami.haffar@urv.cat rami.haffar@urv.cat david.sanchez@urv.cat josep.domingo@urv.cat
    Identificador de l'autor: 0000-0001-7275-7887 0000-0001-7213-4962
    Data d'alta del registre: 2024-10-12
    Versió de l'article dipositat: info:eu-repo/semantics/publishedVersion
    Enllaç font original: https://link.springer.com/article/10.1007/s10489-022-03435-1
    URL Document de llicència: https://repositori.urv.cat/ca/proteccio-de-dades/
    Referència a l'article segons font original: Applied Intelligence. 53 (1): 169-185
    Referència de l'ítem segons les normes APA: Haffar, Rami; Sanchez, David; Domingo-Ferrer, Josep (2023). Explaining predictions and attacks in federated learning via random forests. Applied Intelligence, 53(1), 169-185. DOI: 10.1007/s10489-022-03435-1
    DOI de l'article: 10.1007/s10489-022-03435-1
    Entitat: Universitat Rovira i Virgili
    Any de publicació de la revista: 2023
    Tipus de publicació: Journal Publications
  • Paraules clau:

    Artificial Intelligence,Computer Science, Artificial Intelligence
    Surrogate model
    Random decision forests
    Machine learning
    Federated learning
    Explainability
    Attack detection
    Matemática / probabilidade e estatística
    Interdisciplinar
    Engenharias iv
    Engenharias iii
    Computer science, artificial intelligence
    Ciências ambientais
    Ciências agrárias i
    Ciência da computação
    Biotecnología
    Artificial intelligence
    Administração, ciências contábeis e turismo
  • Documents:

  • Cerca a google

    Search to google scholar