Revistes Publicacions URV: SORT - Statistics and Operations Research Transactions> 2020

Integer constraints for enhancing interpretability in linear regression

  • Datos identificativos

    Identificador: RP:4898
    Handle: https://hdl.handle.net/20.500.11797/RP4898
    Autores:
    Ramírez-Cobo, PepaOlivares-Nadal, Alba V.Carrizosa, Emilio
    Resumen:
    One of the main challenges researchers face is to identify the most relevant features in a prediction model. As a consequence, many regularized methods seeking sparsity have flourished. Although sparse, their solutions may not be interpretable in the presence of spurious coefficients and correlated features. In this paper we aim to enhance interpretability in linear regression in presence of multicollinearity by: (i) forcing the sign of the estimated coefficients to be consistent with the sign of the correlations between predictors, and (ii) avoiding spurious coefficients so that only significant features are represented in the model. This will be addressed by modelling constraints and adding them to an optimization problem expressing some estimation procedure such as ordinary least squares or the lasso. The so-obtained constrained regression models will become Mixed Integer Quadratic Problems. The numerical experiments carried out on real and simulated datasets show that tightening the search space of some standard linear regression models by adding the constraints modelling (i) and/or (ii) help to improve the sparsity and interpretability of the solutions with competitive predictive quality.

  • Otros:

    Autor según el artículo: Ramírez-Cobo, Pepa Olivares-Nadal, Alba V. Carrizosa, Emilio
    Palabras clave: linear regression
    Resumen: One of the main challenges researchers face is to identify the most relevant features in a prediction model. As a consequence, many regularized methods seeking sparsity have flourished. Although sparse, their solutions may not be interpretable in the presence of spurious coefficients and correlated features. In this paper we aim to enhance interpretability in linear regression in presence of multicollinearity by: (i) forcing the sign of the estimated coefficients to be consistent with the sign of the correlations between predictors, and (ii) avoiding spurious coefficients so that only significant features are represented in the model. This will be addressed by modelling constraints and adding them to an optimization problem expressing some estimation procedure such as ordinary least squares or the lasso. The so-obtained constrained regression models will become Mixed Integer Quadratic Problems. The numerical experiments carried out on real and simulated datasets show that tightening the search space of some standard linear regression models by adding the constraints modelling (i) and/or (ii) help to improve the sparsity and interpretability of the solutions with competitive predictive quality.
    Año de publicación de la revista: 2020
    Tipo de publicación: ##rt.metadata.pkp.peerReviewed## info:eu-repo/semantics/publishedVersion info:eu-repo/semantics/article