TY - JOUR
T1 - Capturing between-tasks covariance and similarities using multivariate linear mixed models
AU - Navon, Aviv
AU - Rosset, Saharon
N1 - Publisher Copyright:
© 2020, Institute of Mathematical Statistics. All rights reserved.
PY - 2020
Y1 - 2020
N2 - We consider the problem of predicting several response variables using the same set of explanatory variables. This setting naturally induces a group structure over the coefficient matrix, in which every explanatory variable corresponds to a set of related coefficients. Most of the existing methods that utilize this group formation assume that the similarities between related coefficients arise solely through a joint sparsity struc-ture. In this paper, we propose a procedure for constructing multivariate regression models, that directly capture and model the within-group simi-larities, by employing a multivariate linear mixed model formulation, with a joint estimation of covariance matrices for coefficients and errors via penalized likelihood. Our approach, which we term MrRCE for Multivariate random Regression with Covariance Estimation, encourages structured sim-ilarity in parameters, in which coefficients for the same variable in related tasks share the same sign and similar magnitude. We illustrate the benefits of our approach in synthetic and real examples, and show that the proposed method outperforms natural competitors and alternative estimators under several model settings.
AB - We consider the problem of predicting several response variables using the same set of explanatory variables. This setting naturally induces a group structure over the coefficient matrix, in which every explanatory variable corresponds to a set of related coefficients. Most of the existing methods that utilize this group formation assume that the similarities between related coefficients arise solely through a joint sparsity struc-ture. In this paper, we propose a procedure for constructing multivariate regression models, that directly capture and model the within-group simi-larities, by employing a multivariate linear mixed model formulation, with a joint estimation of covariance matrices for coefficients and errors via penalized likelihood. Our approach, which we term MrRCE for Multivariate random Regression with Covariance Estimation, encourages structured sim-ilarity in parameters, in which coefficients for the same variable in related tasks share the same sign and similar magnitude. We illustrate the benefits of our approach in synthetic and real examples, and show that the proposed method outperforms natural competitors and alternative estimators under several model settings.
KW - Covariance selection
KW - EM algorithm
KW - Multivariate regression
KW - Penalized likelihood
KW - Regularization methods
KW - Sparse precision matrix
UR - http://www.scopus.com/inward/record.url?scp=85098567119&partnerID=8YFLogxK
U2 - 10.1214/20-ejs1764
DO - 10.1214/20-ejs1764
M3 - ???researchoutput.researchoutputtypes.contributiontojournal.article???
AN - SCOPUS:85098567119
SN - 1935-7524
VL - 14
SP - 3821
EP - 3844
JO - Electronic Journal of Statistics
JF - Electronic Journal of Statistics
IS - 2
ER -