TY - JOUR
T1 - Multivariable prediction of functional outcome after first-episode psychosis
T2 - a crossover validation approach in EUFEST and PSYSCAN
AU - Melbourne
AU - Marburg
AU - Madrid
AU - Maastricht
AU - Heidelberg
AU - Galway
AU - Edinburgh
AU - Copenhagen
AU - Cantabria
AU - Amsterdam
AU - Utrecht
AU - London
AU - the PSYSCAN Consortium
AU - Toronto
AU - Seoul
AU - Sao Paulo
AU - Zurich
AU - Vienna
AU - Tel Hashomer
AU - Naples
AU - Slot, Margot I.E.
AU - Urquijo Castro, Maria F.
AU - Winter - van Rossum, Inge
AU - van Hell, Hendrika H.
AU - Dwyer, Dominic
AU - Dazzan, Paola
AU - Maat, Arija
AU - De Haan, Lieuwe
AU - Crespo-Facorro, Benedicto
AU - Glenthøj, Birte Y.
AU - Lawrie, Stephen M.
AU - McDonald, Colm
AU - Gruber, Oliver
AU - van Amelsvoort, Thérèse
AU - Arango, Celso
AU - Kircher, Tilo
AU - Nelson, Barnaby
AU - Galderisi, Silvana
AU - Weiser, Mark
AU - Sachs, Gabriele
AU - Kirschner, Matthias
AU - Lepock, Jenny
AU - Prce, Ivana
AU - Ahmed, Sarah
AU - Maheandiran, Margaret
AU - Gerritsen, Cory
AU - Kiang, Michael
AU - Mizrahi, Romina
AU - Lho, Silvia Kyungjin
AU - Moon, Sun Young
AU - Kim, Minah
AU - Lee, Tae Young
AU - Cho, Kang Ik Kevin
AU - Kwon, Jun Soo
AU - da Cunha, Graccielle Rodrigues
AU - Gadelha, Ary
AU - Zugman, André
AU - Bressan, Rodrigo
AU - Kawohl, Wolfram
AU - Kaiser, Stefan
AU - Huber, Naemi
AU - Stämpfli, Philipp
AU - Burrer, Achim
AU - Maatz, Anke
AU - Prayer, Daniela
AU - Furtner, Julia
AU - Weidenauer, Ana
AU - Sauerzopf, Ullrich
AU - Lenczowska, Marcena
AU - Domany, Yoav
N1 - Publisher Copyright:
© The Author(s) 2024.
PY - 2024/12
Y1 - 2024/12
N2 - Several multivariate prognostic models have been published to predict outcomes in patients with first episode psychosis (FEP), but it remains unclear whether those predictions generalize to independent populations. Using a subset of demographic and clinical baseline predictors, we aimed to develop and externally validate different models predicting functional outcome after a FEP in the context of a schizophrenia-spectrum disorder (FES), based on a previously published cross-validation and machine learning pipeline. A crossover validation approach was adopted in two large, international cohorts (EUFEST, n = 338, and the PSYSCAN FES cohort, n = 226). Scores on the Global Assessment of Functioning scale (GAF) at 12 month follow-up were dichotomized to differentiate between poor (GAF current < 65) and good outcome (GAF current ≥ 65). Pooled non-linear support vector machine (SVM) classifiers trained on the separate cohorts identified patients with a poor outcome with cross-validated balanced accuracies (BAC) of 65-66%, but BAC dropped substantially when the models were applied to patients from a different FES cohort (BAC = 50–56%). A leave-site-out analysis on the merged sample yielded better performance (BAC = 72%), highlighting the effect of combining data from different study designs to overcome calibration issues and improve model transportability. In conclusion, our results indicate that validation of prediction models in an independent sample is essential in assessing the true value of the model. Future external validation studies, as well as attempts to harmonize data collection across studies, are recommended.
AB - Several multivariate prognostic models have been published to predict outcomes in patients with first episode psychosis (FEP), but it remains unclear whether those predictions generalize to independent populations. Using a subset of demographic and clinical baseline predictors, we aimed to develop and externally validate different models predicting functional outcome after a FEP in the context of a schizophrenia-spectrum disorder (FES), based on a previously published cross-validation and machine learning pipeline. A crossover validation approach was adopted in two large, international cohorts (EUFEST, n = 338, and the PSYSCAN FES cohort, n = 226). Scores on the Global Assessment of Functioning scale (GAF) at 12 month follow-up were dichotomized to differentiate between poor (GAF current < 65) and good outcome (GAF current ≥ 65). Pooled non-linear support vector machine (SVM) classifiers trained on the separate cohorts identified patients with a poor outcome with cross-validated balanced accuracies (BAC) of 65-66%, but BAC dropped substantially when the models were applied to patients from a different FES cohort (BAC = 50–56%). A leave-site-out analysis on the merged sample yielded better performance (BAC = 72%), highlighting the effect of combining data from different study designs to overcome calibration issues and improve model transportability. In conclusion, our results indicate that validation of prediction models in an independent sample is essential in assessing the true value of the model. Future external validation studies, as well as attempts to harmonize data collection across studies, are recommended.
UR - http://www.scopus.com/inward/record.url?scp=85206083654&partnerID=8YFLogxK
U2 - 10.1038/s41537-024-00505-w
DO - 10.1038/s41537-024-00505-w
M3 - ???researchoutput.researchoutputtypes.contributiontojournal.article???
C2 - 39375356
AN - SCOPUS:85206083654
SN - 2334-265X
VL - 10
JO - Schizophrenia
JF - Schizophrenia
IS - 1
M1 - 89
ER -