Prediction models have been aiding clinical decisions for many years, but only recently have we been able to unlock their full potential on a grand scale. In the past few decades, we have seen an unprecedented explosion of new data from brain imaging, omics, electronic health records, as well as from wearable devices, smartphones, and the internet. This convergence, coupled with the advancements in high-performance computing technology capable of managing and analysing big data, positions us at the forefront of a remarkable revolution in healthcare and clinical decision-making.
The development of prediction models in psychiatry faces unique challenges compared to other medical research areas. These include the measurement of unobservable complex traits, low genetic variant influence, broad disorder definitions with high comorbidity, and difficulties in patient recruitment. Our group is committed to leading progress in precision medicine in mental health to meet these challenges by promoting an environment of information exchange and collaborative problem-solving.
Precision medicine
Precision medicine has made progress in many medical areas with well-defined diseases such as cancer. Psychiatry, in contrast, has not yet benefited from new technologies that are now integral to other areas of medicine. Only a few clinical prediction models are available, and even these should be regarded as preliminary and are not usually implemented in national health systems such as the NHS.
Why is this the case? The analysis of such data is often more challenging in psychiatry than in other areas of medical research because of the complexity of the conditions, the lack of candidate genes, measurement error in both potential predictor and outcome variables, and challenges in data collection. An incomplete list of potential problems can be found below.
Prediction models in psychiatry
The development of prediction models in psychiatry is often more challenging than in other medical research areas because:
- Psychiatrists study traits which are not easily measurable, and are often measured indirectly, for example via questionnaires (measurement error).
- Although mental disorders typically show a strong heritability, genetic variants for most traits account for far less than 1% of the phenotypic variability.
- Susceptibility genes are often common variants rather than mutations, which means targeting specific genetic changes (as in cancer) is not possible.
- The definition of mental disorders is often very broad and often includes distinct but unknown subcategories. Different mental health problems have similar phenotypes and many mental (and physical) health problems occur together (comorbidity).
- Patient recruitment is difficult and there is a high drop-out rate in many studies. Patients often do not adhere to treatments.
- Restrictive inclusion criteria, self-selection bias or case-healthy control studies can result in samples that are not representative. Studies using such non-random samples are often used in psychiatric research. Models with good internal validity typically show poor external validity and do not generalise to the clinical population.
- Treatment interventions often have several interacting components (complex interventions) and it is often difficult to measure the ‘active ingredients’ of an intervention.
- Key data, such as electronic health records of mental health patients, are typically in narrative text or other unstructured data types. These require non-conventional data analysis technologies such as Natural Language Processing to take full advantage of them.
- Models are often not properly internally validated and calibrated. Often too much emphasis is given to the comparison of different machine learning algorithms than to the development of a clinically useful tool.
Implementation research
Precision psychiatry has developed into a combined area of clinical research, with many individualised prediction models developed and validated on data from more than 3 million participants aged from 1 to 64 years.
There are many validated individualised prediction models available. These can target multiple psychiatric conditions covering psychotic disorders, affective disorders, substance use disorders, anxiety disorders, neurodevelopmental disorders and several clinically relevant outcomes as well.
Despite the progress we’ve seen in developing and validating individualised prediction models for psychiatry, there are some important barriers to the advancement of knowledge.
- The first barrier is that across the overall pool of prediction models developed and published in the broader psychiatric literature, only about 15% is properly validated (10% internal validation and 5% independent external validation).
- The second barrier is that the proportion of prediction models developed and published in psychiatry, only 0.2% is then implemented in clinical routine.
Our Prediction Modelling Group provides innovative approaches to tackle this translational gap and advance implementation science.
Our group is collaborating with other international initiatives such as the European College of Neuropsychopharmacology Network for the Prevention of Mental Disorders and Mental Health Promotion (ECNP PMD-MHP) and the European Brain Area Research Area (EBRA) to develop a coherent and global implementation framework, methodological infrastructures and international implementation infrastructures.
For more information
Please contact Dr Paolo Fusar-Poli, Professor of Preventive Psychiatry on paolo.fusar-poli@kcl.ac.uk or Dr Dominic Oliver, PhD Student on dominic.a.oliver@kcl.ac.uk.