Redefining clinical meaningfulness

A World Dementia Council Summit will this week (Monday) discuss how teams conducting clinical research into Alzheimer’s disease and wider dementia can measure what matters when some interventions clearly benefit patients but may not deliver the dramatic improvements needed to meet current definitions of effectiveness.

The Chief Scientific Officer for Scottish Brain Sciences, Dr John Harrison says the complexity of Alzheimer’s disease and the myriad different ways it presents in different people is often not taken into account by standard approaches to clinical research.

Dr Harrison a psychologist, long term consultant to clinical trials and now CSO of Scottish Brain Sciences is part of a panel discussing ‘clinical meaningfulness’ – the central question researchers ask of any finding in a study: is the effect valuable and important to the patient?

The challenge he says is that nothing is clear-cut in Alzheimer’s disease, and something that’s seemingly normal in one person can be deeply worrying to another:

“The patients we see include individuals in their mid-40s who are still working, who are very sensitive to changes in their cognition. We’ve met actuaries, barristers and a whole bunch of people who, when you measure that cognitively, you can't find anything wrong, but they say, ‘I definitely know that I'm not as good as I used to be’.

“And I think most people that we would interact with, if they had a very small reduction in their executive skills, they would notice it very quickly. But if you’re retired, in your mid-70s and not working – you probably wouldn’t notice. So, by definition, what's clinically relevant for Person A is completely irrelevant for Person B.”

To demonstrate the challenges of establishing clinical meaningfulness in brain illness, Dr Harrison uses the example of effect size, a clinical measure commonly used when discussing the outcomes of research how significant a medicine or treatment plan is for patients. Here, a larger positive effect is represented by a higher number between 0 and 1.

While Dr Harrison says this is a very good statistical measure for thinking about clinical relevance and meaningfulness, the threshold often used may differ from the actual impact on patients:

“Most meta-analyses, when you look at what’s the threshold for de-facto clinical relevance, it’s an effect size of 0.5 – so that seems to be the threshold for saying that we can assume it actually does make a difference for people.

“The challenge comes in that almost all the drugs that we work with for Alzheimer’s disease, these have an effect size between 0.3 and 0.5 so it’s never that they’re completely useless but it’s never that they’re conspicuously worthwhile.”

Dr Harrison says context is also important – for researchers, clinicians and patients:

“I’m always very keen to say it’s contextual – these may sound like ‘small’ effect sizes for these drugs but, for the right person, it’s truly the difference between carrying on working and not. And that’s about as clinically relevant I think anything could possibly be.”

Dr Harrison emphasises that measuring small changes matters; when a person moves from mild cognitive impairment to Alzheimer’s disease, the previous measurement and monitoring separate functions seems to end:

“All that differentiation goes away, and this completely misses the point because if I’ve got somebody who’s presenting with executive dysfunction [task-related problems], that’s a really helpful thing to know about them in terms of their management.”

“In clinical trials, executive dysfunction is only rarely measured. The primary outcome measure used is to look at memory, praxis and language, but there’s no testing of attention, working memory or problem solving.”

This standard test used in clinical trials, called the Alzheimer’s Disease Assessment Scale-Cognitive Subscale (ADAS-Cog), was developed in the 1970-80s and is regarded as the “gold standard” for assessing the efficiency of antidementia treatments.

Yet, Dr Harrison says it’s not up to scratch when considering clinical meaningfulness for people in these early stages:

“It’s so insensitive that people score perfectly on the language and praxis items, even though we know they have a deficiency.

“So, every time a drug shows positive effects of on the ADAS-Cog, I get rung up and I say it’s a memory effect. And they ask, ‘how’d you know that?’ and within a mild population, it couldn’t be anything else because the only tests they’re impaired on are going to be the memory tests.”

Dr Harrison adds that, using this measure, a ‘positive effective drug’ for Alzheimer’s disease is essentially a pro-memory drug. While that remains helpful in the realm of memory, difficulties arise in meeting clinical relevance and clinical meaningfulness for patients impacted in other ways.

He says monitoring the key areas of cognition when people first attend primary care will require an improved assessment – that the current test is based on Alzheimer’s being a memory disorder despite understanding of the disease moving forward since it was first used. This means only part of what matters is measured, and the impact of an intervention on other areas is missed:

“We need a better way of picking up the diversity of that first presentation in primary health care. Then we need to be developing outcome measures in clinical trials that actually index these things – rather than just ignore them.

“So the first order of business is characterising presentation, because there'll be some lessons there. And then when we're looking to fix it, make sure we cover the gamut of cognition.”

He adds that – equipped with the knowledge and research that is already available on the diversity of presentations and levels of cognitive impairment – regulators could be more prescriptive in their regulation of drugs.

Dr Harrison highlights that establishing cognitive clinical meaningfulness is a particular problem when it comes to Alzheimer’s treatments:

“If I look at other indications – we don’t have this problem with depression or dementia with Lewy bodies. It’s peculiar to Alzheimer’s disease, where there’s this strange legacy.”

He says deciding what represents clinical meaningfulness of a treatment may ultimately be down to regulators. But he adds that fixing the system means including more people in research, measuring much more over longer periods, and establishing exactly what matters to patients – an approach that is central to the ambitions of Scottish Brain Sciences.

For more information on the summit, go to Dementia in a new era: summit 2023 | World Dementia Council