Clinical trials - what do we mean by 'no-effect'?

This is a more difficult question than one might think.  The traditional randomised controlled trial (RCT) randomly selects people from a population of interest; then randomly allocates them to one of two or more treatments, one of which may be a placebo or 'dummy' drug.  The purpose of the placebo is to account for the beneficial (or sometimes detrimental) effect that one sometimes gets just through being given a treatment irrespective of whether it works or not.  So far so good..but..

There is a problem.  In most cases the actual clinical treatment options are not new drug versus dummy drug; but new drug versus some other treatment, which may be something or it may be nothing.  New drug versus dummy drug is almost never the actual option given to patients.  Imagine the conversation with your doctor, "the treatment for condition XXX currently is a drug which we think is of no benefit apart from the fact that it tastes nice and comes in a fancy bottle". 

Traditionally the effect that is seen just by being given something has been termed the 'placebo' effect, Latin for 'I shall please'.  This is a link to the Wikipedia page, (but I did actually know the Latin bit).  In some circumstances the pacebo can be a problem, for example if people just feel better without getting better they may miss treatment that they need.  In studies if they feel better from being given a placebo it may hide a benefit from the active drug (this is termed a Type II error - missing a beneficial effect from a treatment).  There are also ethical issues about people not understanding what they are taking.

However, where the desired outcome is comfort, maybe anything that provides comfort is ok, and if that is a placebo maybe that does not matter.  This may, for example be the case in the treatment of children with a fever, where the desired outcome of treatment is not temperature reduction but relief of discomfort.  If a medicine looks nice, tastes nice, and reduces anxiety in parents and children does it actually matter if it is the active ingredient or a placebo effect?

The reason why this is topical now, is that there is a paper in JAMA Pediatrics which looks at this very subject in the treatment of autism, finding a strong placebo effect compared to the active treatment, indeed in some groups the placebo group actually did better.  Puzzling.. 

What to make of this?  Firstly don't underestimate the power of the placebo - it may be as simple as someone taking an interest in the patient making them feel better.  Secondly, as educated informed consumers of research always check, when comparisons are made, to what was the new treatment being compared?  This brings us to a wider point of efficacy v effectiveness - efficacy is can it work in ideal circumstances (as one would see in a RCT) while effectiveness is does it work in practice? - These are often not the same.  Check to see if your new treatment works outside of the lab!

King et al JAMA Pediatr. Published online September 23, 2013. doi:10.1001/jamapediatrics.2013.2698

Does failure to respond to antipyretic drugs tell you anything useful about how serious an infection is? Answer no

This was a question that we considered when drawing up the NICE Guidelines on the treatment of fever in children; and we concluded the answer is no.  A recent review in Archives of Disease in Childhood has looked at 8 papers which have each answered the same question, and concluded the same thing.

It is always good when other people confirm your opinion, particularly so in this case because this is a real 'old chestnut' - "if the temperature comes down quickly they are likely to be ok".  Because serious infection is rare this appears to be true, most children whose temperature comes down quickly are indeed ok; but then so are most children whose temperature does not come down quickly.  They are all likely to be ok.  The key when reading papers which assess diagnostic tools is not to look at the sensitivity and specificity; but rather the positive and negative predictive values (PPV and NPV respectively).  This is not the time or the place to go into the difference, but essentially the predictive values tell you what you want to know.  For example, the PPV answers the question "if I test positive am I likely to be ill?", where as the sensitivity answers the question "if I am ill am I likely to test positive?"  So take care when you are reading or being given sensitivities and specificities.

Back to this paper.  So response to antipyretics is not useful in identifying serious illness in children.  Another couple of observations: Firstly the papers are old - does this matter?  Not really, apart from the appearance of aspirin and the absence of ibuprofen.  Probably not important, there is no particular reason to think ibuprofen should be different.  Secondly, two of the papers are by the same authors, and crucially use the same patients.  Duplicate publication can be a big problem, and including the same patients in meta-analyses where the studies are combined statistically can lead to erroneous conclusions, one study finding that including duplicated data led to a 23% overestimation of the efficacy of one drug.

The problem when reading reviews is that you often don't know this, in this case the authors noticed it and discussed it.  However, when all you have is a forest plot of results it takes a sharp eye to spot sometimes.