Descriptive Results
The ISCRTN search found 65 studies which were eligible (18 musculoskeletal, 22 Mental and Behaviour disorders and 25 cancer). Only 2 PILs were available on the website so the authors (NK,VS,SB) contacted the study teams to request that they provided their study documentation.
Responses were received from 33 studies providing 34 individual PILs (52%) these were broken down by 11 musculoskeletal (61%) 13 Mental and Behaviour disorders (59%) and 10 Cancer (40%). All of these studies were included in the final analysis.
The Flesch Reading Ease Scale range across all three clinical areas were ranged from fairly easy to read too difficult to read. The mean and median scores were interpreted as fairly difficult to read
The mean reading ease across all clinical areas indicated that all clinical areas were classified as fairly difficult to read. This indicates that readers overall would need to have a reading age of a 10th-12th grade which in the UK equates to a 16-18-year-old reading level.
The Flesch-Kincaid Grade minimum to maximum across all three clinical areas were interpreted as average to skilled reading level. The median and the mean interpretation grades are both at the higher level of average.
The mean reading grade across all three clinical areas found that all clinical areas had a higher level than average grade.
The Gunning Fox index range across all three clinical areas were interpreted in the range of High school sophomore (Year 11 in UK) to College senior (2nd Year University in UK). The median score and the mean scores were interpreted as High School Senior reading age (Year 13 in UK)
The reading index mean by individual study type are all interpreted as high school senior reading age (Year 13 in UK). See supplementary table 1 for more details
The total length of the cancer PILs were longer than the information leaflet from the other two clinical areas with a mean of 5434 words in comparison to a mean of 2927 in the musculoskeletal studies and 3289 in the mental and behaviour disorder studies. See Supplementary Table 2 for details.
The length of information leaflet in terms of describing adverse effects and risk in the Cancer studies were longer than the other two clinical areas with a mean number of words of 616 in comparison to a mean of 157 words in the musculoskeletal study and 304 in the Mental & Behaviour disorders studies. This was also reflected in the % of words for adverse effects and risk in comparison to the overall mean.
Table 1 – Length of documents describing adverse effects and risk
|
Minimum and Maximum (Words)
|
Median
(Words)
|
Mean
(Words)
|
% of overall mean (words)
|
1st QR
|
3rd QR
|
IQR
|
Musculoskeletal
|
27-236
|
105
|
157
|
5.4
|
48
|
291
|
242
|
Mental and Behaviour Disorders
|
0-529
|
336
|
304
|
9.2
|
221
|
364
|
143
|
Cancer
|
127-1165
|
559
|
616
|
11.5
|
472
|
643
|
171
|
All clinical areas
|
0-1165
|
326
|
354
|
10.4
|
127
|
516
|
389
|
Overall 30 of the trials that were sampled were using licenced medications (9 Cancer 11 Mental and Behaviour Disorders and 10 Musculoskeletal) however only 14 of these (4 Cancer, 6 Mental and Behaviour and 4 Musculoskeletal) were using the medication within the licensed range of indications, dosage and form.
Qualitative data analysis
Following the start of the study the description of the placebo was further divided into sub themes looking at what the placebo is and why the placebo is being used. We also divided the potential benefits into potential benefits of the study and the potential benefits of the intervention. As there were two PILs from one study [ID 310] which provided elements of study information spread across both PILs, these were combined for the qualitative data analysis (n=33).
Description of the intervention under investigation
Reflecting the range of studies included, the data describing the interventions were extremely heterogeneous. The details varied from providing only the drug name and class (e.g. antidepressant (Sertraline)) to extensive descriptions of the intervention and what it is (or usually is) used for, and what the evidence is to date. Interventions in the study documents for cancer trials were generally more extensive than those in the musculoskeletal or mental health groups of studies.
Description of ‘placebo’ and the nature of the placebo used,
All the included studies involved a placebo intervention as either a substance or a procedure. A small number of the study documents did not provide any information about the placebo used, or what a ‘placebo’ is (5/33 15%). Where a description was provided of what a placebo is, the majority used the term ‘dummy’ (13/28 46%) or described it as not containing an active ingredient (17/28 61%), others described it as being an inactive substance (2/28 7%), or described its physical characterisitics as ‘looking the same’ as the active intervention (11/28 39%) or not being ‘able to tell which is which’ (1/28 4%). The effective characteristics of the placebo was described as having ‘no therapeutic properties’ or ‘no anti-cancer action’ (3/28 11%). Many included a combination of terms e.g. ‘A placebo is sometimes called a dummy treatment - it looks the same as the actual treatment but does not contain any of the active ingredients and will have no effect on you.’ [ID 204]. Two studies which involved a placebo comparator for a minor surgery intervention described the injection as a ‘sham’ procedure, another described it as an imitation injection. There were only two study study documents which described the ingredients of the placebo, one of which was a saline injection and the other a gelatine capsule.
Description of why a placebo is used
The majority of the documents did not provide information about why a placebo was being used (19/33 58%). Of those that did, a number of different reasons were given. Some described it as ensuring that the patient (6/14 43%) and/or their doctor (4/14 29%) would not know whether they had been given the active intervention or not, thus allowing for comparison (3/14 21%) meaning that the effect of just the active intervention can be measured (4/14 29%). A small number explained that this reduces bias (3/14 21%) and means that effects are known to be not just due to chance (1/14 7%) and makes the trial results more robust (3/14 21%). Some used a combination of more than one, e.g. ‘If patients are randomised to placebo tablets rather than no active therapy, the doctor and patients will not know which treatment the patients are taking. There is therefore less scope for bias and this makes the results of the trial more robust.’ [ID 103].
Description of potential adverse effects
As with the intervention itself, the adverse effects data are heterogeneous in nature, with some study documents containing an extensive bullet-pointed list of in excess of 50 side effects [ID 102], whilst others contained a single short sentence which may include those considered most likely to occur e.g. ‘The more common side effects are fatigue, insomnia/abnormal dreams and nausea.’ [ID 222] or a brief paragraph [e.g ID205 and ID 207]. Some studies provided information about how the potential adverse effects are known e.g. ‘Based on research studies and the experience of other people taking [name of investigational medicinal product], some side-effects can be expected.’ [ID 104]. One study document, for the evaluation of a tea-based intervention, did not contain any information about adverse events [ID 210].
Likelihood of adverse effects
Almost all of the documents contained information about the likelihood of any adverse effects occurring (31/33 94%). The two studies which did not provide this information were a medication reduction intervention, and a study which involved a single tendon injection. Of those that did report the likelihood, they were divided between those that used a text description only (16/31 52%) such as ‘rarely’ or ‘commonly’, and those that combined a probability statement with a text description (15/31 48%) such as ‘very rarely (<1 in 1000)’.
Severity of adverse effects
Over half of the study documents do not include any information about the severity of potential adverse effects (18/33 55%). Where the severity of adverse effects are stated, it is described as either ‘mild’ (2/15 13%), ‘serious’ ‘severe’ or ‘major’ (8/15 53%), ‘life-threatening’ or ‘fatal’ (2/15 13%), or a combination of these.
Action to be taken by the participant in the event of adverse effects
Many of the study documents (14/33 42%) did not contain information about what that action the participant should take in the event of an adverse effect occurring. Of those that did, the majority informed the participant that they should contact the study team or doctor or report at the next study visit (12/19 63%), and/or contact another clinician such as their GP or the nearest A&E department (9/19 47%). In a small number the information was contained in a separate information leaflet given to the participant (2/19 11%), the content of which was not included in this study.
Description of the potential beneficial effects of the intervention
A third of the study documents (10/33 30%) did not refer to any potential benefits from the study intervention. One stated that there would be no benefits for participants receiving the placebo beyond those of the weekly cancer treatment that would be the same outside of the trial [ID 107]. Others expressed hope that the participant would derive some benefit, whilst emphasising that any benefits were unknown or uncertain. Of these, most (11/33 33%) reported a specific potential benefit such as delaying the time to progression of your disease, or helping them to feel stronger or experience a reduction in pain. Four described the potential benefits from the intervention in general or non-specific terms such as ‘the medicine may help you and your cancer’.
Description of the potential beneficial effects of the study
The majority of the documents (24/33 73%) included a statement about the study benefitting future patients, or that information from the study will increase understanding about the condition and which treatments are effective, even if the participant doesn’t benefit directly. An additional two stated that some people find it rewarding to take part in medical research. Some of the documents (8/33 24%) reported that participants may find the additional contact with the research or clinical team reassuring or helpful, or that there may be benefits from the additional monitoring that forms part of the study schedule, or that the study assessments could result in finding a previously unknown condition. Four of the documents stated that there may be benefits from the additional blood tests or scans performed as part of the study. Three documents stated the benefits of receiving an intervention that is not routinely offered by the NHS and so is unavailable outside of the study, or that it was provided free of charge (a vaccination). Two studies described the potential benefits that participants might gain from understanding their condition better through the use of particular study measures. Five explicitly stated that the participant will not benefit from taking part in the study, and three did not include any statement about any potential benefits.
Relative position of beneficial and adverse effects
Adverse effects were usually presented before beneficial effects (21/33 64%). Most commonly, the potential adverse effects and beneficial effects were directly adjacent to each other (19/33 58%), less often the sections were presented apart by either one page (5/33 15%) or 2+ pages (4/33 12%), or dispersed in several locations such as brief details in the introduction with additional information later in the document (2/33 6%). One document had a detailed description of the potential adverse effects (and the likelihood of each side effect) in a table in an appendix [ID 220], and one had no description of the potential benefits or adverse effects [ID 210].