Setting
AQUA-12 was designed by members of the Adverse Drug Reaction Review Committee (ADRRC) at Alfred Health in Melbourne, Australia. Alfred Health is a quaternary, university-affiliated health institution that provides, among many clinical services, specialist care in transplantation, human immunodeficiency virus (HIV) infection, cystic fibrosis, haemophilia, trauma and burns. The ADRRC is a multidisciplinary group consisting of senior pharmacists in medication safety and medicines information, and specialist physicians in allergy/immunology, dermatology, clinical pharmacology, infectious diseases and internal medicine. Around 200 ADR episodes per year are reported to ADRRC. The committee meets every two weeks to discuss ADR reports, assign causality, organise further referrals as required (e.g. allergy services) and provide recommendations regarding future medication management. All healthcare professionals within the institution are encouraged to submit ADR reports; approximately 85% of the reports are submitted by hospital pharmacists [11]. Reporting system is predominantly via an electronic form embedded in the electronic medical record (EMR).
The need to design a scoring system to monitor the quality of ADR reports was first identified in mid-2021 when ADRRC began to develop an ADR education program for hospital pharmacists and junior doctors. This sought to improve knowledge, technical skills and competency required to conduct comprehensive assessment of an ADR episode. The tool was intended to assess completeness of information in submitted ADR reports and would be a surrogate marker to assess practical knowledge and technical attributes. Hence, we intend to measure improvement over time, following the planned education program.
Development phase
The primary objectives of AQUA-12 were to assess completeness of data to allow the ADRRC to assess causality and provide effective risk communication to patients. The emphasis was placed on the following data elements for scoring: (i) previous ADR history, (ii) diagnosis or description of actual ADR, (iii) description of key events concerning ADR (i.e. the narrative), (iv) list of suspected medications, (v) consideration of medication timeline relevant to the nature of ADR, (vi) management of ADR episode, and (vii) outcome/sequelae. The rationale behind inclusion of each data element is summarised in Table 1. The data elements also reflect data-fields in the ADR reporting form that are required to be completed by healthcare professionals, and excluding those automatically populated by the EMR, such as patient details, reporter details. Each data element is assigned a maximum score of 2, except for relevant medication timeline (temporality) and management, which are assigned a score of 1 each, giving a total score of 12.
Table 1
Included data elements in AQUA-12 quality assessment tool and underlying principles
Data elements
|
Specific information required
|
Rationale for inclusion
|
Previous ADR history
|
Implicated medication names Description/diagnosis of reaction
|
Previous reaction from exposure to same or structurally similar medications increases strength of causal association
Detailed description/diagnosis of reaction is important to determine the exact nature of past ADR (i.e. intolerance vs. allergic, benign vs. of clinical concern, etc.)
|
Actual reaction
|
Detailed description or diagnosis of index reaction
|
Detailed description or diagnosis provides information regarding the exact nature of current reaction.
This influences management advice regarding avoidance or rechallenge of suspected medications (e.g. complete avoidance in SCARs and anaphylaxis vs. possible rechallenge or treat-through in mild MPE)
Ambiguous descriptions without details (e.g. “rash”) are unhelpful in clinical decision making
|
Description of key events
|
Narrative surrounding the reaction which include: onset, offset, evolution, co-morbidities that may mimic or contribute to the reaction, drug timelines, presence of drug-drug interactions, relevant investigations or information (previous tolerant exposure, re-challenge etc)
|
Detailed information regarding ADR onset, evolution, offset, and timelines aids in causality assessment
Information on relevant investigations (e.g. skin biopsy, presence of eosinophilia, organ dysfunction) aids in confirming the diagnosis as well as causality assessment
Information on co-morbidities is helpful in considering non-ADR related differential diagnoses or underlying factors (i.e. renal impairment) that might have contributed to ADR
Highlighting relevant drug-drug interactions is helpful in determining preventable factors
|
Suspected drugs
|
A list of medications that may have caused ADR. A complete set of information include: medication names, indications, date commenced, date ceased, route, dosage and frequency
|
Comprehensive review of medication list, including over the counter medications, in a patient with ADR aids in identification of a set of possible culprit medications
Indications, commencement and cessation dates, route, dose, frequency helps determine most likely medication(s) causing the reaction
|
Timeline relevant to the nature of ADR
|
Based on the nature of ADR (i.e. Type A or Type B, immediate or delayed hypersensitivity, etc.) all medication(s) that fall within relevant time frame to be included
|
Recognition of timelines specific to the nature of ADR is important to avoid inadvertent omission of medications that may be implicated in the reaction or over-inclusion of medications as suspects when, in actual fact, they fall outside the time frame
This avoids mislabelling of implicated medications
|
Management of reaction
|
Information on how the ADR episode is managed, including: dose reduction, cessation, antidote treatment, re-challenge, monitoring
|
Management narrative aids in causality assessment and helps strengthen causality
|
Outcome/Sequalae
|
Information regarding severity and outcome from the ADR episode
|
This influences management advice regarding avoidance, referral for further investigation/ evaluation or judicious rechallenge of suspected medications
Also determines if certain reports need to be forwarded to the pharmacovigilance authority (i.e Therapeutic Goods Administration)
|
MPE: Maculopapular exanthems. SCARs: Severe cutaneous adverse drug reactions |
In Round 1 of this cross-sectional study, the first version of AQUA-12 was evaluated using 20 randomly selected ADR reports submitted to ADRRC between 11th January to 4th June 2021. Two ADRRC members (AKA, LG) and one junior doctor, who was not part of the ADRRC, independently assessed the reports retrospectively for completeness after ADRRC review had been conducted. The scores were then analysed for inter-rater correlation. Any reports displaying discrepancy in total scores between assessors by more than two points were identified and reasons behind differences discussed. The scoring criteria and wording were refined to make them more concise and easily interpretable.
In Round 2, revised version of AQUA-12 was retrospectively and independently evaluated using the same 20 ADR reports as above, but by a different set of ADRRC members consisting of one clinical pharmacologist (BS), one dermatologist (MG) and one allergist/immunologist (CZ). External to the ADRRC, a clinical pharmacologist and a junior doctor who were not familiar with the current ADRRC review processes, also independently evaluated the reports. Inter-rater correlation analysis was conducted using the scores of 5 assessors from Round 2 and one assessor (AKA) from Round 1 (total n = 6). The tool was then further refined to improve functionality.
In both rounds during the development phase, final ADR diagnoses and management recommendations by ADRRC, as well as further clinical information, were made available to the independent assessors scoring the reports.
The final version of AQUA-12 derived from above process is provided in Appendix 1.
Evaluation phase
In this phase, AQUA-12 was used to assess 50 consecutive reports submitted to ADRRC between 1st Jan 2022 to 18th April 2022. Reports were scored independently by AKA and LG, and inter-rater correlation analysis was conducted. The first assessor (AKA) prospectively scored the ADR reports in a blinded manner prior to scheduled ADRRC review fortnightly. The second assessor (LG) independently scored after further information (diagnosis, investigations and recommendations) was made available post ADRRC review of the reports.
Data variables and outcomes
The following data variables concerning all ADR reports were extracted from electronic medical records: vocation of reporters, treating clinical unit, reaction type, reaction severity and implicated medication classes. Reaction types and implicated medication classes were classified according to the methodology previously described in a publication by ADRRC to maintain consistency [11]. Outcomes of interest were: (i) inter-rater correlation of AQUA-12 scores in both rounds of development phase and in prospective evaluation phase, (ii) proportion of high-quality reports using AQUA-12 tool, and (iii) factors that may be associated with low quality reports.
Data analysis
Summary statistics for discrete variables are presented as counts and proportions. Inter-rater correlation analysis results are presented as intraclass correlation coefficient (Cronbach’s alpha) with 95% confidence intervals. Univariable analyses were conducted to identify any factors that may be associated with poor-quality of reports. For differences in proportions between groups, Fisher’s exact or Chi-square tests were conducted, and statistically significant results are presented as a two-tailed p value of < 0.05. Data analysis was done using SPSS version 28 (IBM Corporation, Armonk, NY, USA).
Ethics approval
Approval to conduct this study as a low-risk research project was granted by the Alfred health human research and ethics committee (project number: 726/21)