Study design
Informed by our recently completed scoping review of factors influencing prosthesis selection (8), we undertook a three-stage mixed methods study to address the objectives. The first stage comprised i) interviews with orthopaedic surgeons to inform the questionnaire design, and ii) piloting the questionnaire. The second stage was conducting a cross-sectional survey with orthopaedic surgeons in Australia. The results of the survey were then reviewed by the expert panel in stage 3, to consolidate a working definition of unwarranted variation for implant selection for THA and TKA.
Stage 1: Development and pilot of questionnaire
Key informant interviews with three orthopaedic surgeons were used to explore the findings of the systematic review and develop a questionnaire for surgeons within the Australian context. Key informant sampling was used, with participants for the interviews and pilot recruited via email through existing professional networks. A semi-structured interview guide was used to explore factors influencing prosthesis selection and what constitutes unwarranted variation (Supplementary File 1). Online piloting of the questionnaire was undertaken to review question wording and structure to ensure clinical relevance, as well as user-friendliness for participants.
Analysis and question development
Initial interviews were audio recorded and transcribed. Framework analysis of the interview transcripts were undertaken using the seven-step process of transcription, familiarisation with the data, coding, framework development, applying the framework, charting the data, and interpreting the data for meaning (11). The initial framework was based on inductive coding and deductive coding relating to the factors derived from the systematic review. Codes were grouped to create themes, then used to formulate the questions. Multiple iterations of the draft questionnaire were shared, both electronically and in-person, with available interview participants and the research team to refine question content, phrasing, and structure of the questionnaire. To pilot the penultimate questionnaire, an orthopaedic surgeon, a health economist, and a healthcare administrator were emailed a link to the questionnaire, with feedback emailed to the study’s Chief Investigator.
Final questionnaire
The final questionnaire had six sections: respondent characteristics, level of autonomy, factors influencing prosthesis selection, changing prosthesis selection, registry results, and unwarranted variation. It contained between 15 to 28 questions, depending on responses. The questionnaire used branching, whereby the number and type of questions depended on whether participants indicated they were a consultant or a registrar; whether they undertook THA, TKA or ‘both’ surgeries; and whether they worked in a public hospital, private hospital, or both. For example, if a respondent indicated that they only conducted THA, then no subsequent questions relating to TKA would be shown. There were no required fields and respondents could choose to not answer any of the questions.
Section 1: Respondent characteristics
The questionnaire was built in Qualtrix and included several questions to describe respondent characteristics, including age and number and location of surgeries performed.
Section 2: Level of autonomy and knowledge of cost
Both the systematic review and the interviews highlighted that there needed to be questions relating to the level of autonomy that surgeons have in relation to prosthesis selection in their context, as well as their awareness of the cost of the prostheses they use, to support the interpretation of results, which were both added to the final survey.
Section 3: Factors influencing prostheses selection
Twelve factors derived from the systematic review and interviews were included in the questionnaire. Pilot and interview participants suggested selecting the top five factors influencing selection would be sufficient to gain useful insight, without being too onerous.
Section 4: Changing prosthesis selection
Interview participants discussed the likelihood that surgeons might select prostheses, and their associated suppliers, based on what they had been trained on. Therefore, questions were included around whether surgeons changed prosthesis in the past, the reasons for doing so, and potential reasons for future changes.
Section 5: Registry results
Australia has a national joint registry which publishes revision rates for implants and combinations of implants, including individual surgeon level data. Questions on the use of registry data were also included.
Section 6: Unwarranted variation
Three questions about unwarranted variation were included using 5-year and 10-year revision rates and cost thresholds informed by national reporting measures and interview data.
Stage 2: Survey of orthopaedic surgeons
Participants
A cross-sectional survey was conducted with:
(1) Orthopaedic surgeons who undertake hip and/or knee surgery in Australia and are registered on the email mailing list of the Arthroplasty Society of Australia, a subspecialty group of the Australian Orthopaedic Association.
(2) Orthopaedic registrars, both training and non-training.
Recruitment
Survey participants were contacted via email sent from the Arthroplasty Society of Australia that contained a link to the questionnaire. The Arthroplasty Society of Australia has 121 members on its mailing list. Twenty registrars were emailed a link to the questionnaire through informal clinical networks, with the request to share with other registrars, whether they were training or non-training. The questionnaire remained active for three months to allow time for completion.
Analysis
The questionnaire responses were analysed using descriptive statistics and use of a numerical score to identify the highest priority factor. For this prioritisation exercise, respondents were asked to select their top five factors from a list of 12 that they consider when selecting an implant. We assigned a numerical score ranging from 5 (highest priority) to 1 (fifth priority), with all other non-prioritised factors receiving a score of 0. We then calculated the mean score for each factor, with higher scores providing an indication of factors that were perceived to have greater importance. All free text comments provided, have been reported.
Survey Results
Respondent characteristics
The survey was completed by 59 participants (50 consultants and 9 registrars) with a response rate of 41% and 45% respectively. Respondent characteristics are outlined in Table 1, noting respondents from all Australian states and territories, except the Northern Territory, and a 75–25% split between those practicing in metropolitan compared to regional hospitals. Most respondents practiced in both public and private hospitals (56% n = 33), with the remainder split evenly between public only and private only. Over half the respondents noted that they were either 15–20 years (n = 12, 20%) or 20 + years (n = 20, 34%,) post orthopaedic specialisation training.
Table 1
Respondent Characteristics
Q1: Which state or territory do you primarily work in?
|
State or territory
|
# of responses
|
% of total (n = 59)
|
QLD
|
20
|
34%
|
NSW
|
19
|
32%
|
VIC
|
7
|
12%
|
SA
|
5
|
8%
|
TAS
|
4
|
7%
|
WA
|
2
|
3%
|
ACT
|
1
|
2%
|
NT
|
0
|
0%
|
Missing
|
1
|
2%
|
Q2: What setting do you primarily work in?
|
Setting
|
# of responses
|
% of total (n = 59)
|
Metropolitan
|
44
|
75%
|
Regional
|
15
|
25%
|
Rural or remote
|
0
|
0%
|
Q3: Are you currently a:
|
Professional role
|
# of responses
|
% of total (n = 59)
|
Consultant orthopaedic surgeon
|
50
|
85%
|
Fellow
|
0
|
0%
|
Non-training registrar
|
5
|
9%
|
Training registrar
|
4
|
7%
|
Q4: How many years post training?
|
Year range
|
# of responses
|
% of total (n = 59)
|
0-5years
|
11
|
19%
|
6–10 years
|
5
|
9%
|
11–15 years
|
2
|
3%
|
15-20years
|
12
|
20%
|
20 + years
|
20
|
34%
|
Missing
|
9
|
15%
|
Q5: What is your age group?
|
Age range
|
# of responses
|
% of total (n = 59)
|
20–34
|
9
|
15%
|
35–49
|
19
|
32%
|
50–64
|
26
|
44%
|
65+
|
4
|
7%
|
Prefer not to say
|
1
|
2%
|
Q6: Do you conduct (select all that apply):
|
Surgery type
|
# of responses
|
% of total (n = 59)
|
THA only
|
1
|
2%
|
TKA only
|
5
|
8%
|
Both THA and TKA
|
53
|
90%
|
Surgery type
|
# of responses
|
% of total (n = 59)
|
Total knee replacement*
|
58
|
98
|
Total hip replacement**
|
54
|
90
|
*If selected include Q7 and enable Q15-18
**If selected include Q8, also enable Q19-21
|
Q7: How many total hip replacements do you carry out per year?
|
THR range
|
# of responses
|
% of total (n = 54)
|
Less than 20
|
11
|
20%
|
20–50
|
7
|
13%
|
50–100
|
18
|
33%
|
100+
|
18
|
33%
|
Q8: How many total knee replacements do you carry out per year?
|
TKR range
|
# of responses
|
% of total (n = 58)
|
Less than 20
|
8
|
14%
|
20–50
|
10
|
17%
|
50–100
|
20
|
34%
|
100+
|
20
|
34%
|
Q9: Do you practice in (select all that apply):
|
Hospital type
|
# of responses
|
% of total (n = 59)
|
Public only
|
13
|
22%
|
Private only
|
13
|
22%
|
Both public and private
|
33
|
56%
|
Hospital type
|
# of responses
|
% of total (n = 59)
|
Public hospital
|
46
|
78%
|
Private hospital
|
46
|
78%
|
Table 1: Respondent characteristics
Level of autonomy and knowledge of cost
For those practicing in private hospitals, all respondents noted full autonomy over prosthesis selection. Approximately half of the consultants practicing in public hospitals reported full autonomy (51%, n = 19), closely followed by departmental consensus (46%, n = 17), with a single participant noting limited input. When combined with registrar responses, departmental consensus became the most selected response overall (n = 21, 46%) for those practicing in public hospitals. Free text comments highlighted that several consultants with full autonomy in public hospitals noted some hospital processes to minimise variation (that were not enforced), whilst registrar choices were directed. Further, the majority of respondents (n = 51, 88%) either knew the cost of the implant/s they use most often (n = 19, 32%) or knew the approximate price range (+/- $1000) (n = 32, 54%).
Table 2: Level of autonomy and cost awareness
Table 2
Level of autonomy and knowledge of cost
Q10: In the public setting, what level of autonomy over choice of implant do you have?
|
Autonomy level
|
Consultant orthopaedic surgeon (n = 37)
|
Others (n = 9)
|
Total (n = 46)
|
Full autonomy
|
19 (51%)
|
0 (0.0%)
|
19 (41%)
|
Departmental consensus
|
17 (46%)
|
4 (44%)
|
21 (46%)
|
Limited input
|
1 (3%)
|
3 (33%)
|
4 (9%)
|
Other
|
0 (0.0%)
|
2 (22%)
|
2 (4%)
|
Free text responses:
Full autonomy
“Department attempts to use similar implants to reduce cost… but there is no pressure to use something else”
“Full autonomy within [state] tender, which most companies are. Needs to be an implant with some track record, but essentially full autonomy”.
“Full autonomy, but free to choose any prosthesis with a threshold price”.
“Have not tested the boundaries on my full autonomy”.
“Implant contract with one large company is threshold based. Although autonomous in choice as there is no pressure to switch to this company”
Departmental consensus
“Choice of two determined by departmental consensus”.
“Patients over 75 have all cemented implant. Have departmental rules regarding over costs for implant”
“[company] agreement in public for hip and knee arthroplasty”
“No restrictions for revisions. Two primary THR and two primary TKR choices available for use”
“Implant tender tied to robot use!!”
Limited input/ Other
“Do what I’m told”
“Implant selected by consultant”
|
Q11: In the private setting, what level of autonomy over choice of implant do you have?
|
Autonomy level
|
# of responses
|
% of total (n = 46)
|
Full autonomy
|
46
|
100
|
Departmental consensus
|
0
|
0
|
Limited input
|
0
|
0
|
Free text responses:
“Required to be in top 10% registry survivorship at minimum 5 years”
|
Q12: Do you know the cost of the implant/s you use most often?
|
Knowledge level
|
# of responses
|
% of total (n = 59)
|
Yes, I know the cost of the implant/s I use
|
19
|
32
|
I know the approximate price range (+/- $1000)
|
32
|
54
|
No, I am not really sure of the cost of the implant/s I use
|
3
|
5
|
Missing (none selected)
|
5
|
9
|
Factors influencing prostheses selection
The factor selected the most often as surgeons’ first priority was revision rate (n = 20). The most frequently “top 5” factors, regardless of position, were familiarity with implant and revision rates (selected 40 times each). When factor ratings were used to provide a relative indicator of perceived factor importance, the highest ranked factor was revision rate (3.6), followed by familiarity of implant (2.6) and implant quality (2.2). Cost, availability of implant at hospitals, and length of time in surgery were the lowest ranked factors. Prioritisation of factors is outlined in Table 3.
Table 3
Q13: Please select your top 5 factors you consider when selecting an implant. Click on the item and drag and drop into the boxes 1–5 (1 = top priority, 2 = second priority, etc).
|
Mean score
Max = 5, Min = 0
|
Revision rate
|
3.6
|
Familiarity with implant
|
2.6
|
Implant quality
|
2.2
|
Patient post-op functionality
|
1.6
|
Ease of use in theatre
|
1.2
|
Education and support provided with implant
|
1
|
Patient age
|
0.9
|
Reason for replacement (e.g.) osteoarthritis, trauma, inflammatory arthritis
|
0.7
|
Relationship with the implant supplier/ company
|
0.6
|
Implant cost
|
0.5
|
Type/brand of implant available at the hospital where I work
|
0.2
|
Length of time in surgery
|
0.1
|
Table 3: Prosthesis selection factor ranking
Changes to prostheses selection
Questions related to changes in prosthesis selection were split into those who conduct THA (n = 44) and those who conduct TKA (n = 49), with some respondents answering questions in both categories. Factors influencing previous changes to implant selection and expected future reasons are summarised in Figs. 1 and 2, and Supplementary Table 1.
Total hip replacement (n = 44)
Post training, the number of different THA implants used had a mixed spread, with more than five different implants the most common response (25%, n = 11). Most (61%, n = 30) respondents had changed away from at least one of the implants they used during training. Technological advancement (67%, n = 18), followed by registry data (56%, n = 15) were the most commonly selected reasons, with the least selected being cheaper alternatives providing similar results (11%, n = 3). For future changes, of the 15 out of 53 THA respondents that answered the question, strong clinical evidence for or against (93%, n = 14) was the most commonly selected response.
Total knee replacement (n = 49)
Post training, the number of different TKA implants used was equally split by one or two implants respectively (22%, n = 11 each), followed by more than five different implants (18%, n = 9). 61% of respondents (n = 27) had changed away from at least one of the implants they used during training. Registry data (67%, n = 20) followed by technological advancement (60%, n = 18) were the most commonly selected, with cheaper alternatives providing similar results (7%, n = 2) the least common. For future changes, of the 17 out of 58 TKA respondents that answered the question, strong clinical evidence for or against (77%, n = 13) was chosen.
Figure 1: Factors influencing decision – THA
Figure 2: Factors influencing decision – TKA.
Registry results
Just over half of consultant participants checked their results once a year (54%, n = 27), with another 32% (n = 16) checking their own results multiple times each year. Two consultant respondents (4%) indicated that they had never checked registry results. When asked how important joint registry outcomes were to them, more than 75% of respondents indicated extremely or very important (extremely important 39% n = 23, very important 37.3% n = 22).
Table 4: Joint registry data use
Table 4
Q24: How often do you check your individual registry results?*
|
Support
|
# of responses
|
% of total (n = 50)
|
Multiple times a year
|
16
|
32
|
Once a year
|
27
|
54
|
Once every 2–3 years
|
2
|
4
|
Once every 3–5 years
|
1
|
2
|
Once every 5 + years
|
0
|
0
|
Never
|
2
|
4
|
Missing
|
2
|
4
|
Q25: How important are joint registry outcomes to you?
|
Importance level
|
Consultant orthopaedic surgeon (n = 50)
|
Others (n = 9)
|
Total (n = 59)
|
Extremely important
|
19 (38%)
|
4 (44%)
|
23 (39%)
|
Very important
|
20 (40%)
|
2 (22%)
|
22 (37%)
|
Moderately
|
7 (14%)
|
0 (0%)
|
7 (12%)
|
Slightly important
|
2 (4%)
|
0 (0%)
|
2 (3%)
|
Not at all important
|
2 (4%)
|
0 (0%)
|
0 (0%)
|
Missing
|
19 (38%)
|
3 (33)
|
5 (9%)
|
*NB: Q24 was answered by those who selected consultant only. |
Unwarranted variation
Cost of prostheses
Participants were asked about unwarranted variation in relation to prosthesis cost (Unwarranted variation summarised in Fig. 3 and supplementary Table 2). Answers were mixed, with 27% (n = 16) indicating that a prosthesis costing 20% more than an equivalent was unwarranted variation, but this was closely followed by 24% (n = 14) indicating that a prosthesis costing 10% more than an equivalent was unwarranted variation. This question was not answered by 14% (n = 8) of participants. The distribution of responses was broadly consistent between consultants and registrars.
Five-year revisions
Fifty-eight percent of respondents (n = 34) indicated that a ‘more than 20%’ increase in five-year revision rates compared to equivalent prostheses constitutes unwarranted variation. This option was selected by 62% (n = 31) of consultants and a third (33%) of registrars. However, nearly a third (29%, n = 17) of respondents selected a lower threshold. Of note, 13.6% of surgeons (n = 8) did not answer this question.
10-year revisions
More than 20% increase in 10-year revision rates was also the most commonly selected response for 10-year revision rates, with 59% (n = 35) selecting this overall, comprising 62% (n = 31) and 44% (n = 4) of consultants and registrars, respectively. Similar to the five-year revision rates, 27% (n = 16) of respondents selected a lower threshold, with the same number of respondents (n = 8) not selecting a threshold at all.
Figure 3: Unwarranted variation
Stage 3: Panel discussion to define unwarranted variation
Survey results were reviewed by a panel of two orthopaedic surgeons and a health economist with expertise in unwarranted variation to consolidate a working definition of unwarranted clinical variation. This definition will be used as part of an economic evaluation using current prosthesis use data. Key outcomes from the panel meeting are summarised below.
1. Revision rates are a key indicator, which should form part of the working definition.
2. Revision rates of 20% or higher than an agreed benchmark rate at 5-year and 10-year intervals were suitable for a working definition. It was noted by participants 20% is similar to international benchmarking.
3. Uncertainty would always be present in estimates of unwarranted variation, for example, if derived from registry data revision rates, confidence intervals could be used for defining conservative narrow and broad definitions of unwarranted variation.
4. Cost is a useful indicator of unwarranted variation. However, given the mixed response in the survey, the panel suggested that multiple cost thresholds, along with revision rates, should be used within the working definition.
5. There was some discussion about other potentially confounding factors, including cemented vs uncemented, or age of implant recipients, and how this ought to be accounted for in future studies investigating unwarranted variation in this field.