Resource Analytics
The PMM website has had 662,827 hits and 262,476 users across 214 countries (data 31st July 2020 - see Fig. 1 and Table 1). Usage has continued to grow since launch (2014) and furthermore following evolution to PMM International (September 2018). Most PMM users are from the US and UK although the total number of countries accessing PMM has increased over time (see additional file 2).
Table 1
Top 50 Countries Accessing PMM Website
Country | Number of Users | Country | Number of Users |
1. US | 79,928 | 26. Spain | 962 |
2. UK | 65,417 | 27. Italy | 890 |
3. Australia | 15,487 | 28. France | 865 |
4. India | 13,895 | 29. Taiwan | 804 |
5. Canada | 10,796 | 30. Sri Lanka | 793 |
6. Malaysia | 5,788 | 31. Colombia | 788 |
7. Ireland | 4,825 | 32. Mexico | 780 |
8. New Zealand | 3,840 | 33. Japan | 779 |
9. Saudi Arabia | 3,534 | 34. China | 749 |
10. Philippines | 3,050 | 35. Israel | 749 |
11. South Africa | 2,836 | 36. Bangladesh | 738 |
12. Pakistan | 2,343 | 37. Sweden | 725 |
13. Indonesia | 2,187 | 38. Greece | 608 |
14. Singapore | 2,163 | 39. Portugal | 581 |
15. Thailand | 1,879 | 40. Norway | 569 |
16. Egypt | 1,760 | 41. Poland | 569 |
17. UAE | 1,491 | 42. Iran | 542 |
18. Brazil | 1,488 | 43. Hungary | 532 |
19. Netherlands | 1,431 | 44. Jordan | 527 |
20. Hong Kong | 1,346 | 45. Belgium | 513 |
21. Germany | 1,337 | 46. Iraq | 512 |
22. South Korea | 1,284 | 47. Czech Republic | 489 |
23. Nigeria | 1,151 | 48. Switzerland | 467 |
24. Kenya | 1,072 | 49. Russia | 448 |
25. Turkey | 1,001 | 50. Nepal | 418 |
Total overall: 262,476 users across 214 countries |
Google Analytic Data from14th November 2014 (go live date) to 31st July 2020 |
The most popular PMM website pages (Table 2) relate to clinical assessment (pGALS, pREMS (34) normal development [gait/milestones], normal variants), frequent falls, fractures, MSK infections, limping child and guidance on when to be concerned (‘red flags’). Users spend approximately 2 minutes on the site (Mean: 01.58, Range: 5 seconds to 5 minutes, 53 seconds) and view 2 pages per session (Mean: 2.08, Range 1–24).
Table 2
Page Title | Page views |
1. Gait and motor milestones | 41,539 |
2. Frequent falls case | 27,661 |
3. Common fractures in children | 20,509 |
4. Septic Arthritis & Osteomyelitis | 20,130 |
5. Limping child - abnormal gait patterns | 15,327 |
6. pGALS | 15,146 |
7. Tip toe walking | 13,558 |
8. Clinical assessment - children differ from adults | 12.627 |
9. pREMS | 9,261 |
10. Causes of foot, heel and ankle pain | 8,621 |
11. Clinical examination | 8,449 |
12. Arthritis module homepage | 7,965 |
13. Non accidental injury | 7,794 |
14. Red flags | 7,782 |
15. Personal dashboard | 6,394 |
16. Normal variants - when to refer | 5,902 |
17. Red flags - knee pain | 5,873 |
18. Kawasaki Disease | 5,761 |
19. Resources | 5,610 |
20. Clinical assessment top tips | 4,842 |
Google Analytic Data from 14th November 2014 (go live date) to 31st July 2020 |
The pGALS app has had 12,670 downloads from 70 countries (data 31st July 2020). The iOS version had 8,067 downloads from 54 countries (Top 3: China (2,372), UK (1,498) and US (706) (see additional file 3). The android version had 4,603 downloads from 63 countries (Top 3: UK (940), Mexico (756) and India (358)) (see additional file 4).
The ELM data includes 150 registered users from 30 countries across Asia, Africa, America, Europe and Oceania (data 30th May 2019, updated data in progress). Most (n = 101, 68%) are from the UK and Ireland with a range of clinical roles: nurse and AHP (n = 50), training doctors (n = 36) and clinicians (n = 35) (see additional files 5 and 6). Most users completed one ELM (n = 128) and the remainder completed > 1 (n = 22); ‘pGALS and clinical examination skills for medical students’ had the highest uptake (n = 130 users) followed by ‘Assessment of childhood MSK presentations in family medicine’ (n = 31 users), and ‘The paediatricians approach to a child with fever ‘ (n = 11 users).
Survey and Interview Data
The response rate to the survey is not clear as the link to the survey was sent to PMM collaborators and forwarded to students and trainees. We do however know that minimum of 592 received the invite (from our random selection of PMM website users and all the ELM registered users at that time) - we received 164 completed responses to the survey, hence we can assume a maximum response rate of 28%.
Survey respondents from 25 countries (across Africa, Asia, Europe, North and South America) comprised a range of roles and levels of experience within community and hospital care (see additional files 7 and 8). Table 3 describes feedback on all three e-resources within the ‘PMM portfolio’ (n = 120) and an additional 44 provided feedback on 2 or less (PMM website and pGALS app (n = 10), PMM website and ELM (n = 3), PMM website alone (n = 31). Some had experience using the e-resources (PMM website n = 103/164, 63%; pGALS app n = 48/131, 37% ELM n = 50/123, 41%) and others did not have experience (PMM website n = 61/164, 37%; pGALS app n = 83/131, 63%; ELM n = 73/123, 59%). Users and non-users held comparable job profiles and were from a similar varied mix of countries (see additional files 7 and 8).
Table 3
Frequency of Reported Use | PMM n (%) | pGALS n (%) | Reported Use | ELM |
Use of Resource | 103 (62.80%) | 48 (36.64%) | Completed short course | 50 (40.65%) |
− Daily | 14 (13.59%) | 6 (12.50%) | − pGALS and clinical examination skills for medical students | 42 (34.15%) |
− Weekly | 15 (14.56%) | 10 (20.83%) | − Assessment of childhood MSK presentations in family medicine | 21 (17.07%) |
− Fortnightly | 7 (6.80%) | 3 (6.25%) | − The paediatricians approach to a child with fever | 14 (11.38%) |
− Monthly | 29 (28.16%) | 11 (22.92%) | | |
− Less often | 32 (31.07%) | 12 (25%) | | |
Other* | 6 (5.83%) | 6 (12.50%) | | |
Not Used Resource | 61 (37.20%) | 83 (63.36%) | Not completed short course | 73 (59.35%) |
| Total n = 164 | Total n = 131 | | Total n = 123 |
*Other included: PMM portfolio recent users accessing resources for the first time on in progress with ELM |
Users of the ‘PMM portfolio’ e-resources who completed the survey comprised mainly AHP (PMM website n = 49/103, 48%; pGALS app n = 17/48, 35%; ELM n = 33/50, 66%), followed by general paediatricians (PMM website n = 13/103, 13%; pGALS app n = 7/48, 15%), paediatric rheumatologists (PMM website n = 16/103, 16%; pGALS app n = 11/48, 23%) and medical students (n = 4/50, 8%) for the ELM. They resided across 24 countries with highest survey uptake in India (PMM website n = 38/98, 39%; pGALS app n = 13/44, 30%; ELM n = 21/44, 48%). See additional files 7 and 8.
Most respondents judged the e-resources to be ‘useful’ or ‘very useful’ and reported being able to use them quickly and easily (Table 4). The main reasons to access the PMM website and ELM were Continuing Professional Development (CPD)/Continuing Medical Education (CME) and ‘to understand a clinical problem’, whereas to ‘help examine a patient’ and ‘teaching’ were reported as the main reasons to access the pGALS app. There was a difference by user group (see Table 5), with students and trainees reporting pGALS guidance relating to patient examination and ‘exam revision’ being the main reasons to access the PMM website and pGALS app; ‘CPD/CME’ and ‘exam revision’ the main reasons to access the ELM. In contrast, clinicians reported ‘teaching’ as the main reasons to access the e-resources with nurses and AHP citing ‘CPD/CME’ (PMM website), ‘to help examine a patient (pGALS app) and ‘to understand a clinical problem’ (ELM) as their main indications.
Table 4
Resource Use and Impact on Education or Clinical Practice
| PMM Website | pGALS App | ELM |
How useful did you find the resource? |
Very useful | 54 (52.43%) | 25 (53.19%) | 23 (46.94%) |
Useful | 46 (44.66%) | 20 (42.55%) | 26 (53.06%) |
Neither | 2 (1.94%) | 1 (2.13%) | 0 |
Not useful | 1 (0.97%) | 1 (2.13%) | 0 |
Not very useful | 0 | 0 | 0 |
| n = 103 | n = 47 (1 did not answer this question) | n = 49 (1 did not answer this question) |
Are you able to use the resource for your required purpose quickly and easily? |
Yes | 92 (89.32%) | 44 (91.67%) | 41 (89.13%) |
No | 11 (10.68%) | 4 (8.33%) | 5 (10.87%) |
| n = 103 | n = 48 | n = 46 (4 did not answer this question) |
Do you feel the resource has or could have any impact on the medical education of yourself or others? |
Yes myself | 69 (67.65%) | 31 (65.96%) | 35 (77.78%) |
Yes others | 26 (25.49%) | 14 (29.79%) | 7 (15.56%) |
No | 7 (6.86%) | 2 (4.26%) | 3 (6.67%) |
| n = 102 (1 did not answer this question) | n = 47 (1 did not answer this question) | n = 45 (5 did not answer this question) |
Do you feel the resource has or could have any impact on your clinical practice? |
Yes | 87 (88.78%) | 40 (85.11%) | 40 (90.91%) |
No | 11 (11.22%) | 7 (14.89%) | 4 (9.09%) |
| n = 98 (5 did not answer this question) | n = 47 (1 did not answer this question) | n = 44 (6 did not answer this question) |
Do you use any of the resources/ information available in the resource within your clinical practice? |
Yes | 66 (66%) | 33 (70.21%) | 33 (78.57%) |
No | 34 (34%) | 14 (29.79%) | 9 (21.43%) |
| n = 100 (3 did not answer this question) | n = 47 (1 did not answer this question) | n = 42 (8 did not answer this question) |
Table 5
Resource Use by User Group
| Training Doctors | Clinicians | Nurses & AHP | Overall |
PMM Website Main Reason of Use |
To find the answer to a clinical problem | 7 (53.85%) | 17 (45.95%) | 26 (49.06%) | 50 (48.54%) |
To find an answer for an educational reason (e.g. essay, MCQ, exam) | 5 (38.46%) | 13 (35.14%) | 15 (28.30%) | 33 (32.04%) |
For Continuing Professional Development (CPD) / For Continuing Medical Education (CME) | 4 (30.77%) | 15 (40.54%) | 38 (71.70%) | 57 (55.34%) |
For exam revision | 8 (61.54%) | 8 (21.62%) | 9 (16.98%) | 25 (24.27%) |
For teaching | 5 (38.46%) | 24 (64.86%) | 15 (28.30%) | 44 (42.71%) |
To access pGALS guidance | 9 (69.23%) | 15 (40.54%) | 15 (28.30%) | 39 (37.86%) |
To access pREMS guidance | 6 (46.15%) | 9 (24.32%) | 14 (26.42%) | 29 (28.16%) |
Other | 0 | 2 (5.41%) General learning. Improve skills in MSK assessment. | 3 (5.66%) Improve knowledge and experience. Assess milestone. | 5 (4.85%) |
| n = 13 | n = 37 | n = 53 | n = 103 |
pGALS App Main Reason of Use |
To help examine a patient | 4 (40%) | 10 (47.62%) | 14 (82.35%) | 28 (58.33%) |
To improve my examination technique | 5 (50%) | 10 (47.62%) | 11 (64.71%) | 26 (54.17%) |
To improve the examination technique of others | 2 (20%) | 12 (57.14%) | 8 (47.06%) | 22 (45.83%) |
For exam revision | 5 (50%) | 8 (38.10%) | 4 (23.53%) | 17 (35.42%) |
For teaching | 4 (40%) | 19 (90.48%) | 5 (29.41%) | 28 (58.33%) |
| n = 10 | n = 21 | n = 17 | n = 48 |
ELM Main Reason of Use |
To understand a clinical problem | 2 (33%) | 4 (40%) | 25 (73.53%) | 31 (62%) |
To learn more for an educational reason (e.g. essay, MCQ, exam) | 3 (50%) | 4 (40%) | 13 (38.24%) | 20 (40%) |
For continuing professional development (CPD)/continuing medical education (CME) | 3 (50%) | 5 (50%) | 23 (67.65%) | 31 (62%) |
For exam revision | 3 (50%) | 4 (40%) | 6 (17.65%) | 13 (26%) |
For teaching | 1 (16.67) | 6 (60%) | 10 (29.41%) | 17 (34%) |
Other | 0 | 0 | 1 (2.94%) Updated. | 1 (2%) |
| n = 6 | n = 10 | n = 34 | n = 50 |
*’Training doctor’ included medical student, general paediatric trainee, paediatric rheumatology trainee, family medicine trainee; ‘Clinician’ included, general paediatrician, paediatric rheumatologist, family medicine doctors, orthopaedic surgeon & clinical lecturer/ research fellow/medical laboratory; ‘Nurses & AHP’ included nurse/nurse practitioner, physiotherapist, podiatrist, occupational therapist, extended scope practitioner & additional needs practitioner – see additional file 7 for breakdown. |
Most users reported that the e-resources ‘have’ or ‘could have’ an impact on their current clinical practice and the education of themselves or others (Table 4); improved clinical skills and knowledge, aiding teaching and increasing awareness about MSK issues in CYP amongst other providers cited as the main benefits (Table 6).
Table 6
Impact of E-resources on Clinical Practice and Learning
1. Improved clinical skills and knowledge in practice |
PMM website | pGALS app | ELM |
• Used to improve knowledge within this area and as a refresher to update and review current knowledge base. Not all countries have access to a paediatric rheumatology specialism and for these people the site enables them to view clinical cases they might otherwise not have access to within their learning environment. • Provides an intuitive source to better inform decision-making and practice, guide patient treatment and aid explaining condition to families. In particular, it Informs users about systematic approach to examination and this in turn thought to enhance confidence and ability to examine children proficiently. | • Informs users about simple systematic approach to examination and serves as useful refresher or revision aid. Increased knowledge gained from the app thought to make examination easier, enhance confidence and improve examination technique. • Equips the user with the necessary knowledge and skills to discern between abnormal and normal, screen asymptomatic and symptomatic patients and distinguish musculoskeletal conditions. | • Used to expand knowledge within a particular area of interest and to consolidate and review current knowledge base or as part of CPD. • Provides content to better inform decision-making and practice and give additional reasoning that can be applied when assessing patients or explaining condition to families. Increased knowledge gained from ELM thought to aid clinical reasoning and make MSK examination easier, improve examination technique and enhance confidence particularly in relation to assessment and examination. |
“It has improved my confidence and skills to facilitate better outcomes”. “In my country no one have a paediatric rheumatology specialty so we can learn a lot about cases from PMM and teach our student”. “When I don’t have any protocol (in Brazil some hospitals doesn’t have at all) to guide me, I choose PMM to help me and solve some problems”. “Its easy, for free and intuitive way to find answers and guide a treatment for a patient”. “It will enhance my capability to check paediatrics efficiently”. | “Making easier the clinical examination”. “Increases my capability in diagnosis”. “Improve in terms of examination, assessment, investigation and management”. | “Gaining wider knowledge of signs, symptoms and examination of a child”. “It helped me improve my technique to perform pGALS”. “I think it came from wanting to consolidate what I had read. It was almost like a test to yourself. Did I actually understand what I read and what would I be inclined to do if I were presented with a certain situation”. |
2. Improved Teaching of others |
PMM website | pGALS app | ELM |
• Used within undergraduate and trainee teaching material and students and trainees directed to site for self-directed learning or review. | • Used within undergraduate and trainee teaching material and students and trainees directed to app for self-directed learning. | • Used to prepare teaching material and inform teaching topics; and students and trainees directed to ELM for self-directed learning. |
“I refer all trainees and CME candidates to it”. “I can revise the knowledge of clinical history and examination skills before my teaching session”. “Clear, focused especially on the basics that were not taught in med school. Therefore this resource is excellent as I want to teach the topics to medical students”. | “I use for teaching and signpost students to it”. | “Acts as an introduction for me before lectures”. “This is very informative and attractive for us…by this we can increase our capabilities to give suggestions to others”. “I really want to do as much online courses as possible to have edge when enrolling for my masters”. |
3. Raised awareness in other providers |
PMM website | pGALS app | ELM |
• Enables clinicians working in different specialties or areas to consider things from a rheumatology perspective. • Highlights key issues with MSK medicine. • Increases awareness of JIA and other rheumatological conditions in children in healthcare providers within and outside of the specialism. | • Increases knowledge in colleagues and AHP. | • Completion of the courses thought to increase awareness of rheumatological conditions in children and encourage those outside the specialism to consider MSK diagnoses when assessing patients. |
“Being an orthopaedic surgeon its useful to see problems from a rheumatological perspective”. “PMM is a very useful website for non paediatric rheumatologists. Highly recommend as a learning resource”. | “Increases knowledge in colleagues and AHP”. | “List out the common MSK problems of paediatrics”. |
Most users reported using the ‘PMM portfolio’ within their own learning, clinical practice or teaching (Table 4). The PMM website content deemed most useful related to clinical assessment and examination skills (e.g. pGALS and pREMS), normal variants, red flags, limping child guidance, links to guidelines and access to videos. The pGALS guidance, ‘Top Tips’ and translations were highly rated in the pGALS app. The ELM users most valued clinical assessment of common MSK presentations, cases and images to illustrate abnormalities and when to be concerned (‘red flags’).
Self-rated confidence (before and after accessing the PMM portfolio, using a Likert scale range: 1 (not very confident) – 5 (very confident)), about MSK knowledge and skills increased for all three e-resources:
-
PMM Website p = < 0.01 t(99)=-6.59. Before: Mean score: 3.51 (S.D. 1.19). After: Mean score: 4.23 (S.D. 0.87)
-
pGALS App: p = < 0.01 t(46)=-3.94. Before: Mean: 3.70 (S.D. 1.38). After: Mean: 4.30 (S.D. 1.02)
-
ELM: p = < 0.01 (t(43)=-4.37. Before: Mean: 3.57 (S.D. 1.21). After: Mean: 4.16 (S.D. 0.91)
Non-users of the e-resources (PMM website n = 61, pGALS App n = 83, ELM n = 73) reported lack of awareness of their existence as the main reason (PMM website: n = 48/60, 80%; pGALS app: n = 66/83, 80%; ELM: n = 48/73, 66%). Most non-users reported that following the study participation they were planning to access the e-resources for their clinical practice and / or teaching.
Increasing awareness of the ‘PMM portfolio’, especially amongst junior clinicians and AHPs, integration into local training systems or curricula, linking with professional organisations, CME/CPD and existing meetings were suggested ways to increase reach and impact of all the e-resources. In addition, expanding the ELM topics, further signposting to key content targeted to user groups, providing offline access (e.g. apps) and maintaining content pertinent to global practice were suggestions for future development of the ‘PMM portfolio’