Development of the knowledge test of medical instruments and materials for cancer chemotherapy

Focusing on medical instruments and materials used for high-risk medicines for cancer treatments, a test on knowledge of medical instruments requiring acquisition of basic knowledge and materials was developed for students in pharmacy school. The aim of the study is to investigate the reliability and validity of the test (medical instruments and materials for cancer treatment-Questionnaire 45; MIMCT-Q45) we developed. Focus group discussion was performed by participants consisting of medical staff considered to have abundant experience of cancer chemotherapy. Content analysis was performed and a list of extracted medical instruments and materials was prepared. A questionnaire survey was performed twice in pharmacy students to conrm reliability employing the retest method. Responses were also collected from nurses and pharmacists to investigate discriminative validity on comparison with the students. Furthermore, diculty and discrimination were estimated using the item response theory (IRT).


Abstract
Background Focusing on medical instruments and materials used for high-risk medicines for cancer treatments, a test on knowledge of medical instruments requiring acquisition of basic knowledge and materials was developed for students in pharmacy school. The aim of the study is to investigate the reliability and validity of the test (medical instruments and materials for cancer treatment-Questionnaire 45; MIMCT-Q45) we developed.

Methods
Focus group discussion was performed by participants consisting of medical staff considered to have abundant experience of cancer chemotherapy. Content analysis was performed and a list of extracted medical instruments and materials was prepared. A questionnaire survey was performed twice in pharmacy students to con rm reliability employing the retest method. Responses were also collected from nurses and pharmacists to investigate discriminative validity on comparison with the students.
Furthermore, di culty and discrimination were estimated using the item response theory (IRT).

Results
Thirteen types of medical instruments and materials were extracted and listed in the knowledge test. In the questionnaire survey, the overall Cronbach's α and interclass correlation coe cient were high, but Cronbach's α was slightly low (0.56-0.58) in some categories. The range of discrimination estimated based on IRT was 0.98-3.09, and that of di culty was -0.91-3.00.

Conclusions
A knowledge test on cancer chemotherapy-related medical instruments and materials including palliative care at home (MIMCT-Q45) was prepared, and its reliability and validity were con rmed. MIMCT-Q45 might serve as a guidance on basic knowledge to be acquired by students and resident pharmacists and be useful to con rm the level of acquired knowledge.

Background
Reduction of incidents and medical errors involving drugs is an important medical issue [1] because it has been clari ed that drugs are involved in about half of incidents and medical errors at medical practice sites [2,3], and prevention of these and minimizing negative in uences of these on patients are likely to contribute to medical safety. Especially, many medicine used for cancer treatment, such as anticancer drugs, psychotropic drugs, and narcotics for medical use, have a high risk [4][5][6], for which concentrated measures should be taken.
Measures to improve the safety and quality of cancer treatment, such as regimen registration system, intervention of wards and outpatient chemotherapy rooms by pharmacists, and certi cation system of pharmacists specialized in the oncology eld, have recently penetrated clinical practice in Japan, with which the previous assignment to pharmacists, which are dispensing and drugs based on prescriptions and delivery within the occupational range, has been shifting to a new form being in charge of following patients from before and after drugs administration to outpatient treatment and treatment at home.
However, previous education concerning medical instruments and materials used to administer drugs was weak, and education concerning dealing with problems with equipment during drug administration, selection of medical materials used for drug administration, and medical devices frequently used in home health care (especially palliative care at home) and instruction of patients inevitably depended on physicians and nurses. Since physicians and nurses are not experts of medical instruments, further involvement of pharmacists in handling medical instruments and materials to administer drugs are needed, and educational targets to promote close involvement of pharmacists in medical instruments and materials were set in the revised core curriculum of pharmaceutical education enforced in 2015. However, the speci c medical instruments and materials requiring acquisition of basic knowledge are not stated and no scale to measure whether the knowledge is presented.
Thus, in this study, focusing on medical instruments and materials used for high-risk treatment for cancer, we developed a knowledge test concerning medical instruments and materials requiring acquisition of basic knowledge for students.

Preparation of questionnaire and knowledge test
Focus group discussion was performed by participants consisting of one surgical oncologist (MD), 3 nurses with cancer-related certi cation, 2 pharmacists, one clinical engineer, and one well experienced and trained researcher (phD) assumed to have abundant experience of cancer treatment, in which speci c medical instruments and materials requiring acquisition of basic knowledge and related knowledge were discussed. The discussion facilitated in accordance with the interview guide. The content of the discussion was entirely recorded. The tape was transcribed and converted to text which was returned to and con rmed by participants and subjected to content analysis.
A list of the medical instruments and materials extracted on content analysis was prepared and de ned as 'medical instruments and materials requiring acquisition of basic knowledge'. In addition, parts in which basic knowledge to be acquired were spoken by the medical instruments and materials were extracted by meaning and classi ed by the medical instruments and materials to prepare categories. In addition to 'correct' and 'false', 'I do not know' was set as a choice to exclude unanswered and accidental correct answers as much as possible, and responses to a knowledge test were collected. Students in the 6th year of Faculty of Pharmacy and Pharmaceutical Sciences, University of Toyama, took the prepared pilot knowledge test as a pretest, and a complete version was prepared after discussion among several researchers. In addition, a questionnaire was prepared by adding questions concerning background of respondents.

Evaluation of validity and reliability of the knowledge test
To investigate reliability of the knowledge test, the questionnaire survey was performed in students twice with a 2-week interval to con rm reliability using the retest method. For the indices of reliability, the interclass correlation coe cient and Cronbach's α were calculated. In addition, using a 2-parameter logistic model of item response theory (IRT) analysis, di culty and discrimination of each item were estimated [7].
The surveys were performed in 4-5th year pharmacy school students of nationwide national, public, and private universities in Japan between January 2015 and March 2015. To verify discriminative validity of the knowledge test [8], the questionnaire survey was similarly performed in nurses and pharmacists (medical staff) belonging to cancer treatment base hospitals in Toyama, Japan to investigate whether the total score of medical staff is higher than that of students. The questionnaire survey was performed using the anonymous mailing method.

Statistics and analysis 1) Statistical analysis
For between-group comparison, the student t-test was used setting the signi cance level at <0.05. For the correlation coe cient, Pearson's product-moment correlation coe cient was used. IRT analysis was performed using EasyEstimation version 2.00 [9], and the other analyses were performed using IBM SPSS version 22 (IBM Japan Ltd., Tokyo).

2) Content analysis
Content analysis was performed based on the method reported by Krippendorff et al [10]. Names of medical instruments and materials and related knowledge stated as those 'requiring acquisition of knowledge' and existing reports and their experience (incident reports, etc.) told by the participants of the discussion were extracted regarding a sematic content as one unit. The extracted sematic contents were inductively classi ed and abstracted following similarity, and several categories (names of medical instruments and materials/ basic knowledge to be acquired) were prepared. The above process was performed independently by several researchers. Discussion was repeated until the results became consistent and the nal conclusion was reached.

Ethical considerations
This study plan was approved by the Ethics Committee of University of Toyama (RIN26-11). All subjects (medical staff and students) were adults requiring no legal representative, and consent to participation in the focus group discussion was obtained after oral explanation of the content using documents. For the questionnaire survey, a consent form was sent with the questionnaire by postal mail to the subjects, and sending back responses to the questionnaire was regarded as consent to the study.

1) Preparation of the knowledge test
The 13 types of medical instruments and materials shown in Table 1 were extracted from the focus group discussion by content analysis and listed in the knowledge test.

2) Questionnaire survey
A total of 2,331 questionnaire forms were distributed to 36 universities per survey, and 14 universities replied (1st: 1,289 forms, 2nd: 1,201 forms). One university replied only once. The overall questionnaire collection rate was 53.4%. The background of the respondents is shown in Table 2. 3) Reliability and validity of the knowledge test (Table 3) Overall, both Cronbach's α and the interclass correlation coe cient were high, but Cronbach's α of 'PCA pump' and 'other medical materials' was slightly low (0.56-0.58).
The mean score of the medical personnel (nurses and pharmacists) widely varied and biases of knowledge were present, whereas the score of the students was consistently lower signi cantly than that of the medical staff, and the overall effect size was 0.56 (medium).
The discrimination estimated by IRT ranged from 0.98 to 3.09, and that of di culty ragged from -0.91 to 3.00. The total score tended to be higher in respondents who could correctly answer to questions with high discrimination.

Discussion
A knowledge test concerning medical instruments and materials of cancer treatment (medical instruments and materials for cancer treatment-Questionnaire 45 (MIMCT-Q45)) was prepared and its reliability and validity were con rmed. MIMCT-Q45, prepared focusing on medical instruments and materials used for high-risk cancer treatment including palliative care at home, may serve as a guidance on basic knowledge to be acquired by students and resident pharmacists and be useful to con rm the level of acquired knowledge.
In the responses from medical personnel collected as a positive control, the percentage of the score was high in items frequently used in routine practice, such as infusion pumps and syringe pumps, but it was low in some domains of items less frequently used, such as home health care-speci c devices and materials, and the rate of correct answer was lower than 10% in 3 questions in the medical staff. Since MIMCT-Q45 is comprised of items considered important by experts of cancer treatment, it was concluded that a low percentage of the score in the medical staff does not indicate that the knowledge is unnecessary. In particular, it might be useful to incorporate it into the educational curriculum of physicians.
The reason for the bias and variation in knowledge among the medical professionals was presumably due to the differences in the types of work they had experienced. The average age of the medical professionals in this study was about 40 years old, but the medical education that these people had received in Japan did not include a curriculum for learning about medical instrument and materials. Hence, they had to acquire the knowledge through hands-on experience. In particular, it is only recently that pharmacists have become involved in cancer chemotherapy and palliative medicine. In addition, the work involved in these areas is subdivided (e.g., those in charge of compounding anticancer drugs, those in charge of giving medication guidance in the wards, those in charge of giving medication guidance in outpatient clinics, and those in charge of home care), and the division of work is clearly de ned, and the work assignment is hardly ever changed. It is suggested that this background of the medical eld might have in uenced the bias in knowledge.
In contrast, the percentage of the score in the students was lower than that in the medical staff in most items, suggesting that discriminative validity was veri ed. The percentage of the score was higher in the students than in the medical staff in a few items. These may have been unfamiliar medical instruments and materials even for medical staff, suggesting the necessity of on the-job-education.
The importance of knowledge related to infusion devices for home health care requiring close involvement of medical staff has also been pointed out [11,12], but it was discussed from viewpoints of physicians and nurses in many cases, in which greater importance is attached to training and knowledge concerning administration of transfusion and drugs [13]. In Japan, pharmacists do not have a right to prescribe or directly administer drugs in response to instruction from physicians, for which required knowledge and training programs (interactions between infusion devices and drugs and troubleshooting) may be different. Accordingly, the systematically constructed knowledge required for resident pharmacists and pharmacy students may be the important rst step toward providing safe medical care.
There were several limitations of this study. Since MIMCT-Q45 was prepared aiming at widely measuring knowledge concerning several medical instruments and materials, important education items may have been overlooked. However, the amount of required knowledge concerning only infusion devices is too large to cover entire knowledge. Therefore, this knowledge test on limited targets may be useful to measure the educational effect. Since MIMCT-Q45 was developed mainly targeting Japanese pharmacy school students, it was prepared so as to match the medical state in Japan. Accordingly, to apply it to other countries and occupations, modi cation may be necessary in several items.

Conclusion
In conclusion, MIMCT-Q45 targeting pharmacy students and resident pharmacists was developed. Development of educational materials concerning 'medical instruments and materials requiring acquisition of knowledge' and measurement of its educational effect using MIMCT-Q45 remain as tasks.

Abbreviations
MIMCT-Q45 medical instruments and materials for cancer treatment-Questionnaire 45 IRT item response theory Declarations Table 3   Table 3 Item analysis and item response theory of MIMCT-Q45