Description of the work environment
Jazan FoM was established in 2001 as the only school of medicine in the Jazan region, southwest of Saudi Arabia, and on the northern borders of Yemen. At the time of the writing of this report, the number of registered medical students is 955 and the number of teaching staff members is 130, working within 11 different academic departments. The administrative hierarchy of FoM, which is similar to other faculties in the country, comprises a deanship, vice-deanships, and departments, where each administrative body has its own administrative responsibilities. Nonetheless, the majority of academic responsibilities are carried out at a departmental level where the dean and vice-deans are the regulating and supervising bodies.
The responsibilities of academic staff members in medical schools include the active participation of all members in teaching, research, clinical, and administrative activities. The clinical activities provided by the teaching staff of Jazan FoM vary between laboratory, preventive and curative health services. Additionally, academic staff members are required to attend activities related to Continuous Professional Developments (CPDs) and participate in scientific conferences and meetings.
Description of collected variables
The responsibilities of the academic staff members in medical schools vary greatly. However, they can be briefly summarized, as is displayed in Table 1. The eight main domains are proposed and detailed descriptions of the activities required within each domain are given. Additionally, different levels, where activities requiring higher effort needed to be achieved are classified as higher levels, have been suggested in order to enable the weighting of each activity. Nonetheless, due to the variability of clinical and community services depending on the specialties, no level classification was made.
Description of variables collection method and data flow
The performed activities detailed in Table 1 can be performed on and off campus. Therefore, it is the duty of each faculty member to keep a record of each performed activity in addition to providing evidence to enable verification. As a part of the Quality Assurance System in Jazan FoM, the Faculty Academic File (FAF) has been developed as the main method for measuring the academic performance of faculty members.
The FAF has five main sections, where each section has a specific function. The first section is a detailed report of all activities listed in Table 1 and is completed by the teaching staff on an annual basis. The second section involves the provision of evidence concerning reported activities, such as a list of publications and research projects details, attendance and contribution to administrative work, committees, and certifications pertaining to activities within CPDs and conference domains, official timetables, and clinical contribution reports within healthcare facilities. The third section involves the verification of reported activities, which is performed by the Heads of Departments (HoDs). HoDs are requested to provide a score up to 100 depending on reporting completion and evidence provision. Teaching staff members who have limited reporting and limited provision of supporting evidence are given lower scores than members who have better reports. The fourth section involves reporting the score estimated via HoD back to the teaching staff to allow rebuttal or endorsement of the score or to provide an additional chance to provide further reporting and supporting evidence. The fifth and last section involves the FoM dean’s final verification.
A summary of the data flow is shown in Figure 1. By the beginning of the last month of each academic year, the Teaching Staff Unit emails all teaching staff members in the college with a request to complete the FAF. Upon the completion of the verification process, the filled in FAFs are returned to the Teaching Staff Unit to perform the required data management and analysis, followed by the production of the annual academic performance report.
Description of data management and display
All completed FAFs are verified by the Teaching Staff Unit to ensure the accuracy of the data entered by the teaching staff. After the verification process, incorrect data are indicated and corrected before data entry. Only verified and correct data are incorporated into the data analysis. Reports of teaching staff not adhering to the FAF reporting policy are produced by the Teaching Staff Unit and communicated with the Dean of the Faculty for monitoring and decision making.
Microsoft Excel 2010 was used for data entry and analysis. The developed Excel file is labeled “Academic Performance Database” and the Teaching Staff Unit is considered its owner. All reported information within the FAFs is extracted into the Academic Performance Database, where all extracted data, apart from identification data of the teaching staff members, are entered in a numerical format to enable the performance of the required statistical descriptive analysis.
Microsoft Excel enables the performance of several statistical testing methods to summarize and compare the staffs’ academic performances. To enable the performance of the statistics necessary to evaluate and monitor this performance, three main basic formulas were utilized. Firstly, totals of event in each measured domain were used either at a departmental level or according to each teaching staff. Secondly, the average of each contribution was calculated to allow comparisons between departments with variable staff numbers. Finally, ranking, either on departmental level or on faculty level, was performed to assess teaching staff activity rankings in specific domains. Nonetheless, to enable appropriate ranking, the levels described in Table 1 were incorporated into the ranking system to enable an appropriate comparison between teaching staffs depending on their effort. Additional file 1 is an example of how each activity was weighed within the research domain, where these weights can be changed, which is automatically reflected on the overall ranking [see Additional file 1].
In 2020, the total number of observations extracted from collected FAFs exceeded 5000 and were inserted into the Academic Performance Database. To enable easier navigation and a display of performance analysis, the developed database display comprises three main components. The first component is a main sheet with all the entered data of the entire teaching staff. This main sheet additionally serves as a source of data with all formulas applied. The second component is an interface sheet designed to facilitate navigation between different generated sheets using the hyperlink function [see Additional file 2]. The third and final component consists of several sheets generated according to all icons displayed in the main interface, where every icon is hyperlinked to a sheet presenting specific information.
To enable an easier display and sorting of data, several display methods have been utilized via the available Excel functions. As displayed in Figure 2, Pivot tables and charts were used to provide numerical and visual representations of data. The Pivot table summarizing the number of teaching staff members with ongoing research activities is used to enable the assessment of variation between departments. In addition to numerical summary, the charts provide an easier visualization of faculty member characteristics, such as age, nationality, gender, departments, and academic titles. Finally, adding slicer icons enables easier comparisons between departments according to the measured teaching staff characteristics.
Limitations
The proposed academic performance monitoring, though conveniently administered via the widely available Microsoft Excel, requires substantial and continuous administrative work and data collection. FAF is currently designed in the Microsoft Word format to enable easy form completion and the addition of requested documents. However, more effort should be invested onto converting the document into an online and secure low-cost method in order to enable easier completion of the file. Finally, this proposed system, though able to provide fair judgment of the quality of certain domains, is currently lacking the ability to incorporate variables related to the quality of teaching, as well as the quality of clinical and community services.
