This would be a mixed explanatory quantitative-qualitative study to evaluate the activities of ISAM. The study was performed in three phases based on the above-mentioned objectives (Figure 1).
Phase 1: Translating of Member Association Capacity Assessment tool (MACAT) from English to Persian, then the validity and reliability assessment
Translation Procedure: The MACAT is a checklist with 96 items in 7 main dimensions including MACAT including governance, management/leadership, financial resources, functions, communication, collaboration, and visibility.
MACAT is a tool developed by ICM in English. It was translated into Persian by one of the researcher team members who is fluent in both English and Persian, and a second time by an independent, nationally accredited translator. When the translators had chosen different terms, alternative wording was reviewed, and preferred wording was agreed on via consensus within the research team.
MACAT is a tool with 96 items with Yes or NO responses, with scores of 1 and 0, respectively. The total scores range from 0 to 96. This tool makes the opportunity to find the needs for further actions to improve the activities of the association and ultimately the quality of midwifery care and maternal health in the country. The data will be analyzed using SPSS-21.
Validity and reliability Assessment: The validity and reliability of the Persian version of MACAT were assessed before the ISAM evaluation process. The validity assessment was performed qualitatively and quantitatively. The reliability assessment was also performed by calculating the Kappa Statistic or Cohen’s* Kappa.
The Validity Assessment: In the qualitative validity assessment of MACAT, 10 midwifery and reproductive health faculty members were asked to provide feedback on the questionnaire regarding grammaticality, appropriate word use, and appropriate placement of phrases. The experts were also instructed to qualitatively assess the adequacy index of the items of MACAT to determine whether all the items and the number of items related to each dimension of the instrument are sufficient to measure the association capacity (17).
The quantitative content validity: The quantitative content validity of the MACAT was investigated based on the opinions of 10 experts regarding relevance, simplicity, and clarity. In the quantitative content validity assessment, the content validity ratio (CVR) and content validity index (CVI) were calculated.
Content validity ratio (CVR): The content validity ratio CVR was assessed by 10 experts. The participants scored the items based on a 3-point Likert scale including "not essential", "useful but not essential", and "essential" which scored 1 to 3 respectively. Then, CVR was calculated through the following formula (18).
The accepted value was determined based on Lawshe’s table and the number of experts (18). The opinions of the 10 experts were referred to in evaluating the CVR, with 0.62 regarded as acceptable.
Content validity index (CVI): CVI was calculated using Waltz and Bausell’s (1983) criteria to ensure that the items of the instrument are excellently designed to measure content. The experts evaluated the relevance, clarity, and simplicity of the items by a four-point Likert scale. Relevance pertains to whether a statement is irrelevant, requires modifications, is relevant but needs revision, or is completely relevant and appropriate, which are scored form of 1 to 4, respectively. Simplicity refers to whether a statement is complicated, needs modifications, is simple but needs revision, or is completely simple and fluent which are scored form of 1 to 4, respectively. Clarity denotes whether a statement is vague, needs modifications, is clear but needs revision, or is completely clear and sensible which are scored form of 1 to 4, respectively.
The CVI score of each statement was calculated by dividing the number of experts agreeing with each statement using scores 3 and 4 on the Likert scale by the total number of experts. Based on this index, an entire statement was initially measured in terms of relevance, after which its acceptability was determined according to the following criteria (19, 20).
- A CVI score over 0.79 is regarded as adequate.
- A CVI score of 0.7 to 0.79 is considered questionable and calls for revision and modification.
- A CVI score of 0.7 denotes unacceptability, and an item must be removed.
Finally, correction of the CVI formula to verify the probability of chance agreement was also calculated. It is important because CVI may match the chance agreement. Polit et al. (2007) (21), suggested the modified Kappa statistics of K* to show the possibility of a chance of agreement. To calculate K*, the probability of chance agreement should first be computed through the following formula used for binominal random variables:
In which N indicates the number of raters, and A represents the number of agreements regarding relevance.
Then, K* is calculated using the ratio of agreement on relevance or the individual CVI and the probability of chance agreement. K* is evaluated as excellent (K > 0.74), good (K = 0.60–0.74), or relatively good (K = 0.40–0.59).
The Reliability Assessment:
The reliability of MACAT was assessed using the intra-rater reliability assessment method and calculating the Kappa agreement coefficient between the evaluators. The facilities, equipment, and documents were observed and scored in the MACAT checklist by two evaluators. Then, the scores of two evaluators were used to calculate intra-rater reliability by calculating the Kappa agreement coefficient between them (22).
Then, the valid and reliable Persian Version of MACAT was used for ISAM capacity evaluation, describing the current situation, determining the needs, and providing evidence-based strategies and activities for the next section and the qualitative phases of the study.
Phase 2. Evaluation of Iran Scientific Association of Midwifery (ISAM)
The quantitative phase was a descriptive study using the valid and reliable Persian Version of MACAT. Two trained evaluators attended the site of the ISAM and assessed the facilities, equipment, and documents using the MACAT. Also, after completing the MACAT, the current status of the association was described and the existing challenges, problems, and needs were extracted. Then, the determined needs were examined in the third qualitative phase of the study to propose the necessary strategies for ISAM capacity improvement.
Phase 3. Explaining the needs and the related strategies and actions to improve ISAM
The results of the ISAM capacity evaluation demonstrated the problems and needs. Then the needs and the related strategies and actions to improve ISAM were explained in the qualitative phase of the study.
Study design: This was a qualitative study with a content analysis approach. The content analysis approach is usually applied to describing a phenomenon when existing theory or literature on a phenomenon is limited (23, 24). Since the problems of ISMA could have complex dimensions and require specific strategies and actions, the qualitative study using in-depth- interviews and focus group discussions with key informants was used for data collection.
The Participants: The participants were midwives and key informants who participated in a focus group discussion or depth-individual interview. The inclusion criteria were having at least two years of work experience in midwifery care provision, education, research, or management.
Sampling: Purposive sampling method was performed until data saturation was reached.
Data collection: We used Focus Group Discussion (FGD) and semi-structured individual in-depth interviews, documents review, field notes, and observation. All the interviews were audio recorded and transcribed. The in-depth interviews and FGDs began with semi-structured based on the problems extracted from the ISAM evaluation phase in 7 dimensions of MACAT including governance, management, financial resources, functions, communication, collaboration, and visibility. Therefore, the guide questions were "how do you explain the reasons for the problem in this dimension of ISAM" and "what strategies and actions can be suggested to solve the problems and meet the needs to improve ISAM capacity"? The duration of FGD was 2 hours and the interviews lasted 40 to 180 minutes. All the interviews were recorded after obtaining the participants' written consent forms.
The rigor of the Data: Lincoln and Guba’s criteria were used to confirm the rigor and trustworthiness of the data. To increase credibility, adequate time was allocated to data collection and frequent reviewing of the data. In addition, the integration of data collection methods, i.e. FGD, individual interviews and observations, document review, and field visits increased the credibility of the data. Some codes extracted from interviews were reviewed by two colleagues who had not participated in the study to ensure that the results were accurately depicted by the participants. To confirm the dependability of the data, code-recode and external checking were used. To assess the confirmability of the data, the researchers abandoned all their assumptions and thoughts and carefully documented all the research steps, and allowed external auditors to investigate all the steps. To ensure transferability and comprehensiveness, a clear explanation of the methods of collecting and analyzing the data was presented along with examples of the statements made by the participants (25).
Data analysis: The directed content analysis was conducted for data analysis. With a directed content analysis approach, the analysis starts with a theory or relevant research findings as guidance for initial codes (26).
After each interview, responses were typed verbatim on MS Word and imported to MAXQDA software. Then, data were analyzed under the steps proposed by Mayring (27, 28), who suggested a seven-step method for directed qualitative content analysis that distinctively differentiated between inductive and deductive methods as follows: (1) determination of the research question and theoretical background, (2) definition of the category system such as main categories and subcategories based on the previous theory and research, (3) establishing a guideline for coding, considering definitions, anchor examples, and coding rules, (4) reading the whole text, determining preliminary codes, adding anchor examples and coding rules, (5) revision of the category and coding guideline after working through 10– 50% of the data, (6) reworking data if needed, or listing the final category, and (7) analyzing and interpreting based on the category frequencies and contingencies. We used the seven dimensions of the MACAT including governance, management, financial resources, functions, communication, collaboration, and visibility, for categorizing the extracted codes.
Ethics approval and consent to participate: Before data collection, approval was obtained from the Ethics Committee of Shahid Beheshti University of Medical Sciences [Code: IR.SBMU.PHARMACY.REC.1401.173]. Ethical considerations (i.e., informed consent, objective and procedures of the study, confidentiality and non-disclosure of personal information, voice recording, prevention of financial burden on the participants, the right to withdraw from participation at any point in the study) were explained to the participants.