The stages for the development and validation of an instrument encompass four distinct phases: planning, construction, quantitative analysis, and validation [9,10,11]. Figure 1 outlines the development of the tool. The instrument planning and construction phases consisted of an extensive literature review of the existing integrated and individual registry forms available online and by gathering information from the regional functional registries and expert opinion.
2.1 Instrument development:
Based on the literature review, the first version of the instrument was elaborated, consisting of 125 items. Items were divided into four domains: “General data of registered patient,” “Patient sociodemographic details,” “Patient history (including behavioural risk factors for NCDs as Tobacco use, alcohol use, diet, dietary salt, physical activity, anthropometric measurements, family history, obstetric history),” and “Medical condition details (Young diabetes, Stroke, Cancer and Acute coronary artery events).”
In the second phase, the face and the content of the instrument was validated by a panel of specialist judges/experts. The Specialists were selected on the basis of the criteria of area of specialization in the field of specific NCDs or those who are already running the individual NCD registries or have field experience of running a population-based registry or have been working in their respective areas for a period of more than one year, or who have scientific publications in the relevant field. Those who did not respond to the invitation to participate within the established deadline, were excluded as an expert. Total of 10 experts were approached and 6 responded. The qualitative judgement of the experts was also taken as an open-ended question.
This step entails confirmation by a specific number of experts, indicating that instrument items and the entire instrument have content validity. For this purpose, an expert panel is contacted. This includes Content validity and Face validity.
2.2.1 Content Validation
Content Validity Index
The researchers asked the panel of experts to give their view points on the items generated for the construct of integrated NCD registry tool. The CVI was calculated for all individual items (I-CVI) and the overall scale (S-CVI)12. For CVI, the panel of experts was asked to rate each scale item in terms of its relevance to the underlying construct. A 4-point scale was used to avoid a neutral point. The four points used along the item for rating continuum (1 = not relevant, 2 = somewhat relevant, 3 = quite relevant, 4 = highly relevant). For each item, I-CVI was computed as the number of experts giving a rating of 3 or 4, divided by the total number of experts. For example, an item rated 3 or 4 by four out of five experts has I-CVI of 0.80. It is advised that I-CVI should be 1.00 in case of five or fewer judges and in case of six or more judges; I-CVI should not be less than 0.78. The S-CVI was computed for ensuring content validity of the overall scale. It can be conceptualized in two ways—S-CVI (universal Agreement) and S-CVI (Average). S-CVI (Universal agreement) reflects the proportion of items on an instrument that achieved a rating of 3 or 4 by all the experts in the panel. S-CVI (Average) is the liberal interpretation of Scale validity Index, and it is computed by using average I-CVI. S-CVI (Average) emphasizes on average item quality rather than on average performance of the experts. It is recommended that a minimum S-CVI should be 0.8 for reflecting content validity [12,13,14]. Six experts participated in the first phase of content validation, the acceptable CVI value for each item ranged from 1.00 to 0.78, and the Kappa Modified value from 1.00 to 0.65 .
Kappa Statistic coefficient
CVI is extensively used by researchers for determining the content validity. However, it does not consider the inflated values that may occur because of possibility of chance agreement. Therefore, computation of Kappa coefficient ensures better understanding of content validity as it removes any random chance agreement. Kappa statistic is a consensus index of interrater agreement that supplements CVI to ensure that the agreement among experts is beyond chance. Computation of Kappa Statistic requires the calculation of probability of chance agreement, that is, Pc = [N! / A! (N – A)!] × 0.5N. In this formula, N =number of experts in the panel, A=number of experts in the panel who agree that the item is relevant. Kappa statistic is then calculated as K= (I-CVI – Pc) / (1 – Pc). Evaluation criteria for Kappa is that values above 0.74, between 0.6 and 0.74, and the ones between 0.4 and 0.59 are considered to be excellent, good, and fair, respectively [9,13].
Content Validity Ratio (CVR)
CVR according to the Lawshe test is computed to specify whether an item is necessary for operating a construct in a set of items or not. For this, the expert panel was asked to give a score of 1 to 3 to each item ranging from essential, useful but not essential, and not necessary. The formula for computation of CVR= (Ne – N / 2) / (N / 2) in which Ne is the number of panelists indicating “essential” and N is the total number of panelists. The numeric value of CVR ranges from -1 to 1 (Lawshe, 1975). High scores of CVR indicate the agreement of members on the necessity of an item in the instrument (Ayre & Scally, 2014). A positive CVR indicates that atleast half of the panelists agree on the necessity of the item for the construct [15,16].
2.2.2 Face validity
Face validity answers this question whether an instrument apparently has validity for subjects, patients and/or other participants. Face validity means if the designed instrument is apparently related to the construct underlying study. The face validity of the tool was assessed using the calculation of item impact score. The experts were required to evaluate the items with respect to 10 domains i.e. Matches Understanding Level, Technical content ok, Has logical sequence of questions, Continuity of items is fine, Language is understandable, Terminology, Given options are simple to understand, Is not loaded with unnecessary information, Permits answering properly and Is useful with practical value using the 4 point Likert scale i.e. strongly disagree, disagree, agree and strongly agree ranging from 1 to 4. All the questionnaires were collected and analysed, the impact score was computed for each item using the formula: Impact Score = Frequency (proportion of raters who scored 3&4) * Importance (mean score for the importance on the basis of domains). If the item impact of an item is equal to or greater than 1.5 (which corresponds to a mean frequency of 50%), it is maintained in the instrument; otherwise it is eliminated [9,17].