The Psychometric Properties and Performances of Teacher-Made Tests in Measuring Students’ Academic Performance in Ethiopian Public Universities: Baseline Survey Study

doi:10.21203/rs.3.rs-3095433/v1

Download PDF

Research Article

The Psychometric Properties and Performances of Teacher-Made Tests in Measuring Students’ Academic Performance in Ethiopian Public Universities: Baseline Survey Study

https://doi.org/10.21203/rs.3.rs-3095433/v1

This work is licensed under a CC BY 4.0 License

Version 1

posted

You are reading this latest preprint version

All education levels today require quality tests to infer students' scores in judging the quality of teaching and learning. In this judgment, as many scholars argue, teacher-made tests (TMTs) are significant and more promissory than other tests. Besides their benefits, TMTs are prone to measurement error if they do not satisfy assumptions about the psychometric properties and performance of tests. Thus, this study aimed to survey the psychometric properties and performance of teacher-made tests (TMTs) in measuring students’ academic performance using pieces of evidence from three Ethiopian public universities. The researchers employed the post-positivist research paradigm, quantitative research approach, cross-sectional research design, purposive sampling size technique, CTT as the theoretical framework, and SPSS version 28 software as a statistical analysis tool. Empirical data reveal Ethiopian public universities' TMTs quality is below the expected CTT test item standard qualities [content validity, S-CVI=.65<.83, IC reliability, α=.57<.6-.7, difficulty (p)=.49<.5-.6, Skewness=-.300 and D=-0.01<.2], requiring curriculum developers to design test tables, teachers to demonstrate professional accountability, and course offering departments to have a TMTs validation team. Future studies need to include intervention and longitudinal evidence to clarify the association between TMTs and academic performance.

Educational Psychology

Psychometric properties

Public universities

Students’ academic performances

Teachers-Made Tests

Measurement is the knowledge that enables us to quantify things, and measures are the instruments we employ to quantify those traits. Estimation refers to the calculations that produce our measure and ensure the accuracy of our quantification (Mohammad, 2022).

Since its inception, educational measurement has been the cornerstone of all educational practices, and results on assessments are critical in determining the effectiveness of teaching and learning (Bijlsma, 2021; Mohammad, 2022). Despite the fact that tests have been criticized by academics for measuring extremely small samples of behavior, it is still possible that they will remain important measures for a long time to come because it is difficult to obtain other measures of behavior that would resemble testing.

According to Dickson (2020), tests might be diagnostic, teacher-made, or standardized. The teacher uses diagnostic tests to get information about the students' academic progress. By dividing the subjects into units, the teacher approaches this during the learning process. Teacher-made tests (TMTs) can take the form of a comprehensive exam, a weekly test, a unit test, or a term-ending exam. Such tests are unique in that they are created by classroom teachers and measure learning outcomes related to a particular course, but standardized tests (STs) are an overly comprehensive test that is administered at the beginning or end of the educational program (Mohammad, 2021).

In contrast, many studies show that teachers' accountability for developing useful measurement tools has been questioned at all educational levels (Adeyemo, 2021; Quansah, 2019; Singh, 2022). This is true even though many academics have pronounced the importance of TMTs above STs. These teachers' actions lessen the value of the test results since the existence of these teacher-related factors introduces errors into students' scores. Accordingly, academics contend that TMTs must meet expectations for a set of theoretical attributes that are officially referred to as the test's psychometric properties and performances (Cordova, 2018; Mohammad, 2022; Suppiah, 2020).

Psychometric properties are concerned with the validity, reliability, objectivity, fairness, and usability of tests (Cordova, 2018; Mohammad, 2022). Validity and reliability are the core characteristics of the tests when it comes to psychometric attributes (Espinoza, 2021). Content validity, for instance, is the most pertinent one in this circumstance, even though other aspects of validity are significant in teaching and learning since they allow for the evaluation of the success of each syllabus's and course's contents (Mohammad, 2022; Simachew, 2019). Otherwise, students will neglect paying close attention to, studying, and honing their skills in those courses, especially those that focus on language and other skills that do not appear or appear less frequently on exams (Mekonnen, 2017; Simachew, 2019). Similar to content validity, of forms of reliability, internal consistency (IC) is feasible for classroom test context (Mohammad, 2022). It is applied not to one item but to groups of items that are thought to measure different aspects of the same construct.

Regarding test performances, difficulty level, discrimination power, and distractors effectiveness are the basic concerns (Suppiah, 2020). Satisfying these theoretical assumptions is about ensuring the test’s quality and those students are assessed properly, fairly, and objectively (Butakor, 2022).

According to Lemecha's (2020) study, the student evaluations of teaching (grading system) in HEIs in Ethiopia followed the curriculum revisions. Various foreign influences also had an impact on these curriculum adjustments. Let's begin with the British effect on Ethiopian education in the 1950s. The majority of the exam formats at the time were essay formats. The evaluation process included criteria-referenced scoring. The objective system dominated the tests in the 1960s, when the US had a significant influence on Ethiopian education. The norm-referenced assessment approach, popularized in the United States, predominated the evaluation standards up until the fall of the Soviet Union. After that, in particular starting in 2004, the HEIs measures of validity, trustworthiness, fairness, performances, and students’ academic performance problems in Ethiopian history started escalating (Lemecha, 2020; Metages, 2019; Tesfamariam, 2021).

Despite Ethiopian universities' efforts to improve the quality and relevance of higher education to the needs of the market and the nation's development, including the use of standardized curricula and the introduction of criterion-referenced and continuous assessments, the desired impact on students' competence has not been as great as anticipated (Eyob, 2022; Metages, 2019). In this, there are some gaps in focusing solely on the curriculum and mode of assessment while ignoring the assumptions about the quality of the measurements.

The psychometric features and performances of TMTs in measuring students' academic performance in full-fledged courses are not clearly evident in research on Ethiopian scholars (Sewagegn, 2019). For instance, Metages' (2019) study examined how TMTs' content validity affected teachers' and students' perceptions, attitudes, and motivations, as well as the activities and choice of teaching-learning materials in the English language course at Ambo University. This is where there is a gap because TMTs also need to have reliability, a degree of difficulty, and discrimination power in addition to content validity (Marjorie, 2016). Similar to this, Motuma (2019) made an effort to investigate the content validity of EFL TMTs at Ambo University. The overall result of his study revealed that the TMTs' content validity was extremely low (0.211), which indicates adverse consequences of the assessments on the course's teaching-learning process. Here, the gap is comparable to that of Metages' (2019) study, which found that it was challenging to assess the quality level of TMTs solely by looking at their content validity. The same gap is revealed by a similar study on the content validity of English language courses conducted by Simachew (2019) at Debre Work Preparatory School, East Gojjam, in the pre-university previous Ethiopian educational structure. In that study, the author himself forwarded implications for further study, stating that considering other qualities of TMTs like reliability, practicability, other validity types, item difficulty, and discrimination power could be an agenda for other research.

Since higher education is now regarded as a requirement for success in a technologically, economically, and politically advanced world, troubling questions about the measuring tools used to measure students' academic performance in Ethiopian HEIs have prompted researchers to conduct research in this area. To accomplish this, the researchers determined the study context by choosing one university from each of the three categories of Ethiopian public universities: Jimma, Wollega, and Ambo from established, new, and emerging universities, respectively, taking into consideration their representativeness of the other universities because Ethiopian public universities are all almost uniform in their curricula and in their similar directives, policies, rules, or regulations of assessments and exam procedures established by the MOE (Eyob, 2022; Metages, 2019).

The study's evidence for the learning subject (course), English language communicative skill-I, which is offered as a common course for all departments, was purposefully chosen because language is thought to be an important factor in teaching and learning (Le Thi, 2021); in addition, Ethiopian public universities use English as their medium of instruction. In keeping with this, numerous academics have looked into and argued that English continues to be taught in universities even when other languages are employed to teach the curriculum's content (Alghadri, 2019; Ehsan, 2019).

As a result, the researchers were motivated to conduct a baseline survey study on the psychometric properties and performances of English language communicative skill TMTs in measuring undergraduate students' academic performance in Ethiopian public universities. They also intended to conduct an interventional study on the corresponding issue. In order to achieve this, the following research questions (RQs) are created:

1. What are the psychometric properties of TMTs in measuring students’ academic performances?

2. What are the performances of TMTs in measuring students’ academic performances?

A research methodology is a philosophical structure with crucial suppositions directing the research (Kivunja, 2017; Oluseye, 2022). Thus, this part discusses the research paradigm, approach, design, and methods of sampling, data collection tools, and data quality control of this baseline survey study.

2.1. The Research Paradigm

A research paradigm is a conceptual lens through which researchers evaluate the methodological components of their research project to form and choose the study approach, design, and methods that will be applied. (Creswell, 2018). The post-positivist paradigm, which is one of the more popular ones, is used in this study. Other prevalent paradigms include positivist/post-positivist, constructivism/interpretive, transformative/critical, and pragmatic (Creswell, 2018). This is due to the fact that the investigation in the current study focuses on the reality of how accurately TMTs are evaluated and what they were intended to measure in terms of students' academic performance in comparison to the theoretical assumptions of tests. As a result, post-positivism is the paradigm that can work since it allows for the analysis of human behavior (Kivunja, 2017).

Post-positivists

acknowledge the imperfections of reality and the probabilistic nature of truth. It permits making observations without performing experiments or coming up with and testing hypotheses. In light of this, the post-positivist paradigm has a tendency to offer the worldview for the majority of research on human behavior undertaken in educational environments (Kivunja, 2017). As a result, this paradigm would be appropriate for gathering and examining student test results to assess the effectiveness of TMTs. The fact that post-positivism acknowledges that reality can only ever be approximated and never fully comprehended is another rationale for using it in the current investigation (Kivunja, 2017). This paradigm permits additional intervention studies because the purpose of the current baseline survey study necessitates further exploration. In the current study, ontology, epistemology, methodology, and axiology must be followed in order to articulate the post-positivist aspects because determining the reality of students' academic performance using the measuring tool's qualities is hard.

Ontology

Post-positivists contend that although reality exists, it can only be partially understood because of researcher constraints (critical realism) (Kivunja, 2017). They allow researchers to uncover the truth within a particular region or realm of probability. The current researcher therefore articulated and described reality about the characteristics of TMTs from the perspective of objective truth, using statistical results' assumptions to control the likelihood of the researcher's own opinions and values influencing what are being examined.

Epistemology: According to the post-positivist paradigm, relationships in research are best determined by what the researcher believes is acceptable for that specific study (Kivunja, 2017). The academic performance of students as measured by TMTs in the current study is expressed in terms of indirect numerical scores; teachers' relationships with tests are direct but indirect compared to what students actually know. Taking these into account, the researcher communicated from knowledge sources (intuitive, authoritative, logical, and empirical knowledge) the patterns of TMTs data, the responses of previous TMTs students, and the authoritative scholars' theories in order to infer the truth of the study from various perspectives.

Methodology

According to Kivunja (2017), the post-positivist research paradigm favors a quantitative research strategy. The researchers suggested using a quantitative method to solve this situation. The researchers chose a cross-sectional survey design over the alternatives for the research design. The cross-sectional survey research design philosophy is exceptional because it offers a reliable database for various types of research regarding the present status of the situation or event, correlated events related to the issue, etc. The researchers chose to adhere to the cross-sectional survey research design philosophy since it is compatible with the current investigation.

Axiology

is the idea of how researchers should prevent or minimize danger or harm, whether it be psychological, legal, societal, economic, or other types. According to the post-positivist perspective, ethics is the ethical responsibility of the researcher to carry out high-quality research. According to this paradigm, excellent research entails the suppression of personal bias, thorough data collection and reporting, and the open acknowledgment of the limitations of the scientific dependability of empirical studies (Kivunja, 2017). The three guiding ethical principles are beneficence, respect, and justice, as this author explains. According to the concept of beneficence, good outcomes should be maximized for science, mankind, and each research participant, while unneeded risk, injury, or wrongdoing should be minimized or avoided. Respect means exhibiting consideration for others, even those who lack autonomy.

2.2. Research Approach

Research approaches, according to Creswell (2018), are strategies and guidelines for conducting studies that cover everything from general hypotheses to specific techniques for gathering, analyzing, and interpreting data. There are a number of choices in this strategy, and they don't all have to be made in the order that makes sense to us or to our target group. Which approach should be employed to study a topic is the main decision. This choice should be based on the researcher's philosophical presuppositions, the research designs, and the particular data collection, analysis, and interpretation techniques that were used.

Creswell (2018) and Oluseye (2022) identified three research approaches. These are mixed, qualitative, and quantitative approaches. The kind of research topic or issue being addressed, the researchers' individual experiences, and the study's target audiences are all taken into consideration when choosing a research approach. As a result, this baseline survey study employed a quantitative approach to inquiry. Quantitative research is an approach for testing objective ideas by looking at the relationship between variables (Creswell, 2018). To enable statistical analysis of numbered data, these variables can be measured, often using instruments. An introduction, literature and theory, method, results, and reflection make up the predetermined format of the final written report. A quantitative approach to research is appropriate because the current study, the psychometric qualities, and the performances of TMTs in measuring students' academic performance are more prone to testing theoretical presumptions in nature.

2.3. Design of the Study

Research designs, according to Creswell (2018), are types of inquiry that fall under the qualitative, quantitative, and mixed approaches and offer precise guidance for steps in a research study. Since this study takes a quantitative approach, the researchers chose a non-experimental quantitative design over an experimental one. Prefer cross-sectional survey design over longitudinal survey design from non-experimental (longitudinal and cross-sectional surveys). According to Creswell (2018), the most common type of survey design used in education is the cross-sectional design, which offers a quantitative description of the present attitudes, opinions, or practices of a population or checks for associations between variables in that population by looking at a sample of that population. Researchers can also respond to three different kinds of questions using a cross-sectional survey design: questions that are descriptive, questions that examine the correlations between variables, and questions that examine the predictive power of those relationships over time. The researchers used a cross-sectional survey design in the current study since the goal was to evaluate the measuring quality status of TMTs and then draw conclusions for subsequent research.

2.4. Methods

The types of data collection, analysis, interpretation, and control that researchers propose for their investigations, as well as sampling-based study methods (Creswell, 2018). As a result, the procedures used in this investigation are all covered below.

2.4.1. Area of the Study and Sampling

Borden (2018) asserts that the study issue the researcher is addressing, as well as the location's possible costs, convenience, and ethical implications, all influence the choice of setting. Similar to this, Creswell (2014) (as cited in Bekalu, 2021) asserts that it is crucial to identify the sample frame of prospective respondents in the population since doing so aids the researcher in choosing the sampling methods and resources to employ. In order to do this, the sample frame for this baseline survey study was specifically held in a few chosen public universities in Ethiopia, particularly Jimma, Wollega, and Ambo universities. These three public universities were chosen for a variety of reasons. The first is that they can also represent the other public universities in Ethiopia because these learning institutions share curricula and similar MOE directives, policies, rules, or laws on assessments and examination practices (Eyob, 2022; Metages, 2019). The second factor is that the researchers have spent more than 20 years living and working in this city, and the corresponding author completed both his undergraduate and master's degrees at two distinct public universities. As a result, the researchers have firsthand knowledge of the issue and are familiar with how it affects the targeted area. Additionally, they are more practical for the researchers and situated near where they do their jobs. This enables the researchers to simply manage the study's plans while carrying out the investigation.

In addition to the aforementioned factors, the researchers also kept the sample representative by taking into account the different types of Ethiopian public universities. For instance, according to the Bekalu (2021) study, Ethiopian public universities can be divided into three generations based on their establishment year and student population sizes: established, new, and emerging. Established universities are those founded between 1950 and 2000 and have a relatively larger student and teacher population and better facilities when compared to emerging and new universities. New universities were established between 2006 and 2009 and are in the middle between established and emerging universities in terms of student population and educational infrastructure. Emerging universities are relatively younger universities (established in 2010 and 2011) with a relatively smaller number of students and teachers and undeveloped educational facilities. Accordingly, the researchers considered three universities, one from each category: Jimma, Wollega, and Ambo are from established, new, and emerging universities, respectively, with two classrooms each as a sample. Moreover, it is difficult to manage a large number of students from different universities and to analyze test items for each student in different universities when the scope of the study goes beyond the selected university.

The English language is specifically chosen to act as the study's evidence with regard to learning subject (course) selection. As a foundation, language is thought to be a crucial factor in teaching and learning (Le Thi, 2021). In addition to this, the English language is used as a medium of instruction in Ethiopian public universities. Accordingly, a number of studies have been conducted, and it has been discovered that English continues to be taught in universities even when other languages are utilized to teach the curriculum's content (Alghadri, 2019; Ehsan, 2019). Foreign languages play a significant role as an evolving international standard (Le Thi, 2021). English is a key foreign language for both native speakers and non-native speakers (Hariharasudan, 2018; Sukarno, 2020). In other words, English is classified as a lingua franca (ELF) since it is widely utilized in fields such as technology, science, and commerce (Hariharasudan, 2018) and is a common language among people who do not share the same language or culture. It makes sense to pay special attention to English language instruction given its significance, necessity, and intermediary nature (Görkem, 2021). Thus, it is suggested that the study's supporting data come from an English language communicative skills course that is available to all departments.

In terms of respondent sampling, which involves choosing TMTs sheets, the researcher used a purposive and full population sampling technique in the scenario that the number of TMTs item responders is low. The full class was used for the survey because less than 100 pupils were anticipated in one classroom.

2.4.2. Target Population

Creswell (2018) contends that defining the sample frame of prospective responders in the population is crucial because it aids the researcher in choosing the appropriate resources and sampling methods. Accordingly, the English language communicative skill-I TMTs sheets and first-year student scores from the 2022 G.C. academic year at Jimma, Wallaggaa, and Ambo universities serve as the sample frame for the study.

2.4.3. Data Collection Instruments

2.4.3.1. Test

TMTs are the data-gathering instruments in the present baseline survey study. A test takes a sample of the respondent’s behavior to evaluate the individual’s more general understanding competence. Instead, test results are an accurate reflection of the course, the teacher, and the test itself (Bijlsma, 2021; Polat, 2022). Similar to this, item analysis (P-level and D-power) was empirically demonstrated by Bichi (2018) and Suppiah (2020) to be crucial to any test development. Item analysis analyzes students' responses to individual test items to evaluate the quality of those items and the quality of the test as a whole. The psychometrics of tests apply a variety of statistical analyses to the precision of performance measurements and give the tester in-depth knowledge of the test's validity and reliability (Cordova, 2018; Mohammad, 2022). These imply that the effectiveness of measuring tools can be assessed based on the psychometric characteristics and performance of TMTs.

As a result, by using TMTs as a tool for data collection in the current baseline survey study, researchers were able to deduce the CVI from experts' TMT content validation as well as the IC reliability, P-value, and D-power of TMTs from students' replies to TMTs items. In order to do this, subject matter instructors were asked to submit examinations they had created for the 2022 academic year.

2.4.4. Methods of Data Analysis

The statistical analysis and interpretation assumptions used by Creswell (2018) served as the basis for the researcher's data analysis method, which was mostly CTT-rooted. In this case, the researcher first gathered data from the databases of the English language departments of the three institutions (Jimma, Wallaggaa, and Ambo) and then used CTT statistical equation models to evaluate the data to preserve statistical validity.

Accordingly, to estimate the psychometric properties and performances of TMTs in measuring students’ academic performance, all TMTs obtained from respective universities' content relevance was estimated based on four rating scales (i.e., not relevant, somewhat relevant, quite relevant, and highly relevant) that were rated by six experts following the CVI formula and assumptions shown in Yusoff (2019), IC reliability by Cronbach alpha, and tests’ performances (P-level and D-power) were estimated based on students test item response in line with the formula and assumptions shown in Butakor (2022) study.

2.4.5. Data Quality Controlling Mechanisms (Rigor) of the Study

Quantitative data usually rely on quality criteria, including validity and reliability (Cecilia, 2021).

Research Validity: By validity, we mean that a research study, its components, the conclusions reached, and the applications based on them can be of high, low, or in between quality. The validity issue, at least as we use the term, is not about singular truths, and it most certainly is not limited to quantitative measurement. Research validity is often classified into four dimensions: internal validity, external validity, construct validity, and statistical validity (Analay, 2019), and in this study, they were maintained. In this situation, to maintain internal validity, the study employed homogenous participants (the same department, year, semester, and class), whereas external validity was maintained by taking samples from each generation (established, new, and emerging universities) to ensure representativeness. Concerning construct validity, the choice of instrument, data analysis, and interpretation were done in line with the CTT. To maximize the statistical validity of the study, descriptive and inferential statistics were utilized with their necessary assumptions aligned to the RQs formulated.

Accordingly, in RQ₁, the psychometric properties of TMTs in measuring students’ academic performance were estimated by CVI (for content validity) and Cronbach's alpha (for IC reliability). RQ₂ about the performances of TMTs (p-level and D-power) in measuring students’ academic performance was estimated using the proportion formulas of item analysis. These are to follow the view of the previous study by Analay (2019), which says statistical validity refers to the quantitative evaluation of the soundness of the conclusions drawn from the results of a study. Thus, steps are believed to maximize the predictive validity of the research tools and the soundness of the findings.

Research Reliability: Repetition-based consistency is referred to as reliability (Daud, 2018). Therefore, a study is deemed reliable if it can be duplicated under comparable circumstances using the same methods and produces results that are equivalent to those of earlier studies. There are two types of reliability: external and internal. External reliability is the degree to which an independent investigator can repeat a study and come up with conclusions similar to those drawn in the previous study. To this end, the researchers discussed all findings, whether they were consistent with previous empirical studies or not. Internal reliability refers to the consistency of data collection and interpretation procedures. That is, internal reliability is a question of whether research replicated through the same methods and procedures leads to the same findings observed during the first investigation. Again, to this end, the researchers followed the scenic procedures of the area and maintained the interdependency of the study components from end to end.

2.5. Ethical Issues

For ethical issues, the study was conducted with the consent of the universities, explaining that the objective of this study is purely academic and supported by the university’s official letter. In all processes, the researchers considered ethical issues like trustworthiness, balance, fairness, and respect for human relations between researchers and universities. As with any researcher, the researchers attempted to present a balanced report of the findings for all stakeholders, in particular for that course-offering university; they submitted the study’s final report and findings to Jimma University (JU) in softcopy and hardcopy for oral presentation to get recognition from the university before releasing it as a permanent research document.

The analysis, discussion, and interpretation of the data in this part deal with the quality of TMTs in measuring students’ academic performance in Ethiopian public universities. The result presentation is based on data collected from different public university categories: established, new, and emerging universities, by employing English language communicative skills (I TMTs) sheets for the 2022 academic year. In this, three public universities, one from each category, with one natural and one social sciences classroom, a total of three social and three natural science classrooms in a grand total of six classrooms, were involved. Fortunately, in all universities, English language communicative skills are developed at the English language department level.

Technically, the analysis relies on students’ scores, item responses, and experts in the area’s validation. Content validity index (CVI), Cronbach alpha, difficulty level (P), and discrimination power (D) have been used to indicate the TMTs quality status in measuring students’ academic performance. For analysis, percentages have been used to indicate the characteristics of the study population. The percentage is also used to indicate the magnitude that each score possesses.

For the sake of systematic analysis, this part is organized into two parts: while the first part describes the respondents, the second part looks into the background information on the sample respondents to build up the findings. To address the research's basic questions, the second part of this part is organized into two main categories: the psychometric properties of TMTs and the performance of TMTs in measuring students’ academic performances in Ethiopian public universities.

3.1. Characteristics of the Respondents

The necessary data for this research were obtained from the Ethiopian public universities’ English language communicative skills I TMTs sheets. In all universities, TMTs are developed in groups at the English language department level. Hence, it can be said that information obtained from TMTs at the same time represents both teachers and universities' measuring qualities. To capture the full picture, the researcher targeted both natural and social sciences students. Again, a content relevancy rating scale (not relevant, somewhat relevant, quite relevant, and highly relevant) was prepared for experts in the area of the Yusoff (2019) study. Accordingly, based on the responses obtained from the respondents (TMT sheets and experts) regarding their background information, some major characteristics, such as established universities, new and emerging categories, students’ departments, and experts' educational levels, are presented hereafter.

3.1.1. Descriptions of Respondents by University Categories

For the sake of obtaining representative data, universities were targeted according to their categories. Criteria for the category used were those of the Bekalu (2021) study, which indicates that based on their year of establishment and sizes in terms of the student population, Ethiopian public universities are categorized into three generations: established, new, and emerging. In the present baseline survey study, Jimma University, Wollega University, and Ambo University were selected as established, new, and emerging universities, respectively.

Table 4.1

Descriptions of Respondents by University
University	Respondents	Frequency in terms of classroom	Percentage (%) in terms of classroom	Frequency in terms of students number	Percentage (%) in terms of students number
Established	Social science	1	16.67	40	16.46
Established	Natural science	1	16.67	40	16.46
New	Social science	1	16.67	34	13.99
New	Natural science	1	16.67	48	19.75
Emerged	Social science	1	16.67	40	16.46
Emerged	Natural science	1	16.67	41	16.87
Grand Total	Social science	3	50	114	47
	Natural science	3	50	129	53
	Social and Natural sciences	6	100	243	100

Table 4.1 depicts the nature of the participants in terms of universities. As one can witness from the table, of the total participants, 2 classrooms (1 classroom = 16.67% social and 1 classroom = 16.67% natural sciences) were from each university category (i.e., 33.34% from established, new, and emerging universities, respectively), and in total, 6 classrooms (3 classrooms = 50% social and 3 classrooms = 50% natural sciences), whereas in number 243 (47% social and 53% natural sciences) students were the respondents of the study. This enables the study to capture representative data and maintain proportionality in the analysis.

3.1.2. Descriptions of Experts

In this study, experts were required to validate the relevancy of the content. It can be agreed that for content validation, the minimum acceptable expert number is two; however, most recommendations propose a minimum of six experts (Yusoff, 2019). Considering the recommendations and the researchers’ experience, the number of experts for content validation should be at least six and not exceed 10. Thus, six experts have been used in this study.

Table 4.2

Descriptions of Experts
Experts
Department	Educational Level	Number (N)
English Language	PhD	6

We can infer from Table 4.2 that the number of experts involved in content validation was recommended at six, and their educational preparation (PhD) would help them validate the quality of TMTs accordingly, which directly has a positive contribution to the CVI. Therefore, what can be concluded from the backgrounds of the experts' collected data is that it is trustful to infer generalizations about the content validity of Ethiopian public universities’ TMTs.

3.2. Major Findings

For the sake of data result analysis, this part of the study was further categorized into two themes with two sub-categories, each considering the basic RQs on which the data of the study was collected. The categories are the psychometric properties of TMTs (content validity and IC reliability) and the performances of TMTs (P-level and D-power) in measuring students’ academic performances.

The raw data collected on the RQs was entered into SPSS version 28 software. The quantitative data was obtained from TMT sheets, and the researcher first edited, categorized, and finally described it using statistical techniques such as percentage, mean, and standard deviation. CVI, Cronbach alpha, difficulty level (P), and discrimination power (D) have been used to indicate the TMT's quality status.

The cut-off point assumptions followed are as indicated in Butakor's (2022) and Yusoff's (2019) studies. For CVI, it is as indicated in Yusoff's (2019) study. That is, if the judgment is by two experts at least 0.80, if three to five experts should be 1.00, at least six experts at least 0.83, six to eight experts at least 0.83, and at least nine experts at least 0.78, For IC reliability, according to the assumptions shown in Butakor (2022), a Cronbach alpha of 0.6–0.7 indicates acceptable reliability. Similarly, the assumptions shown in Butakor (2022) were utilized for both P and D. For P, to determine the proportion of the number of students who were able to correctly and incorrectly answer all the questions tested, the cut-off score utilized was P between 0.25–0.75, with an average P = 0.6 as ideal. Whereas for D, if D < 0.20, it is interpreted as rejected, if D = 0.20-.39, it is accepted, and if D > 0.40, it is interpreted as excellent.

3.2.1. The Psychometric Properties of TMTs in Measuring Students’ Academic Performances in Ethiopian Public Universities

The present analysis produces information about the Ethiopian public universities TMT's effectiveness status in measuring students’ academic performance based on the parameters of items or sets of questions, namely content validity and internal consistency reliability. This objective is supported by Espinoza (2021), who argues validity and reliability are the two most important and fundamental qualities of psychometric properties of measurement scales.

3.2.1.1. The Content Validity of TMTs in Measuring Students’ Academic Performances in Ethiopian Public Universities

All TMTs obtained from Ethiopian public universities (established, new, and emerging universities) were estimated based on four rating scales (i.e., not relevant, somewhat relevant, quite relevant, and highly relevant) that were rated by six experts following the CVI procedures, formula, and assumptions shown in Yusoff (2019). Accordingly, six steps of content validation were followed by the researcher. The first step was to prepare the content validation form to ensure the review panel of experts would have clear expectations and an understanding of the English language's communicative skills. The second step was selecting a review panel of experts. Considering the recommendations about CVI and the researchers’ experience, the number of experts for TMTs that CVI utilized was six. Step three was conducting content validation. Taking into account the experts’ educational preparation, the non-face-to-face approach was identified as very efficient since a systematic follow-up was in place to improve the response rate and time. In step four, the experts were provided and requested to read the course materials, like the course’s curriculum, students’ modules, and course outline, in order to critically review the course’s domain and its TMT items before providing a score on each item. In step five, the experts were requested to provide a score on each item independently based on the relevant scale (not relevant = 1, somewhat relevant = 2, quite relevant = 3, and highly relevant = 4). The last (six) step was calculating CVI at item level (I-CVI) and scale level (S-CVI). Before the calculation of CVI, the relevance rating was recorded as 1 (relevance scale of 3 or 4) or 0 (relevance scale of 1 or 2). In this, I-CVI is the expert in agreement divided by the number of experts, whereas S-CVI/Ave (based on I-CVI) is the average of I-CVI scores across all items. Based on these calculations, the researcher concluded the fitness of the I-CVI and S-CVI/Ave cut-off scores sat by CTT. This cut-off point for CVI is as indicated in Yusoff (2019). Thus, since the judgment was by six experts, the cut-off was decided to be at least 0.83.

Table 4.3

The Content Validity of TMTs in Measuring Students’ Academic Performances in Ethiopian Public Universities
University	Number of Item	S-CVI
Established	75	0.66
New	81	0.60
Emerged	72	0.69
Grand Mean		0.65

Table 4.3 shows all universities as individuals or groups with an S-CVI < .83 (established = .66, new = .6, emerging = .69, and their grand mean = .65), which is below the acceptable cut-off score of CVI. This leads to the conclusion that, although content validity is the most important because it can measure the entire material that has been taught, Ethiopian public universities’ TMTs content validity is below expectations.

3.2.1.2 The Internal Consistence (IC) Reliability of TMTs in Measuring Students’ Academic Performances in Ethiopian Public Universities

The IC reliability of TMTs in measuring students’ academic performance in Ethiopian public universities (established, new, and emerging universities) was computed by Cronbach’s alpha based on students' test item response data obtained from TMTs sheets in line with the formula and assumptions shown in the Butakor (2022) study. To this end, the researchers entered all students’ test item responses into SPSS-28 and computed the TMTs IC reliability for each university separately with Cronbach's alpha. Lastly, their grand mean was computed to judge the overall status of Ethiopian public universities' TMTs IC reliability in measuring students’ academic performance.

Table 4.4

The IC Reliability of TMTs in Measuring Students’ Academic Performances in Ethiopian Public Universities
University	Number of Item	Cronbach's alpha
Established	75	0.57
New	81	0.59
Emerged	72	0.55
Grand Mean		0.57

Table 4.4 reveals all universities as individuals or groups, their TMTs IC reliability α < .6 (Established = .57 < .6, New = .59 < .6, Emerging = .55 < .6, and their grand mean = .57 < .6) which is below the commonly-accepted rule that an alpha of 0.6–0.7 indicates acceptable reliability. This leads to the conclusion that, although TMTs IC reliability is vital to ensure overall reliability in measuring students’ academic performance, Ethiopian public universities’ TMTs IC reliability is below expectations.

3.2.2. The Performances of TMTs in Measuring Students’ Academic Performances in Ethiopian Public Universities

The analysis produces information about the Ethiopian public universities' TMT quality status in measuring students’ academic performance based on the parameters of items or sets of questions, namely the difficulty level (P) and discriminating power (D). This objective is consistent with Suppiah's (2020) argument that item analysis is essential to any test construction as it examines students’ responses to individual test items to assess the quality of those items and the quality of the test as a whole.

3.2.2.1. The Difficulty level (P) of TMTs in Measuring Students’ Academic Performances in Ethiopian Public Universities

The Ethiopian public universities’ TMTs difficulty level (p-value) was estimated based on students' test item responses obtained from established, new, and emerging universities' objective type of TMTs sheets in line with the formula and assumptions shown in Butakor (2022) study. This P-level was done for the respective universities to determine the proportion of the number of students who were able to answer correctly and incorrectly answer from all the questions tested. The P-value obtained for each item is computed further as grand mean or aggregation to test level (university level) P-value.

Table 4.5

The Difficulty level (P) of TMTs in Measuring Students’ Academic Performances in Ethiopian Public Universities
University	Number (N) of Items	Difficulty Level (P = value)
		Distribution (Frequency)			Aggregation	Skewness
		P < 0.25 (Difficult)	P = .25 − .75 (Moderate)	P > .75 (Easy)	Aggregation	Skewness
Established	75	5(6.66%)	67(89.33%)	3(4%)	0.51	− .583
New	81	20(24.69%)	58(71.60%)	3(3.70%)	0.44	− .056
Emerging	72	4(5.55%)	59(81.94%)	9(12.50%)	0.53	− .200
Grand Mean					0.49	− .300

Table 4.5 above shows that even though all universities’ aggregation p-values (Established = .51, Skewness = − .583, New = .44, Skewness = − .056, and Emerging = .53, Skewness = − .200, and their grand p-level = .49, Skewness =-.300) were in the cut-off score p between 0.25–0.75 and negatively skewed, they did not fit the assumption of normal distribution expected (in which difficult and easy items are expected to be 25% each in amount). In this situation, since the established university’s difficult = 6.66% < 25% and easy items were 4% < 25%, the new university’s difficult items were 24.69%, almost equal to 25%, but easy items were 3.70% < 25%, and the emerging university’s difficult items were 5.55% < 25%, and easy items were 12.50% < 25%) we can say it was not normally distributed. Moreover, in all universities, as individuals or groups, their TMTs P-value is below the recommended ideal p = .6. Hence, it can be said that Ethiopian public universities’ TMTs yield some score distribution that makes it hard to identify reliable differences in academic performance between the students who have done well and those who have done poorly.

3.2.2.2. The Discrimination (D) power of TMTs in Measuring Students’ Academic Performances in Ethiopian Public Universities

In this dissertation, the Ethiopian public universities’ TMTs D-power was computed based on students' test item responses obtained from the established university (75 items), the new university (81 items), and the emerging university (72 items) objective type of TMTs sheets in line with the formula and assumptions shown in Butakor's (2022) study. Accordingly, to involve each examinee in the computation, the mathematical formula utilized was a point-biserial correlation coefficient (r_pbi): \({\mathbf{r}}_{\mathbf{p}\mathbf{b}\mathbf{i}}= \frac{{\text{M}}_{\text{p}}- {\text{M}}_{\text{q}}}{{\text{S}}_{\text{t}}}\sqrt{\text{p}\text{q}}\) where, Mp = whole-test mean of examinees who answered an item correctly, Mq = whole-test mean for examinees who answered an item incorrectly, St = standard deviation for the whole test, and p = proportion of examinees answering correctly q = proportion of examinees answering incorrectly.

Table 4.6

The D-power of TMTs in Measuring Students’ Academic Performances in Ethiopian Public Universities
University	Number (N) of Items	Discrimination power (D = value)
		Distribution			Aggregation
		D < 0.20 (Rejected)	D = 0.20–0.39 (Accepted)	D > 0.40 (Excellent)	Aggregation
Established	75	52(69%)	18(24%)	5(7%)	0.03
New	81	55(67.90%)	20(24.69%)	6(7.41)	-0.12
Emerging	72	44(61.11%)	17(23.61%)	11(15.28%)	0.07
Grand Mean					-0.01

Table 4.6 above depicts TMTs in all universities with a high percentage of D < .2 (Established = 69%, New = 67.90%, and Emerging = 61.11%), which is below the cut-off score of accepted discrimination power, and a few D > .2, including an excellent (Established = 31%, New = 32.10%, and Emerging = 38.89%). This implies that Ethiopian public universities’ TMTs could not sufficiently discriminate against students based on their academic performance.

The discussion of the study is to interpret the quantitative findings, linking them with the theoretical framework and empirical studies. This is to show to what extent these quantitative findings support or challenge the working theoretical framework and what kind of research ought to be done in the future to build up the study. To this end, the researchers compared and contrasted the study's basic RQ results with CTT assumptions and previous studies.

Accordingly, the statistical results of RQ₁: "What are the psychometric properties of TMTs in measuring students’ academic performances in Ethiopian public universities?" reveals that both content validity (S-CVI = .65 < .83) and IC reliability (α = .57 < .6) are below the cut-off score standards set in CTT. The finding implies that although content validity is the most important because it can measure the entire material that has been taught (Nana, 2018) and IC is vital to ensure overall reliability (Mohammad, 2022), Ethiopian public universities’ TMTs content validity and IC reliability in measuring students’ academic performance are below the cut-off score commonly accepted by the CTT rule that was indicated in Butakor (2022) and Yusoff (2019) studies. This finding is consistent with the Espinoza (2021) study, which argues that unless a test measures consistently whatever it does measure, it cannot hope to accurately measure what it claims to measure, and for a test to be perfectly valid, it must be perfectly reliable. Therefore, based on the CTT cut-off criteria and Tengku's (2020) study argument, decisions made based on the Ethiopian public universities students’ learning competence reports are questionable.

The RQ₂ is: "What are the performances of TMTs in measuring students’ academic performances in Ethiopian public universities?" The statistical result reveals that the grand mean of p = .49 .6, the Skewness of p = − .300, and D = -0.01 < .2 are indicators, as effective test items were not constructed in Ethiopian public universities for measuring students’ academic performance. In this situation, even though all universities’ difficulty level (P) grand mean = .49 is acceptable by Butakor (2022) to be in the accepted cut-off score p between 0.25–0.75, it has already lost its effectiveness as an ‘ideal’ best item, which should be between 0.5 < p < 0.6 (Suppiah, 2020), and it also does not fit the CTT assumption of normal distribution expected, in which difficult and easy items are expected to be 25% each in amount. Hence, it can be said that Ethiopian public universities’ TMTs yield some score distribution that makes it hard to identify reliable differences in academic performance between the students who have done well and those who have done poorly (Suppiah, 2020). Similarly, in all universities, a high percentage, 66% of D < .2 and the grand mean D = -0.01 < .2 is acceptable by Bichi (2018), Butakor (2022), and Suppiah (2020), which is below the cut-off score of accepted D-power. This implies that Ethiopian public universities’ TMTs could not sufficiently discriminate against the students based on their academic performance.

In sum, it was learned from the overall findings that the present study confirmed to accept the previous studies’ realities, like that of Metages (2019) and Chala (2022), that describe the image of Ethiopian education as severely tarnished as a result of poor quality test provision, in particular, that of Mesfin (2021) and Tesfamariam (2021) arguments, which say the Ethiopian universities undergraduate students' competence testing is questionable. On the other side, where Ethiopian public universities have almost harmonized curricula and similar directives, policies, rules, or regulations of assessments and examination procedures (Metages, 2019), the psychometric properties and performances of TMTs in measuring students' academic performance are equally questionable in all public universities, which convinces us to conduct further interventional studies by categorizing them as experimental and control groups.

5.1. Conclusion

This part concludes the study by summarizing the key research findings in relation to research objectives and basic research questions (RQs), as well as the value contributions thereof. It also reviews limitations and proposes opportunities for future research.

This study aimed to conduct a baseline survey study to investigate the psychometric properties and performances of TMTs in measuring students’ academic performances in Ethiopian public universities using evidence from some selected Ethiopian public universities' English language communicative skill classrooms. To this end, the researcher employed a post-positivist research paradigm and a quantitative approach with a cross-sectional design.

To capture the current image of Ethiopian public universities' TMTs, the researchers reviewed the literature to identify gaps. Based on these gaps, the researchers formulated two RQs to collect empirical data. Accordingly, the literature review showed Ethiopian public university graduates’ scores are raising questions among society because the competence the graduates possess has always contradicted their GPA. In attempting to tackle this problem, no full-fledged study about the qualities, in particular the psychometric properties and performances of TMTs, has been conducted. Giving attention to one of these qualities while ignoring the other creates a gap in educational measurement. Further, this gap becomes the failure of the education system in general. Ultimately, failure in education is failing the country, and failing the country is a burden to the globe. Thus, this area is a big issue that needs investigation in Ethiopia's public universities because it is the basis for their teaching-learning process.

Empirical data collected from established, new, and emerging universities showed that the current Ethiopian public universities' TMT quality status is below the expected CTT standard in measuring students’ academic performances.

In addition to revealing its strengths, the study has indicated its potential limitations for interested parties. The limitations are outlined since the trustworthy approach was followed, but it also has shortcomings that are beyond the researchers’ control. In this regard, three circumstances, which were beyond the control of the researchers, were pivotal. First, the study conducted on one subject is challenging to generalize to the whole course given at the university level. However, the researchers limited their study by considering the importance, necessity, and intermediacy of the English language as logical evidence for other subjects. Second, due to financial constraints, the study couldn’t enrich all Ethiopian HEIs. This brings up the issue of the generalizability of the findings to the Ethiopian context. However, to minimize this challenge, the researchers made efforts by delimiting the study to public universities and then selecting one university from each generation. Third, universities failed to provide the TMT sheets of the many academic years, which would have been the tested item response source of many batches of data. To solve this problem, the researchers framed the 2022 academic calendar and analyzed it exhaustively.

Above all, due to the findings of homogeneity among universities, researchers in the area can probably confidently conduct further different interventional studies on assessments by categorizing universities as experimental and control groups.

5.2. Recommendations

5.2.1. For Public Universities’ Stakeholders

In all Ethiopian public universities, the qualities of TMTs in measuring students’ academic performance were found to be below the expected standards. Thus, all stakeholders, like curriculum developers and teachers, are expected to give critical attention to public universities’ TMTs quality enhancement because it is the base for the teaching-learning process in enhancing instruction, curriculum, and consequently computing grades for the student’s future career.

Curriculum developers should design a table of specifications that can assist teachers in constructing classroom tests.
Teachers are expected to demonstrate professional accountability and be governed by a table of specifications when practicing tests to measure students’ academic performance.
Departments: course-providing departments are expected to have an individual teacher test validation and updating team.

Competing interests: The authors declare no competing interests.

Adeyemo Emily Oluseyi (2021). Assessing Teachers’ Competence in Items Development Through Evidence of Convergent Validity of Test Scores from Alternate Examinations, International Journal of Psychology and Behavioral Sciences, 11(5), 79–83. doi: 10.5923/j.ijpbs.20211105.01.
Alghadri, A. M. A. (2019). Students’ Academic Performance Compared With Their Entry Level Academic Results: A Case Of Islamic University Of Technology (IUT) (M.SC.TE dissertation, Department of Technical and Vocational Education (TVE), Islamic University of Technology, Gazipur, Bangladesh).
Analay Perez. (2019). Assessing Quality in Mixed Methods Research: a Case Study Operationalizing the Legitimation Typology.
Bekalu Ferede et al. (2021). Determinants of instructors’ educational ICT use in Ethiopian higher education. Springer Science.
Bichi, A. A., & Embong, R. (2018). Evaluating the quality of Islamic civilization and Asian civilizations examination questions. Asian People Journal, 1(1), 93–109.
Bijlsma, R. van der Lans, T. Mainhard, and P. den Brok (2021). A Reflection on Student Perceptions of Teaching Quality from Three Psychometric Perspectives: CCT, IRT and GT. https://doi.org/10.1007/978-3-030-75150-0_2
Butakor P.K. (2022). Using Classical Test and Item Response Theories to Evaluate Psychometric Quality of Teacher-Made Test in Ghana. European Scientific Journal, ESJ, 18(1), 139. https://doi.org/10.19044/esj.2022.v18n1p139
Cecilia, Pedro, and Raquel. (2021). Mixed-method research protocol: Development and evaluation of a nursing intervention in patients discharged from the intensive care unit. 8(6), 36666–3676.
Chala, L., & Agago, M. (2022). Exploring national examination malpractice mechanisms and countermeasures: An Ethiopian perspective. International Journal of Instruction, 15(3), 413–428. https://doi.org/10.29333/iji.2022.15323a
Cordova, C., & Tan, DA. (2018). Mathematics Proficiency, Attitude and Performance of Grade 9 Students in Private High School in Bukidnon, Philippines‖. Asian Academic Research. Research Journal of Social Sciences and Humanities, 5(2), 103–116.
Creswell, J. W., & Plano Clark, V. L. (2018). Designing and conducting mixed methods research. 3rd ed. Thousand Oaks. SAGE Publications, Inc.
Dickson Adom, Jephtar Adu Mensah, Dennis Atsu Dake. (2020). Test, measurement, and evaluation: Understanding and use of the concepts in education. International Journal of Evaluation and Research in Education (IJERE), 9, 109–119.
Ehsan, N., Vida, S., & Mehdi, N. (2019). The impact of cooperative learning on developing speaking ability and motivation toward learning English. Journal of Language and Education, 5(3 (19).
Eyob Ayenew, Abreham Gebre Yohannes (2022). Assessing Higher Education Exit Exam in Ethiopia: Practices, Challenges and Prospects. American Journal of Applied Psychology, 10(2), pp. 79–86. doi: 10.11648/j.sjedu.20221002.15
F. E. Espinoza Molina, B. d. V. Arenas Ramirez, F. Aparicio Izquierdo, and D. C. Zu´ñiga Ortega. (2021). Road safety perception questionnaire (RSPQ) in Latin America: a development and validation study. International Journal of Environmental Research and Public Health, 18(5), 2433.
Görkem Erdoğan and Enisa Mede (2021). Evaluating an English Preparatory Program using CIPP Model and Exploring the Motivational Beliefs for Learning. Journal of Education and Educational Development, 8(1), 53–76
Hariharasudan, A., & Kot, S. (2018). A scoping review on Digital English and Education 4.0 for Industry 4.0. Social sciences, 7(11), 227.
Kivunja, C., & Kuyini, A. B. (2017). Understanding and Applying Research Paradigms in Educational Contexts. International Journal of Higher Education, 6(5), 26–41.
Le Thi Ngoc Diep. (2021). English Language Learning Quality At Tertiary Institutions From Learners’ Perspective. Journal of Legal, Ethical and Regulatory Issues, 24(1), Van Lang University
Lemecha Geleto Wariyo (2020). Applying grade inflation adjustment mechanism in an Ethiopian university: Differences in nominal and real grades. Technium Vol. 2 (7), 77–93, College of Education, Addis Ababa University, Sidist Kilo, Ethiopia.
Marjorie. C. Kirkland (2016). The Effects of Tests on Students and Schools. 41(4), Troy State University.
Mekonnen Yibrah (2017). Assessing Content Validity of the EGSEC English Examinations. Haramaya University, Ethiopia.
Mesfin Molla Demissie et al. (2021). Graduates’ Unemployment and Associated Factors in Ethiopia: Analysis of Higher Education Graduates’ Perspectives. Hindaw.
Metages Gebeyehu Alebachew. (2019). Washback Effect of EFL Teacher-Made Test on Teaching-Learning Process of Communicative English Skills Course at Ambo University. International Journal Online of Humanities (IJOHMN), 5(6), ISSN: 2395–5155.
Mohammad Ali Salmani Nodoushan. (2022). Psychometrics Revisited: Recapitulation of the Major Trends in TESOL.
Mohammad Hasan Razmi, Masoud Khabir & Shouket Ahmad Tilwani. (2021). A Meta-Analysis on the Predictive Validity of Graduate Record Examination (GRE) General Test. Tabaran Institute of Higher Education. International Journal of Language Testing, 11(2).
Motuma Hirpassa Minda. (2019). Content Validity of EFL Teacher-Made Assessment: The Case of Communicative English Skills Course at Ambo University. East African Journal of Social Sciences and Humanities, 3(1), 41–62.
Nana Erna, Baso Jabu, Kisman Salija (2018). An Analysis of English Teacher-Made Tests. Faculty of Language and Education, State University of Makassar, St. Gunungsari Baru. Bonto Langkasa, Makassar, Indonesia.
Oluseye Olugboyega. (2022). Confirmatory sequential research design for theory building and testing: proposal for construction management research’s paradigm and methodology. Obafemi Awolowo University, Ile-Ife, Nigeria.
Pham Duy. (2019). Bringing Learning Back In: Examining Three Psychometric Models Bringing Learning Back In: Examining Three Psychometric Models for Evaluating Learning Progression Theories.
Polat, M. (2022). Comparison of performance measures obtained from foreign language tests according to item response theory vs classical test theory. International Online Journal of Education and Teaching (IOJET), 9(1), 471–485.
Quansah, F., Amoako, I., & Ankomah, F. (2019). “Teachers’ test construction skills in Senior High Schools in Ghana: Document Analysis,”. International Journal of Assessment Tools in Education, 6(1), 1–8.
Sewagegn, A.A. (2019). A Study on the Assessment Methods and Experiences of Teachers at an Ethiopian University. International Journal of Instruction, 12(2), 605–622.
Simachew Gashaye and Yibeltal Degwale. (2019). The Content Validity of High School English Language Teacher Made Tests: The Case of Debre Work Preparatory School, East Gojjam, Ethiopia. International Journal of Research in Engineering, IT and Social Sciences, ISSN 2250 – 0588, 09 (11), 41–50
Singh, C. K. S., Singh, H. K. J., Singh, T. S. M., Moneyam, S., Abdullah, N. Y., & Zaini, M. F. (2022). ESL teachers’ assessment literacy in classroom: A review of past studies. Journal of Language and Linguistic Studies, 18(1), 01–17.
Sukarno, S. (2020). Enhancing English language teaching and learning in Industrial Revolution 4.0 Era: Methods, Strategies and Assessments
Suppiah Shanmugam, S. K., Wong, V., & Rajoo, M. (2020). Examining the quality of English test items using psychometric and linguistic characteristics among grade six pupils. Malaysian Journal of Learning and Instruction, 17(2), 63–101.
Tengku Zatul. (2020). Investigating reliability and validity of student performance assessment in Higher Education using Rasch Model. Journal of Physics: Conference Series 1529 042088.
Tesfamariam Shimekit., JeiluOumer. (2021). Ethiopian Public Universities Graduates Employability Enhancement at the Labor Market: Policies, Strategies, and Actions in Place. Academy of Educational Leadership Journal, 25(7), 1–18.
Yusoff MSB. (2019). ABC of content validation and content validity index calculation. Education in Medicine Journal, 11(2):49–54. https://doi.org/10.21315/eimj2019.11.2.6

Download PDF

Version 1

posted

You are reading this latest preprint version

The Psychometric Properties and Performances of Teacher-Made Tests in Measuring Students’ Academic Performance in Ethiopian Public Universities: Baseline Survey Study

Status:

Version 1

Abstract

1. INTRODUCTION

2. THE METHODOLOGY OF THE STUDY

2.1. The Research Paradigm

2.2. Research Approach

2.3. Design of the Study

2.4. Methods

2.4.1. Area of the Study and Sampling

2.4.2. Target Population

2.4.3. Data Collection Instruments

2.4.3.1. Test

2.4.4. Methods of Data Analysis

2.4.5. Data Quality Controlling Mechanisms (Rigor) of the Study

2.5. Ethical Issues

3. RESULTS OF THE BASELINE SURVEY

3.1. Characteristics of the Respondents

3.1.1. Descriptions of Respondents by University Categories

3.1.2. Descriptions of Experts

3.2. Major Findings

3.2.1. The Psychometric Properties of TMTs in Measuring Students’ Academic Performances in Ethiopian Public Universities

3.2.1.1. The Content Validity of TMTs in Measuring Students’ Academic Performances in Ethiopian Public Universities

3.2.2. The Performances of TMTs in Measuring Students’ Academic Performances in Ethiopian Public Universities

3.2.2.1. The Difficulty level (P) of TMTs in Measuring Students’ Academic Performances in Ethiopian Public Universities

3.2.2.2. The Discrimination (D) power of TMTs in Measuring Students’ Academic Performances in Ethiopian Public Universities

4. DISCUSSION

5. CONCULUSION AND RECOMMENDATION

5.1. Conclusion

5.2. Recommendations

5.2.1. For Public Universities’ Stakeholders

Declarations

References

Status:

Version 1