DOI: https://doi.org/10.21203/rs.3.rs-2117958/v1
Medical educators and programs are deeply interested in understanding and projecting the longitudinal developmental trajectories of medical students after these students are matriculated into medical schools so appropriate resources and interventions can be provided to support students’ learning and progression during the process. As students have different characteristics and they do not learn and progress at the same pace, it is important to identify student subgroups and address their academic needs to create more equitable learning opportunities. Using latent class growth analysis, this study explored students’ developmental trajectories and detected group differences based on their coursework performance in Anatomy within the two years of preclinical education in one medical school. Four subgroups were identified with various intercepts and slopes. There were significant group differences between these groups and their standardized scores in MCAT and UCMLE Step 1. The study provides evidence about the heterogeneity of the student population and points out future research directions.
Medical educators and programs are deeply interested in understanding the longitudinal developmental trajectories during the journey from being medical students to competent physicians. Researchers projected various models about learning curves of this journey with the purpose to enhance teaching, assessment, and research activities effectively (Howard, et. al., 2021; Pusic, et. al., 2014). In the same way, it is also important to understand learning trajectories of various subgroups within the population during this journey because it is very likely that students and trainees do not achieve competencies at the same speed and one learning curve oversimplifies the complexity. Because students and trainees have different characteristics and they frequently do not learn and progress at the same pace, it is important to examine whether student subgroups exist so their academic needs can be addressed to create more equitable learning opportunities. By doing so, appropriate resources and interventions can be provided to support learning and progression accordingly. As such, this study intended to explore students’ developmental trajectories and detect group differences based on their coursework performance in anatomy at the preclinical education in one medical school in the middle west of United States using latent growth mixture models. The study also drilled down to investigate how students’ background information and standardized scores were related to their group differences. The study used latent class growth analysis, an under-researched area, to identify unobservable student subgroups and detect group differences on the student growth trajectory.
Learning trajectories and latent growth modeling
Various research examined learning curves and described the rate of learning over an extended period of time in the areas such as psychology, medicine, and education. For example, Pusic and his collaborators (2017) monitored learning processes of 38 participants including medical students, residents, and fellows. They identified learning curve statistical models and recognized the complexity of group and individual trajectories within the same ‘‘multi-level’’ model. Jackson and Sher (2005) identified different subgroups and concluded heterogeneity of trajectories in developmental course by examining initial level (intercept) and growth of alcohol involvement using a growth curve modeling approach. Using latent growth modeling, Costa et. al., (2013) found undergraduate medical students’ empathy, controlled by gender, openness, and agreeableness, did not decline over time. Overall, these theoretical and empirical publications indicate that assuming homogenous of the whole class population of medical students oversimplifies the complexity of growth trajectories among members of different student subgroups.
The literature shows that latent growth mixture modeling provides a promising approach to fully understand students’ growth trajectories and under-observed heterogenous groups within the whole population over time. Latent growth modeling such as latent class growth analysis (LCGA) is a technique to obtain information about inter-individual differences in intra-individual change over time (Nesselroade, 1991). The focus is to examine the relationships among individuals and stratify individuals into distinct groups or categories based on individual response patterns so that individuals within a group are more similar than individuals between groups (Jung & Wickrama, 2008). LCGA generally requires an extensive dataset which might include multiple cohorts and longitudinal data (MacCallum & Austin, 2000). LCGA is different from conventional growth modeling in that it assumes that individuals come from different subgroups and that a single growth trajectory cannot adequately approximate an entire population. The student characteristics (e.g., prior knowledge, socioeconomic classes, gender, and learning styles) that affect the growth trajectory influence each individual in a different way.
LCGA is generally used for the exploratory, post hoc purpose when researchers are not certain about the underlying drivers of distinct developmental courses (van der Nest, et. al., 2020). LCGA assumes that the population is heterogeneous, and it consists of distinct, latent subgroups or classes. It is conducted through the inclusion of K latent classes with different mathematical models for the trajectory. The assignment of individuals to different classes is based on the degree of similarity of developmental courses among individuals. LCGA assumes that the error variance to be the same for all classes and all time points. The identification of these classes allows for discrete individual differences by letting fixed effects (given by the trend) different between classes. In other words, while one class might have a linear, increasing growth curve, the other remains steady or has a linear, decreasing growth curve.
Context
Central Michigan University College of Medicine (CMED) has established since 2013 with a mission to improve access to high-quality health care in Michigan emphasizing rural and medically underserved regions. CMED has about 104 students per class with generally equal number of genders. Students come from different undergraduate and/or graduate academic backgrounds such as mathematics, biochemistry, business, neuroscience, and physics. Its preclinical education lasts for two years, and the curriculum is delivered to students through “a series of integrated, interdisciplinary, systems-based course blocks during the pre-clerkship curriculum” (Thompson, et. al., 2020). Nine foundational and organ-system courses (e.g., Foundational Sciences, Cardio/pulmonary Wellness and Disease, Neuroscience Behavior, and Hematology) are designed and delivered with deliberate considerations of the sequence, breath, and depth of topics, along with two longitudinal courses focusing on medicine in society and essential clinical skills in Year 1 and Year 2. Anatomy has been embedded in all the nine courses of the integrated curriculum.
This study used coursework performance in anatomy of the class of 2023 in the nine courses during the Years 1-2 preclinical education. While there were about 45 questions (mostly cadaver-based questions) for the class of 2023 in their 1st year prior to Covid 19, some changes were made in their 2nd year due to the pandemic and curriculum restructure. After the pandemic, cadaver-based anatomy laboratory has been changed to synchronous and asynchronous learning approaches such as slide sets, videos, image decks, narrated PPT files, and pre-readings. However, anatomy examinations remained to take place in-person in the laboratory environment with 24 test items and the class was divided into groups of 24 or less due to the social distancing requirement. Various of sources of validity evidence continued to be accumulated during the testing process such as test content and internal structure of the test (Song, 2021). Testing items were distributed across content in the course such that they emphasized clinically relevant anatomic structures, spaces, and function. Students were allotted 60 seconds per station plus 15 additional seconds per station for revisiting items at the end of the exam (30 minutes total). Ninety-nine students were included in the analysis.
To ensure comparability of test scores among these 9 courses, the percent-correct raw scores were first transformed to relative scores using equipercentile equating (Livingston, 2014). Equipercentile equating is an approach to equate test scores on the new form to scores on the reference form using the same percentile rank. It identifies the equating relationship as one where a score could have an equivalent percentile on either form. After the score transformation, LCGA with maximum likelihood parameter estimates with robust standard errors was completed in MPlus. We replicated the best log likelihood and ensured the global solutions. The initial step was to specify a univariate growth model. After that, a series of models with a different number of classes were performed until non-significant results achieved. In order to identify the best fitting model, models were compared iteratively based on the following criterion which were used to guide optimal class solution: (1) lower information criterion values (Akaike Information Criterion [AIC], Bayesian Information Criterion [BIC] values, and sample-size adjusted BIC values [SABIC]) indicate a better model fit (Nylund, Asparouhov, & Muthén, 2007); (2) Lo-Mendell-Rubin [LMR] (Lo, Mendell, & Rubin, 2001) Likelihood ratio test and Bootstrapped Likelihood ratio Test [BLRT] (McLachlan 1987) where a smaller p value (e.g., p < .05) for LMR and BLRT test indicates a better model fit of the k classes model compared to the k-1 classes model (Tein, Coxe, and Cham, 2013); higher entropy values close to 1 indicate an excellent classification of subjects into its corresponding latent classes; (4) theoretically meaningfulness and (5) size of the smallest derived class < 1.0% of total sample size or fewer than 25 subjects (Lubke & Neale, 2006). When deciding the number of classes, besides statistical fit indices, research questions, model parsimony, theoretical justification, and class size are also considered (Bauer, & Curran, 2003; Muthén, 2003).
A univariate growth curve modeling was first fitted to the data and achieved a good model fit. Then an unconditional latent class growth analysis was carried out to identify underlying subgroups. According to the criterion we have set up in the method, the results showed that four class model was selected as the best fitting model (see Table 1). Four class model had a significant p value for LMR test, smaller AIC and sample adjusted BIC, and higher entropy than three class model. Additionally, inspection of the graph plots of four class model verse three class model, four class model made more sense in clinical meaningfulness (see Fig. 1). Table 2 showed descriptive statistics for the four latent classes. Each class varied in their initial status—percentile rank (i.e., intercept) and rate of change over time (i.e., slope) (see Fig. 1).
|
Loglikelihood value |
AIC |
BIC |
SABIC |
BLRT |
Pblrt |
LMR |
Plmr |
Entropy |
|
|---|---|---|---|---|---|---|---|---|---|
|
Univariate* |
-4101.98 |
8272.76 |
8309.09 |
8264.88 |
-- |
-- |
-- |
-- |
-- |
|
2 class |
-4127.06 |
8282.11 |
8318.44 |
8274.23 |
210.29 |
< .001 |
196.07 |
< .001 |
0.89 |
|
3 class |
-4118.48 |
8270.95 |
8315.07 |
8261.38 |
17.16 |
< .001 |
16.00 |
0.20 |
0.76 |
|
4 class |
-4113.80 |
8267.59 |
8319.50 |
8256.33 |
9.36 |
0.09 |
8.73 |
0.02 |
0.78 |
|
5 class |
-4111.12 |
8268.25 |
8327.93 |
8255.30 |
5.35 |
0.14 |
4.99 |
0.19 |
0.76 |
| * univariate growth curve model | |||||||||
| BLRT = Bootstrapped likelihood ratio test |
|||||||||
| LMR = LO-MENDELL-RUBIN adjusted LRT test |
|||||||||
Small p value indicating better model fit of k class model compared to (k-1) class model
|
n(%) |
Intercept |
Slope |
|
|---|---|---|---|
|
class 1 |
11(11%) |
53.65 |
-3.65 |
|
class 2 |
17(17%) |
42.22 |
1.98 |
|
class 3 |
38(38%) |
23.90 |
0.81 |
|
class 4 |
33(33%) |
76.37 |
-2.5 |
To further detect group differences, students’ background information including their GPA, non-Sciences GPA, Sciences GPA, science credits, and MCAT as well as their USMLE Step 1 scores were examined (See Table 3). One-way ANOVA showed significant differences with MCAT and Step 1. There was a significant difference of four classes on MCAT and Step 1 performance at the p < .05 level [F (3, 95) = 5.979, p = .001; F (3, 93) = 7.886, p < .001].
|
N |
GPA |
Non-Sciences GPA |
Sciences GPA |
Sciences credit |
MCAT* |
Step 1* |
|
|---|---|---|---|---|---|---|---|
|
Class 1 (red) |
11 |
3.63 |
3.76 |
3.52 |
77.8 |
503 |
222 |
|
Class 2 (blue) |
17 |
3.62 |
3.85 |
3.47 |
78.4 |
511 |
230 |
|
Class 3 (green) |
38 |
3.59 |
3.75 |
3.48 |
74.9 |
506 |
217 |
|
Class 4 (pink) |
33 |
3.71 |
3.82 |
3.63 |
69.0 |
509 |
237 |
The study confirmed different learning trajectories with this cohort using LCGA. There were four learning patterns within the class of 2023 with a larger variance when these students first started their medical school. The variance decreased from 53 to 25 points by the end of their 2nd year. This result is consistent with what is anticipated as the result of the carefully planned instruction and curriculum activities during the two years. The ANOVA results also indicated the statistically significant differences between anatomy and MCAT as well as between anatomy and Step 1, demonstrating the alignment of the anatomy curriculum with the USMLE standardized exams.
As showed in Fig. 1, the group of Class 4 (pink) started the strongest 76.37 and ended strong. This is the strongest group with the highest GPA and Sciences GPA as well as the Step 1 score. Class 2 (blue) started lower 42.22, but they ended up strong with a positive slope. In comparison, the group of Class 1 (red) dropped down rapidly and Class 3 (green) made limited progress during the two years of learning. The group of Class 1 (red line) started with relatively strong performance, and they ended with the lowest performance. Class 3 (green) started lowest and they remained at the lower end. When students’ background information was put side by side, we found that both groups, Class 1 and Class 3, had relatively lower non-sciences GPA. As presented in Table 3, both groups have lower MCAT and USMLE Step 1. These results indicate the potential connection between non-sciences GAP and grades in course work/standardized examinations. Results also showed that two groups, Class 1 (red) and Class 2 (blue), with similar background characteristics, started in the middle but ended quite differently with one decreasing and one increasing during the two years of learning. What would be the potential drivers to their differences on the learning trajectories is not clear. Overall, it is likely that knowledge, skills, and dispositions developed from non-sciences courses during their pre-medical education might contribute to different trajectories with this class. This warrants further investigations with more background information.
The study found different developmental trajectories among students in preclinical medical education at CMED. There were significant differences between group performance and standardized testing in MCAT and Step 1. One limitation is the possible impact of the pandemic on learning outcomes in this study. Because covariates (e.g., academic background in pre-med, learning skills, motivation) might potentially influence the growth factor and class, future research could include those covariates in a conditional latent class model to explain heterogeneity of students’ development in their learning outcomes. Moreover, multiple cohorts may be utilized to replicate the findings.
The study has two significant implications. First, latent growth mixture modeling can be a valuable tool for classifying multiple trajectories that have not been observed in the localized setting. It allows inter-individual variability between students within latent classes through the inclusion of random effects. Second, meaningful longitudinal analyses of student performance help to explain how students’ progress during their two-year preclinical education. Considering students’ heterogeneous backgrounds and changing realities is intensified by an emphasis on cultivating equitable educational systems in the local and national landscape. By doing so, more equitable and innovative learning opportunities can be provided accordingly so all medical students are progressing to become competent physicians in the future.