Transforming in-service high school biology teachers’ acceptance of evolution through professional development

doi:10.21203/rs.3.rs-4858530/v1

Download PDF

Research Article

Transforming in-service high school biology teachers’ acceptance of evolution through professional development

https://doi.org/10.21203/rs.3.rs-4858530/v1

This work is licensed under a CC BY 4.0 License

You are reading this latest preprint version

Background: Evolution plays a significant role in the understanding of biology. Educators must acknowledge the importance of this theory for effective biology teaching, but there are several educational gaps both in the classroom and in in-service teachers that need to be addressed to help improve the acceptance and understanding of the theory of evolution. In this study, the Measure of Acceptance of the Theory of Evolution (MATE) was translated, adapted, and evidence was gathered for its use with Spanish-speakers, as means to study how in-service teachers’ acceptance of the theory of evolution changes after participating in a professional development program (PD), framed in the genome-to-phenome overarching theme.

Results: Evidence was gathered to evaluate content validity, response process, internal structure, and reliability of the MATE instrument for it use with Spanish speakers (i.e., MATE-E). A pilot test was conducted, enabling an Exploratory Factor Analysis to be performed. Five factors were identified. When administered to teachers that participated in a PD program, descriptive statistics showed that all teachers had higher scores in the posttest. A Wilcoxon matched pairs and signed ranks test produced a Zscore of -2.023 and a p value of .043.

Conclusions: MATE’s translation to Spanish, as well as its adaptation, and the evidence-gathering process for the instrument’s possible use with Spanish-speaking participants are discussed. A PD was implemented to develop concepts related to the theory of evolution for in-service teachers. MATE-E was administered to five teachers before and after participating in the PD program. A Wilcoxon matched pairs and signed ranks test was carried out to test if treatment was effective in changing their acceptance of the theory of evolution. The null hypothesis was rejected; and it can be confirmed that the treatment was both significant and positive in changing participants’ acceptance of the theory of evolution.

Biology education

Evolution

Evolution acceptance

Evolution education

In-service teachers

Measure of Acceptance of the Theory of Evolution

Professional development program

Theory of evolution

Instrument translation

Instrument validation

Evolution plays a significant role in the understanding of biology. Thus, when teaching biology, educators must acknowledge the importance of developing curriculum and planning lessons to effectively teach subject-area concepts while simultaneously keeping this theory relevant. Dobzhansky (1973) famously stated that the theory of evolution grants sense and meaning to biology. Nevertheless, there are several factors that do not contribute to the understanding or acceptance of the theory of evolution.

One of the more prominent reasons of low acceptance of the theory of evolution comes from individuals’ religious views. Students often identify discrepancies between their creationist religious views and the theory of evolution (Wingert et al. 2023). Additionally, misconceptions make the theory more difficult to understand and, consequently, more difficult to accept (Newall and Reiss, 2023). Thus, teachers face the challenge of providing students with theory-clad lessons that both close educational gaps and support students’ acceptance of the theory of evolution.

There are additional obstacles that educators confront when teaching evolution. Nehm and Schonfeld (2007) identified that in-service teachers present similar misconceptions about natural selection to those of K-12 students. Additionally, teachers also face similar difficulties accepting the theory of evolution, which translates into ineffective biology teaching. Professional development programs are instrumental to address and close these gaps (Friedrichsen et al. 2016).

Although multiple studies confirm that professional development programs help high school biology teachers in improving their acceptance of the theory of evolution, as well as their pedagogical practices, it is also true that there is still a need for in-service teachers’ learning of evolution (Romine et al. 2017). The project Genomic Logic Underlying Morphological Adaptive Divergence (NSF #1736026) designed a professional development (PD) program for high school biology teachers in Puerto Rico. This project’s objective is to improve the teachers’ ability to effectively teach the topic of evolution to their students through a two-year-long professional development program under the overarching theme of the genome to phenome relationship.

The program consisted of two phases: one educational, followed by an implementation phase. During the educational phase, master teachers carried out professional development sessions in which participants were taught about several concepts related to the theory of evolution. A sample of the topics discussed throughout these monthly sessions is shown on Table 1. Participating teachers also had the opportunity to spend one and a half weeks doing fieldwork and research during the summer. Furthermore, teachers were mentored into preparing classroom activities in which they would transfer their acquired knowledge and experience.

Table 1 Sample topics discussed throughout the professional development program phase 1 monthly sessions

Table 1

Sample topics discussed throughout the professional development program phase 1 monthly sessions
Learning module	Title	Sample topics
1	Natural selection	Species, biological interactions, biodiversity
2	Adaptation	Genetic variation, environmental conditions
3	Evolution	Speciation, speciation mechanisms
4	Heredity	Genetics, chromosome, mutations
5	Gene expression	Genome, phenome

The PD program's second phase involved the implementation and transfer of acquired knowledge into lessons and activities that the teachers would do in their classroom. The teachers would also receive follow-up sessions with the mentor teachers. Figure 1 summarizes the PD program.

This PD program was prepared for Spanish-speaking high school biology teachers. To this extent, it is essential to examine Spanish-speaking high school biology teachers’ gaps in the understanding and acceptance of the theory of evolution to provide a pathway into better biology education. At the time of planning and conducting this study, no research instruments were found suitable for this purpose. In an alternative to creating new materials, the extant literature can be reviewed to evaluate if there are content-appropriate instruments, notwithstanding any language barrier (Bravo-Vick et al, 2019). To address the need for valid culturally sensitive research instruments in the participants’ target language (Bravo, 2003; Bravo-Vick et al, 2019), the team relied on the translation to Spanish, adaptation, and validity evidence gathering process of the Measure of Acceptance of the Theory of Evolution (MATE), originally developed and validated by Rutledge and Warden (1999).

In this paper, the process for evidence gathering for the MATE translation, adaptation, and validation is described. The resulting instrument, the MATE-E (MATE-Español) was administered in pre-posttest format to the teachers that participated in the professional development program. The results of this administration are discussed regarding the program’s effectiveness in changing participants’ acceptance of the theory of evolution.

Participants

Five high school students participated in the focus group phase of the evidence gathering process. For the pilot study, 281 middle and high school students were invited to participate, of which 185 assented to their participation. All student participants attend the University of Puerto Rico, Rio Piedras Campus’ Laboratory Secondary School. Additionally, five Puerto Rican, Spanish-speaking, high school biology teachers from public secondary schools in Puerto Rico participated in the PD program, and thus, were administered the MATE-E instrument.

This protocol was approved by the University of Puerto Rico Rio Piedras Campus’ Institutional Review Board (CIPSHI #2021-018).

Genome-to-phenome PD program description

The program was designed to provide an opportunity for in-service high school biology teachers to address different concepts related to the theory of evolution. Given the topic's complexity and the need for a multi-dimensional approach, a two-year-long program was designed. It was developed using the train-the-trainer model, which means that master science teachers were identified to develop and design (in collaboration with the researchers) and offer learning modules on topics related to evolution. Besides the content component, the PD program includes summer research experience, mentoring, and in-classroom application. This PD program's development and design yielded five learning modules on natural selection, adaptation, evolution, heredity, and gene expression. These modules were used during monthly PD sessions for participants. These modules were carried out as three-hour PD sessions offered on Saturdays. During these sessions, trainers taught in-service teachers on the topics. They also addressed misconceptions, provided learning materials, modeled a sample classroom lesson, carried out learning activities, and offered ideas to modify lessons and activities to accommodate diverse students.

The research experience took place during the summer immediately after all monthly PD sessions concluded. Participants were assigned in pairs to a research laboratory at the University of Puerto Rico, Rio Piedras Campus in which the genome to phenome theme is investigated. The laboratory’s team assisted in developing participants’ laboratory research skills. Furthermore, they helped in demonstrating the practical relevance of learning these skills to transfer their acquired knowledge into their classrooms. In addition, participants had field experience in which they explored the impact of global warming on different species. Lastly, the designed PD program also includes a mentoring component for in-classroom application. The trainers mentored participating teachers to develop lessons and activities that would help them transfer their experience into their classrooms. They also had follow-up mentoring sessions with participating teachers while they were carrying out the planned lessons and activities.

Translation, validation, and adaptation of the Measure of Acceptance of the Theory of Evolution (MATE)

The Measure of Acceptance of the Theory of Evolution (Rutledge & Warden, 1999) was translated, validated, and adapted from English to Spanish (MATE-E). As it is imperative to gather sound evidence that demonstrates that MATE scores’ interpretation is comparable to its intended use (AERA, APA, NCME, 2014), there must be a structure to set in place the process to gather said evidence. For this instrument, the guidelines of AERA, APA, NCME, (2014) and Creswell’s (2012) recommendations were followed to gather evidence for content validity, response process, internal structure validity, and instrument reliability.

Content validity evidence for MATE-E

Content validity evidence was gathered to analyze the relationship between the instrument’s content and constructs (AERA, APA, NCME, 2014). Content validity evidence gathering was divided into two parts. The first one was through the translation/back-translation technique for semantic equivalence. The translation/back-translation process was carried out as follows: (1) a bilingual, native Spanish speaker and fluent English speaker with education expertise translated the instrument from English to Spanish; and (2) a bilingual, native English speaker and fluent Spanish speaker with biology expertise translated the instrument from Spanish back to English. All three versions of the instrument (original, translation, and back-translation) were then prepared for the second part of the content validity evidence gathering process.

The second part consisted of evaluation by an expert panel. The expert panel consisted of four judges, all of them experts in biology. Two of them are also experts in education and one of them is an expert in instrument design. Additionally, all experts were bilingual. The judges were tasked with evaluating each item in terms of accuracy and clarity of its translation. The judges received an item evaluation sheet that contained all three versions of each item, a scale to evaluate accuracy, a scale to evaluate clarity, and a section to provide comments for each item. After receiving each judge’s evaluations, an online meeting was scheduled to discuss them and update the translated version for its use during the response process evidence gathering phase.

Response process evidence for MATE-E

Response process refers to the evidence that corresponds to the relationship between the construct and how individuals responded (AERA, APA, NCME, 2014). For this phase, high school students who had previously taken a biological sciences course were identified. Five high school students agreed to participate in this process. Besides having taken a course at the middle-school level, they were taking high-school biology courses at the time of their participation. They completed the MATE-E instrument and then took part in a focus group in which they were asked about the test’s content, as well as their thought process when answering each item. Additionally, students also provided recommendations to further improve items’ accuracy and clarity. Said recommendations were consulted with the expert panel once more to receive their input and decide which of these to adopt. After this meeting with the expert panel, the MATE-E was modified once more. This version was used during the internal structure evidence gathering phase (pilot study).

MATE-E Pilot study and internal structure evidence

A pilot study was conducted with secondary-level students. It is important to notice that high school students were the participants during this phase because this test will be administered to both high school teachers and students. Therefore, the assumption in this case is that, if students understand the content of the instrument, teachers must also be able to understand it. The sampling used for this pilot study was non-probabilistic and by convenience. A total of 281 8th, 10th, and 11th grade students from the University of Puerto Rico, Rio Piedras Campus’ Laboratory Secondary School were invited to participate in this pilot study, of which 185 assented to their participation, for a total response rate of 65.8%. MATE-E was self-administered, and it was done online. The participants’ responses were anonymous, and they did not receive any incentive for their participation. Out of the 185 respondents, only 174 were valid for analysis. Eleven respondents submitted answers with missing data. The valid responses make up 94% of all responses and 61.9% of all invited participants.

Statistics for MATE-E pilot study

The descriptive statistics and an Exploratory Factor Analysis (EFA) for the Spanish Measure of Acceptance of the Theory of Evolution were performed using IBM® Statistical Package for the Social Sciences (SPSS®) version 29.0. Prior to conducting the Exploratory Factor Analysis, the responses were submitted to several tests. Data from survey administration was used for descriptive statistical analysis. Data was also tested for normality using the Kolmogorov-Smirnov test. A reliability test was performed as well, and Cronbach’s alpha was used to estimate MATE-E’s internal consistency. Additionally, before the EFA, adequacy tests were carried out, The Keiser-Meyer-Olkin Measure of Sampling Adequacy and Bartlett’s Test of Sphericity were performed. EFA with Varimax rotation was used to reduce factors using Eigenvalue, Scree testing, factor loading, and cumulative percent of variance.

Administration to PD participants

A call for applications was sent to public high school biology teachers from a database provided by the Center for Science and Math Education Research at the University of Puerto Rico, Rio Piedras Campus. A total of six1 educators were selected for the first cohort. The sample size is limited to this amount because there was a maximum capacity of participants that laboratories would accept for the summer research experience. All voluntarily agreed to participate in the professional development program. The participants are all female, and at the time of their professional development they were teaching high-school level biology at public schools in Puerto Rico. Pretest and posttest data were analyzed using IBM® Statistical Package for the Social Sciences (SPSS®) version 29.0. Descriptive statistics were conducted. A non-parametric Wilcoxon (1945) matched pairs and signed ranks statistical test was also performed with this dataset to determine if there were statistically significant changes in PD participants’ acceptance of the theory of evolution.

Six teachers were selected to participate in this professional development program, but only five teachers completed it. Only data from the five teachers that completed their experience will be reported.

It is imperative to gather validity evidence for an instrument translated and culturally sensible to support its interpretation and to determine that it is being used for its intended purpose. Content, response process, and internal structure validity evidence was gathered to determine whether the Spanish translation of the MATE, i.e., the MATE-E, was adequately translated and adapted for Puerto Rican Spanish-speakers.

It was equally essential to conduct an Exploratory Factor Analysis (EFA) that would assess the instrument’s internal reliability, as well as the structure in which the items within MATE-E are grouped. The EFA also facilitates the comparison of how the items are grouped in relation to the original instrument, and it facilitates determining if items measure the construct. After the translation and adaptation process, the MATE-E was administered to the PD participants on two occasions: before and after the educational phase of the PD program. Due to the sample size (n = 5), a Wilcoxon signed ranks and matched pairs test was performed to determine if there were significant changes in their acceptance of the theory of evolution upon treatment.

Content validity

There was a fundamental need to identify an instrument in Spanish that could measure the acceptance of the theory of evolution to conduct our research. Upon thorough revision, none were identified in the target language. The items of the MATE instrument were adequate for the research. The original instrument consists of 20 items (Cronbach’s alpha coefficient value of 0.98) that, after factor analysis, were grouped into a single factor corresponding to evolutionary theory (Rutledge & Warden, 1999). In another adaptation of the MATE done by Rutledge and Sadler (2007), they categorized the instrument into six concepts: process of evolution, scientific validity of evolutionary theory, evolution of humans, evidence of evolution, scientific community’s view of evolution, and age of the Earth. Consequently, Rutledge and colleagues' work provided the theoretical framework for translating and adapting the MATE instrument for Puerto Rican Spanish speakers.

Behling & Law (2000) explained that accurate translations reflect a thorough process of examining similarity in the meaning of each item in its original language and its target language. Given this context, the translation/back-translation technique was the first of two steps taken to gather evidence of content validity for MATE-E. The process was carried out as follows: (1) a bilingual, native Spanish speaker and fluent English speaker with education expertise translated the instrument from English to Spanish; and (2) a bilingual, native English speaker and fluent Spanish speaker with biology expertise translated the instrument from Spanish back into English. This process yielded three versions of the same instrument: the original, a Spanish translation, and a back-translation.

All three versions of the instrument (original, translation, and back-translation) were then prepared for the second part of the content validity evidence gathering process, which consisted in an evaluation from an expert panel. An item evaluation sheet was prepared and sent to the expert panel, for them to judge each item; and evaluate accuracy and clarity of its translation.

After the panel members evaluated the items, an online meeting was scheduled to discuss their evaluations. After this meeting, an updated version of the MATE’s Spanish translation (i.e., MATE-E) was generated that incorporated the panel’s comments and evaluations. Table 2 shows an example of how one of the items was evaluated by the expert panel.

Table 2 Item evaluation sheet sample

Table 2

Item evaluation sheet sample
Item #	Type	Statement	Translation		Clarity		Comments*
Item #	Type	Statement	A	I	Clear	Confusing	Comments*
16	O	Evolutionary theory is supported by factual, historical, and laboratory data.					E1: La traducción no es 100% correcta, pero el mensaje es esencialmente el mismo. (Translation is not 100% correct, but the message is essentially the same) E2: ¿Laboratorios fácticos? (Factual laboratories?) E3: De la manera que se tradujo no mantiene el sentido de los tres tipos de datos incluidos en la premisa original. Sugiero: La teoría evolutiva es respaldada/apoyada por datos factuales, históricos y de laboratorio. (The way it was translated does not maintain the meaning of the three types of data included in the original premise. I suggest: Evolutionary theory is supported by factual, historical and laboratory data.)
	T	La teoría de la evolución está apoyada por datos históricos y de laboratorio fácticos.		X (E1, E2, E3)	X (E1)	X (E2, E3)
	B	The theory of evolution is supported by historical data and laboratory results.
O = Original item, T = Translated item, B = Back-translated item; A = Adequate translation, I = Inadequate translation; E1 = Expert 1, E2 = Expert 2, E3 = Expert 3 For clarity purposes, expert comments are translated into English in italics*.

Response process

A focus group was carried out to gather evidence for MATE-E’s response process. Five high school students assented to their participation in this focus group. The students belonged to the University of Puerto Rico, Rio Piedras Campus’ Laboratory Secondary School. They had previously taken a middle-school biology course and were taking a high-school biology course at the time of their participation. As part of the focus group, they were asked to complete the MATE-E and then answer questions related to the items, their process to select an answer, and the wording on the instrument.

Participants deemed the instrument’s translated name and its instructions appropriate. They provided some recommendations to improve wording for some items and the instrument’s scale. A sample of the recommendations given by the focus group participants can be found in Table 3. Upon completing the focus group data analysis, a final online meeting with the expert panel was conducted to present the recommendations given by the students/ participants during the focus group and confer whether to accept these suggestions. The expert panel accepted the recommendations that lead to the final MATE-E instrument that was used for the pilot study and the administration to PD participants.

Table 3 Sample of focus group’s recommendations

Table 3

Sample of focus group’s recommendations
Item #	Statement evaluated by focus group	Recommendations	Statement as recommended
16	La teoría evolutiva está apoyada por datos fácticos (verificables) históricos y de laboratorio. *(Evolutionary theory is supported by factual, historical, and laboratory data.)	1. Sustituir “teoría evolutiva” por “teoría de la evolución” (Substitute “evolutionary theory” for “the theory of evolution”) 2. Sustituir “datos fácticos (verificables)” por “evidencia.” (Substitute “factual (verifiable) data” for “evidence”) 3. Sustituir “de laboratorio” por “científica.” (Substitute “laboratory” for “scientific”)	La teoría de la evolución está apoyada por evidencia histórica y científica.
For clarity purposes, the original item, as well as translations for recommendations were added in italics*.

Pilot study and internal structure

To gather evidence for an instrument’s internal structure, there should be a correspondence between the test scores and its conceptual framework. To analyze this, Creswell (2012) recommended to perform “statistical procedures to determine the relationship among test item and test parts” (p. 163). A pilot study was done to gather evidence of the instrument’s reliability of responses and internal consistency of the scores. The MATE-E instrument was administered to students from the University of Puerto Rico, Rio Piedras Campus’ Laboratory Secondary School.

The research team invited 281 students from 8th, 10th, and 11th grade to participate in the pilot study. The pilot study took place via online meetings. The instrument was self-administered, responses were anonymous, and students did not receive any incentive for their participation. 185 students assented to their participation, of which 174 cases were valid. This means that 11 responses had missing data. Table 4 summarizes the profile of the pilot study participants.

Table 4 Pilot study participant demographics

Table 4

Pilot study participant demographics
Groups	Male	Female	Total
8th grade	27	25	52
10th grade	6	13	19
11th grade	8	16	24
11th grade	13	22	35
11th grade	13	7	20
11th grade	11	24	35
Total groups: 6	Total male: 78	Total female: 107	Grand total: 185*
*There were only 174 valid cases among the 185 participants; 11 cases had missing data.

Table 5 summarizes the descriptive statistics for the MATE-E pilot study. A Kolmogorov-Smirnov test was performed to measure normality. Test results show that data is normally distributed, D(174), = 0.065, p 0.066.

A reliability analysis was performed to check for MATE-E’s internal consistency. Since the test’s items are scored as continuous variables, the appropriate test for reliability is Cronbach’s coefficient alpha (1984). Results show that MATE-E’s Cronbach’s alpha based on standardized items is .879. George and Mallery (2003) stated that a Cronbach’s alpha of .879 is well within the range of a good (0.80 ≥ α < 0.90) internal consistency.

Table 5 Descriptive statistics for pilot study administration

Table 5

Descriptive statistics for pilot study administration
N	Valid	174
N	Missing	11
Mean		76.48
Std. Error of Mean		.997
Median		77.50
Std. Deviation		13.146
Variance		172.806
Skewness		− .786
Std. Error of Skewness		.184
Kurtosis		.980
Std. Error of Kurtosis		.366
Range		5
Minimum		0
Maximum		5

The Cronbach’s alpha for MATE-E, though, is lower than the one reported in the original instrument, which is 0.98 (Rutledge & Warden, 1999). This may be due to factors such as a lower dataset than the original or due to cultural differences in translation and adaptation. However, it is noteworthy to point out that Cronbach’s alpha constantly remains in the range of a good internal consistency even if other items are deleted.

Additionally, the Kaiser-Meyer-Olkin Measure of Sampling Adequacy index was calculated. Data shows that the sample used was adequate (KMO = .857). Similarly, a Bartlett’s Test of Sphericity was performed, and results were significant (p < .001). Both tests confirmed adequacy, which means an Exploratory Factor Analysis could be conducted.

Exploratory Factor Analysis for MATE-E

An Exploratory Factor Analysis (EFA) was conducted to study the underlying factors present in the MATE-E version of the instrument. Since this is a translated and adapted instrument, an EFA would be helpful in discovering the underlying factor structure of the instrument after its translation and adaptation. As part of the EFA, a Principal Component Analysis was conducted to identify MATE-E test’s factors.

Upon analysis, five components with Eigenvalues greater than 1 were identified. Despite Rutledge and Warden’s (1999) initial reporting of a standalone factor of evolution acceptance for the original MATE test, more recent research, such as analyses done by Metzger et al. (2018) point to the instrument having more than one factor (Barnes et al. 2019). Furthermore, Rutledge and Sadler (2007) discussed that the instrument has six concepts, and each item is grouped into only one of them.

Therefore, the extracted components may be considered for MATE-E factor categorization. Rutledge and Sadler’s (2007) concepts were considered when categorizing these factors. These five components also represent 59.472% of the total variance explained (see Table 6).

The cumulative percentage of the total variance explained (59.472%) for MATE-E’s EFA closely approximates Pituch and Stevens’ (2016) recommended 60% threshold of total variance explained for factor analysis. Figure 2 shows a Scree plot with the Eigenvalues from each component as extracted from the Principal Component Analysis.

Table 6 Total variance explained

Table 6

Total variance explained
Component	Initial Eigenvalues			Extraction Sums of Squared Loadings			Rotation Sums of Squared Loadings
Component	Total	% of Variance	Cumulative %	Total	% of Variance	Cumulative %	Total	% of Variance	Cumulative %
1	6.282	31.408	31.408	6.282	31.408	31.408	3.925	19.624	19.624
2	2.024	10.120	41.528	2.024	10.120	41.528	2.934	14.670	34.294
3	1.482	7.410	48.938	1.482	7.410	48.938	2.462	12.311	46.605
4	1.099	5.495	54.433	1.099	5.495	54.433	1.467	7.336	53.941
5	1.008	5.039	59.472	1.008	5.039	59.472	1.106	5.532	59.472
6	.857	4.285	63.757
7	.822	4.110	67.868
8	.784	3.918	71.785
9	.749	3.745	75.530
10	.696	3.482	79.011
11	.621	3.103	82.115
12	.600	3.002	85.117
13	.565	2.823	87.940
14	.463	2.313	90.253
15	.398	1.991	92.244
16	.390	1.948	94.192
17	.320	1.600	95.791
18	.303	1.517	97.309
19	.294	1.468	98.777
20	.245	1.223	100.000
Extraction Method: Principal Component Analysis.

The Principal Component Analysis additionally informed how the MATE-E’s items correlated with each of the five components. The rotation method used for this analysis was Varimax with Kaiser Normalization and the test was adjusted to show correlations greater than .40. Results shown on Table 7 demonstrate that eight items correlate strongly with the first component, five items correlate strongly with the second component, four items correlate strongly with the third component, two items correlate strongly with the fourth component and one item correlates strongly with the fifth component.

Two of these items have strong correlations with more than one component. Item number 2 correlates strongly with the first and second component, while item number seven correlates strongly with the second and third component. However, item 2 has a stronger correlation to the first component, while item number 7 has a stronger correlation with the third component. Therefore, upon review, the team decided to group items 2 and 7 into the first and third component, respectively.

It is also important to point out that only one item correlates strongly with component 5. Despite this, the team has decided not to discard this component, nor this item. The item possesses the strongest correlation among all observed data. Table 7 shows the rotated component matrix. Reported results include correlations greater than .40 and items with overlapping component correlations.

Table 7 Rotated component matrix

Table 7

Rotated component matrix
Item #	Component
Item #	1	2	3	4	5
3	.727
20	.724
16	.694
18	.658
1	.654
8	.625
12	.563
2	.515	.440
10		.790
4		.727
6		.657
17		.515
14		.448
9			.828
15			.753
19			.682
7		.453	.455
13				.733
5				.653
11					.853
Extraction Method: Principal Component Analysis. Rotation Method: Varimax with Kaiser Normalization. Rotation converged in 9 iterations.

Figure 3 shows a path diagram for MATE-E’s Exploratory Factory Analysis. To further help visualize this diagram, the Eigenvalues, rotation sums of squared loadings and percent of variance explained is shown for each factor. Additionally, factors are named (names are also translated to English in italics for clarity) and the correlation for each item is also shown. As mentioned, Rutledge and Sadler’s (2007) concepts were considered when naming these factors.

Upon conducting Exploratory Factor Analysis, items were grouped into five components. After reviewing how the items were grouped within the components, as well as research results using the MATE instrument each factor was named as follows: Validez del proceso de evolución (Validity of the evolutionary process), Evidencia de la evolución (Evidence of evolution), Evolución de los humanos y otras especies (Evolution of humans and other species), Punto de vista de la comunidad científica en torno a la evolución (Scientific community’s view of evolution), and Edad de la Tierra (Age of Earth). These factors, its corresponding items, and some sample items are shown on Table 8. The final translated version of the MATE-E instrument is available on Additional File 1.

Table 8 Construct naming for MATE-E’s factors

Table 8

Construct naming for MATE-E’s factors
Factor	Construct names in Spanish (English translation in parenthesis)	Items	Sample items	Items in English
1	Validez del proceso de evolución (Validity of the evolutionary process)	1, 2, 3, 8, 12, 16, 18, 20	(20) La evolución es una teoría científica válida.	(20) Evolution is a scientifically valid theory.
2	Evidencia de la evolución (Evidence of evolution)	4, 6, 10, 14, 17	(14) La teoría de la evolución no puede ser correcta ya que difiere del relato bíblico de la creación.	(14) The theory of evolution cannot be correct since it disagrees with the Biblical account of creation.
3	Evolución de los humanos y otras especies (Evolution of humans and other species)	7, 9, 15, 19	(15) Los humanos existen hoy en día esencialmente de la misma forma en que siempre lo han hecho.	(15) Humans exist today in essentially the same form in which they always have.
4	Punto de vista de la comunidad científica en torno a la evolución (Scientific community’s view of evolution)	5, 13	(13) La teoría de la evolución genera predicciones comprobables en relación con las características de la vida.	(13) Evolutionary theory generates testable predictions with respect to the characteristics of life.
5	Edad de la Tierra (Age of Earth)	11	(11) La edad de la tierra es de al menos 4000 millones de años.	(11) The age of the earth is at least 4 billion years.

The MATE-E test was translated to Spanish, adapted, and validity evidence was gathered with Puerto Rican Spanish-speaking high school students, based on the assumption that biology teachers will be at least as familiar with the language and content knowledge of the instrument. Gathering validity evidence with students has the advantage of allowing follow-up studies that measure the change in students’ acceptance of the theory of evolution, as well as measuring the effectiveness of educators in transferring the acquired knowledge into classes and classroom activities.

Administration to PD participants

Genome-to-phenome Professional Development Program description

The program was designed to provide a professional development opportunity for high school biology teachers on concepts related to evolution. The PD was developed using the train-the-trainer model. Master science teachers collaborated with Biology faculty from the University of Puerto Rico, Rio Piedras Campus to identify, design, develop, and offer learning modules on five topics related to evolution. This collaboration yielded five modules on the following topics: natural selection, adaptation, evolution, heredity, and gene expression. These modules were used during monthly PD sessions for participants.

Throughout the professional development sessions, master teachers instructed participants on the different topics, addressed misconceptions, provided learning materials, modeled sample classroom lessons, carried out learning activities, and offered ideas to modify lessons and activities to accommodate for diverse students. In addition to these monthly sessions, teachers participated in summer research experience in which they spent one week in dedicated laboratories at the University of Puerto Rico, Rio Piedras Campus and spent three additional days doing field work and exploring climate change’s impact on different species. Participants were assigned to a research laboratory and each laboratory personnel were responsible for the participants’ development of laboratory research skills, as well as helping them establish the relevance of learning these skills to transfer their knowledge into the classroom and help their students understand and accept the theory of evolution.

Study question

To determine PD program’s effectiveness in changing participants’ acceptance of the theory of evolution, the following question was formulated: Is the designed professional development program effective in changing in-service high school biology teachers’ acceptance of the theory of evolution? The MATE-E test administration occurred on two occasions during the first phase of the PD program to address this question. The first administration was in December 2021, before the professional development experience began. The second one was in June 2022, when the teachers culminated their participation in the professional development sessions and the summer research experience.

Summary of the descriptive statistics

For the pretest, data from five participants were analyzed. The minimum score for these cases was 74, while the maximum was 95. According to the rating scales of the Measure of Acceptance of the Theory of Evolution (Rutledge, 1996), the levels of acceptance fluctuated between moderate and very high acceptance. On average, however, the score is 83.20, which places the acceptance average at a high level. The standard deviation of this data is 7.98, while its variance is 63.70 (see Table 9).

Table 9 Descriptive statistics for MATE-E administration to PD participants

Table 9

Descriptive statistics for MATE-E administration to PD participants
Min. Score	Max. Score	Avg.	Std. Dev.	Variance	Skewness		Kurtosis
Min. Score	Max. Score	Avg.	Std. Dev.	Variance	Statistic	Std. Error	Statistic	Std. Error
74	95	83.20	7.98	63.70	.598	.913	.408	2.000
80	99	91.40	7.54	56.80	− .941	.913	.079	2.000

In relation to the posttest, data from five participants were analyzed. The minimum score of these cases was 80, while the maximum was 99. The five cases presented increases in their score between the pretest and the posttest (see Fig. 4). According to the classification scales of the original MATE instrument (Rutledge, 1996), acceptance levels fluctuated between high and very high acceptance. Compared to the pretest data, there were three participants whose acceptance level increased: one case from moderate acceptance to high acceptance level and two cases from high acceptance to very high.

Wilcoxon matched pairs and signed ranks test

Upon carrying out the Wilcoxon matched pairs and signed ranks test, it was found that all ranks are positive. This indicates that there was an increment in all participants’ acceptance of the theory of evolution, Statistical analysis for the test also showed that its Z score is -2.023 and its p value is .043. Data is summarized in Tables 10 and 11.

Table 10 Summary of signed ranks for MATE-E administration to PD participants

Table 10

Summary of signed ranks for MATE-E administration to PD participants
		N	Average of ranks	Sum of ranks
Posttest – Pretest	Negative ranks	0^a	.00	.00
	Positive ranks	5^b	3.00	15.00
	Tied ranks	0^c
	Total	5
^a Posttest < Pretest ^b Posttest > Pretest ^c Posttest = Pretest

Table 11 Wilcoxon matched pairs and signed ranks test for MATE-E administration to PD participants

Table 11

Wilcoxon matched pairs and signed ranks test for MATE-E administration to PD participants
	Posttest – Pretest
Z score	-2.023^a
Sig. (2-tailed hypothesis)	.043
^a Z score based on negative ranks

Various published instruments (Nadelson and Southerland, 2012; Rutledge and Warden, 1999; Smith et al. 2016) could have been considered for this study to assess the content and acceptance of the theory of evolution. We chose to use and translate the MATE survey because its use is well documented in the literature (Rutledge, 1996; Rutledge & Sadler, 2007; Rutledge & Warden, 1999), and it was the instrument that best aligned with the project and the PD program’s objectives regarding the improvement of acceptance of the theory of evolution. The need to translate it into Spanish is a consequence of the lack of instruments that could help us assess in-service high school biology teachers’ acceptance of the theory of evolution in the target language.

Evidence was obtained on the translation, adaptation, and validation of the MATE in Spanish (MATE-E). A translation/back-translation framework, complemented by an evaluation done by an expert panel provided precise content validity for the instrument. Furthermore, a focus group was carried out to gather evidence for the response process. During the focus group, topics such as the test’s content, and the participants’ response process were discussed. These results and their subsequent evaluation by the expert panel provided more validity evidence and insight into the changes needed to improve the instrument.

A pilot study was carried out with secondary school students (n = 185) to gather evidence of internal structure and reliability. Descriptive statistics confirmed test normality, which made it possible to perform reliability tests. These tests showed that the translated, adapted, and validated MATE-E instrument possesses good internal consistency and additional tests for sampling adequacy and sphericity confirmed that the data could undergo an exploratory factor analysis.

Upon exploratory factor analysis, five components were found to explain 59.472% of the instrument’s variance. Due to each item’s strong correlation to their respective components, no items were excluded from the Spanish translation of the MATE instrument. Factors were also found to behave in a comparable manner as Rutledge and Sadler’s (2007) discussion of MATE’s concepts. Some items aligned with the components similarly in the Spanish translation. These concepts were considered when naming MATE-E’s factors. All evidence considered; the MATE-E instrument is found to be fit for its use with Puerto Rican Spanish speakers. This version was administered to five Puerto Rican, Spanish-speaking, public high school biology teachers before and after participating in a professional development program.

The overarching theme of genome to phenome framed the design of the professional development program to target a broad range of concepts related to evolution. Each learning module's design considered the content's depth and breadth to address any misconceptions on key content from the theory of evolution. Such complexity was addressed by designing an extensive two-year-long PD program with a multi-dimensional approach that included experiences such as training in content knowledge using best teaching practices, summer research, mentoring, and support for in-classroom transfer and application. The MATE-E was administered before the training began and after the summer research ended.

The Wilcoxon test assessed significant changes in teachers' acceptance of the theory of evolution theory after participating in the professional development program using data gathered through the pre and posttest administration of the MATE-E. A positive and significant difference is observed in the teachers’ levels of acceptance of the theory of evolution after participating in the project's PD program. This suggests that the PD program was effective in changing the participants’ acceptance of the theory of evolution.

Our results suggest that the intensive and hands-on approach involved in these experiences was pivotal in improving their acceptance of the theory of evolution. For future studies we will investigate the effects of the program in changing students’ acceptance of the theory of evolution. The participants were mentored into developing lessons and activities to instruct their students about the same topics they learned. The MATE-E will help us assess the change in their students’ acceptance of the theory of evolution before and after they participated in their teachers’ lessons and activities.

We recognize that the use of other instruments could give different results and/or lead to other conclusions (Barnes et al. 2019). Hence, for future research, we recommend the use of different or additional instruments that could provide a better scope on how a PD program can help improve in-service high school teachers’ acceptance of the theory of evolution. Furthermore, several authors, including Wagler and Wagler (2013), and Romine et al. (2018), have pointed out that the MATE has some limitations which include the addition of context in its items, its dimensionality, and its measurement invariance. To address these limitations during our study, we made sure to follow best practices for the sound translation and adaptation of the instrument (Bravo, 2003).

It is noteworthy that before this report, Barnes et al. (2022) published a revised version of the MATE instrument, the MATE 2.0. The revised instrument contains nine items instead of twenty and has a different scoring system (Barnes et al. 2022). We could consider the use of the MATE 2.0 for further studies; however, it should first be translated to Spanish and adapted for our purposes. Evidence for its validation should also be gathered to determine if the instrument is appropriate for use with the target audience.

Other limitations of the study include the fact that all research activities were conducted using online tools and platforms as this part of the project took place during the COVID-19 pandemic lockdown period. Hence, it should be noted that the expert panel meetings, focus group, pilot study, and the pre-posttest administration of the MATE-E were all online. Other researchers who plan to replicate this study may opt to carry out research activities in person. Additionally, there was potential sampling bias during the pilot study phase, because all participants were from the University of Puerto Rico, Rio Piedras Campus’ Laboratory Secondary school. Researchers who may seek to perform a confirmatory factor analysis to test whether data fits the measurement model are recommended to further expand the population to include students from different schools in their sample. Lastly, regarding the data obtained from teacher participants, as there were few participants (n = 5, they were all female, and from Puerto Rico) these results cannot be generalized because it does not meet sample size and sample representation criteria. A larger, more representative sample size should be considered for future studies, to confirm if results can be generalized.

The present study provided evidence for the reliability and validity of the Spanish translation of the instrument Measure of Acceptance of the Theory of Evolution. We gathered validity evidence to support MATE-E use in research and evaluation with Puerto Rican, Spanish-speaking, secondary-level students, and biology teachers. Furthermore, this study provided evidence for the effectiveness of a professional development program in changing teachers’ acceptance of the theory of evolution. Although results cannot be generalized, this study further proves that professional development for in-service educators is essential to enhance their acceptance of the theory of evolution.

EFA

Exploratory Factor Analysis

MATE

Measure of Acceptance of the Theory of Evolution

MATE-E

Measure of Acceptance of the Theory of Evolution-Español

Professional Development

Ethics approval and consent to participate

This protocol was approved by the University of Puerto Rico, Rio Piedras Campus’ Institutional Review Board (CIPSHI #2021-018). The research team prepared consent and assent forms that were presented and discussed accordingly.

Consent for publication

Not applicable

Competing interests

The authors declare that they have no competing interests

Author details

¹ College of Education, University of Puerto Rico, Rio Piedras Campus. ² College of Natural Sciences, University of Puerto Rico, Rio Piedras Campus

Funding

This work was supported by the National Science Foundation (NSF #1736026)

Author Contribution

ÁEPV: conducted all data gathering, analysis and interpretation and drafted the initial manuscript. MB: conceptualized the study, designed the methodology, supervised the project and contributed to the manuscript revisions. ÁEPV and MB: obtained and managed all necessary IRB approvals required for the research. RP: acquired funding, provided research experience for participants and their students; and provided overall project vision. All authors read and approved the final manuscript.

Acknowledgement

We would like to thank Ms. Minnuette Rodríguez and Ms. María Ortíz for the development of the learning modules, as well as the training and mentoring of teacher participants. We would also like to acknowledge Dr. Marta Fortis, Ms. Brenda Santiago and Ms. Jomarie Ortiz-Álvarez at the Center for Science and Math Education research for their contributions to the implementation of the PD program and administrative support. Additional thanks to Dr. Alfredo Ghezzi and Dr. José A. Rodríguez-Martínez and their laboratories’ personnel for providing the summer research experience to the participants. We would also like to thank Dr. Keyla Soto for her help with the research activities done at the University of Puerto Rico Rio Piedras Campus Laboratory Secondary School. Special thanks to Dr. Manuel Leal from the University of Missouri, for providing field experience and workshops for participating teachers and their students. Finally, our appreciation to all the students and teachers that agreed to participate in this study.

Data Availability

The MATE-E instrument is included in this publication as Additional File 1 and is available for use, provided proper citation is given. The datasets generated and analyzed during the current study are not publicly available due to the University of Puerto Rico, Rio Piedras Campus’ Institutional Review Board only authorizing aggregate data to be reported (CIPSHI #2021-018). Please contact MB for more information.

American Educational Research Association, American Psychological Association. National Council on Measurement in Education. Standards for educational and psychological testing. Washington, DC: American Educational Research Association; 2014.
Barnes ME, Dunlop HM, Holt EA, Zheng Y, Brownell SE. Different evolution acceptance instruments lead to different research findings. Evol Educ Outreach. 2019;12(4):1–17. https://doi.org/10.1186/s12052-019-0096-z.
Barnes ME, Misheva T, Supriya K, Rutledge M, Brownell SE. A revised Measure of Acceptance of the Theory of Evolution: introducing the MATE 2.0. Life Scie Ed. 2022;21(1):1–14. https://doi.org/10.1187/cbe.21-05-0127.
Behling O, Law K. Translating questionnaires and other research instruments. Problems and solutions. Thousand Oaks: Sage Publications, Inc.; 2000.
Bravo M. Instrument development: Cultural adaptations for ethnic minority research. In: Bernal G, Trimble JE, Burley AK, Leong FTL, editors. Handbook of racial and ethnic minority psychology. Thousand Oaks: Sage; 2003. pp. 220–38.
Bravo-Vick M, Padro-Collazo P, Borrero M. Mathematics and science attitude inventory: validation for use in a Spanish speaking context. R Est Inv Psico y Educ. 2019;6(2):167–78. https://doi.org/10.17979/reipe.2019.6.2.5687.
Creswell JW. Educational research: Planning, conducting and evaluating quantitative and qualitative research. Boston: Pearson; 2012.
Cronbach LJ. Essentials of psychological testing. New York: Harper & Row; 1984.
Dobzhansky T. Nothing in biology makes sense except in the light of evolution. Am Biol Teach. 1973;35(3):125–9. https://doi.org/10.2307/4444260.
Friedrichsen PJ, Linke N, Barnett E. Biology teachers’ professional development needs for teaching evolution. Sci Educ. 2016;25(1):51–61. https://eric.ed.gov/?id=EJ1110254.
George D, Mallery P. SPSS for Windows step by step: A simple guide and reference. 11.0 update. Boston: Allyn & Bacon; 2003.
King BM, Rosopa PJ, Minium EW. Statistical reasoning in behavioral sciences. Hoboken: Wiley; 2011.
Metzger KJ, Montplaisir D, Haines D, Nickodem K. Investigating undergraduate health sciences students’ acceptance of evolution using MATE and GAENE. Evolution. 2018;11(1):10. https://doi.org/10.1186/s12052-018-0084-8.
Nadelson LS, Southerland S. A more fine-grained measure of students’ acceptance of evolution: development of the inventory of student evolution acceptance—I-SEA. Int J Sci Educ. 2012;34(11):1637–66. https://doi.org/10.1080/09500693.2012.702235.
Nehm RH, Schonfeld IS. Does increasing biology teacher knowledge of evolution and the nature of science lead to greater preference for the teaching of evolution in schools? J Sci Teach Educ. 2007;18:699–723. http://dx.doi.org/10.1007/s10972-007-9062-7.
Newall E, Reiss MJ. Evolution hesitancy: challenges and a way forward for teachers and teacher educators. Evol Educ Outreach. 2023;16(5):1–17. https://doi.org/10.1186/s12052-023-00183-9.
Pituch KA, Stevens JP. Applied multivariate statistics for social sciences. New York: Rutledge; 2016.
Romine WL, Barnett E, Friedrichsen PJ, Sickel AJ. Development and evaluation of a model for secondary evolution educators’ professional development needs. Evol Educ Outreach. 2014;7(27):1–10. https://doi.org/10.1186/s12052-014-0027-y.
Romine WL, Todd AN, Walter EM. A closer look at the items within three measures of evolution acceptance: analysis of the MATE, I-SEA, and GAENE as a single corpus of items. Evol Educ Outreach. 2018;11(17):1–20.
https://doi.org/10.1186/s12052-018-0093-7
Rutledge ML. Indiana high school biology teachers and evolutionary theory: Acceptance and understanding. (Doctoral dissertation). Ball State University; 1996.
Rutledge ML, Sadler KC. Reliability of the Measure of Acceptance of the Theory of Evolution (MATE) instrument with university students. Am Biol Teach. 2007;69(6):332–5. https://doi.org/10.1662/0002-7685(2007)69[332:ROTMOA]2.0.CO;2.
Rutledge ML, Warden MA. The development and validation of the measure of acceptance of the theory of evolution instrument. Sch Sci Math. 1999;99(1):13–8. https://doi.org/10.1111/j.1949-8594.1999.tb17441.x.
Siegel S, Castellan J. Nonparametric statistics for the behavioral sciences. New York: McGraw-Hill; 1988.
Smith MU, Snyder SW, Devereaux RS. The GAENE—generalized acceptance of evolution evaluation: development of a new measure of evolution acceptance. J Res Sci Teach. 2016. https://doi.org/10.1002/tea.21328.
Wagler A, Wagler R. Addressing the lack of measurement invariance for the measure of acceptance of the theory of evolution. Int J Sci Educ. 2013;35:2278–98. https://doi.org/10.1080/09500693.2013.808779.
Wilcoxon F. Individual comparisons by ranking methods. Biometrics Bull. 1945;1(6):80–3. https://doi.org/10.2307/3001968.
Wingert JR, Basset GM, Terry CE, Cheng K, Lee J. The effects of religious views and creationism on teleological reasoning, acceptance and understanding of natural selection: a preliminary mixed-methods study. Evol Educ Outreach. 2023;16(10):1–17. https://doi.org/10.1186/s12052-023-00186-6.

No competing interests reported.

AdditionalFile1.docx

Download PDF

Editorial decision: Revision requested
08 Sep, 2024
Reviews received at journal
05 Sep, 2024
Reviews received at journal
29 Aug, 2024
Reviewers agreed at journal
09 Aug, 2024
Reviewers agreed at journal
09 Aug, 2024
Reviewers invited by journal
07 Aug, 2024
Editor assigned by journal
05 Aug, 2024
Submission checks completed at journal
05 Aug, 2024
First submitted to journal
04 Aug, 2024

You are reading this latest preprint version

Transforming in-service high school biology teachers’ acceptance of evolution through professional development

Status:

Version 1

Abstract

Figures

Introduction

Methods and Materials

Content validity evidence for MATE-E

Response process evidence for MATE-E

MATE-E Pilot study and internal structure evidence

Statistics for MATE-E pilot study

Administration to PD participants

Results

Genome-to-phenome Professional Development Program description

Study question

Summary of the descriptive statistics

Wilcoxon matched pairs and signed ranks test

Discussion and conclusion

Abbreviations

Declarations

Ethics approval and consent to participate

Competing interests

Author details

Funding

Author Contribution

Acknowledgement

Data Availability

References

Additional Declarations

Supplementary Files

Status:

Version 1