DOI: https://doi.org/10.21203/rs.3.rs-2873136/v1
The globalization of education is becoming increasingly competitive. The convenience of online learning helps countries actively respond to this challenge by overcoming geographic barriers and, during medical emergencies like the COVID-19 pandemic, facilitating safe, accessible, high-quality education. This two-stage study developed a tool for assessing online learning literacy for physical education (PE) teachers (OLLPET) using two rounds of the Delphi method and one round of expert ranking consultations with 15 PE experts from universities, primary and secondary schools, and teaching-research staff to draw up, revise, and finalize an evaluation index system for OLLPET. Our OLLPET evaluation index system includes three first-level indicators—online learning values (OLV), online learning essential character (OLEC), and online learning key competencies (OLKC)—with equal weighting given to OLV (0.367) and OLKC (0.367) and slightly less given to OLEC (0.267). The OLLPET is a theoretical yet practical tool that governments, schools, and teachers can use to evaluate PE teachers’ online learning literacy to improve their learning capacity in a targeted manner.
With the globalization of education, international educational exchanges and competition have become more frequent, and countries worldwide have increased their attention on teachers’ literacy—including online literacy. At the same time, information technology (IT) penetrated all corners of our lives [1, 2]. The COVID-19 pandemic fueled a shift from purely offline learning to blended online and offline learning to make student and teacher education safe, accessible, and convenient [3]. Physical education (PE) teachers must also reinforce their core literacy through online learning to renew teaching methods and approaches in PE classrooms [4]. Although a full understanding of the online learning literacy of physical education teachers (OLLPET) could enhance the quality of PE teaching [5], current research on OLLPET is limited. Therefore, we constructed an online learning literacy assessment tool applicable to PE teachers to provide scientific evaluation and feedback to advance PE teachers’ online learning.
The Strategic Plan for Teachers (2022–2025) published by the United Nations Educational, Scientific and Cultural Organization (UNESCO) proposed using multiple approaches, including information and communications technology (ICT), to enhance the quality of teaching and learning and to foster teachers’ professional development [6]. Economic growth has led to an increase in global IT and changes in learning styles, expanding the popularity of online learning and making theoretical research on online learning support a hot topic. The rising need for inter-regional exchanges has driven people to seek ways of learning that transcend time and space constraints. PE teachers’ endogenous demand for online learning is the fundamental driving force behind the construction of online learning literacy assessments [7]. The current level of IT use among PE teachers remains inadequate, and their online learning literacy is particularly important in the face of major public crises like the COVID-19 pandemic, which caused a global health crisis and created barriers to offline communication [8].
Online learning is highly efficient and immediate, breaking through the time and space constraints of traditional learning in ways that align with the work-related characteristics of primary and secondary school teachers. Online communication and training allow for new experiences but also place great demands on teachers’ online learning skills. Trends in the macro environment and PE teachers’ fundamental need for online learning urgently call for an in-depth study of PE teachers’ online learning literacy [9]. Based on our search of the EBSCO and China National Knowledge Internet (CNKI) databases for articles on learning literacy, research on learning literacy has become increasingly popular since 2007. However, most relevant studies have focused on students’ learning literacy; relatively few have examined teachers’ learning literacy. Fewer still have focused on PR teachers, despite the importance of evaluating online learning for PE teachers. Thus, there is a real need for more research on OLLPET.
Research on learning literacy, teachers’ literacy, and PE teachers’ literacy has evolved from defining concepts and connotations to building assessments [10]. Constructing an assessment tool for OLLPET has several benefits. First, it would enrich the theoretical research on PE teachers, refine research on their core literacy, lay the foundation for evaluating their learning, and provide a reference for future research on their learning literacy. Second, an assessment would help government departments and schools evaluate OLLPET while fostering PE teachers’ self-perceptions of their online learning literacy. Finally, it would help identify suggestions for PE teachers to strengthen their online learning skills and optimize their learning styles, enhancing their teaching skills and abilities. Thus, we constructed an OLLPET assessment tool to provide a basis for evaluating and promoting the development of OLLPET.
To develop an assessment for OLLPET, we needed to divide the study into three stages. In the first stage, we established a definition of OLLPET through a literature search; we also had to select evaluation indicators for OLLPET. In the second stage, we sought expert advice using the Delphi method to identify the evaluation indicators. In the third stage, we determined the weights of the evaluation indicators via the expert-ranking method (Fig. 1). The Ethics Review Committee of East China Normal University (#HR 096-2021) approved the study’s protocol, all methods were performed in accordance with the relevant guidelines.
This study’s core concept is the relatively new concept of OLLPET. Giving OLLPET a clear and practical definition is crucial and directly determines the research direction. A literature search revealed no standard definition for OLLPET. However, the key components in OLLPET, online learning and literacy learning, have been well defined. We integrated those to synthesize a working definition for OLLPET, described below.
PE teachers’ work revolves around various teaching activities, and the logical starting point for educational work is developing students’ core literacy [11]. In 2018, China’s Ministry of Education defined subject core literacy as correct values, essential character, and key competencies that students gradually acquire about a given subject [12]. Thus, Yin proposed that the core competency framework for the new era of PE teachers included three first-level indicators: correct values, essential character, and key competencies [13]. Referencing the above points, we identified online learning values (OLV), online learning essential character (OLEC), and online learning key competencies (OLKC) as the first-level indicators of OLLPET. By reviewing the literature and China’s national policies, we determined that OLLPET has three levels of indicators: The first- and second-level indicators should be concise and precise, reflecting a strong sense of framework and logic, and the third-level indicators should be detailed and specific, reflecting the characteristics of OLLPET. We selected 43 initial indicators: three first-, seven second-, and 33 third-level.
The Delphi method, also known as the expert survey method, uses anonymous feedback from experts to reach expert consensus and develop professional guidelines. After several rounds of consultation and feedback, the expert members’ opinions gradually converge, resulting in a collective judgment with a high accuracy rate. The Delphi method relies on rotational iteration; the investigation is incomplete until the experts reach a consensus. The Delphi method helped us ensure the objectivity and accuracy of the OLLPET evaluation indicators. We used an online survey approach, identifying the evaluation indicators through two Delphi rounds, described below.
We needed PE teaching or research experts, preferably university research scholars, primary and secondary school PE teachers, or PE teaching-research staff. The detailed criteria for selecting the experts were these: (1) the university research scholars and teaching-research staff had to have published at least two papers on PE; (2) the primary and secondary school PE teachers had to have more than five years of teaching experience; and (3) all the experts had to be familiar with the core literacy research process and actively support the study.
The survey included a section for the experts’ essential information; we asked them to rate their familiarity with the indicators and provide the basis for their judgments. Each option had a corresponding score (Table 1) that we used to calculate the level of the experts’ authority as follows:
Authority coefficient = \(\frac{\mathbf{s}\mathbf{c}\mathbf{o}\mathbf{r}\mathbf{e} \mathbf{f}\mathbf{o}\mathbf{r} familiarity + score for basis of judgement}{2}\).
The general rule is that when an expert’s authority coefficient is at or above 0.70, the expert’s opinion regarding the survey is authoritative.
| Type | Option | Score |
|---|---|---|
| Familiarity with the indicators | Very familiar | 1 |
| Relatively familiar | 0.8 | |
| Somewhat familiar | 0.6 | |
| Relatively unfamiliar | 0.4 | |
| Very unfamiliar | 0.2 | |
| Basis for judgement | Theoretical analysis | 1 |
| Practical experience | 0.75 | |
| Peer understanding | 0.5 | |
| Intuitive perception | 0.25 |
In a Delphi survey, the study’s background and relevant research materials need to be explained to the experts, who then judge and select the importance of each indicator. Based on a 5-point Likert scale, the indicators are classified into five levels of importance, from high to low—very important, important, somewhat important, unimportant, and very unimportant—with the five options assigned scores of 5, 4, 3, 2, and 1, respectively. The indicators’ scores are usually measured by three kinds of data: the average score, the standard deviation, and the coefficient of variation. By reviewing master’s and PhD theses and statistical references, we found that if the score of an indicator satisfied the criteria of a “mean score greater than 3.0, [a] standard deviation [of] less than 1, [and a] coefficient of variation [of] less than 0.2,” the indicator had a high level of confidence and could be retained [14].
After counting and analyzing the results of the first round of the survey administered to the experts, we screened and eliminated indicators that did not meet the requirements, revised indicators with reference to the experts’ comments, and prepared and distributed the second survey to the experts. After recovering and collating the results of the second survey, we needed to verify the convergence with the results of the first round. If there was convergence at this stage, the survey was closed; otherwise, it was necessary to conduct another survey until the results converged.
The expert-ranking method also uses surveys to elicit experts’ opinions on indicators. Experts do not need to give specific values for each indicator; they only need to rank the importance of the peer indicators. Our formula for calculating the indicator weights was as follows: (aj) = \(\frac{2\left[\mathbf{M}\right(1+N)-Rj]}{\mathbf{M}\mathbf{N}(1+\mathbf{N})}\), where M was the number of experts participating in the survey and N was the number of indicators at the same level; aj denoted the weight of the indicator; and Rj represented the rank-sum of the jth indicator. The rank sum was the sum of the ranking numbers of m experts for a given indicator. In the case of two or more indicators of equal importance, we used the same ranking number, deferring the ranking number of the indicator that came after it [15]. For example, if an expert ranked five indicators as 1, 2, 3, 3, and 5 (i.e., two indicators were tied as the third-most important), we recorded the ranking number of both tied indicators as 3.5 because (3 + 4) / 2 = 3.5. This juxtaposition did not affect the calculation of the indicator weights; the sum of the weights of all indicators at each level was always equal to 1.
We performed two rounds using the Delphi method and one round using the expert-ranking method. We distributed 20 surveys in the first Delphi round and received responses from 15 experts. We sent the second Delphi round surveys only to the experts who participated in the first round; all 15 experts completed the second-round surveys and the expert-ranking method surveys. Thus, 15 experts fully participated in the two rounds of the Delphi method and the single round of the expert-ranking method (Table 2). The expert authority coefficient was 0.78, demonstrating a good level of authority across the community of experts.
| Characteristic | Descriptors | N |
|---|---|---|
| Sex | Male | 11 |
| Female | 4 | |
| Age (years) | 30–39 | 6 |
| 40–49 | 9 | |
| Education level | Undergraduate | 4 |
| Master’s | 2 | |
| PhD | 9 | |
| Title | Senior | 2 |
| Sub-senior | 10 | |
| Intermediate | 3 | |
| Years of work experience | < 10 | 3 |
| 10–19 | 6 | |
| 20–29 | 6 | |
| Position at work | University teacher | 10 |
| Teaching-research staff | 1 | |
| Primary and secondary school teachers | 4 |
Many scholars worldwide have defined the concept of online learning relatively clearly. Online learning is distinguished in a broad and a narrow sense. Broadly speaking, online learning refers to browsing information or digital content on the internet to gain knowledge or experience. Narrowly speaking, it means purposeful and planned learning activities that learners undertake over a fairly concentrated period to complete a specific task or goal [16]. Integrating the two views, we defined PE teachers’ online learning as the process by which they use the internet to access learning resources or interact with other learners to supplement and reinforce their knowledge for professional growth or to solve practical problems in teaching.
Scholars have studied learning literacy far longer than online learning. Learning literacy refers to the systematic and profound qualities that individuals develop during the learning process through repeated, ongoing practice [17]. Thus, it means the combination of competencies and qualities individuals exhibit when faced with complex learning situations. It helps individuals identify their learning needs, choose appropriate learning methods and strategies, and adjust and evaluate their learning processes.
Having clarified the notions of online learning and literacy, we needed to assign a clear definition to OLLPET that was detailed and practical; its superordinate concept had to be the core literacy of PE teachers [18]. Fu et al. wrote that PE teachers’ core literacy was the most critical quality they could possess in the 21st century because it promotes lifelong development and teaches correct values, essential character, and essential competencies [19]. The definition of OLLPET considers the characteristics of online learning and PE. Ultimately, we defined OLLPET as the comprehensive competencies or qualities PE teachers demonstrate by accessing learning resources and refining knowledge online. OLLPET helps PE teachers identify their learning needs, select learning strategies, and adapt and evaluate the learning process in three critical areas: values, essential character, and key competencies.
The first draft of the OLLPET evaluation system encompassed three first-level, seven second-level, and 30 third-level indicators. The three first-level indicators were OLV, OLEC, and OLKC. OLV describe the value perspective and identification of PE teachers with online learning activities; OLEC reflects the extent to which PE teachers adhere to online learning and maintain their learning character; and OLKC reflects the knowledge, skills, and execution capabilities required for PE teachers to conduct online learning activities successfully [20]. Developing second-level indicators requires considering PE teachers’ ideological awareness, learning process, and practical workplace demands. We identified seven second-level indicators of OLPET: (1) career view (CV); (2) = learning view (LV); (3) learning spirit (LS); (4) learning character (LC); (5) ability to discover learning resources (ADLR); (6) ability to plan the learning process (APLP); and (7) ability to apply learning outcomes (AALO). We followed these indicators with several rounds of surveys with the experts involving changes; therefore, we will not present the third level of evaluation indicators in detail here.
After distributing the survey and receiving the experts’ ratings of the importance of the indicators, we calculated the mean, standard deviation, and coefficient of variation of the scores for each indicator (Table 3).
| Level | Indicator name | Score | SD | C.V |
|---|---|---|---|---|
| Level 1 Indicators | Online learning values | 4.80 | 0.41 | 0.09 |
| Online learning essential character | 4.67 | 0.62 | 0.13 | |
| Online learning key competencies | 4.73 | 0.46 | 0.10 | |
| Level 2 Indicators | Career view | 4.53 | 0.52 | 0.11 |
| Learning view | 4.67 | 0.62 | 0.13 | |
| Learning spirit | 4.87 | 0.35 | 0.07 | |
| Learning character | 4.73 | 0.59 | 0.13 | |
| The ability to discover learning resources | 4.80 | 0.41 | 0.09 | |
| The ability to plan the learning process | 4.67 | 0.49 | 0.10 | |
| The ability to apply learning outcomes | 4.80 | 0.56 | 0.12 | |
| Level 3 Indicators | PE teachers’ professional development teachers through online learning | 4.27 | 0.88 | 0.21 |
| Online learning can enhance PE teachers’ teaching skills | 4.53 | 0.52 | 0.11 | |
| Online learning facilitates access to cutting-edge information in PE | 4.67 | 0.62 | 0.13 | |
| Online learning can improve the quality of PE teachers’ work | 4.40 | 0.83 | 0.19 | |
| Have a strong interest in online learning in sports | 4.67 | 0.49 | 0.10 | |
| Have a positive attitude toward online learning in sports | 4.67 | 0.49 | 0.10 | |
| Have awareness of lifelong adherence to online learning in sports | 4.73 | 0.59 | 0.13 | |
| Active adaptation to IT development in PE | 4.67 | 0.49 | 0.10 | |
| Stick to long-term online learning in sports | 4.60 | 0.63 | 0.14 | |
| Critically question online learning content in sports | 4.67 | 0.49 | 0.10 | |
| Have the courage to overcome the challenges in online learning in sports | 4.67 | 0.49 | 0.10 | |
| Reflect on the effectiveness and problems of online learning in sports | 4.67 | 0.49 | 0.10 | |
| Adhere to basic online ethics in online learning in sports | 4.67 | 0.49 | 0.10 | |
| Maintain a sense of social responsibility in online learning in sports | 4.60 | 0.51 | 0.11 | |
| Do not misuse learning resources and competencies acquired in online learning in sports | 4.53 | 0.64 | 0.14 | |
| Ability to operate a computer related to online learning in sports | 4.93 | 0.26 | 0.05 | |
| Ability to take the initiative to join a sports online learning network group | 4.33 | 0.82 | 0.19 | |
| Ability to find PE learning resources on national and international websites | 4.53 | 0.52 | 0.11 | |
| Access online learning resources for sports (e.g., micro-courses and courses of national quality) | 4.80 | 0.41 | 0.09 | |
| Explore the skills of using modern multimedia equipment (e.g., electronic whiteboards) | 4.27 | 0.80 | 0.19 | |
| Ability to explore skills using sports-related IT (e.g., pedometers, heart rate monitors, etc.) | 4.13 | 0.92 | 0.22 | |
| Be able to set reasonable online learning goals in sports based on one’s own circumstances | 4.60 | 0.63 | 0.14 | |
| Ability to select appropriate sports-related online learning content based on goals | 4.67 | 0.72 | 0.16 | |
| Ability to choose appropriate online learning tools based on the content | 4.60 | 0.74 | 0.16 | |
| Ability to effectively implement online learning plans in sports | 4.60 | 0.63 | 0.14 | |
| Ability to assess the effectiveness of one’s own online learning in sports | 4.53 | 0.74 | 0.16 | |
| Be able to apply PE online learning resources | 4.80 | 0.41 | 0.09 | |
| Ability to apply PE online learning resources in PE classroom teaching | 4.93 | 0.26 | 0.05 | |
| Ability to apply PE online learning resources in extra-curricular sports activities | 4.73 | 0.46 | 0.10 | |
| Ability to apply sports online learning resources in after-school sports training | 4.73 | 0.46 | 0.10 | |
| Ability to apply sports online learning resources in school sports competitions | 4.67 | 0.49 | 0.10 | |
| Ability to apply sport online learning resources in sports research work | 4.87 | 0.35 | 0.07 | |
| Ability to conduct sports learning activities such as lectures and exchanges through the Internet | 4.60 | 0.63 | 0.14 |
By comparing the criteria for the retention of indicators, we found that all the first- and second-level indicators met the conditions for retention after the first round of the Delphi method; thus, we did not adjust the first- and second-level indicators. Regarding the third-level indicators, the two indicators’ coefficients of variation of scores were greater than 0.2, which did not meet the statistical requirements (Table 4). After analyzing the specific expressions of these two indicators, we found that the first one on promoting PE teachers’ professional development was too general and failed to reflect the requirements of concision and specificity; the indicator for the skill of using PE IT was more closely related to the teaching work of PE teachers and might be somewhat less relevant to online learning. After careful consideration, we decided to remove those two indicators.
| Name of the indicator | Average score | Standard deviation | Coefficient of variation | Final decision |
|---|---|---|---|---|
| PE teachers’ professional development teachers through online learning | 4.27 | 0.88 | 0.21 | Remove indicator |
| Ability to explore skills using sports-related IT (e.g., pedometers, heart rate monitors, etc.) | 4.13 | 0.92 | 0.22 | Remove indicator |
In addition to the two indicators that did not meet the statistical requirements, the experts suggested making three types of changes to the indicators: (1) strengthen the correlation between indicators at different levels and reinforce the linkage between upper- and lower-level indicators; (2) clarify logical relationships and avoid inclusion relationship between indicators at the same level; and (3) revise the language to make them clearer. The Kendall coordination coefficient W for the first survey (0.327; p = 0.000 < 0.01) left room for improvement since it was between 0 and 1; the larger the coefficient, the higher the expert consensus. Combining the experts’ comments, we refined the indicators once more, eventually removing four third-level indicators, adding one new third-level indicator, and revising the statements of some indicators, as detailed below.
After the first round of the Delphi method, we removed four third-level indicators (two did not meet the statistical requirements, and two were not logically related) and added one new third-level indicator. We also revised the presentation of the third-level indicators to make them more concise and specific. We compiled the revised three first-level, seven second-level, and 30 third-level indicators into a survey and sent it to the experts a second time. Analysis of the second-round Delphi data showed significant improvement in the indicators’ mean scores. Also, the standard deviations and coefficients of variation were significantly lower than in the first round, implying a significant increase in the experts’ consensus. The results of the second survey demonstrated that the mean, standard deviation, and coefficient of variation of all indicator scores met the retention requirements and had more desirable scores and stability. The Kendall coordination coefficient W for the second survey was 0.460 (p = 0.000 < 0.01), which was acceptable; the experts did not suggest making further changes to the indicators. Thus, we concluded the Delphi method surveys and determined the OLLPET evaluation indicators (Table 5).
| Level | Indicator name | Score | SD | C.V |
|---|---|---|---|---|
| Level 1 Indicators | Online Learning Values | 4.87 | 0.34 | 0.07 |
| Online Learning Essential Character | 4.93 | 0.25 | 0.05 | |
| Online Learning Key Competencies | 4.93 | 0.25 | 0.05 | |
| Level 2 Indicators | Career View | 4.93 | 0.25 | 0.05 |
| Learning View | 4.93 | 0.25 | 0.05 | |
| Learning Spirit | 4.87 | 0.34 | 0.07 | |
| Learning Character | 4.87 | 0.34 | 0.07 | |
| Discover Learning Resources Ability | 4.87 | 0.34 | 0.07 | |
| Plan Learning Process Ability | 4.87 | 0.34 | 0.07 | |
| Apply Learning Outcomes Ability | 4.87 | 0.34 | 0.07 | |
| Level 3 Indicators | Online learning can enhance PE teachers’ teaching skills | 4.73 | 0.44 | 0.09 |
| Online learning facilitates access to cutting-edge information in PE | 4.87 | 0.34 | 0.07 | |
| Online learning can improve the quality of PE teachers’ work | 4.80 | 0.40 | 0.08 | |
| Have a strong interest in online learning in sports | 4.73 | 0.44 | 0.09 | |
| Have a positive attitude toward online learning in sports | 4.93 | 0.25 | 0.05 | |
| Have awareness of lifelong adherence to online learning in sports | 4.87 | 0.34 | 0.07 | |
| Active adaptation to IT development in PE | 4.73 | 0.44 | 0.09 | |
| Stick to long-term online learning in sports | 4.80 | 0.40 | 0.08 | |
| Maintain communication during online learning | 4.73 | 0.44 | 0.09 | |
| Critically question online learning content in sports | 4.80 | 0.40 | 0.08 | |
| Have the courage to overcome the challenges in online learning in sports | 4.87 | 0.34 | 0.07 | |
| Reflect on the effectiveness and problems of online learning in sports | 4.87 | 0.34 | 0.07 | |
| Adhere to basic online ethics in online learning in sports | 4.80 | 0.40 | 0.08 | |
| Maintain a sense of social responsibility in online learning in sports | 4.87 | 0.34 | 0.07 | |
| Do not misuse learning resources and competencies acquired in online learning in sports | 4.73 | 0.57 | 0.12 | |
| Ability to operate a computer related to online learning in sports | 4.93 | 0.25 | 0.05 | |
| Ability to take the initiative to join a sports online learning network group | 4.53 | 0.62 | 0.14 | |
| Ability to find PE learning resources on national and international websites | 4.87 | 0.34 | 0.07 | |
| Access online learning resources for sports (e.g., micro-courses and courses of national quality) | 4.80 | 0.40 | 0.08 | |
| Be able to set reasonable online learning goals in sports based on one’s own circumstances | 4.87 | 0.34 | 0.07 | |
| Ability to select appropriate sports-related online learning content based on goals | 4.87 | 0.34 | 0.07 | |
| Ability to choose appropriate online learning tools based on the content | 4.80 | 0.40 | 0.08 | |
| Ability to effectively implement online learning plans in sports | 4.87 | 0.34 | 0.07 | |
| Ability to assess the effectiveness of one’s own online learning in sports | 4.60 | 0.61 | 0.13 | |
| Ability to apply PE online learning resources in PE classroom teaching | 4.87 | 0.34 | 0.07 | |
| Ability to apply PE online learning resources in extra-curricular sports activities | 4.47 | 0.62 | 0.14 | |
| Ability to apply sports online learning resources in after-school sports training | 4.60 | 0.49 | 0.11 | |
| Ability to apply sports online learning resources in school sports competitions | 4.80 | 0.40 | 0.08 | |
| Ability to apply sport online learning resources in sports research work | 4.80 | 0.40 | 0.08 | |
| Ability to conduct sports learning activities such as lectures and exchanges through the Internet | 4.47 | 0.50 | 0.11 |
The survey format for the expert-ranking method was slightly different than for the Delphi method, which asked experts to judge the importance of indicators. In the expert-ranking method, the experts ranked the peer indicators in order of importance. We explained the calculation of the indicator weights earlier. Here, we provide an example of calculating indicator weights using the first-level indicator OLV. From the survey data, we can calculate the rank-sum (Rj) of OLV as 27, where the number of first-level indicators N is 3, and the number of experts M is 15: (aj) = \(\frac{2\left[\mathbf{M}\right(1+N)-Rj]}{\mathbf{M}\mathbf{N}(1+N)}\). This calculation produced a weighting of 0.367 for OLV. We computed the weight of the second-level indicator CV (0.600) using the same method. However, since CV was a subordinate indicator of OLV, the final weight of CV was 0.367 × 0.600 = 0.220. We used Eq. (3) to compute the weights for the remaining indicators. The Kendall coordination coefficient W for the survey of the expert-ranking method was 0.679 (p = 0.000 < 0.01), meaning there was strong consistency in the experts’ indicator rankings.
After calculating the indicator weights, we outlined the overall indicator framework and checked the arithmetic results. In the correct framework, the sum of the weights of all indicators at the same level should be equal to 1; otherwise, they should be recalculated. Thus, we derived the complete evaluation indicators and weights for OLLPET (Table 6).
| Level 1 indicators | Level 2 indicators | Level 3 indicators |
|---|---|---|
| Online learning values (0.367) | Career view (0.220) | Online learning can enhance PE teachers’ teaching skills (0.103). |
| Online learning facilitates access to cutting-edge information in PE (0.051). | ||
| Online learning can improve the quality of PE teachers’ work (0.066). | ||
| Learning view (0.147) | Have a strong interest in online learning in sports (0.058). | |
| Have a positive attitude toward online learning in sports (0.040). | ||
| Have awareness of lifelong adherence to online learning in sports (0.026). | ||
| Active adaptation to IT development in PE (0.022). | ||
| Online learning essential character (0.267) | Learning spirit (0.166) | Stick to long-term online learning in sports (0.064). |
| Maintain communication during online learning (0.039). | ||
| Critically question online learning content in sports (0.031). | ||
| Have the courage to overcome the challenges in online learning in sports (0.019). | ||
| Reflect on the effectiveness and problems of online learning in sports (0.013). | ||
| Learning character (0.101) | Adhere to basic online ethics in online learning in sports (0.047). | |
| Maintain a sense of social responsibility in online learning in sports (0.030). | ||
| Do not misuse learning resources and competencies acquired in online learning in sports (0.024). | ||
| Online learning key competencies (0.367) | The ability to discover learning resources (0.179) | Ability to operate a computer related to online learning in sports (0.069). |
| Ability to take the initiative to join a sports online learning network group (0.043). | ||
| Ability to find PE learning resources on national and international websites (0.036). | ||
| Access online learning resources for sports (e.g., micro-courses and courses of national quality) (0.031). | ||
| The ability to plan the learning process (0.106) | Be able to set reasonable online learning goals in sports based on one’s own circumstances (0.032). | |
| Ability to select appropriate sports-related online learning content based on goals (0.022). | ||
| Ability to choose appropriate online learning tools based on the content (0.023). | ||
| Ability to effectively implement online learning plans in sports (0.013). | ||
| Ability to assess the effectiveness of one’s own online learning in sports (0.016). | ||
| The ability to apply learning outcomes (0.081) | Ability to apply PE online learning resources in PE classroom teaching (0.023). | |
| Ability to apply PE online learning resources in extra-curricular sports activities (0.016). | ||
| Ability to apply sports online learning resources in after-school sports training (0.014). | ||
| Ability to apply sports online learning resources in school sports competitions (0.012). | ||
| Ability to apply sport online learning resources in sports research work (0.011). | ||
| Ability to conduct sports learning activities such as lectures and exchanges through the Internet (0.006). |
After two Delphi rounds and one expert-ranking round, we determined the evaluation indicators and weights for OLLPET. We divided the final index system into three levels with 40 indicators that met the requirements for developing an evaluation index system with weighted indicators for the different levels.
OLV is the value content and ideals that individuals hold in relation to online learning that is positive or meets certain accepted standards; it reflects individuals’ motivations or innate drive for online learning and their awareness of online learning [21]. In the final evaluation index system, the weight of OLV was 36.7%, reflecting the vital role of OLV in online learning. PE teachers’ online learning behaviors usually have two goals: (1) professional development and self-improvement; and (2) solutions to a specific task or problem. Thus, we included two secondary indicators under the OLV dimension: CV and LV. CV related to online learning’s contribution to professional growth or enhancing professional skills and work quality. LV related to PE teachers’ conscious (versus instinctive) self-directed learning. CV contained three third-level indicators: PE teachers’ development of their teaching skills, access to cutting-edge information in teaching, and improving teaching quality through online learning. LV contained four third-level indicators: PE teachers’ interests, attitudes, awareness, and inclinations toward online learning.
OLEC is an individual’s spirit, habit, or character that facilitates or enhances the effectiveness of their online learning; it is a stable characteristic or performance. The learning characteristics of teachers and students are universally linked and present in teaching and learning [22]. OLEC comprised 26.7% of the evaluation index system, which was slightly lower in weight than the other two first-level indicators but still reflected its integral role in individuals’ online learning behaviors. OLEC expressed people’s spiritual cultivation and moral character in online learning, so we divided it into two dimensions: LS and LC. LS means PE teachers’ ability to critically question and reflect over time to fulfill the strenuous demands of long-term online learning; overcome difficulties in online learning; and enhance the effectiveness of online learning. LC means PE teachers’ compliance with rules, preserving a sense of social responsibility, and promoting academic ethics in the online learning process. LS contained five third-level indicators used to demonstrate the strength of individuals’ spirit to persist in online learning and maintain communication and critical questioning while reflecting on the challenges of online learning. LC contained three third-level indicators used to demonstrate PE teachers’ sense of social responsibility, academic ethics, and the ability to apply resources properly in online learning.
OLKC encompassed the competencies individuals needed to respond to online learning situations, solve problems in online learning, and complete online learning behaviors successfully. We divided OLKC into general knowledge, information survival, and subject integration. OLKC required basic competence for applying information equipment, general learning, and the ability to learn with the support of modern information equipment [23]; its weight was 36.7% in the evaluation index system, meaning it was as crucial as OLV. We can roughly divide online learning activities according to time processes into discovering learning resources, planning the learning process, and applying learning outcomes. Thus, OLKC comprised three second-level indicators. ADLR means the capacity to find and access online resources to analyze, judge, and extract information. This determines the source and quality of online learning materials. APLA means designing and managing the online learning process, which directly determines learning outcomes. AALO means linking online learning behavior to work and society, which determines the application of learning outcomes [24]. ADLR comprised four third-level indicators describing an individual’s capacity to access and acquire cutting-edge online learning resources for sports and join learning communities. The ALPA comprised five third-level indicators that focused on controlling learning objectives, content, plans, and outcomes. AALO comprised six third-level indicators describing PE teachers’ ability to apply online learning resources and learning outcomes in various teaching contexts, research, and lecture exchanges.
In sum, we identified the framework and connotations of OLLPET and considered further applications. At the government level, the OLLPET evaluation scale could be developed to facilitate scientific evaluation and provide effective regulations at the macro level. At the school level, the evaluation of OLLPET should be carried out and targeted to improve PE teachers’ learning behaviors and enhance the overall quality of teaching and learning in schools. At the individual teacher level, OLLPET could enhance the value and character of online learning in accordance with the indicators to improve online learning skills and foster professional growth.
This study had some limitations. First, the primary selection of the initial indicators was based on China’s national policies and literature; thus, our assessment tool might not be generalizable to other cultures or countries without refinements. Furthermore, the use of more normative interviews or rooted theory could be considered a validation. Second, all our experts (e.g., teachers and researchers) were from Chinese cultural backgrounds. Future research should replicate the study with experts from other cultures and countries and experts from more diverse backgrounds. Third, although the evaluation indicators underwent a standardized screening and weighting process, actual measurement of OLLPET has yet to be carried out to verify their impact.
The assessment indicator system of OLLPET is a theoretical tool that can be used for practical measurements. To the best of our knowledge, this is the first study to evaluate OLLPET in China. This study’s OLLPET covered three first-level, seven second-level, and 30 third-level indicators, all weighted by level. Governments, schools, and teachers can use this system to evaluate PE teachers’ online learning literacy to enhance their learning capacity in a targeted manner.
Author Contributions: Conceptualization and study design: H..T., Z..Y., and F..L.; Data collection: H..T. and H..L.; Manuscript: H..T. and Z..Y.; Supervision: M..S.. All authors have read and agreed to the published version of the manuscript.
Funding: This research was funded by the Later funded projects of the China National Social Science Foundation, grant number 21FTYB006; and China Postdoctoral Science Foundation, grant number 2021M701932.
Institutional Review Board Statement: This study has been approved by Ethics Board at East China Normal University (HR 096-2021, 14 March 2021).
Informed Consent Statement: The Delphi Experts panel has provided informed consent with research ethics and protocols prior to data in manuscript file.
Data Availability Statement: Data can be shared upon reasonable request to the corresponding author.
Acknowledgments: The authors acknowledge the valuable time and insightful feedbacks provided by all the experts that took part in the Delphi study.
Conflicts of Interest: The authors declare no conflict of interest.