Quality Assessment of Regional Primary English Education Based on Big Data: A Case Study of Gansu Province


 Introduction: Recently, big data attract great attention from industries, academia, and governments. Education big data from teaching and learning processes used to assess elementary education level is scarce in China. Case description: Gansu province, located in the northwest China, is one of the most backward economy areas where the development of elementary English education falls far behind developed areas of the country. Taking Gansu province as a case study and English subject in senior high schools as the study object, we use score data of English academic proficiency test (2018-2020) from 322 senior high schools to explore spatial-temporal differences of English education level among those schools in order to promote the balanced development of regional education. First we briefly describe the generation of English big data from senior high schools, and then, we mine the big data to analyze the difference of the English education level using the anomaly method, the comparison method, the cluster analysis method, and hot point analysis method. English excellence degree is developed as an indicator to assess elementary English education level. Discussion and Evalution: The results show that: (1) there exists obvious difference in English education level among the selected senior high schools: the highest average value of English excellence degree is 759 times the lowest, and the number of schools with negative anomalies accounts for more than 50%; (2) English excellence degree displays an obvious agglomeration characteristic in Lanzhou city, the provincial capital, that is, the hot spots (high English level) of English excellence degree are located in Lanzhou, while the cold spots (low English level) are concentrated in minority and poor areas such as Linxia autonomous region, the major part of Longnan and the eastern part of Qingyang; (3) The investigation in the form of questionnaires confirms the fact that disadvantage schools in English level are usually lack of good teachers who have a preference to teach in areas with good conditions. Conclusion: From the results, we can draw the conclusion that big data play an important role in quantitatively assessing basic education level, which lays foundation for improving the regional education level. Individuals’ value orientation of study, attitude towards study and the study behavior will be investigated in the future.


Introduction
At present, there is no consensus on the de nition of big data. It is generally accepted that it refers to data sets that are di cult to obtain, manage and process using common software tools within tolerable time slot. Big data is usually characterized by its huge volume in size, great variety in data types and velocity in fast data creating, processing and analyzing [3,28]. Li & Cheng think that big data is as an important strategic resource as natural resources and human resources [21]. The US government believes that big data is the "new oil of the future" [7], so it takes the lead in launching the "Big Data Development Plan" to make breakthroughs in multiple research and application elds, aiming at generating inestimable amount of value [33].
In terms of education, big data will promote the reform of teaching, educational research, educational management and educational evaluation [16,37]. Especially in educational evaluation, big data can make it more comprehensive, objective and accurate [34]. Accurate assessment results let students know how well they have grasped the knowledge, understand their personality preference, make up for shortcomings and seek help on the nose. The assessment also lets teachers nd out which knowledge has not been mastered by students and thus needs to be repeated or emphasized, and what kinds of problems are most prone to happen, so as to improve teaching methods. Moreover, it lets researchers explore the correlation between variables---learning content, learning time and learning method----and the nal learning result [1,22] in order to nd main factors impeding student learning. Last but not the least, it lets policy makers realize the imbalanced development in education environment, education resources, education opportunities and education quality of regional education, and devote to establish a comprehensive, real-time and dynamic quality monitoring system of education [32]. It can be seen that the evaluation based on big data plays a huge role in the eld of education. However, for the problem of serious imbalance of educational resources between urban and rural areas, among regions and across schools, there is still a lack of study on how we can creatively use big data to accurately evaluate the education quality so as to promote the balanced development of regional education.
Gansu is located in the western region of China, whose educational environment, especially that of the poor areas and minority areas, is signi cantly different from that of other regions. However, the contrast is only an empirical judgment. How to quantify the contrast is beyond the grasp of education policymaking departments. If there is no scienti c, reasonable and systematic evaluation of teaching quality, but only the use of empirical generalization and subjective estimation to evaluate the difference in education, mistakes will be made in policy making. Through mining, analyzing, modeling and evaluating big data in education, managers can timely and accurately understand the regional teaching situation, nd typical vulnerable schools and groups, and investigate the causes of the vulnerability, so as to more scienti cally balance the allocation of education resources. English is an important subject in basic education in China [36]. Taking English education as an example, we aim to achieve the following objectives through the application of big data: (1) Scienti cally re ect the spatial and temporal differences of regional English teaching quality; (2) Find some typical vulnerable areas (cold spots in English education) through big data mining. The research results provide methods for the education department to evaluate the regional education level, and help to make better education decisions in order to promote the balanced development of basic education.

Case Description
Study area Gansu province is located at the intersection of the Loess Plateau, Qinghai-Tibet Plateau and Inner Mongolia Plateau, between 32°11'-42°57'N and 92°13'-108°46'E. Administratively, it has 14 cities including 86 counties. As of 2021, the permanent resident population is 25.0198 million, and the regional Gross domestic product (GDP) is 824.61 billion yuan, or 31,336 yuan per capita based on resident population [11]. Out of the 56 ethnic groups in China, there are 16 in Gansu, all of which have a population of more than 1,000. Due to multiple reasons, Gansu has been a relatively backward area in the development of basic education in China.
According to the statistics of Gansu Provincial Department of Education in 2019 [10], there were 376 senior high schools in the province (Figure 1), with 526270 students and an average population of 1399 in each school. As can be seen from Figure 1, the distribution of senior high schools in Gansu is extremely uneven, scattered in the western and southwestern part, and concentrated in the central and southeastern part.

Data collection
English education big data is composed of four parts: historical score data, basic information data, teaching data, and learner personal data (see Figure 2). Historical score data records the performance of the students generated in the learning process. Basic information data records basic information of schools, teachers, and students, such as the location of school, its education resource, teachers' and students' archive, etc. Teaching data includes the information of teaching materials, tools (internet) and teaching activities. Learner personal data contains information of students' individual character, preference and habits of study. In this study, we mainly use the historical score data for analysis.
Learning data varies spatiotemporally. Spatially, learning processes differ among schools in the region. Temporally, the scores uctuate in mid-term, nal and periodic test. We only use basic information data concerning school location points of interest (POI) (data source: http://www.databox.store/) and historical score data of English academic pro ciency test (2018-2020) coming from Gansu Education Examination Institute for evaluating the English education level. Some data is collected from questionnaires.

Methodology
Through a variety of methods of data mining, we can identify some typical vulnerable schools, and then, design programs in line with the school's own development, such as teacher training programs and shortterm exchange programs with high-quality schools, adjust the allocation of education funds, and implement a number of policies that favor vulnerable schools. The data mining methods and the paths to improve education quality are shown in Figure 3. In the next section, we introduce the analysis methods used to evaluate the English education level in the study area.

The anomaly method
The anomaly method is used to re ect the degree to which the performance data are away from the central value. Some typical vulnerable schools and vulnerable individuals can be distinguished according to this degree, which is represented by the anomaly percentage (k'), calculated as follows: where, k i is an exam score in school i, k i is the average of the exam score in all schools.

Comparison method
In order to assess the level of English in each school, we developed an index named English excellence degree (EED). The EED (X) is calculated as follows: where: M j is the average score of English in the j school; E j is the passing rate of English subject in the j school.
According to the EED from 2018 to 2020, we use the percentage change rate (in every two years) as the index to compare and analyze the changes in English teaching quality in each school, and look for schools with great changes.
where: ΔEED is the percentage change rate between year t and year t-1.
We use the quartile method to compare and analyze the changing trend of English quality. In this method, we arrange data from the smallest to the largest in each year. The rst quartile is de ned as the middle number between the smallest number (minimum) and the median of the data set. The second quartile is the median of a data set. The third quartile is the middle value between the median and the highest value of the data set. We can use those quartile values to build a violin plot. A violin plot is a method for graphically depicting groups of numerical data through their quartiles. Moreover, it also shows the probability density of the data at different values. We use the GraphPad Prism 8.0 software to make the violin plot, which by re ecting the distribution characteristics of EED every year, helps us compare the English level of all schools in three years.

Cluster analysis method
Clustering is the process of grouping data into groups or clusters, so that data within the same group retain high level of similarity but groups differ from each other on measuring index. Here, similarities and differences are assessed based on the EED of schools, which often involves distance measures [35]. We use the Euclidean distance method to determine how different EED are among schools in this study. There are n clusters (here referring to schools), d ij represents the distance of EED between school I i and school I j which can be calculated by the following formula: After d ij is calculated, a distance matrix D = d ij n × n is obtained. Given the distance matrix D, the procedure for grouping n schools proceeds in the following steps. Merge a pair of schools whose d ij is the smallest, leaving n-1 clusters for the next step. Next, repeat step one on (n-1) clusters, and get n-2 clusters in total. Then, continue doing this procedure-reducing one cluster at each step-until only one cluster, containing n schools, is formed. The groupings process is shown in the tree diagram (the dendrogram).

Hot spot analysis method
Hot spot analysis based on Kernel density estimation (KDE) Kernel density method is an important statistical analysis method for extracting the distribution characteristics of geospatial attributes. It is based on the rst law of geography, namely, the law of distance attenuation effect. The closer the things are, the more correlated they are to each other. The Kernel density is calculated by the quadratic kernel function f (x, y)[36], whose expression is: where, x i and y i are the coordinates of grid i, n is the total number of schools, h is the bandwidth parameter, that is, the distance attenuation threshold. (x-x i ) 2 + (y-y i ) 2  study area is grided, the density contribution of each school to the center point of each grid in the range h can be calculated using the Kernel function (f(x,y)). With multiple schools, a density map is generated by spatial superposition of the density values of each grid cell. The area with high density value is a hot spot, otherwise, it is a cold spot area.
GIS based spatial hot spot detection and analysis method (Getis ord G i ) GIS based spatial hot spot detection analysis is a method of spatial autocorrelation at the local scales [32] based on distance weight. It is used to measure the correlation, which is represented by the G i index, between an attribute value at a location and the same attribute value at a neighboring location (here the attribute value is represented by EED). When G i is high, it indicates the presence of a hot spot. When G i is low, it indicates a cold spot instead. The G i index is calculated as follows: where: x j is the attribute value (EED) of school j; n is the total number of schools; W ij is the spatial weight between school i and school j within distance d. G i can be standardized in this way: where E(G i ) is the mathematical expected value and VAR(G i ) is the coe cient of variation. When Z(Gi) is positive, it indicates that the value around the spatial unit i is relatively large, which means the spatial unit is a hot spot area; When Z(Gi) is negative, it indicates that the value around the spatial unit i is relatively small, which means the spatial unit is a cold spot area. A shnet with a 5000×5000m grid is created with Create Fishnet tool in the ArcGIS, and the then it is cut by the boundary of Gansu province. After that, Zonal Statistics as Table tool and Join tool are used to link the distribution data of schools with EED attributes to shnet les. Finally, Hot Spot Analysis with Rendering is used to obtain the hotspot distribution map.

Discussion and Evaluation
Evaluation Analysis of differences in English education level A total of 322 senior high schools in the province are selected. Their average EED in the three years ranges from 0.21 to 91.12. Schools that have EED above the average account for 43.37% of all schools, ( ) ( ) √ ( ) and those that have EED below the average account for 56.63%. In 2018, the schools with positive anomalies account for 43.48% of the total, and those with negative anomalies account for 56.52%. The highest anomaly percentage is +188.89%, and the lowest anomaly percentage is -98.55%. In 2019, there are 42.55% of the schools with positive anomalies and 57.45% with negative anomalies. The highest anomaly percentage was +185.53% and the lowest was -99.47%. In 2020, 44.10% of the schools are with positive anomalies and 55.90% are with negative anomalies. The highest percentage anomaly is +165.69%, while the lowest is -100% (Fig. 4). The data of three years of anomaly shows that English performance varies greatly. The lower the anomaly value is, the more disadvantaed the school is. According to our analysis, 71 schools have anomaly values below -70% and 32 schools have anomaly values below -90%, which indicates that they are poor in terms of English education quality.
The schools from cluster analysis are divided into four categories. There are 33 schools in category four, whose EED values range from 71.90 to 91.85, with the mean being 80.22. Category three includes 76 schools that EED of these schools vary from 46.49 to 69.92 with the mean value being 57.62. In category two, there are 74 schools. EED in these schools are from 27.07 to 45.14, with the average being 35.16. Schools in category one are 139 whose EED range from 0.27 to 26.16, with the mean being 10.81. From cluster analysis, we see that many "poverty" schools in English education (Fig. 5) that distribute in the southeast part of Gansu province.

Analysis on the interannual change of English education level
There is a signi cant difference between the percentage change of EED in 2018-2019 and that in 2019-2020 (see Fig. 6). From year 2018 to 2020, 186 schools have improved their English teaching quality, accounting for 57.76% of the total number of schools, while 136 schools have decreased their English teaching quality, accounting for 42.24% of the total number of schools. We regard year 2018-2019 and year 2019-2020 as two stages, calculate the change rate of each stage, and then the change rate of two stages. The schools whose absolute change rate of two stages is more than 100 are shown in Table 1. The school that has declined the most in English teaching level is No. 321, and that has improved the most is No. 293. We use three violin plots to show the distribution of EED for all schools from 2018 to 2020 below (See Figure 7). We can see that the plot of 2018 is similar to that of 2019. The plot of 2020 looks different in that its EED values shift up, indicating an overall improvement in English teaching level.

Regional Differences in English education level
We use Kernel density tool in ArcGIS 10.6 software to analyze the spatial distribution of Kernel density in the study area (Fig. 8). The results show that the density is highest in Lanzhou, the provincial capital, and second highest in Tianshui, Pingliang, Wuwei and Jiuquan. High-density areas represent hot spots of EED, which are regions with high English education quality.
With the help of the spatial hot spot detection and analysis tool in ArcGIS10.2 software, we obtain the distribution of Z(Gi) values.Hot spot and cold spot regions are indicated by Z(Gi) (Figure 7). If Z(Gi)>2, the region is a hot spot region with high English teaching level; if Z(Gi)<-2, the region is a cold spot region with low English teaching level. As can be seen from Figure 9, the hot spots are mainly concentrated in Lanzhou and the central part of Hexi Corridor, while the cold spots are concentrated in the southern Gansu, mainly in Linxia city, the major part of Longnan and the eastern part of Qingyang.

Differences in English education level in the study area
From the English pro ciency test results, English education level varies greatly among different schools.
In 322 senior high schools, the highest average value of EED is 759 times the lowest. Schools with EED below the average value account for more than half of the total number of schools (56.63%) in 2018-2021. 71 schools have anomaly values below -70% and 32 schools have anomaly values below -90%, which indicates that they are poor in terms of English education quality. We carried out investigations on those 32 schools and found that most of them are located in remote mountainous areas and the residental areas of ethnic minority groups. Disparities in educational achievements resulted from ethnic and racial factors are a core issue of educational research. English as a global language is widely used in the world [31], and it is one of the compulsory courses in primary education in China [37]. Students in ethnic areas learn not only their mother tongue, but also English and Chinese as well. Therefore, they encounter more challenges when learning English. First, ethnologue and English belong to different language categories, having different phonetics and grammar characteristics. Second, the learning environment of students in ethnic areas is poor---with a lack of available teaching resources and professionally well-trained English teachers who, if do work there, are asked to speak and understand the ethnologue [6,9]. Third, there exists negative attitude of the schools and patents toward English learning. These challenges make students in ethnic areas learn in a helpless and anxious way, and such mindset also impact the learning performance of subjects other than foreign language [16, 28, 41]. Some researchers think that teachers and their teaching practice are prominent factors in exacerbating or alleviating learned helplessness symptoms and behavior [4,14]. Some suggest that negative attitude or no involvement of parent results in poor academic performance [21,24]. In summary, it is di cult for students in ethnic areas to learn English. In order to resolve the problem and to eliminate the big differences in English education level in the study area, Chinese State Education Commission has released many educational policies such as increasing salaries, implementing a rotation system, giving priority to professional title promotion and so on [19].  [5]. Generally, teacher resources keep stable in high-quality schools, while low-quality schools have a higher turnover rate of teachers than high-quality schools, and private schools have a higher turnover rate than state schools [2]. The results from questionnaries in the study is the consistent with that in literature [2], in addition, the teachers who quit are relatively younger and have shorter teaching experience compared to before. Teacher turnover is disruptive for students' academic attainment [15]. Teaching environment, salary and welfare guarantee are responsible for teacher turnover and loss [25]. Besides, the principal shortage and weak leadership are other contributors to the loss of teachers [13], whose impact is even more severe in schools with high concentrations of ethnic minority students and poor students [30]. The educational regulation has an important in uence on the improvement of teacher allocation in basic education in Gansu Province. Local governments have increased funding for education, improved educational facilities and increased the spots for teachers in ethnic and poor areas. The intensity of improvement was the greatest in 2018 [10]. Corresponding improvement of English academic test scores might happen during 2019-2020.

Aggregation pattern in English education level
From our analysis of the hot spot, we see that the regional differences in English level are signi cant. The English education quality decreases as we move away from the provincial capital (Lanzhou) to the periphery cities. On the provincial level, the regions with high English level are concentrated in Lanzhou and the central part of Hexi Corridor, while the regions with low English level are mainly concentrated in Linxia city, most parts of Longnan and eastern part of Qingyang. This shows an aggregation pattern. The spatial inequality of education can be between provinces, between rural and urban areas, or between districts or counties [12,27]. Peng points out the spatial difference is mainly due to unbalanced economic   Comparison of English teaching level in the two stages in 322 senior high schools Annual differences in EED. The white dots present the median, the thick vertical bars represent the interquartile range, and the thin vertical lines extend to 1.5 times the interquartile range. The density plot width of the violin area presents the frequency distribution.  Hot spot maps showing of spatial clustering patterns for EED: red color represents hot spots; blue color expresses cold spots.